Linux kernel Synchronization

Kernel Synchronization Introduction

Author: Harold Wang 

 

http://blog.csdn.net/hero7935

 

 

Many kernel tasks access and modify share data, like kernel thread, bottom-half, hardware interupt, system call, etc. These operations may involve data overwrite, unconsistent, unstable or even dead lock. It is difficult to find and debug.

Review some basic conceptions:

Critical regin: code that access or modify share data.
--Critical regin should take mutex or synchronization into consideration.

Race condition: Two or more running thread in the same Critical region.

Synchronization: Mechnism that avoid the concurrency and prevent from Race condition.

NOTE: Data is the target should be protected, NOT the CODE!

DeadLock: A set of processes is deadlocked if each process in the set is waiting for an event that only another process in the set can cause.

Important Principle:
1. Lock contention: lock that are contented highly are the bottleneck of the system’s efficiency.
2. Scalability: make the granularity from coarse into fine-grained, as simple as possible.

 

Kernel Synchronization Methods

Per-CPU variable
--an array of data structures, one element per each CPU in the system
--a CPU should not access the elements of the array corresponding to the other CPUs; on the other hand, it can freely read and modify its own element
--can be used only when it makes sense to logically split the data across the CPUs of the system
--provide protection against concurrent accesses from serveral CPUs, they do not provice protection against accesses from asynchronous functions(interrupt handlers and deferrable functions)
As a general rule: a kernel control path should access a per-CPU variable with kernel preemption disabled

Atomic operations
--interrupt-disable can be guaranteed during the operation.
--Linux provide us two mechnisam: atomic_t for int, and for bit operations. Typically used for counter.(note that when you can you atomic, don’t s use  something more complex.
Example:
atomic_t u;
atomic_t v=ATOMIC_INT(0);
atomic_set(&v,4);
atomic_add(2,&v);
atomic_inc(&v);
//If you need to cast atomic_t to int, just use atomic_read(), like:
printk(“%d/n”,atomic_read(&v));
//you can set the value and test it at the same time use operations like, int atomic_sub_and_test, etc.
Example: set_bit(0,&word);

Spin locks
--Spin lock can only be hold by single executable thread. If a thread tries to obtain a spin lock in contention, he will in the state of busy-looping, so holding a spin lock for a long time is obviously a bad idea.
Example:
spinlock_t mr_lock=SPIN_LOCK_UNLOCKED;
spin_lock(&mr_lock);
/*Critical Region*/
spin_unlock(&mr_lock);
NOTE:
1. You should use spin locks in the process of interrupt handler(Using semaphore is not right because it may lead thread to sleep).
2. Before you use spin lock, make the interrupt diable is really necessary(Kernel provide the interface that can obtain lock and diable the interrupt at the same time, use spin_lock_irqsave() and spin_unlock_irqestore() ).

Read_Write_lock
--reader first
--rwlock_t is a 32 bit lock, including 24 bit for counting(concurrent reading operations)
--initialize: rwlock_t mr_rwlock=RW_LOCK_UNLOCKED;

Seqlock
--writer first
--writer don’t need to wait when there are reader in the critical region.
--writer obtain Seqlock and increase the sequence, when finishing, increate the sequence again.
--reader should test the sequence’s value between the former one and the latter one(before reading and after reading).Odd number or inconsistent means the previous reading is invalid.
Example:
write_seqlock(&seqlock_a); 
/*write operation*/
write_seqlock(&seqlock_a);
do
{
seqnum=read_seqbegin(&seqlock_a);
/*read operation*/
}while(read_seqretry(&seqlock_a,seqnum));

RCU(read_copy_update)
--new in 2.6, used in network level and VFS(virtual file system)
--allow multiple readers and writers
--Don’t use lock
--constrains: protect only the data structure that are dynamic allocated and referened by ptr.  Sleep is NOT allowed.

Semaphore
--kernel semaphore and System V IPC semaphore. Semaphore in linux is a kind of sleep lock. The definition in the kernel is as followed:
struct semaphore
{
atomic_t count;
int sleepers;
wait_queue_head_t wait;
}
Example:
static DECLARE_MUTEX(mr_sem);
if(down_interruptible(&mr_sem));
/*Critical Region*/
up(&mr_sem);

Read_Write_Semaphore
--just like read_write_lock and semaphore.
static DECLARE_RWSEM(name);
down_read_trylock();
down_write_trylock();

Competion Variable
--simple method to replace semphore in some situations.
struct completion
{
unsigned int done;
wait_queue_head_t wait;
}
DECLARE_COMPLETION(mr_comp);
init_completion();
wait_for_completion();
completion();

Memory Barrier
--typically used for SMP system. see another article of mine here which provide very specific description.

Author: Harold Wang
http://blog.csdn.net/hero7935
Thank you.