Linux Debugging 1 - Kernel Introduction

1. linux kernel version

/home/a/j/nomad2:uname -r 2.6.24-22-generic

Major.Minor.Release[extra]

 

Minor: Even minor numbers denote stable "production" kernel, whereas odd ones are reserved for development kernels.

 

API changes in the 2.6 kernel series refer to http://lwn.net/Articles/2.6-kernel-api/

 

解压kernel代码：

 

>>gzip -cd linux-2.6.26.tar.gz | tar xvf - >>tar zxvf linux-2.6.26.tar.gz

打补丁命令：

>>gzip -cd patch.gz | patch -p0 >>bzip2 -cd patch.gz | patch -p0

 

2. Linux Kernel的Feature

1) Implement generic UNIX kernel structure

2) Monolithic, thread kernel

3) user threads are LWP

4) kernel is preemptive (in 2.6)

5) small footprint

6) Modular and Extensible

 

3. 2.6的内核实现了抢占调度，抢占点在于：

- On return from Interrupt handlers

- On blocking Kernel calls

- when the Kernel preemption is reenabled (call preempt_enable)

- when a kernel thread voluntarily yields by calling schedule.

 

4. Schedulers

refer to : http://lwn.net/Articles/230574/

 

1) in 2.4 version, scheduler is O(n), while in 2.6 it is O(1), which is more efficient and provide natural CPU affinity.  It has one runqueue per processor(每个CPU都有自己的runqueue), and each runqueue contains 2 priority arrays. And it use bitmap to find the highest priority task(进程或者线程) to run.

We can check the codes from LXR.

 

RT task priority is from 0 to 100, and normal task priority is from 101 to 139 (from sched.h)

510/* 511 * Priority of a process goes from 0..MAX_PRIO-1, valid RT 512 * priority is 0..MAX_RT_PRIO-1, and SCHED_NORMAL/SCHED_BATCH 513 * tasks are in the range MAX_RT_PRIO..MAX_PRIO-1. Priority 514 * values are inverted: lower p->prio value means higher priority. 515 * 516 * The MAX_USER_RT_PRIO value allows the actual maximum 517 * RT priority to be separate from the value exported to 518 * user-space. This allows kernel threads to set their 519 * priority to a value higher than any user task. Note: 520 * MAX_RT_PRIO must not be smaller than MAX_USER_RT_PRIO. 521 */ 522 523#define MAX_USER_RT_PRIO 100 524#define MAX_RT_PRIO MAX_USER_RT_PRIO 525 526#define MAX_PRIO (MAX_RT_PRIO + 40)

 

调度算法(from sched.c)

2198new_array: 2199 /* Start searching at priority 0: */ 2200 idx = 0; 2201skip_bitmap: 2202 if (!idx) 2203 idx = sched_find_first_bit(array->bitmap); 2204 else 2205 idx = find_next_bit(array->bitmap, MAX_PRIO, idx); 2206 if (idx >= MAX_PRIO) { 2207 if (array == busiest->expired && busiest->active->nr_active) { 2208 array = busiest->active; 2209 dst_array = this_rq->active; 2210 goto new_array; 2211 } 2212 goto out; 2213 } 2214 2215 head = array->queue + idx; 2216 curr = head->prev; ...

 

看一下sched_find_first_bit的实现，常数时间即可选择出最小的不为0的bit，所以为O(1)调度算法。

7/* 8 * Every architecture must define this function. It's the fastest 9 * way of searching a 140-bit bitmap where the first 100 bits are 10 * unlikely to be set. It's guaranteed that at least one of the 140 11 * bits is cleared. 12 */ 13static inline int sched_find_first_bit(const unsigned long *b) 14{ 15#if BITS_PER_LONG == 64 16 if (unlikely(b[0])) 17 return __ffs(b[0]); 18 if (likely(b[1])) 19 return __ffs(b[1]) + 64; 20 return __ffs(b[2]) + 128; 21#elif BITS_PER_LONG == 32 22 if (unlikely(b[0])) 23 return __ffs(b[0]); 24 if (unlikely(b[1])) 25 return __ffs(b[1]) + 32; 26 if (unlikely(b[2])) 27 return __ffs(b[2]) + 64; 28 if (b[3]) 29 return __ffs(b[3]) + 96; 30 return __ffs(b[4]) + 128; 31#else 32#error BITS_PER_LONG not defined 33#endif 34}

 

问题：高优先级的任务总是先被调度，低优先级的任务无法被调度，starvation.

 

2) In 2.6.23 release, a CFS (Completely Fair Scheduler) is added, see this from wiki (http://en.wikipedia.org/wiki/Completely_Fair_Scheduler), and we can get more info from http://kerneltrap.org/node/8059

 

In contrast to the previous O(1) scheduler used in older Linux 2.6 kernels, the CFS scheduler implementation is not based on run queues. Instead, a red-black tree implements a "timeline" of future task execution. Additionally, the scheduler uses nanosecond granularity accounting, the atomic units by which an individual process' share of the CPU was allocated (thus making redundant the previous notion of timeslices). The fair queuing CFS scheduler has a scheduling complexity of O(log N), where N is the number of tasks in the runqueue. Choosing a task can be done in constant time, but reinserting a task after it has run requires O(log N) operations, because the runqueue is implemented as a red-black tree.

 

High priority tasks still get larger slices, but tasks are ordered by "vruntime", which means schedule cpu-starved threads first, low priority threads get their smaller slice but at least execute.

 

3) IO scheduler

2.6 kernel has 4 I/O schedulers, No-op I/O scheduler, Anticipatory I/O scheduler(default), Deadline I/O scheduler与CFQ I/O scheduler. In 2.4, use the "Linus Elevator" (No-op I/O scheduler).问题主要是“请求饥饿”。

/home/a/j/nomad2:pwd /sys/block/sda/queue /home/a/j/nomad2:ls iosched max_hw_sectors_kb max_sectors_kb nr_requests read_ahead_kb scheduler /home/a/j/nomad2:cat scheduler noop anticipatory deadline [cfq]

 

5. Kernel Architecture

 

old system call is using "int $0x80", and 2.6 use "sysenter" and the syscall number is in EAX. see syscall_table.S

 

6. Devices

分类：字符设备(例如：内存，终端，IO ports,键盘，鼠标等), 块设备 (buffered) device(硬盘,闪存等).

 

例如： mknod /dev/testdev c 254 0创建一个字符设备，major号(the number of the device registered in the kernel)是254，minor号(the number of device registered in the driver)是0.

 

/home/a/j/nomad2:pwd /proc /home/a/j/nomad2:cat devices Character devices: 1 mem 2 pty 3 ttyp 4 /dev/vc/0 4 tty 4 ttyS 5 /dev/tty 5 /dev/console 5 /dev/ptmx 6 lp 7 vcs 10 misc 13 input 21 sg 29 fb 128 ptm 136 pts 180 usb 189 usb_device 253 usb_endpoint 254 rtc Block devices: 1 ramdisk 8 sd 11 sr 65 sd 66 sd 67 sd 68 sd 69 sd 70 sd 71 sd 128 sd 129 sd 130 sd 131 sd 132 sd 133 sd 134 sd 135 sd