目录
一、锁的使用
当操作共享可写数据时,需要加锁进行保护。
1、使用锁的缺点
- 性能会受到影响,更糟糕的是,这些不利影响可能会在数百核的高端多核系统上成倍增加。现实生活中的一种场景,是繁忙高速公路上的单一收费站或繁忙十字路口的红绿灯,严重影响通行性能。
- 锁的竞争:增加系统内锁的数量,有利于降低两个或多个进程/线程之间对特定锁的争用。
但出现死锁的机会大增。 - 一些列问题:性能问题,死锁,优先级转换风险、优先级高的需要等到优先级低的进程等。
2、解决方法
lock-free技术:per-CPU与lock-free数据结构RCU
原理:通过拷贝一份变量
二、per-CPU使用
头文件 #include <linux/percpu.h>
有两种申请方式:动态申请和静态申请
1)动态
申请
- alloc_percpu()
- alloc_percpu_gfp()
- devm_alloc_percpu(),第一个参数dev
释放
- void free_percpu(void __percpu *__pdata)
2)静态
- DEFINE_PER_CPU(int, pcpa);
3)注意事项
void *p;
val = get_cpu_var(pcpa);
p = vmalloc(20000);
pr_info("cpu1: pcpa = %+d\n", val);
put_cpu_var(pcpa);
vfree(p);
- get_cpu_var()/put_cpu_var() 之间的数据必须是原子的并且不能阻塞
- 所以禁用内核抢占,不允许任何类型的阻塞(或休眠)
- vmalloc、printk() 或者pr_foo<>是 可能会睡眠
4)引用使用
get_cpu_var()会引用preempt_disable(),禁止内核竞争
put_cpu_var()会引用preempt_enable()
增加per-CPU的变量
get_cpu_var(pcpa) ++;
put_cpu_var(pcpa);
或者per_cpu(var,cpu) 如遍历每个CPU核的pcpa的变量
for_each_online_cpu(i) {
val = per_cpu(pcpa, i);
pr_info(" cpu %2d: pcpa = %+d\n", i, val);
}
通过指针指向变量
{get,put}_cpu_ptr()
三、per-CPU在内核中的使用
current 变量
// arch/x86/include/asm/current.h
struct task_struct;
DECLARE_PER_CPU(struct task_struct *, current_task);
static __always_inline struct task_struct *get_current(void)
{
return this_cpu_read_stable(current_task);
}
#define current get_current()
current_task的变量什么时候更新? 上下文切换的时候
//源码文件arch/x86/kernel/process_64.c
__visible __notrace_funcgraph struct task_struct *
__switch_to(struct task_struct *prev_p, struct task_struct *next_p)
{
[ ... ]
this_cpu_write(current_task, next_p);
[ ... ]
}
四、实例
两个线程使用共享变量,使用per-CPU,最后各自的输出是正确的
#define THRD0_ITERS 3
static int thrd_work(void *arg)
{
int i, val;
long thrd = (long)arg;
struct drv_ctx *ctx;
if (set_cpuaffinity(thrd) < 0) {
pr_err("setting cpu affinity mask for our kthread %ld failed\n", thrd);
return -ENOSYS;
}
SHOW_CPU_CTX();
if (thrd == 0) { /* kthread #0 runs on CPU 0 */
for (i=0; i<THRD0_ITERS; i++) {
/* Operate on our perpcu integer */
val = ++ get_cpu_var(pcpa);
pr_info(" thrd_0/cpu0: pcpa = %+d\n", val);
put_cpu_var(pcpa);
ctx = get_cpu_ptr(pcp_ctx);
ctx->tx += 100;
pr_info(" thrd_0/cpu0: pcp ctx: tx = %5d, rx = %5d\n",
ctx->tx, ctx->rx);
put_cpu_ptr(pcp_ctx);
}
} else if (thrd == 1) { /*kthread #1 runs on CPU 1 */
for (i=0; i<THRD1_ITERS; i++) {
val = -- get_cpu_var(pcpa);
pr_info(" thrd_1/cpu1: pcpa = %+d\n", val);
ctx = get_cpu_ptr(pcp_ctx);
ctx->rx += 200;
pr_info(" thrd_1/cpu1: pcp ctx: tx = %5d, rx = %5d\n",
ctx->tx, ctx->rx);
put_cpu_ptr(pcp_ctx);
}
}
程序输出
benshushu:2_percpu# insmod percpu_var.ko
[ 3180.878394] percpu_var: loading out-of-tree module taints kernel.
[ 3180.888442] percpu_var: module verification failed: signature and/or required key missing - tainting kernel
[ 3180.912374] percpu_var:init_percpu_var(): inserted
[ 3180.928186] percpu_var:thrd_work(): *** kthread PID 837 on cpu 1 now ***
[ 3180.928853] percpu_var:thrd_work(): thrd_1/cpu1: pcpa = -1
[ 3180.933631] percpu_var:thrd_work(): *** kthread PID 836 on cpu 0 now ***
[ 3180.939866] percpu_var:thrd_work(): thrd_0/cpu0: pcpa = +1
[ 3180.939947] percpu_var:thrd_work(): thrd_0/cpu0: pcp ctx: tx = 100, rx = 0
[ 3180.939968] percpu_var:thrd_work(): thrd_0/cpu0: pcpa = +2
[ 3180.939977] percpu_var:thrd_work(): thrd_0/cpu0: pcp ctx: tx = 200, rx = 0
[ 3180.939984] percpu_var:thrd_work(): thrd_0/cpu0: pcpa = +3
[ 3180.939992] percpu_var:thrd_work(): thrd_0/cpu0: pcp ctx: tx = 300, rx = 0
[ 3180.940691] percpu_var:disp_vars(): 000) [thrd_0/0]:836 | ...0 /* disp_vars() */
[ 3180.940786] percpu_var:disp_vars(): cpu 0: pcpa = +3, rx = 0, tx = 300
[ 3180.940801] percpu_var:disp_vars(): cpu 1: pcpa = -1, rx = 0, tx = 0
[ 3180.940811] percpu_var:disp_vars(): cpu 2: pcpa = +0, rx = 0, tx = 0
[ 3180.940822] percpu_var:disp_vars(): cpu 3: pcpa = +0, rx = 0, tx = 0
[ 3180.940838] percpu_var:thrd_work(): Our kernel thread #0 exiting now...
[ 3180.956032] percpu_var:thrd_work(): thrd_1/cpu1: pcp ctx: tx = 0, rx = 200
[ 3180.956454] percpu_var:thrd_work(): thrd_1/cpu1: pcpa = -2
[ 3180.956821] percpu_var:thrd_work(): thrd_1/cpu1: pcp ctx: tx = 0, rx = 400
[ 3180.959607] percpu_var:thrd_work(): thrd_1/cpu1: pcpa = -3
[ 3180.960655] percpu_var:thrd_work(): thrd_1/cpu1: pcp ctx: tx = 0, rx = 600
[ 3180.963535] percpu_var:disp_vars(): 001) [thrd_1/1]:837 | .N.0 /* disp_vars() */
[ 3180.965357] percpu_var:disp_vars(): cpu 0: pcpa = +3, rx = 0, tx = 300
[ 3180.969093] percpu_var:disp_vars(): cpu 1: pcpa = -3, rx = 600, tx = 0
[ 3180.970835] percpu_var:disp_vars(): cpu 2: pcpa = +0, rx = 0, tx = 0
[ 3180.971269] percpu_var:disp_vars(): cpu 3: pcpa = +0, rx = 0, tx = 0
[ 3180.971561] percpu_var:thrd_work(): Our kernel thread #1 exiting now...
解析
- 两个线程共享的数据 val与ctx,互不干扰
- 线程0执行后,val=3; ctx->tx += 100, 最后结果 300
- 线程1执行后,val=-3;ctx->rx += 200; 最后结果 600