linux中的内存分配接口简析(五)

1.概览

  那内存分配对于驱动工程师来说就是指的DRAM，不包括iROM/SRAM/CPU cache等SOC内部的内存。下面提到的内存管理，其所管理的主体就是DRAM。在linux中对内存的描述单位是页page，它对应的数据结构为struct page，定义如下

//include\linux\mm_types.h
struct page {
    struct {	/* Page cache and anonymous pages */
        	/* See page-flags.h for PAGE_MAPPING_FLAGS */
			struct address_space *mapping;
            ...
    };
    struct {	/* slab, slob and slub */
    ...};
    struct {	/* ZONE_DEVICE pages */
    };
    ...
    void *virtual;			/* Kernel virtual address (NULL if
};

  一般而言的我们申请内存的时候喜欢按byte，但是对kernel来说它是按page来描述内存的，即一次分配的最小单位就页，页的大小在编译的时候是可配置的一般都是4KB没页。

//arch/arm64/Kconfig
config ARM64_PAGE_SHIFT
	int
	default 16 if ARM64_64K_PAGES
	default 14 if ARM64_16K_PAGES
	default 12

//arch\arm64\include\asm\page-def.h
#define PAGE_SHIFT		CONFIG_ARM64_PAGE_SHIFT
#define PAGE_SIZE		(_AC(1, UL) << PAGE_SHIFT)

  可见在没有设置ARM64_PAGE_SHIFT的时候，PAGE_SHIFT也就是12因此PAGE_SIZE为2的12次方即4096(4KB)。kernel管理内存的最小单位为page，其最小单位都是4KB，显然这太大了，对于设备驱动开发，大部分场景并不需要如此巨大的内存，因为他们也就存存几个结构体罢了。所以，kernel开发者又基于page开发了slab等辅助函数，类似于libc库，里面提供的接口则支持最小单位字节方式的内存分配，以及下面提到的kmem_xx系列分配接口都是来自slab库提供的。
  讨论内存分配接口前，读者需要注意下面几个概念
    a)过进程上下文，即本次的内核调用是由用户进程触发陷入的。
    b)每一个用户进程的寻址空间由两部分组成，其一为用户地址空间，其二为内核地址空间。并且这两个地址空间对于的物理内存在没有做映射的情况下是不允许相互访问的。
    c)因为linux中，每一个进程的寻址空间在虚拟地址的概念上是完全一样的，所以在不同的进程中即使是同一个虚拟地址所对应的物理内存也是不一样的。每个进程都会维护一个其专属的页表，用于表示虚拟地址和物理地址的对应关系，在进程切换的时候，页表的基地址也会随之切换。
    d)此处所讨论的地址没有特别说明的化都是指代虚拟地址并且已建立页表。

2.使用页(Page)分配内存

  此处的page也就是上面提到过的内存管理的最小单位了，所以分配页的话最小单位就是4KB了。

2.1 下面是使用页的示例代码

struct my_page {
	u8 mem[PAGE_SIZE - sizeof(int)];
	int size;
};
static struct my_page *mp;
//分配
alloc_memory_test_drv_init
	struct page *tmp_page = NULL;
    tmp_page = alloc_page(GFP_KERNEL);
    mp = page_address(tmp_page);
    mp->size = PAGE_SIZE - sizeof(int);
	sprintf(mp->mem, "This is a free mem from page which size is %d, [PAGE_SIZE]%d.", mp->size, PAGE_SIZE);
	ALLOC_MEMORY_TEST_INFO("%s", mp->mem)
//释放
alloc_memory_test_drv_exit
	free_page((unsigned long)mp);
	mp = NULL;

  自定义了my_page来管理page，后续使用内存就不需要再像kernel申请了，并且这种方式分配到内存不论在虚拟还是物理地址上都是连续的。使用GFP_KERNEL标志分配内存代表其可能会进入休眠，下面是常用的标志
    GFP_ATOMIC
    users can not sleep and need the allocation to succeed. A lower watermark is applied to allow access to “atomic reserves”.
    GFP_KERNEL
    it is typical for kernel-internal allocations. The caller requires %ZONE_NORMAL or a lower zone for direct access but can direct reclaim.

2.2 运行结果如下

[11829.469443] allocMemoryTest:alloc_memory_test_drv_init,163:This is a free mem from page which size is 4092, [PAGE_SIZE]4096.

3.使用slab分配内存–内存物理地址连续

3.1 kmalloc

  该接口则是支持最小以字节为单位分配内存的，和上层的malloc使用类似。

3.1.1 下面是示例代码

struct flagstaff_mem {
	char desc[60];
	struct list_head list;
};

alloc_memory_test_drv_init
    struct flagstaff_mem *my_mem = kmalloc(sizeof(struct flagstaff_mem), GFP_KERNEL);
    sprintf(my_mem->desc, "flagstaffmem create by kmalloc");
    list_add_tail(&my_mem->list, &flagstaff_mem_head);
alloc_memory_test_drv_exit
	struct flagstaff_mem *my_mem_tmp = NULL;
	my_mem = list_first_entry(&flagstaff_mem_head, struct flagstaff_mem, list);
    list_del(&my_mem->list);
	kfree(my_mem);

  此处使用kmalloc分配一个flagstaff_mem，并将其挂到全局队列flagstaff_mem_head中去。kmalloc的使用也很简单，内存获取成功则为非NULL值，入参一为申请的大小单位为字节，入参二用于指示申请内存过程的行为，例如GFP_ATOMIC则不会造成休眠，这可以用在原子及中断上下文中。使用kmalloc需要和kfree配对使用，否则会造成内存泄漏。

3.1.2 下面是运行结果

[11829.469431] allocMemoryTest:alloc_memory_test_drv_init,145:show_list:[0xffffff882dc48a80]flagstaffmem create by kmalloc

3.2 kmem_cache

  该接口方便用户分配自定义类型的数据结构，类似于面向对象中的new。

3.2.1 示例代码如下

static struct kmem_cache *kc;
alloc_memory_test_drv_init
	kc = kmem_cache_create("flagstaff_mem",
		sizeof(struct flagstaff_mem),
		0,
		SLAB_PANIC,
		flagstaff_kmem_alloc);
#define FLAGSTAFF_MEM_COUNT_MAX		1
	for(count = 0 ; count < FLAGSTAFF_MEM_COUNT_MAX; ++count) {
		my_mem = kmem_cache_alloc(kc, GFP_KERNEL);
		if(!IS_ERR_OR_NULL(my_mem)) {
			list_add_tail(&my_mem->list, &flagstaff_mem_head);
		}
	}

	list_for_each_entry(my_mem, &flagstaff_mem_head, list)
		ALLOC_MEMORY_TEST_INFO("show_list:[0x%llx]%s", my_mem, my_mem->desc);

alloc_memory_test_drv_exit
	struct flagstaff_mem *my_mem = NULL;
	struct flagstaff_mem *my_mem_tmp = NULL;
    list_for_each_entry_safe(my_mem, my_mem_tmp, &flagstaff_mem_head, list) {
        ALLOC_MEMORY_TEST_INFO("free:%s", my_mem->desc);
        list_del(&my_mem->list);
        kmem_cache_free(kc, my_mem);

  kmem_cache_create的定义如下

//mm\slab_common.c
struct kmem_cache *
kmem_cache_create(const char *name, unsigned int size, unsigned int align,
		slab_flags_t flags, void (*ctor)(void *))

  其入参说明如下
  name
    A string which is used in /proc/slabinfo to identify this cache.
  size
    The size of objects to be created in this cache.每个对象的大小。
  align
    The required alignment for the objects。内部的对齐方式。
  flags
    SLAB flags.这里设置为SLAB_PANIC，意味着如果创建失败就直接panic 内核。
  ctor
    A constructor for the objects.在object创建的时候回调，可以做对象的初始化操作，和c++中的构造函数没什么区别。
  下面就是构造函数，我们只简单的设置了下desc用于后续打印区分不同的object。

static void flagstaff_kmem_alloc(void *data) {
	static int alloc_count = 0;
	struct flagstaff_mem *fmem = data;
	sprintf(fmem->desc, "flagstaff_obj_%d", alloc_count++);
	ALLOC_MEMORY_TEST_INFO("The new flagstaff_mem is [0x%llx]%s", (u64)fmem, fmem->desc);
}

3.2.2 运行结果

[11829.469158] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005eb58]flagstaff_obj_0
[11829.469172] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e2c0]flagstaff_obj_1
[11829.469180] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005eec8]flagstaff_obj_2
[11829.469185] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e210]flagstaff_obj_3
[11829.469190] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005eb00]flagstaff_obj_4
[11829.469195] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e160]flagstaff_obj_5
[11829.469201] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005edc0]flagstaff_obj_6
[11829.469207] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005ed10]flagstaff_obj_7
[11829.469213] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005ef78]flagstaff_obj_8
[11829.469219] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e790]flagstaff_obj_9
[11829.469224] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e528]flagstaff_obj_10
[11829.469231] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005ef20]flagstaff_obj_11
[11829.469238] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005ed68]flagstaff_obj_12
[11829.469244] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e7e8]flagstaff_obj_13
[11829.469250] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e948]flagstaff_obj_14
[11829.469255] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e108]flagstaff_obj_15
[11829.469261] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e0b0]flagstaff_obj_16
[11829.469266] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e840]flagstaff_obj_17
[11829.469271] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e478]flagstaff_obj_18
[11829.469277] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e1b8]flagstaff_obj_19
[11829.469282] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e000]flagstaff_obj_20
[11829.469288] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005ecb8]flagstaff_obj_21
[11829.469293] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e898]flagstaff_obj_22
[11829.469298] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005ee18]flagstaff_obj_23
[11829.469304] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005ec08]flagstaff_obj_24
[11829.469310] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e580]flagstaff_obj_25
[11829.469315] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e318]flagstaff_obj_26
[11829.469321] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e3c8]flagstaff_obj_27
[11829.469326] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e688]flagstaff_obj_28
[11829.469331] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005ebb0]flagstaff_obj_29
[11829.469336] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e9f8]flagstaff_obj_30
[11829.469342] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005ee70]flagstaff_obj_31
[11829.469350] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e9a0]flagstaff_obj_32
[11829.469358] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e6e0]flagstaff_obj_33
[11829.469365] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e5d8]flagstaff_obj_34
[11829.469369] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e268]flagstaff_obj_35
[11829.469375] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e630]flagstaff_obj_36
[11829.469381] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e058]flagstaff_obj_37
[11829.469386] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e420]flagstaff_obj_38
[11829.469391] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005eaa8]flagstaff_obj_39
[11829.469397] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e370]flagstaff_obj_40
[11829.469403] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e8f0]flagstaff_obj_41
[11829.469408] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e4d0]flagstaff_obj_42
[11829.469414] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005ec60]flagstaff_obj_43
[11829.469419] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005ea50]flagstaff_obj_44
[11829.469424] allocMemoryTest:flagstaff_kmem_alloc,100:The new flagstaff_mem is [0xffffff890005e738]flagstaff_obj_45
[11829.469443] allocMemoryTest:alloc_memory_test_drv_init,163:This is a free mem from page which size is 4092, [PAGE_SIZE]4096.

  kmem_cahe 中内存的分配遵循以下两个条件
    a)惰性创建，即只有对缓存队列执行第一次分配操作后，才会向系统申请内存。
    b)object的申请数量实际上会大于等于目前真正需要的，所以如果第一次调用接口 kmem_cache_alloc即使只分配1个，那么缓存队列所拥有的对象总是大于1的。这也就是缓存的命名吧。从上面的log中也能看出来，对然只请求分配1个object，但是构造函数实际上被调用了46次，这也意味着缓存管理着46个object大小的内存。

4.使用vmalloc–进程内虚拟地址连续

 该接口只能保证虚拟地址上的连续，这意味着这些虚拟地址的对应的的物理地址并不一定连续的。但相较于kmalloc可以申请的内存更加巨大。

4.1 代码示例

alloc_memory_test_drv_init
   	vfm = vmalloc(sizeof(struct flagstaff_mem));
	sprintf(vfm->desc, "This memory is from vmalloc.", vfm);
	ALLOC_MEMORY_TEST_INFO("[vfm->desc]%s", vfm->desc);

4.2 执行结果

[11829.469480] allocMemoryTest:alloc_memory_test_drv_init,173:[vfm->desc]This memory is from vmalloc.

5.完整源码如下

alloc_memory_test

https://gitee.com/solo-king/linux-kernel-base-usage/blob/master/flagstaff/alloc_memory_test.c