NUMA初探

最新推荐文章于 2024-04-18 10:38:29 发布
最新推荐文章于 2024-04-18 10:38:29 发布
阅读量500 收藏
点赞数
分类专栏： kernel numa
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/raykwok1150/article/details/118368660
版权

工具

numactl

github地址：https://github.com/numactl/numactl
在ubuntu上可以直接安装。

(base) [root@localhost test]# numactl --hardware
available: 8 nodes (0-7)
node 0 cpus: 0 1 2 3 4 5 6 7
node 0 size: 65431 MB
node 0 free: 35258 MB
node 1 cpus: 8 9 10 11 12 13 14 15
node 1 size: 65535 MB
node 1 free: 62369 MB
node 2 cpus: 16 17 18 19 20 21 22 23
node 2 size: 65535 MB
node 2 free: 64002 MB
node 3 cpus: 24 25 26 27 28 29 30 31
node 3 size: 65535 MB
node 3 free: 63536 MB
node 4 cpus: 32 33 34 35 36 37 38 39
node 4 size: 65535 MB
node 4 free: 63747 MB
node 5 cpus: 40 41 42 43 44 45 46 47
node 5 size: 65535 MB
node 5 free: 63872 MB
node 6 cpus: 48 49 50 51 52 53 54 55
node 6 size: 65535 MB
node 6 free: 64035 MB
node 7 cpus: 56 57 58 59 60 61 62 63
node 7 size: 65535 MB
node 7 free: 61804 MB
node distances:
node   0   1   2   3   4   5   6   7 
  0:  10  16  16  16  28  28  22  28 
  1:  16  10  16  16  28  28  28  22 
  2:  16  16  10  16  22  28  28  28 
  3:  16  16  16  10  28  22  28  28 
  4:  28  28  22  28  10  16  16  16 
  5:  28  28  28  22  16  10  16  16 
  6:  22  28  28  28  16  16  10  16 
  7:  28  22  28  28  16  16  16  10 
(base) [root@localhost test]# 
(base) [root@localhost test]# numastat
                           node0           node1           node2           node3
numa_hit                17238639         2348706          571063          748099
numa_miss                      0               0               0               0
numa_foreign                   0               0               0               0
interleave_hit             27721           27729           27726           27734
local_node              17210141         2216993          493770          671023
other_node                 28498          131713           77293           77076

                           node4           node5           node6           node7
numa_hit                 1034606          876289          562762         1165051
numa_miss                      0               0               0               0
numa_foreign                   0               0               0               0
interleave_hit             27715           27744           27724           27727
local_node                956360          751729          472855          735365
other_node                 78246          124560           89907          429686
(base) [root@localhost test]#

MLC

Intel 的Memory Latency Checker是用来检测numa 内存延迟和带宽的工具，一般使用下面两个命令：

在这里插入代码片

rocm_bandwidth_test

AMD的rocm软件栈中也有一个类似MLC的工具，用于测试GPU和CPU之间的带宽，一般使用方式如下：

(base) [root@localhost centos7.6]# ./rocm-bandwidth-test 
............................................................................................................................................................................................................................................
          RocmBandwidthTest Version: 2.3.11

          Launch Command is: ./rocm-bandwidth-test (rocm_bandwidth -a + rocm_bandwidth -A)


          Device: 0,  
          Device: 1,  
          Device: 2,  
          Device: 3,  
          Device: 4,  
          Device: 5,  
          Device: 6,  
          Device: 7,  
          Device: 8,  C888888,  c5:0.0
          Device: 9,  C888888,  c8:0.0
          Device: 10,  C888888,  cb:0.0
          Device: 11,  C888888,  ce:0.0

          Inter-Device Access

          D/D       0         1         2         3         4         5         6         7         8         9         10        11        

          0         1         1         1         1         1         1         1         1         1         1         1         1         

          1         1         1         1         1         1         1         1         1         1         1         1         1         

          2         1         1         1         1         1         1         1         1         1         1         1         1         

          3         1         1         1         1         1         1         1         1         1         1         1         1         

          4         1         1         1         1         1         1         1         1         1         1         1         1         

          5         1         1         1         1         1         1         1         1         1         1         1         1         

          6         1         1         1         1         1         1         1         1         1         1         1         1         

          7         1         1         1         1         1         1         1         1         1         1         1         1         

          8         1         1         1         1         1         1         1         1         1         1         1         1         

          9         1         1         1         1         1         1         1         1         1         1         1         1         

          10        1         1         1         1         1         1         1         1         1         1         1         1         

          11        1         1         1         1         1         1         1         1         1         1         1         1         


          Inter-Device Numa Distance

          D/D       0         1         2         3         4         5         6         7         8         9         10        11        

          0         0         16        16        16        28        28        22        28        42        42        42        42        

          1         16        0         16        16        28        28        28        22        48        48        48        48        

          2         16        16        0         16        22        28        28        28        48        48        48        48        

          3         16        16        16        0         28        22        28        28        48        48        48        48        

          4         28        28        22        28        0         16        16        16        36        36        36        36        

          5         28        28        28        22        16        0         16        16        36        36        36        36        

          6         22        28        28        28        16        16        0         16        20        20        20        20        

          7         28        22        28        28        16        16        16        0         36        36        36        36        

          8         42        48        48        48        36        36        20        36        0         40        40        40        

          9         42        48        48        48        36        36        20        36        40        0         40        40        

          10        42        48        48        48        36        36        20        36        40        40        0         40        

          11        42        48        48        48        36        36        20        36        40        40        40        0         


          Unidirectional copy peak bandwidth GB/s

          D/D       0           1           2           3           4           5           6           7           8           9           10          11          

          0         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         8.235595    8.234948    8.234949    8.235756    

          1         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         8.238830    8.238668    8.239155    8.238830    

          2         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         8.238830    8.237859    8.239155    8.238830    

          3         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         8.239154    8.238830    8.238831    8.240125    

          4         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         13.080635   13.020941   13.023370   13.079411   

          5         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         13.081043   13.020941   13.022561   13.081451   

          6         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         13.022558   13.020941   13.022561   13.081859   

          7         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         13.021750   13.020941   13.080230   13.021750   

          8         7.550506    7.538700    7.549148    7.549148    14.106568   14.106568   14.106093   14.105619   587.026452  14.106568   14.106093   14.105619   

          9         7.550370    7.538157    7.549147    7.548875    14.106568   14.105619   14.106568   14.106568   14.106568   585.796648  14.106093   14.106568   

          10        7.551459    7.538430    7.549148    7.549148    14.106096   14.106571   14.106571   14.106571   14.106571   14.106571   586.615944  14.108469   

          11        7.550371    7.538158    7.549148    7.549012    14.105619   14.106568   14.105144   14.105144   14.106093   14.106568   14.105619   586.615944  


          Bdirectional copy peak bandwidth GB/s

          D/D       0           1           2           3           4           5           6           7           8           9           10          11          

          0         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         11.942445   11.976716   12.012391   11.968857   

          1         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         11.960665   12.067514   12.057456   12.037557   

          2         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         12.024616   12.061441   12.047932   12.067515   

          3         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         12.003796   12.075679   12.070295   12.037212   

          4         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         22.000032   22.024291   22.037603   22.068907   

          5         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         22.087501   22.019088   22.090995   22.081687   

          6         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         22.227966   22.330926   22.292356   22.303021   

          7         N/A         N/A         N/A         N/A         N/A         N/A         N/A         N/A         21.940764   21.974099   22.007539   21.992534   

          8         11.942445   11.960665   12.024616   12.003796   22.000032   22.087501   22.227966   21.940764   N/A         27.127089   27.107968   27.126924   

          9         11.976716   12.067514   12.061441   12.075679   22.024291   22.019088   22.330926   21.974099   27.127089   N/A         27.743500   27.713025   

          10        12.012391   12.057456   12.047932   12.070295   22.037603   22.090995   22.292356   22.007539   27.107968   27.743500   N/A         27.713946   

          11        11.968857   12.037557   12.067515   12.037212   22.068907   22.081687   22.303021   21.992534   27.126924   27.713025   27.713946   N/A         


(base) [root@localhost centos7.6]#

API

在安装完numactl或者libnuma-dev以后系统中应该会有如下两个头文件/usr/inclue/numa.h和/usr/include/numaif.h，二者的功能有一些重叠，笔者暂时还没弄清楚为啥这么设置。
详细函数的使用咱先跳过，直接看代码。

#include <stdio.h>
#include <stdlib.h>
#include <numa.h>
#include <numaif.h>
#include <errno.h>
#include <sys/mman.h>
#ifndef MPOL_F_STATIC_NODES
/* Bug in numaif.h, this should be defined in there. Definition copied
 * from linux/mempolicy.h.
 */
#define MPOL_F_STATIC_NODES     (1 << 15)
#endif

void print_bitmask(struct bitmask* mask) {
    printf("bitmask size: %lu \n", mask->size);
    printf("bitmask val: 0x%lx \n", *mask->maskp);
}

void print_nodes(int node_count) {
    long long fsz = 0;
    for (int i = 0; i < node_count; i++) {
        long long sz = numa_node_size64(i, &fsz);
        printf("node: %d, size: %lldMB, free: %lldMB \n", i, sz >> 20, fsz >> 20);
    }
}

int main(int argc, const char* argv[])
{
    if (-1 == numa_available()) {
        printf("numa support not available \n");
        goto out;
    }

    int maxnode = numa_max_node();
    print_nodes(maxnode + 1);

    struct bitmask *node_mask;
    node_mask = numa_bitmask_alloc(maxnode + 1);
    if (!node_mask) {
        printf("numa bit mask alloc failed \n");
        goto err;
    }

    print_bitmask(node_mask);

    // bind the memory to node 1
    numa_bitmask_setbit(node_mask, 1);
    print_bitmask(node_mask);

    size_t len = 10UL << 30;
    char *tenGB = (char *) mmap(tenGB, len, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
    if (NULL == tenGB) {
        printf("failed to malloc memory \n");
    }
    // int mode = MPOL_F_STATIC_NODES | MPOL_BIND;
    // int ret = mbind(tenGB, len, mode, node_mask->maskp, maxnode + 2, 0);
    // if (ret < 0) {
    //     printf("mbind failed, error: %d \n", errno);
    // }
    memset(tenGB, 0x0, len);

    print_nodes(maxnode + 1);

    numa_bitmask_free(node_mask);

err:
out:
    return 0;
}


(base) [root@localhost test]# ./mynuma 
node: 0, size: 65431MB, free: 35245MB 
node: 1, size: 65535MB, free: 62371MB 
node: 2, size: 65535MB, free: 64003MB 
node: 3, size: 65535MB, free: 63537MB 
node: 4, size: 65535MB, free: 63744MB 
node: 5, size: 65535MB, free: 63884MB 
node: 6, size: 65535MB, free: 64035MB 
node: 7, size: 65535MB, free: 61804MB 
bitmask size: 8 
bitmask val: 0x0 
bitmask size: 8 
bitmask val: 0x2 
node: 0, size: 65431MB, free: 35245MB 
node: 1, size: 65535MB, free: 62371MB 
node: 2, size: 65535MB, free: 64003MB 
node: 3, size: 65535MB, free: 63537MB 
node: 4, size: 65535MB, free: 63744MB 
node: 5, size: 65535MB, free: 53624MB 
node: 6, size: 65535MB, free: 64035MB 
node: 7, size: 65535MB, free: 61804MB 
(base) [root@localhost test]# 
(base) [root@localhost test]# 
(base) [root@localhost test]# 
(base) [root@localhost test]# ./mynuma 
node: 0, size: 65431MB, free: 35237MB 
node: 1, size: 65535MB, free: 62371MB 
node: 2, size: 65535MB, free: 64000MB 
node: 3, size: 65535MB, free: 63535MB 
node: 4, size: 65535MB, free: 63745MB 
node: 5, size: 65535MB, free: 63882MB 
node: 6, size: 65535MB, free: 64035MB 
node: 7, size: 65535MB, free: 61805MB 
bitmask size: 8 
bitmask val: 0x0 
bitmask size: 8 
bitmask val: 0x2 
node: 0, size: 65431MB, free: 35237MB 
node: 1, size: 65535MB, free: 62371MB 
node: 2, size: 65535MB, free: 64000MB 
node: 3, size: 65535MB, free: 54496MB 
node: 4, size: 65535MB, free: 62525MB 
node: 5, size: 65535MB, free: 63882MB 
node: 6, size: 65535MB, free: 64035MB 
node: 7, size: 65535MB, free: 61805MB 
(base) [root@localhost test]# 
(base) [root@localhost test]# 
(base) [root@localhost test]# ./mynuma 
node: 0, size: 65431MB, free: 35237MB 
node: 1, size: 65535MB, free: 62372MB 
node: 2, size: 65535MB, free: 64000MB 
node: 3, size: 65535MB, free: 63534MB 
node: 4, size: 65535MB, free: 63745MB 
node: 5, size: 65535MB, free: 63882MB 
node: 6, size: 65535MB, free: 64035MB 
node: 7, size: 65535MB, free: 61805MB 
bitmask size: 8 
bitmask val: 0x0 
bitmask size: 8 
bitmask val: 0x2 
node: 0, size: 65431MB, free: 35233MB 
node: 1, size: 65535MB, free: 62371MB 
node: 2, size: 65535MB, free: 64000MB 
node: 3, size: 65535MB, free: 63534MB 
node: 4, size: 65535MB, free: 63743MB 
node: 5, size: 65535MB, free: 53622MB 
node: 6, size: 65535MB, free: 64035MB 
node: 7, size: 65535MB, free: 61805MB 
(base) [root@localhost test]# 
(base) [root@localhost test]# ./mynuma 
node: 0, size: 65431MB, free: 35235MB 
node: 1, size: 65535MB, free: 62371MB 
node: 2, size: 65535MB, free: 64000MB 
node: 3, size: 65535MB, free: 63534MB 
node: 4, size: 65535MB, free: 63744MB 
node: 5, size: 65535MB, free: 63880MB 
node: 6, size: 65535MB, free: 64035MB 
node: 7, size: 65535MB, free: 61805MB 
bitmask size: 8 
bitmask val: 0x0 
bitmask size: 8 
bitmask val: 0x2 
node: 0, size: 65431MB, free: 35235MB 
node: 1, size: 65535MB, free: 62371MB 
node: 2, size: 65535MB, free: 64000MB 
node: 3, size: 65535MB, free: 63534MB 
node: 4, size: 65535MB, free: 53483MB 
node: 5, size: 65535MB, free: 63880MB 
node: 6, size: 65535MB, free: 64035MB 
node: 7, size: 65535MB, free: 61805MB 
(base) [root@localhost test]#

可以看出来，每次分配的内存都不在同一个节点上，我们用numactl绑定节点再试试看

(base) [root@localhost test]# numactl --preferred=2 ./mynuma
node: 0, size: 65431MB, free: 35237MB 
node: 1, size: 65535MB, free: 62368MB 
node: 2, size: 65535MB, free: 64001MB 
node: 3, size: 65535MB, free: 63410MB 
node: 4, size: 65535MB, free: 63744MB 
node: 5, size: 65535MB, free: 63862MB 
node: 6, size: 65535MB, free: 63862MB 
node: 7, size: 65535MB, free: 61677MB 
bitmask size: 8 
bitmask val: 0x0 
bitmask size: 8 
bitmask val: 0x2 
node: 0, size: 65431MB, free: 35241MB 
node: 1, size: 65535MB, free: 62368MB 
node: 2, size: 65535MB, free: 53741MB 
node: 3, size: 65535MB, free: 63538MB 
node: 4, size: 65535MB, free: 63616MB 
node: 5, size: 65535MB, free: 63857MB 
node: 6, size: 65535MB, free: 63862MB 
node: 7, size: 65535MB, free: 61677MB 
(base) [root@localhost test]# 
(base) [root@localhost test]# numactl --preferred=2 ./mynuma
node: 0, size: 65431MB, free: 35241MB 
node: 1, size: 65535MB, free: 62367MB 
node: 2, size: 65535MB, free: 64000MB 
node: 3, size: 65535MB, free: 63538MB 
node: 4, size: 65535MB, free: 63616MB 
node: 5, size: 65535MB, free: 63857MB 
node: 6, size: 65535MB, free: 63862MB 
node: 7, size: 65535MB, free: 61677MB 
bitmask size: 8 
bitmask val: 0x0 
bitmask size: 8 
bitmask val: 0x2 
node: 0, size: 65431MB, free: 35241MB 
node: 1, size: 65535MB, free: 62367MB 
node: 2, size: 65535MB, free: 53741MB 
node: 3, size: 65535MB, free: 63538MB 
node: 4, size: 65535MB, free: 63616MB 
node: 5, size: 65535MB, free: 63857MB 
node: 6, size: 65535MB, free: 63862MB 
node: 7, size: 65535MB, free: 61677MB 
(base) [root@localhost test]# 
(base) [root@localhost test]# numactl --preferred=2 ./mynuma
node: 0, size: 65431MB, free: 35241MB 
node: 1, size: 65535MB, free: 62366MB 
node: 2, size: 65535MB, free: 64001MB 
node: 3, size: 65535MB, free: 63538MB 
node: 4, size: 65535MB, free: 63744MB 
node: 5, size: 65535MB, free: 63728MB 
node: 6, size: 65535MB, free: 63862MB 
node: 7, size: 65535MB, free: 61676MB 
bitmask size: 8 
bitmask val: 0x0 
bitmask size: 8 
bitmask val: 0x2 
node: 0, size: 65431MB, free: 35241MB 
node: 1, size: 65535MB, free: 62366MB 
node: 2, size: 65535MB, free: 53740MB 
node: 3, size: 65535MB, free: 63538MB 
node: 4, size: 65535MB, free: 63744MB 
node: 5, size: 65535MB, free: 63728MB 
node: 6, size: 65535MB, free: 63862MB 
node: 7, size: 65535MB, free: 61676MB 
(base) [root@localhost test]#

可以看出来这样每次分配内存都分配到了节点2上。

我们将代码中mbind相关代码注释取消再来运行

(base) [root@localhost test]# numactl --preferred=2 ./mynuma
node: 0, size: 65431MB, free: 35255MB 
node: 1, size: 65535MB, free: 62365MB 
node: 2, size: 65535MB, free: 64000MB 
node: 3, size: 65535MB, free: 63538MB 
node: 4, size: 65535MB, free: 63740MB 
node: 5, size: 65535MB, free: 63844MB 
node: 6, size: 65535MB, free: 64028MB 
node: 7, size: 65535MB, free: 61803MB 
bitmask size: 8 
bitmask val: 0x0 
bitmask size: 8 
bitmask val: 0x2 
node: 0, size: 65431MB, free: 35257MB 
node: 1, size: 65535MB, free: 52126MB 
node: 2, size: 65535MB, free: 63981MB 
node: 3, size: 65535MB, free: 63538MB 
node: 4, size: 65535MB, free: 63740MB 
node: 5, size: 65535MB, free: 63844MB 
node: 6, size: 65535MB, free: 64028MB 
node: 7, size: 65535MB, free: 61803MB 
(base) [root@localhost test]# 
(base) [root@localhost test]# numactl --preferred=2 ./mynuma
node: 0, size: 65431MB, free: 35254MB 
node: 1, size: 65535MB, free: 62364MB 
node: 2, size: 65535MB, free: 63999MB 
node: 3, size: 65535MB, free: 63537MB 
node: 4, size: 65535MB, free: 63741MB 
node: 5, size: 65535MB, free: 63844MB 
node: 6, size: 65535MB, free: 64028MB 
node: 7, size: 65535MB, free: 61803MB 
bitmask size: 8 
bitmask val: 0x0 
bitmask size: 8 
bitmask val: 0x2 
node: 0, size: 65431MB, free: 35254MB 
node: 1, size: 65535MB, free: 52124MB 
node: 2, size: 65535MB, free: 63979MB 
node: 3, size: 65535MB, free: 63537MB 
node: 4, size: 65535MB, free: 63741MB 
node: 5, size: 65535MB, free: 63844MB 
node: 6, size: 65535MB, free: 64028MB 
node: 7, size: 65535MB, free: 61803MB 
(base) [root@localhost test]#

可以发现api的优先级比numactl高。

#include <stdio.h>
#include <stdlib.h>
#include <numa.h>
#include <numaif.h>
#include <errno.h>
#include <sys/mman.h>
#include <numacompat1.h>
#ifndef MPOL_F_STATIC_NODES
/* Bug in numaif.h, this should be defined in there. Definition copied
 * from linux/mempolicy.h.
 */
#define MPOL_F_STATIC_NODES     (1 << 15)
#endif

void print_bitmask(struct bitmask* mask) {
    printf("bitmask size: %lu \n", mask->size);
    printf("bitmask val: 0x%lx \n", *mask->maskp);
}

void print_nodes(int node_count) {
    long long fsz = 0;
    for (int i = 0; i < node_count; i++) {
        long long sz = numa_node_size64(i, &fsz);
        printf("node: %d, size: %lldMB, free: %lldMB \n", i, sz >> 20, fsz >> 20);
    }
}

int main(int argc, const char* argv[])
{
    if (-1 == numa_available()) {
        printf("numa support not available \n");
        goto out;
    }

    int maxnode = numa_max_node();
    print_nodes(maxnode + 1);

    struct bitmask *node_mask;
    node_mask = numa_bitmask_alloc(maxnode + 1);
    if (!node_mask) {
        printf("numa bit mask alloc failed \n");
        goto err;
    }

    print_bitmask(node_mask);

    // bind the memory to node 1
    numa_bitmask_setbit(node_mask, 1);
    print_bitmask(node_mask);

    size_t len = 10UL << 30;
    char *tenGB = (char *) mmap(tenGB, len, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
    if (NULL == tenGB) {
        printf("failed to malloc memory \n");
    }

    nodemask_t memmask = numa_get_membind();
    printf("memmask: 0x%lx \n", memmask);

    int mode = MPOL_F_STATIC_NODES | MPOL_BIND;
    int ret = mbind(tenGB, len, mode, node_mask->maskp, maxnode + 2, 0);
    if (ret < 0) {
        printf("mbind failed, error: %d \n", errno);
    }
    memset(tenGB, 0x0, len);

    print_nodes(maxnode + 1);

    numa_bitmask_free(node_mask);

err:
out:
    return 0;
}

修改代码加入numa_get_membind函数，检测当前memory是否有绑定node，再次运行

(base) [root@localhost test]# numactl --preferred=2 ./mynuma
node: 0, size: 65431MB, free: 35198MB 
node: 1, size: 65535MB, free: 62362MB 
node: 2, size: 65535MB, free: 63982MB 
node: 3, size: 65535MB, free: 63539MB 
node: 4, size: 65535MB, free: 63740MB 
node: 5, size: 65535MB, free: 63878MB 
node: 6, size: 65535MB, free: 64037MB 
node: 7, size: 65535MB, free: 61806MB 
bitmask size: 8 
bitmask val: 0x0 
bitmask size: 8 
bitmask val: 0x2 
memmask: 0xff 
node: 0, size: 65431MB, free: 35205MB 
node: 1, size: 65535MB, free: 52123MB 
node: 2, size: 65535MB, free: 63966MB 
node: 3, size: 65535MB, free: 63539MB 
node: 4, size: 65535MB, free: 63740MB 
node: 5, size: 65535MB, free: 63876MB 
node: 6, size: 65535MB, free: 64037MB 
node: 7, size: 65535MB, free: 61806MB 
(base) [root@localhost test]# 
(base) [root@localhost test]# 
(base) [root@localhost test]# 
(base) [root@localhost test]# 
(base) [root@localhost test]# 
(base) [root@localhost test]# 
(base) [root@localhost test]# 
(base) [root@localhost test]# numactl --membind=2 ./mynuma
node: 0, size: 65431MB, free: 35192MB 
node: 1, size: 65535MB, free: 62364MB 
node: 2, size: 65535MB, free: 64000MB 
node: 3, size: 65535MB, free: 63539MB 
node: 4, size: 65535MB, free: 63743MB 
node: 5, size: 65535MB, free: 63873MB 
node: 6, size: 65535MB, free: 64037MB 
node: 7, size: 65535MB, free: 61806MB 
bitmask size: 8 
bitmask val: 0x0 
bitmask size: 8 
bitmask val: 0x2 
memmask: 0x4 
node: 0, size: 65431MB, free: 35192MB 
node: 1, size: 65535MB, free: 52124MB 
node: 2, size: 65535MB, free: 63980MB 
node: 3, size: 65535MB, free: 63539MB 
node: 4, size: 65535MB, free: 63743MB 
node: 5, size: 65535MB, free: 63873MB 
node: 6, size: 65535MB, free: 64037MB 
node: 7, size: 65535MB, free: 61806MB 
(base) [root@localhost test]#

发现numactl的–membind其实就是制定了当前mem所需要绑定的node。

后期代码有些许修改，请参阅https://github.com/raykwok1150/numa-demo

网络编码
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
Linux NUMA 进化
07-19
Linux NUMA 进化Linux NUMA 进化Linux NUMA 进化Linux NUMA 进化Linux NUMA 进化
Oracle-NUMA绑定脚本
07-21
Oracle-NUMA绑定脚本Oracle-NUMA绑定脚本Oracle-NUMA绑定脚本Oracle-NUMA绑定脚本Oracle-NUMA绑定脚本Oracle-NUMA绑定脚本
libnuma详解(A NUMA API for LINUX)
叶子心情你不懂
08-05 9401
白皮书 numa3手册 numa3手册 一、什么是NUMA 关于这个很多博客都有，基本都是这个白皮书的翻译或者扩充。 在传统的SMP(对称多处理)系统中，计算机有一个由所有cpu共享的内存控制器。当所有处理器同时访问内存时，这种单一内存连接常常成为瓶颈。对于具有更多cpu的大型系统，它也不能很好地扩展。由于这个原因，越来越多的现代系统正在使用CC/NUMA(缓存一致性/非一致性内存访问)架构。例如AMD* Opteron*、IBM* Power5*、HP* Superdome和SGI* Altix*。
numa API
runshui27的博客
05-04 295
4-5 +0-34-5 +0-3。
AMD ROCM软件栈 -clr项目分析
最新发布
光明顶魔教工程师
04-18 987
项目层级结构fill:#333;color:#333;color:#333;fill:none;AMD CLR：Comgr 库提供用于编译和检查 AMDGPU 代码对象的应用程序接口CLR项目目录结构。
numa的内存按节点分配和线程绑定
ming5432ming的博客
10-23 1万+
前段时间在做1.5亿条数据的比对， 需要多线程和内存的访问问题， 由于内存的访问与numa节点的分配有关，即：cpu访问本地节点的内存速度要快。 现将numa的几个库函数做一下说明： int main() { if(numa_available()<0) { printf("your system does not support numa api \n"); return
linux numa 内存,在Linux中NUMA内存页面迁移开销
weixin_42476912的博客
05-08 342
我必须在Linux下找到与NUMA内存页迁移相关的开销.你能告诉我哪些工具可以使用吗？如果可能,你能展示一个例子.如果您想了解您的系统是否正在进行过多的远程节点内存访问并且您正在使用intel CPU,Intel’s PMU有一个名为vtbwrun的实用程序来报告QPI /非核心活动.如果要查看执行页面迁移所需的时间,可以测量对numa_move_pages的调用持续时间(由numactl包提供)...
ROCm平台简介及使用汇总
weixin_45206081的博客
04-03 4933
ROCm是AMD的一个软件平台，用来加速GPU计算A卡上编程模型使用的是HIP或者OpenCL，而运行环境是ROCmN卡上，编程模型是CUDA，运行环境也是CUDA链接: [https://rocmdocs.amd.com/en/latest/]
numa.rar_NUMA
09-14
NUMA（Non-Uniform Memory Access）是现代多处理器系统中的一种内存访问架构，它旨在提高大规模多核心系统的性能。在NUMA系统中，每个处理器都有其本地内存，访问本地内存的速度通常比访问远程处理器的内存快。NUMA...
numa_emulation.rar_NUMA_V2
09-23
NUMA（Non-Uniform Memory Access，非统一内存访问）是一种计算机架构，用于处理多处理器系统中的内存访问问题。在NUMA系统中，不同处理器访问内存的速度可能会有所不同，因为它们与不同的内存节点之间的物理连接...
计算机体系结构-NUMA
11-05
Un-shared memory access.最新的X86 CPU体系结构。多核多线程，共享cache访问。
fatal error: numa.h: No such file or directory
草原狼的专栏
03-21 1682
dpdk编译numa.h: No such file or directory问题
numactl -hardware源码分析
jason的笔记
10-28 2802
https://github.com/jmesmon/numactl/blob/0df3f720e606a3706700e0487ba19d720f50c4b8/numactl.c 可以看到numactl总共支持的命令如下：struct option opts[] = {     {"interleave", 1, 0, 'i' },     {"preferred", 1, 0, 'p'
NUMA node
zhuangshu_feng'‘每天进步一点点
04-07 1万+
现在的机器上都是有多个CPU和多个内存块的。以前我们都是将内存块看成是一大块内存，所有CPU到这个共享内存的访问消息是一样的。这就是之前普遍使用的SMP模型。但是随着处理器的增加，共享内存可能会导致内存访问冲突越来越厉害，且如果内存访问达到瓶颈的时候，性能就不能随之增加。NUMA（Non-Uniform Memory Access）就是这样的环境下引入的一个模型。比如一台机器是有2个处理器，有
LINUX平台上的NUMA API（A NUMA API for LINUX）（第一部分）
热门推荐
vah101的专栏
10-29 1万+
翻译自 http://developer.amd.com/wordpress/media/2012/10/LibNUMA-WP-fv1.pdf
numa.h：No such file or directory 解决方法
weixinhmz的博客
04-30 895
numa.h：No such file or directory 解决方法
使用源码编译安装AMD ROCm
u014756627的博客
02-15 2917
本文介绍了使用源码来安装AMD的ROCm组件。
centos如何编译带有numa.h头文件的程序
Robin思源的博客
10-07 2806
最近需要用numa api进行编程，整理一下如何编译一个带有numa.h的cpp文件 下载numa.h文件，利用numactl里的 将numa.h拷贝到程序相关的目录中 编写numatest.cc如下 #include <numa.h> #include <iostream> int main() { if(numa_available() < 0) { printf("your
numa,numa node 和物理内存
msdnchina的专栏@JiNan,ShanDong
04-20 4954
numa,numa node 和物理内存
bios numa
07-27
NUMA（Non-Uniform Memory Access）是一种计算机体系结构，它允许多个处理器访问共享内存的方式不同。在NUMA架构中，系统内存被划分为多个节点，每个节点包含一组处理器和与之关联的本地内存。每个处理器可以直接...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值