如何本地构建一个 dpdk l2fwd snap 包？

目标

基于 dpdk-19.11 l2fwd 程序及其依赖库本地构建一个 snap 包，能够在 ubuntu 20.04 桌面环境中安装并测试运行。

编写 snap 包 yaml 描述文件

yaml 文件：

name: test # you probably want to 'snapcraft register <name>'
base: core22 # the base snap is the execution environment for this snap
version: '0.1' # just for humans, typically '1.2+git' or '1.3.2'
summary: Single-line elevator pitch for your amazing snap # 79 char long summary
description: |
  This is my-snap's description. You have a paragraph or two to tell the
  most important story about your snap. Keep it under 100 words though,
  we live in tweetspace and your description wants to look good in the snap
  store.

grade: devel # must be 'stable' to release into candidate/stable channels
confinement: devmode # use 'strict' once you have the right plugs and slots

apps:
 l2fwd:
  command: bin/l2fwd
  
parts:
 file-copy:
   plugin: dump
   source: /home/longyu/snap/l2fwd
   stage:
      - bin
      - lib

上述描述文件使用 file-copy 将指定目录的二进制文件内容拷贝到 snap 包中，使用这些文件生成一个 snap 包文件。

原始文件目录结构：

root@ubuntu:/home/longyu/snap/l2fwd# tree ./bin ./lib/
./bin
└── l2fwd
./lib/
└── x86_64-linux-gnu
    ├── libcrypto.so.1.0.0
    └── libnuma.so.1
    
1 directory, 3 files

编译生成 snap 包

编译过程日志如下:

root@ubuntu:/home/longyu/snap/l2fwd/snap# snapcraft --destructive-mode --debug
Executed: pull file-copy                                                                                                                                                           
Executed: build file-copy                                                                                                                                                          
Executed: stage file-copy                                                                                                                                                          
Executed: prime file-copy                                                                                                                                                          
Executed parts lifecycle                                                                                                                                                           
Generated snap metadata                                                                                                                                                            
Running linter: library                                                                                                                                                            /bin/bash: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /snap/core22/current/lib/x86_64-linux-gnu/libtinfo.so.6)
/bin/bash: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /snap/core22/current/lib/x86_64-linux-gnu/libtinfo.so.6)
Unable to determine library dependencies for 'lib/x86_64-linux-gnu/libcrypto.so.1.0.0'                                                                                             /bin/bash: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /snap/core22/current/lib/x86_64-linux-gnu/libtinfo.so.6)

Unable to determine library dependencies for 'lib/x86_64-linux-gnu/libnuma.so.1'                                                                                                   
Created snap package test_0.1_amd64.snap

snapcraft 指定了 debug 参数能够输出 snap 包构建中的一些调试信息，出现问题时可以根据调试信息进行排查。

安装运行 l2fwd snap 包

安装命令：

snap install --dangerous ./test_0.1_amd64.snap  --devmode

snap 默认从官方源安装 snap 包，要安装一个本地制作的 snap 包，需要指定 --dangerous，同时 --devmode 指定使用开发模式，在这种模式下 snap 内部的权限控制仅输出日志并不会做实际的阻断，便于测试。

运行日志示例：

root@ubuntu:/home/longyu/snap/l2fwd# /snap/bin/test.l2fwd 
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 8086:100f net_e1000_em
EAL: PCI device 0000:02:06.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 8086:100f net_e1000_em
EAL:   using IOMMU type 8 (No-IOMMU)
EAL: Ignore mapping IO port bar(4)

dmesg 相关信息：

[518163.136314] kauditd_printk_skb: 712 callbacks suppressed
[518163.136317] audit: type=1326 audit(1693798734.950:73568): auid=1000 uid=0 gid=0 ses=286 subj=snap.test.l2fwd pid=274885 comm="l2fwd" exe="/snap/test/x1/bin/l2fwd" sig=0 arch=c000003e syscall=172 compat=0 ip=0x7f4a886fdb3b code=0x7ffc0000
[518163.138624] audit: type=1400 audit(1693798734.950:73569): apparmor="ALLOWED" operation="open" profile="snap.test.l2fwd" name="/run/dpdk/rte/config" pid=274885 comm="l2fwd" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0
...........................................................................................................................................................
[518496.329159] audit: type=1400 audit(1693799068.140:74046): apparmor="ALLOWED" operation="unlink" profile="snap.test.l2fwd" name="/run/dpdk/rte/mp_socket" pid=275040 comm="l2fwd" requested_mask="d" denied_mask="d" fsuid=0 ouid=0
[518496.329171] audit: type=1326 audit(1693799068.140:74047): auid=1000 uid=0 gid=0 ses=286 subj=snap.test.l2fwd pid=275040 comm="l2fwd" exe="/snap/test/x1/bin/l2fwd" sig=0 arch=c000003e syscall=49 compat=0 ip=0x7fbb4b8ff66b code=0x7ffc0000
[518496.546520] vfio-pci 0000:02:06.0: vfio-noiommu device opened by user (l2fwd:275040)

从 dmesg 的日志看，apparmor 监控到了 l2fwd 运行过程中的许多资源访问过程，仅仅输出了日志，并没有阻断，程序能够正常运行。

strict 模式运行 l2fwd

dmesg 中的 audit 信息：

[612862.262407] audit: type=1400 audit(1693893435.219:74550): apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap-update-ns.test-user-core" pid=285736 comm="apparmor_parser"
[612862.343653] audit: type=1400 audit(1693893435.303:74551): apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap.test-user-core.l2fwd" pid=285737 comm="apparmor_parser"
[612884.622773] audit: type=1326 audit(1693893457.551:74552): auid=1000 uid=0 gid=0 ses=339 subj=snap.test-user-core.l2fwd pid=285770 comm="l2fwd" exe="/snap/test-user-core/x1/bin/l2fwd" sig=0 arch=c000003e syscall=172 compat=0 ip=0x7fdf39e16b3b code=0x50000
[612884.766348] audit: type=1400 audit(1693893457.719:74553): apparmor="DENIED" operation="open" profile="snap.test-user-core.l2fwd" name="/run/dpdk/rte/config" pid=285770 comm="l2fwd" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0

上述日志是以 strict 模式运行时内核输出的部分信息，在这种模式下， l2fwd 会按照默认的 apparmor 规则严格执行，apparmor 阻断了 l2fwd 打开 /run/dpdk/rte/config 文件的过程，l2fwd 无法正常运行。

要让 l2fwd 正常运行，需要修改 l2fwd 的 apparmor 规则，这里已经达成了目标，暂时跳过。

参考链接

https://askubuntu.com/questions/822765/snap-install-failure-error-cannot-find-signatures-with-metadata-for-snap