linux acct移植到arm,docker containerd 交叉编译 移植到arm64v8

可以把docker移植到嵌入式的aarch64 linux系统中，比如hi3559等芯片

一. docker编译

docker以及所依赖的containerd、runc等可能没法自己交叉编译，没有相关文档，并且东西太多太复杂，不过好在docker官方为不同平台提供编译好的程序，其中包括：

containerd、ctr、docker-init、dockerd、containerd-shim、docker、docker-proxy、runc

相关说明：

https://docs.docker.com/engine/install/binaries/

其中包括下载地址：

https://download.docker.com/linux/static/stable/aarch64/

二. 移植iptable

如果系统中没有iptable命令，则需要移植

0.打开内核选项：

Networking support ---> Networking options ---> Network packet filtering framework (Netfilter)

选中Core Netfilter Configuration以及下面所有的，IP: Netfilter Configuration以及下面所有的。

iptable又依赖libmnl、libnftnl，需要按顺序编译：

1.libmnl

git clone https://git.netfilter.org/libmnl/

./configure CC=aarch64-himix100-linux-gcc --host=aarch64-himix100-linux --prefix=$PWD/bin --enable-static

make

make install

2.libnftnl

git clone https://git.netfilter.org/libnftnl/

LIBMNL_CFLAGS="-I/home/a/share/libmnl/bin/include" \

LIBMNL_LIBS="-L/home/a/share/libmnl/bin/lib -lmnl" \

./configure CC=aarch64-himix100-linux-gcc --host=aarch64-himix100-linux --prefix=$PWD/bin --enable-static

make

make install

3.iptables

git clone https://git.netfilter.org/iptables/

libmnl_CFLAGS="-I/home/a/share/libmnl/bin/include" \

libmnl_LIBS="-L/home/a/share/libmnl/bin/lib -lmnl" \

libnftnl_CFLAGS="-I/home/a/share/libnftnl/bin/include" \

libnftnl_LIBS="-L/home/a/share/libnftnl/bin/lib -lnftnl" \

./configure CC=aarch64-himix100-linux-gcc --host=aarch64-himix100-linux --prefix=$PWD/bin --enable-static --without-cli

make

make install

最后把 iptables/bin 目录下生成的东西放到板子上。

三. 内核配置

内核需要打开很多选项，具体见此文章最后的docker依赖的内核选项列表。

此列表是用这个脚本检测并生成的 https://github.com/moby/moby/blob/master/contrib/check-config.sh

四. 运行服务

1.配置eth0和lo：

ifconfig lo 127.0.0.1 netmask 255.255.255.0

2.挂载cgroup并重启

vi /etc/fstab

在最后一行增加以下配置信息：

none        /sys/fs/cgroup        cgroup    defaults    0    0

3. 把下载的docker安装包里的文件都放在 /usr/bin 下

4. 先执行 containerd &，然后执行 dockerd &

五. 运行镜像

1.下载镜像

一般要在单板上用 docker pull arm64v8/ubuntu 来下载镜像，还要配置好 /etc/docker/daemon.json，如果pull有问题的话可以先用另外的方法下载镜像：

找一个正式的ubuntu系统通过上面的命令下载镜像

用命令 docker save arm64v8/ubuntu > ubuntu_image.tar 来保存镜像，把ubuntu_image.tar放到单板上

在单板上用 docker load < ./ubuntu_image.tar 来导入镜像，这时候 docker image ls 就能看到这个镜像了

2.执行 docker run -it arm64v8/ubuntu /bin/sh 就能运行容器，并进入这个容器的shell环境

附：docker依赖的内核选项:

Generally Necessary:

- cgroup hierarchy: properly mounted [/sys/fs/cgroup]

- CONFIG_NAMESPACES: enabled

- CONFIG_NET_NS: enabled

- CONFIG_PID_NS: enabled

- CONFIG_IPC_NS: enabled

- CONFIG_UTS_NS: enabled

- CONFIG_CGROUPS: enabled

- CONFIG_CGROUP_CPUACCT: enabled

- CONFIG_CGROUP_DEVICE: enabled

- CONFIG_CGROUP_FREEZER: enabled

- CONFIG_CGROUP_SCHED: enabled

- CONFIG_CPUSETS: enabled

- CONFIG_MEMCG: enabled

- CONFIG_KEYS: enabled

- CONFIG_VETH: enabled (as module)

- CONFIG_BRIDGE: enabled (as module)

- CONFIG_BRIDGE_NETFILTER: enabled (as module)

- CONFIG_NF_NAT_IPV4: missing

- CONFIG_IP_NF_FILTER: enabled (as module)

- CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module)

- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled (as module)

- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)

- CONFIG_NETFILTER_XT_MATCH_IPVS: enabled (as module)

- CONFIG_IP_NF_NAT: enabled (as module)

- CONFIG_NF_NAT: enabled (as module)

- CONFIG_NF_NAT_NEEDED: missing

- CONFIG_POSIX_MQUEUE: enabled

Optional Features:

- CONFIG_USER_NS: enabled

- CONFIG_SECCOMP: enabled

- CONFIG_CGROUP_PIDS: enabled

- CONFIG_MEMCG_SWAP: enabled

- CONFIG_MEMCG_SWAP_ENABLED: enabled

(cgroup swap accounting is currently enabled)

- CONFIG_BLK_CGROUP: enabled

- CONFIG_BLK_DEV_THROTTLING: enabled

- CONFIG_IOSCHED_CFQ: missing

- CONFIG_CFQ_GROUP_IOSCHED: missing

- CONFIG_CGROUP_PERF: enabled

- CONFIG_CGROUP_HUGETLB: enabled

- CONFIG_NET_CLS_CGROUP: enabled

- CONFIG_CGROUP_NET_PRIO: missing

- CONFIG_CFS_BANDWIDTH: enabled

- CONFIG_FAIR_GROUP_SCHED: enabled

- CONFIG_RT_GROUP_SCHED: enabled

- CONFIG_IP_NF_TARGET_REDIRECT: enabled (as module)

- CONFIG_IP_VS: enabled (as module)

- CONFIG_IP_VS_NFCT: enabled

- CONFIG_IP_VS_PROTO_TCP: enabled

- CONFIG_IP_VS_PROTO_UDP: enabled

- CONFIG_IP_VS_RR: enabled (as module)

- CONFIG_EXT4_FS: enabled (as module)

- CONFIG_EXT4_FS_POSIX_ACL: enabled

- CONFIG_EXT4_FS_SECURITY: enabled

- Network Drivers:

- "overlay":

- CONFIG_VXLAN: enabled (as module)

- CONFIG_BRIDGE_VLAN_FILTERING: enabled

Optional (for encrypted networks):

- CONFIG_CRYPTO: enabled

- CONFIG_CRYPTO_AEAD: enabled

- CONFIG_CRYPTO_GCM: enabled

- CONFIG_CRYPTO_SEQIV: enabled

- CONFIG_CRYPTO_GHASH: enabled

- CONFIG_XFRM: enabled

- CONFIG_XFRM_USER: enabled

- CONFIG_XFRM_ALGO: enabled

- CONFIG_INET_ESP: enabled (as module)

- CONFIG_INET_XFRM_MODE_TRANSPORT: missing

- "ipvlan":

- CONFIG_IPVLAN: missing

- "macvlan":

- CONFIG_MACVLAN: enabled (as module)

- CONFIG_DUMMY: enabled (as module)

- "ftp,tftp client in container":

- CONFIG_NF_NAT_FTP: enabled (as module)

- CONFIG_NF_CONNTRACK_FTP: enabled (as module)

- CONFIG_NF_NAT_TFTP: enabled (as module)

- CONFIG_NF_CONNTRACK_TFTP: enabled (as module)

- Storage Drivers:

- "aufs":

- CONFIG_AUFS_FS: missing

- "btrfs":

- CONFIG_BTRFS_FS: enabled (as module)

- CONFIG_BTRFS_FS_POSIX_ACL: enabled

- "devicemapper":

- CONFIG_BLK_DEV_DM: enabled (as module)

- CONFIG_DM_THIN_PROVISIONING: enabled (as module)

- "overlay":

- CONFIG_OVERLAY_FS: enabled (as module)

- "zfs":

- /dev/zfs: missing

- zfs command: missing

- zpool command: missing