iptables交叉编译及使用

从https://www.netfilter.org/projects/iptables/downloads.html下载新版本的iptables，拷贝、解压后config:

./configure --prefix=/home/nfsshare/hisi/iptables/iptables --host=arm-linux CC=arm-himix200-linux-gcc

执行出现：

checking for libmnl... no
*** Error: No suitable libmnl found. ***
    Please install the 'libmnl' package
    Or consider --disable-nftables to skip
    iptables-compat over nftables support.

搜了下看起来不需要，改成：

./configure --prefix=/home/nfsshare/hisi/iptables/iptables --host=arm-linux CC=arm-himix200-linux-gcc --disable-nftables
make 
make install

使用出现：

/mnt/homeshare/hisi/iptables/iptables/sbin # ./iptables -L
Fatal: can't open lock file /run/xtables.lock: No such file or directory

未搜索到此问题的解决办法，仔细看之后发现 /run 目录就不存在，于是创建 /run 目录后能执行，考虑到我这里内核一般把锁放在 /var/lock 下，所以修改了iptables的源码。
再编译依旧出现：

/mnt/homeshare/hisi/iptables/iptables/sbin # ./iptables -L
iptables v1.8.4 (legacy): can't initialize iptables table `filter': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

已经设置了一些内核配置，但是还缺一些：


编译内核烧写之后，再试试就能跑起来了。
再次执行出现：

/mnt/homeshare/hisi/iptables/iptables/sbin # ./iptables -t nat -A POSTROUTING -s
 192.168.44.204/24 -j MASQUERADE
iptables v1.8.4 (legacy): Couldn't load target `MASQUERADE':No such file or directory
Try `iptables -h' or 'iptables --help' for more information.

内核MASQUERADE相关项再检查一遍，make clean之后重新编译烧写内核。
依旧不行，到底是什么原因呢？网上去查找这个相关的文章，翻遍了没有找到，看MASQUERADE的HOW TO文件，内核配置里边查找任何相关的配置，仔细检查命令没有问题，将iptables版本换成1.4.21版本，去源码中查找这个问题也没有找到，始终不行，都提示这个错误。怎么去查找这个错误呢？这个问题卡了很长时间。
后来想到使用strace跟踪，跟踪结果如下(中间结果太长删除掉部分)：

/mnt/homeshare/strace/bin # export LD_LIBRARY_PATH=/mnt/homeshare/hisi/iptables/iptables/lib:$LD_LIBRARY_PATH
/mnt/homeshare/strace/bin # ./strace /mnt/homeshare/hisi/iptables/iptables/sbin/iptables -w -t nat -A POSTROUTING -s 192.168.44.204/24 -j MASQUERADE
execve("/mnt/homeshare/hisi/iptables/iptables/sbin/iptables", ["/mnt/homeshare/hisi/iptables/ipt"..., "-w", "-t", "nat", "-A", "POSTROUTING", "-s", "192.168.44.204/24", "-j", "MASQUERADE"], 0xbea2cd24 /* 10 vars */) = 0
brk(NULL) = 0x83000
uname({sysname="Linux", nodename="(none)", ...}) = 0
mmap2(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6f59000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/home/nfsshare/hisi/iptables/iptables/lib/tls/v7l/neon/vfp/libip4tc.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
...
...
...
mprotect(0x30000, 4096, PROT_READ) = 0
mprotect(0xb6f5c000, 4096, PROT_READ) = 0
stat64("/home/nfsshare/hisi/iptables/iptables/lib/xtables/libxt_MASQUERADE.so", 0xbefa4808) = -1 ENOENT (No such file or directory)
stat64("/home/nfsshare/hisi/iptables/iptables/lib/xtables/libipt_MASQUERADE.so", 0xbefa4808) = -1 ENOENT (No such file or directory)
brk(NULL) = 0x83000
brk(0xa4000) = 0xa4000
socket(AF_UNIX, SOCK_STREAM, 0) = 3
bind(3, {sa_family=AF_UNIX, sun_path=@"xtables"}, 10) = 0
socket(AF_INET, SOCK_RAW, IPPROTO_RAW) = 4
fcntl64(4, F_SETFD, FD_CLOEXEC) = 0
getsockopt(4, SOL_IP, 0x40 /* IP_??? */, "nat\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., [84]) = 0
getsockopt(4, SOL_IP, 0x41 /* IP_??? */, "nat\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., [992]) = 0
stat64("/home/nfsshare/hisi/iptables/iptables/lib/xtables/libxt_MASQUERADE.so", 0xbefa4808) = -1 ENOENT (No such file or directory)
stat64("/home/nfsshare/hisi/iptables/iptables/lib/xtables/libipt_MASQUERADE.so", 0xbefa4808) = -1 ENOENT (No such file or directory)
write(2, "iptables v1.4.21: ", 18iptables v1.4.21: ) = 18
write(2, "Couldn't load target `MASQUERADE"..., 60Couldn't load target `MASQUERADE':No such file or directory
) = 60
write(2, "\n", 1
) = 1
write(2, "Try `iptables -h' or 'iptables -"..., 61Try `iptables -h' or 'iptables --help' for more information.
) = 61
exit_group(2) = ?
+++ exited with 2 +++
/mnt/homeshare/strace/bin # 

仔细观察上边的打印，关注设置的PATH和open、stat64等调用的路径：

export LD_LIBRARY_PATH=/mnt/homeshare/hisi/iptables/iptables/lib:$LD_LIBRARY_PATH
stat64("/home/nfsshare/hisi/iptables/iptables/lib/xtables/libipt_MASQUERADE.so", 0xbefa4808) = -1 ENOENT (No such file or directory)

发现两个路径根本就不匹配，/mnt/homeshare/路径是我的共享目录，我在主机上此目录交叉编译后安装在这里，telnet到板子上mount后，设置path后运行在共享目录的程序，正常情况下是没有问题的，我iptables-1.4.21的编译命令为：

./configure --prefix=/home/nfsshare/hisi/iptables/iptables --host=arm-linux CC=arm-himix200-linux-gcc --with-kernel=/home/work/Hisi/Hi3519AV100_SDK_V2.0.1.0/osdrv/opensource/kernel/linux-4.9.y-smp
make
make install

正常情况下，根据configure prefix的使用说明，prefix是独立于体系结构的安装路径，应该是不影响运行时依赖的路径的，但是不知道是不是iptables误用了或者是我没有注意到哪里的使用说明。
那这里建立 /home/nfsshare/hisi/iptables/iptables/lib/xtables/ 目录，将 /mnt/homeshare/hisi/iptables/iptables/lib 目录的文件拷贝到哪里，重新运行就可以了，当然正式代码还是把prefix目录改成实际lib目录。

/mnt/homeshare/strace/bin # /mnt/homeshare/hisi/iptables/iptables/sbin/iptables -t nat -A POSTROUTING -s 192.168.44.204/24 -j MASQUERADE
/mnt/homeshare/strace/bin # /mnt/homeshare/hisi/iptables/iptables/sbin/iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination         

Chain INPUT (policy ACCEPT)
target prot opt source destination         

Chain OUTPUT (policy ACCEPT)
target prot opt source destination         

Chain POSTROUTING (policy ACCEPT)
target prot opt source destination         
MASQUERADE all -- 192.168.44.0/24 anywhere     

再运行就正常了，这里时间主要耗费在MASQUERADE报错这里，主要纠结于这个本来是内核 netfilter 的一个功能，所以一直误以为这个是内核配置有问题，所以反复纠结检测验证内核配置，strace确实是个好工具…