今天轮到讨论安全问题了。 python 代码中包含有害内容该怎么办?常用技术是沙箱(Sandboxing)。本文从一些基础设施讲起。
目录
- 如何在 C++ 中调用 python 解析器来执行 python 代码(一)?
- 如何在 C++ 中调用 python 解析器来执行 python 代码(二)?
- 如何在 C++ 中调用 python 解析器来执行 python 代码(三)?
- 如何在 C++ 中调用 python 解析器来执行 python 代码(四)?
- 如何在 C++ 中调用 python 解析器来执行 python 代码(五)?
- 如何在 C++ 中调用 python 解析器来执行 python 代码(六)?
基础设施:seccomp-bpf
seccomp is a computer security facility in the Linux kernel. seccomp allows a process to make a one-way transition into a “secure” state where it cannot make any system calls except exit, sigreturn, read and write to already-open file descriptors.
Seccomp BPF 全称 SECure COMPuting with filters,它产生的背景是:操作系统给应用层暴露了数百个系统调用接口,但是大部分应用程序只需要访问其中一个子集。Seccomp BPF 提供了一个过滤器接口,用于描述允许应用程序使用哪些系统调用接口。
prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, prog);
其中,prog 指向一个 struct sock_fprog,里面定义了过滤器。考虑到 Berkeley Packet Filter (BPF) 已经在 socket 过滤领域使用多年,拥有非常强大的描述能力,接口过滤器也使用了 BPF 格式(kernel design choice)。
BPF 比较有意思,它有一套自己的指令集,用于编写 FILTER 程序,举个例子(来自这里),下面这段程序禁止execve系统调用:
#include <stdio.h>
#include <sys/prctl.h>
#include <linux/seccomp.h>
#include <linux/filter.h>
#include <stdlib.h>
#include <unistd.h>
int main()
{
struct sock_filter filter[] = {
BPF_STMT(BPF_LD+BPF_W+BPF_ABS,0), //将帧的偏移0处,取4个字节数据,也就是系统调用号的值载入累加器
BPF_JUMP(BPF_JMP+BPF_JEQ,59,0,1), //判断系统调用号是否为59(execve),是则顺序执行,否则跳过下一条
BPF_STMT(BPF_RET+BPF_K,SECCOMP_RET_KILL), //返回KILL
BPF_STMT(BPF_RET+BPF_K,SECCOMP_RET_ALLOW), //返回ALLOW
};
struct sock_fprog prog = {
.len = (unsigned short)(sizeof(filter)/sizeof(filter[0])),//规则条数
.filter = filter, //结构体数组指针
};
prctl(PR_SET_NO_NEW_PRIVS,1,0,0,0); //设置NO_NEW_PRIVS
prctl(PR_SET_SECCOMP,SECCOMP_MODE_FILTER,&prog);
write(0,"test\n",5);
system("/bin/sh");
return 0;
}
小结:有了 seccomp-bpf 后,我们就可以针对系统调用做一些定制化的约束,在安全和功能之间取得平衡。
关于 seccomp-bpf 更多讨论,这篇文章非常好:https://xz.aliyun.com/t/11480
基础设施: Linux Namespaces
Namespaces are a feature of the Linux kernel that partitions kernel resources such that one set of processes sees one set of resources while another set of processes sees a different set of resources. The feature works by having the same namespace for a set of resources and processes, but those namespaces refer to distinct resources. Resources may exist in multiple spaces. Examples of such resources are process IDs, host-names, user IDs, file names, some names associated with network access, and Inter-process communication
.
Namespaces are a fundamental aspect of containers in Linux.
从这篇文章里摘来一个总结,包含了比较全面的 namespace 资源:
关于 namespace 的更多概念,参考 wiki
NSJail
编译依赖
bison 3.0+
libnl3-devel.x86_64
protobuf-devel.x86_64
特别注意:protobuf 的 library 版本要和 protoc 文件的版本一致,不然会各种链接报错。
编译命令: PATH=/.vos/.dep_cache/7d6d26725ac1e91bc824e1be337cf31e/bin/:/share/nsjail/bison/bin/:$PATH make -j
在我的系统上,clone(flags=CLONE_NEWUSER) 还不支持,所以需要用 --disable_clone_newuser
把这个 flag 过滤掉。
[xiaochu.yh ~/tools/nsjail] (master) $sudo LD_LIBRARY_PATH=/.vos/.dep_cache/7d6d26725ac1e91bc824e1be337cf31e/var/usr/local/gcc-5.2.0/lib64/ nsjail -Mr --chroot / -R /tmp/ --user 99999 --group 99999 --disable_clone_newuser -- /bin/sh -i
[I][2023-03-07T21:56:30+0800] Mode: STANDALONE_RERUN
[I][2023-03-07T21:56:30+0800] Jail parameters: hostname:'NSJAIL', chroot:'/', process:'/bin/sh', bind:[::]:0, max_conns:0, max_conns_per_ip:0, time_limit:0, personality:0, daemonize:false, clone_newnet:true, clone_newuser:false, clone_newns:true, clone_newpid:true, clone_newipc:true, clone_newuts:true, clone_newcgroup:true, clone_newtime:false, keep_caps:false, disable_no_new_privs:false, max_cpus:0
[I][2023-03-07T21:56:30+0800] Mount: '/' -> '/' flags:MS_RDONLY|MS_BIND|MS_REC|MS_PRIVATE type:'' options:'' dir:true
[I][2023-03-07T21:56:30+0800] Mount: '/proc' flags:MS_RDONLY type:'proc' options:'' dir:true
[I][2023-03-07T21:56:30+0800] Uid map: inside_uid:99999 outside_uid:0 count:1 newuidmap:false
[I][2023-03-07T21:56:30+0800] Gid map: inside_gid:99999 outside_gid:0 count:1 newgidmap:false
[W][2023-03-07T21:56:30+0800][1] initNs():223 prctl(PR_CAP_AMBIENT, PR_CAP_AMBIENT_CLEAR_ALL): Invalid argument
[I][2023-03-07T21:56:30+0800] Executing '/bin/sh' for '[STANDALONE MODE]'
sh: cannot set terminal process group (-1): Inappropriate ioctl for device
sh: no job control in this shell
sh-4.2$ ls
bin boot data dev etc home lib lib64 lost+found media mnt ob opt proc root run sbin share srv sys tmp u01 usr var
sh-4.2$ ps wuax
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
99999 1 0.1 0.0 13760 1744 ? SNs 21:59 0:00 /bin/sh -i
99999 2 0.0 0.0 49556 1716 ? RN 21:59 0:00 ps wuax
sh-4.2$ id
uid=99999 gid=99999 groups=99999
sh-4.2$ echo "abc" > /tmp/abc.txt
sh: /tmp/abc.txt: Read-only file system
nsjail 的选项如下:
Usage: nsjail [options] -- path_to_command [args]
Options:
--help|-h
Help plz..
--mode|-M VALUE
Execution mode (default: 'o' [MODE_STANDALONE_ONCE]):
l: Wait for connections on a TCP port (specified with --port) [MODE_LISTEN_TCP]
o: Launch a single process on the console using clone/execve [MODE_STANDALONE_ONCE]
e: Launch a single process on the console using execve [MODE_STANDALONE_EXECVE]
r: Launch a single process on the console with clone/execve, keep doing it forever [MODE_STANDALONE_RERUN]
--config|-C VALUE
Configuration file in the config.proto ProtoBuf format (see configs/ directory for examples)
--exec_file|-x VALUE
File to exec (default: argv[0])
--execute_fd
Use execveat() to execute a file-descriptor instead of executing the binary path. In such case argv[0]/exec_file denotes a file path before mount namespacing
--chroot|-c VALUE
Directory containing / of the jail (default: none)
--no_pivotroot
When creating a mount namespace, use mount(MS_MOVE) and chroot rather than pivot_root. Usefull when pivot_root is disallowed (e.g. initramfs). Note: escapable is some configuration
--rw
Mount chroot dir (/) R/W (default: R/O)
--user|-u VALUE
Username/uid of processes inside the jail (default: your current uid). You can also use inside_ns_uid:outside_ns_uid:count convention here. Can be specified multiple times
--group|-g VALUE
Groupname/gid of processes inside the jail (default: your current gid). You can also use inside_ns_gid:global_ns_gid:count convention here. Can be specified multiple times
--hostname|-H VALUE
UTS name (hostname) of the jail (default: 'NSJAIL')
--cwd|-D VALUE
Directory in the namespace the process will run (default: '/')
--port|-p VALUE
TCP port to bind to (enables MODE_LISTEN_TCP) (default: 0)
--bindhost VALUE
IP address to bind the port to (only in [MODE_LISTEN_TCP]), (default: '::')
--max_conns VALUE
Maximum number of connections across all IPs (only in [MODE_LISTEN_TCP]), (default: 0 (unlimited))
--max_conns_per_ip|-i VALUE
Maximum number of connections per one IP (only in [MODE_LISTEN_TCP]), (default: 0 (unlimited))
--log|-l VALUE
Log file (default: use log_fd)
--log_fd|-L VALUE
Log FD (default: 2)
--time_limit|-t VALUE
Maximum time that a jail can exist, in seconds (default: 600)
--max_cpus VALUE
Maximum number of CPUs a single jailed process can use (default: 0 'no limit')
--daemon|-d
Daemonize after start
--verbose|-v
Verbose output
--quiet|-q
Log warning and more important messages only
--really_quiet|-Q
Log fatal messages only
--keep_env|-e
Pass all environment variables to the child process (default: all envars are cleared)
--env|-E VALUE
Additional environment variable (can be used multiple times). If the envar doesn't contain '=' (e.g. just the 'DISPLAY' string), the current envar value will be used
--keep_caps
Don't drop any capabilities
--cap VALUE
Retain this capability, e.g. CAP_PTRACE (can be specified multiple times)
--silent
Redirect child process' fd:0/1/2 to /dev/null
--stderr_to_null
Redirect child process' fd:2 (STDERR_FILENO) to /dev/null
--skip_setsid
Don't call setsid(), allows for terminal signal handling in the sandboxed process. Dangerous
--pass_fd VALUE
Don't close this FD before executing the child process (can be specified multiple times), by default: 0/1/2 are kept open
--disable_no_new_privs
Don't set the prctl(NO_NEW_PRIVS, 1) (DANGEROUS)
--rlimit_as VALUE
RLIMIT_AS in MB, 'max' or 'hard' for the current hard limit, 'def' or 'soft' for the current soft limit, 'inf' for RLIM64_INFINITY (default: 4096)
--rlimit_core VALUE
RLIMIT_CORE in MB, 'max' or 'hard' for the current hard limit, 'def' or 'soft' for the current soft limit, 'inf' for RLIM64_INFINITY (default: 0)
--rlimit_cpu VALUE
RLIMIT_CPU, 'max' or 'hard' for the current hard limit, 'def' or 'soft' for the current soft limit, 'inf' for RLIM64_INFINITY (default: 600)
--rlimit_fsize VALUE
RLIMIT_FSIZE in MB, 'max' or 'hard' for the current hard limit, 'def' or 'soft' for the current soft limit, 'inf' for RLIM64_INFINITY (default: 1)
--rlimit_nofile VALUE
RLIMIT_NOFILE, 'max' or 'hard' for the current hard limit, 'def' or 'soft' for the current soft limit, 'inf' for RLIM64_INFINITY (default: 32)
--rlimit_nproc VALUE
RLIMIT_NPROC, 'max' or 'hard' for the current hard limit, 'def' or 'soft' for the current soft limit, 'inf' for RLIM64_INFINITY (default: 'soft')
--rlimit_stack VALUE
RLIMIT_STACK in MB, 'max' or 'hard' for the current hard limit, 'def' or 'soft' for the current soft limit, 'inf' for RLIM64_INFINITY (default: 'soft')
--rlimit_memlock VALUE
RLIMIT_MEMLOCK in KB, 'max' or 'hard' for the current hard limit, 'def' or 'soft' for the current soft limit, 'inf' for RLIM64_INFINITY (default: 'soft')
--rlimit_rtprio VALUE
RLIMIT_RTPRIO, 'max' or 'hard' for the current hard limit, 'def' or 'soft' for the current soft limit, 'inf' for RLIM64_INFINITY (default: 'soft')
--rlimit_msgqueue VALUE
RLIMIT_MSGQUEUE in bytes, 'max' or 'hard' for the current hard limit, 'def' or 'soft' for the current soft limit, 'inf' for RLIM64_INFINITY (default: 'soft')
--disable_rlimits
Disable all rlimits, default to limits set by parent
--persona_addr_compat_layout
personality(ADDR_COMPAT_LAYOUT)
--persona_mmap_page_zero
personality(MMAP_PAGE_ZERO)
--persona_read_implies_exec
personality(READ_IMPLIES_EXEC)
--persona_addr_limit_3gb
personality(ADDR_LIMIT_3GB)
--persona_addr_no_randomize
personality(ADDR_NO_RANDOMIZE)
--disable_clone_newnet|-N
Don't use CLONE_NEWNET. Enable global networking inside the jail
--disable_clone_newuser
Don't use CLONE_NEWUSER. Requires euid==0
--disable_clone_newns
Don't use CLONE_NEWNS
--disable_clone_newpid
Don't use CLONE_NEWPID
--disable_clone_newipc
Don't use CLONE_NEWIPC
--disable_clone_newuts
Don't use CLONE_NEWUTS
--disable_clone_newcgroup
Don't use CLONE_NEWCGROUP. Might be required for kernel versions < 4.6
--enable_clone_newtime
Use CLONE_NEWTIME. Supported with kernel versions >= 5.3
--uid_mapping|-U VALUE
Add a custom uid mapping of the form inside_uid:outside_uid:count. Setting this requires newuidmap (set-uid) to be present
--gid_mapping|-G VALUE
Add a custom gid mapping of the form inside_gid:outside_gid:count. Setting this requires newgidmap (set-uid) to be present
--bindmount_ro|-R VALUE
List of mountpoints to be mounted --bind (ro) inside the container. Can be specified multiple times. Supports 'source' syntax, or 'source:dest'
--bindmount|-B VALUE
List of mountpoints to be mounted --bind (rw) inside the container. Can be specified multiple times. Supports 'source' syntax, or 'source:dest'
--tmpfsmount|-T VALUE
List of mountpoints to be mounted as tmpfs (R/W) inside the container. Can be specified multiple times. Supports 'dest' syntax. Alternatively, use '-m none:dest:tmpfs:size=8388608'
--mount|-m VALUE
Arbitrary mount, format src:dst:fs_type:options
--symlink|-s VALUE
Symlink, format src:dst
--disable_proc
Disable mounting procfs in the jail
--proc_path VALUE
Path used to mount procfs (default: '/proc')
--proc_rw
Is procfs mounted as R/W (default: R/O)
--seccomp_policy|-P VALUE
Path to file containing seccomp-bpf policy (see kafel/)
--seccomp_string VALUE
String with kafel seccomp-bpf policy (see kafel/)
--seccomp_log
Use SECCOMP_FILTER_FLAG_LOG. Log all actions except SECCOMP_RET_ALLOW). Supported since kernel version 4.14
--nice_level VALUE
Set jailed process niceness (-20 is highest -priority, 19 is lowest). By default, set to 19
--cgroup_mem_max VALUE
Maximum number of bytes to use in the group (default: '0' - disabled)
--cgroup_mem_memsw_max VALUE
Maximum number of memory+swap bytes to use (default: '0' - disabled)
--cgroup_mem_swap_max VALUE
Maximum number of swap bytes to use (default: '-1' - disabled)
--cgroup_mem_mount VALUE
Location of memory cgroup FS (default: '/sys/fs/cgroup/memory')
--cgroup_mem_parent VALUE
Which pre-existing memory cgroup to use as a parent (default: 'NSJAIL')
--cgroup_pids_max VALUE
Maximum number of pids in a cgroup (default: '0' - disabled)
--cgroup_pids_mount VALUE
Location of pids cgroup FS (default: '/sys/fs/cgroup/pids')
--cgroup_pids_parent VALUE
Which pre-existing pids cgroup to use as a parent (default: 'NSJAIL')
--cgroup_net_cls_classid VALUE
Class identifier of network packets in the group (default: '0' - disabled)
--cgroup_net_cls_mount VALUE
Location of net_cls cgroup FS (default: '/sys/fs/cgroup/net_cls')
--cgroup_net_cls_parent VALUE
Which pre-existing net_cls cgroup to use as a parent (default: 'NSJAIL')
--cgroup_cpu_ms_per_sec VALUE
Number of milliseconds of CPU time per second that the process group can use (default: '0' - no limit)
--cgroup_cpu_mount VALUE
Location of cpu cgroup FS (default: '/sys/fs/cgroup/cpu')
--cgroup_cpu_parent VALUE
Which pre-existing cpu cgroup to use as a parent (default: 'NSJAIL')
--cgroupv2_mount VALUE
Location of cgroupv2 directory (default: '/sys/fs/cgroup')
--use_cgroupv2
Use cgroup v2
--detect_cgroupv2
Use cgroupv2, if it is available. (Specify instead of use_cgroupv2)
--iface_no_lo
Don't bring the 'lo' interface up
--iface_own VALUE
Move this existing network interface into the new NET namespace. Can be specified multiple times
--macvlan_iface|-I VALUE
Interface which will be cloned (MACVLAN) and put inside the subprocess' namespace as 'vs'
--macvlan_vs_ip VALUE
IP of the 'vs' interface (e.g. "192.168.0.1")
--macvlan_vs_nm VALUE
Netmask of the 'vs' interface (e.g. "255.255.255.0")
--macvlan_vs_gw VALUE
Default GW for the 'vs' interface (e.g. "192.168.0.1")
--macvlan_vs_ma VALUE
MAC-address of the 'vs' interface (e.g. "ba:ad:ba:be:45:00")
--macvlan_vs_mo VALUE
Mode of the 'vs' interface. Can be either 'private', 'vepa', 'bridge' or 'passthru' (default: 'private')
--disable_tsc
Disable rdtsc and rdtscp instructions. WARNING: To make it effective, you also need to forbid `prctl(PR_SET_TSC, PR_TSC_ENABLE, ...)` in seccomp rules! (x86 and x86_64 only). Dynamic binaries produced by GCC seem to rely on RDTSC, but static ones should work.
--forward_signals
Forward fatal signals to the child process instead of always using SIKGILL.
Examples:
Wait on a port 31337 for connections, and run /bin/sh
nsjail -Ml --port 31337 --chroot / -- /bin/sh -i
Re-run echo command as a sub-process
nsjail -Mr --chroot / -- /bin/echo "ABC"
Run echo command once only, as a sub-process
nsjail -Mo --chroot / -- /bin/echo "ABC"
Execute echo command directly, without a supervising process
nsjail -Me --chroot / --disable_proc -- /bin/echo "ABC"