ARM端交叉编译python3.7及第三方库

---

前言

在PC端交叉编译python 3.7及第三方库，并移植到ARM开发板上。

---

一、开发部署环境

开发环境	软件版本
PC系统版本	Ubuntu 18.04LTS
开发板型号	Firefly AIO-3399 ProC开发板 (3+16GB)
开发板操作系统	Ubuntu 18.04LTS
开发板固件版本	AIO-RK3399PROC-UBUNTU18.04-GPT-20200525-1016.img
交叉编译工具链	gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu
Python版本	Python 3.7.10
zlib版本	1.12.11
numpy版本	1.16.3
pyserial版本	3.4

二、安装前的相关包下载及环境准备

1. 交叉编译工具链环境搭建

参考《Firefly AIO-3399ProC开发板与PC端部署Qt5.12.2交叉编译环境》
文章，部署好gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu交叉编译工具链。

2.安装环境所需依赖包

• 安装编译所需依赖包

sudo apt-get install gcc make cmake -y
sudo apt-get install libffi-dev

编译前必须安装libffi包，否则后续python导入模块时会出现以下错误：

Traceback (most recent call last):
  File "setup.py", line 415, in <module>
    setup_package()
  File "setup.py", line 392, in setup_package
    from setuptools import setup
  File "/home/user/Python-3.7.10/build/lib/python3.7/site-packages/setuptools/__init__.py", line 19, in <module>
    from setuptools.dist import Distribution
  File "/home/user/Python-3.7.10/build/lib/python3.7/site-packages/setuptools/dist.py", line 34, in <module>
    from setuptools import windows_support
  File "/home/user/Python-3.7.10/build/lib/python3.7/site-packages/setuptools/windows_support.py", line 2, in <module>
    import ctypes
  File "/home/user/Python-3.7.10/build/lib/python3.7/ctypes/__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'

• 编译zlib包

host端（x64架构）

#host端zlib
./configure --prefix=/home/user/zlib-1.2.11/host
make
sudo make install

target端（aarch64架构）

./configure --prefix=/home/user/zlib-1.2.11/target --enable-shared 

• 若无报错，编辑文件夹下的Makefile文件，将CC、AR、RANLIB改为交叉编译工具的命令后保存。

sudo vim Makefile
CC=aarch64-linux-gnu-gcc

CFLAGS=-O3 -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN
#CFLAGS=-O -DMAX_WBITS=14 -DMAX_MEM_LEVEL=7
#CFLAGS=-g -DZLIB_DEBUG
#CFLAGS=-O3 -Wall -Wwrite-strings -Wpointer-arith -Wconversion \
#           -Wstrict-prototypes -Wmissing-prototypes

SFLAGS=-O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN
LDFLAGS=
TEST_LDFLAGS=-L. libz.a
LDSHARED=aarch64-linux-gnu-gcc -shared -Wl,-soname,libz.so.1,--version-script,zlib.map
CPP=aarch64-linux-gnu-gcc -E

STATICLIB=libz.a
SHAREDLIB=libz.so
SHAREDLIBV=libz.so.1.2.11
SHAREDLIBM=libz.so.1
LIBS=$(STATICLIB) $(SHAREDLIBV)

AR=aarch64-linux-gnu-gcc-ar
ARFLAGS=rc
RANLIB=aarch64-linux-gnu-gcc-ranlib
LDCONFIG=ldconfig
LDSHAREDLIBC=-lc
TAR=tar
SHELL=/bin/sh
EXE=

ESC 后输入:wq后保存并退出。

• make & make install

make
make install

3.编译host端python 3.7及第三方库

• 编译python 3.7

host端python编译

cd /home/user/Python-3.7.10/Modules
sudo vim Setup.dist
#338行左右
zlib zlibmodule.c -I/home/user/zlib-1.2.11/include -L/home/user/zlib-1.2.11/lib -lz

ESC 后输入:wq后保存并退出。

./configure --prefix=$PWD/build --enable-optimizations
make -j8
sudo make install

完成后配置python运行环境

sudo rm /usr/bin/python3
sudo ln -s /home/user/Python-3.7.10/build/bin/python3.7 /usr/bin/python3

测试python是否正常运行：

python3
Python 3.7.10 (default, May 18 2021, 12:33:36) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

• 将numpy库安装到编译的host端python中

cd /home/user/numpy-1.16.3
sudo python3 setup.py install --prefix=/home/user/Python-3.7.10/build

测试numpy能否正常导入

python3
Python 3.7.10 (default, May 18 2021, 12:33:36) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.__version__
>>> 1.16.3

至此host端python3.7 编译完成。

4.交叉编译ARM端python 3.7及第三方库

• 编译python 3.7

arm端python编译

sudo cp /home/user/zlib-1.2.11/target/include/ /opt/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu
sudo cp /home/user/zlib-1.2.11/target/lib/ /opt/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu

cd /home/user/Python-3.7.10/Modules
sudo vim Setup.dist
#338行左右
zlib zlibmodule.c -I/opt/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/include -L/opt/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/lib -lz

ESC 后输入:wq后保存并退出。

```bash
user@admin-machine:~/Python-3.7.10$ sudo make clean
[sudo] user 的密码： 
find . -depth -name '__pycache__' -exec rm -rf {} ';'
find . -name '*.py[co]' -exec rm -f {} ';'
find . -name '*.[oa]' -exec rm -f {} ';'
find . -name '*.s[ol]' -exec rm -f {} ';'
find . -name '*.so.[0-9]*.[0-9]*' -exec rm -f {} ';'
find build -name 'fficonfig.h' -exec rm -f {} ';' || true
find build -name '*.py' -exec rm -f {} ';' || true
find build -name '*.py[co]' -exec rm -f {} ';' || true
rm -f pybuilddir.txt
rm -f Lib/lib2to3/*Grammar*.pickle
rm -f Programs/_testembed Programs/_freeze_importlib
find build -type f -a ! -name '*.gc??' -exec rm -f {} ';'
rm -f Include/pydtrace_probes.h
rm -f profile-gen-stamp

user@admin-machine:~/Python-3.7.10$ ./configure CC=aarch64-linux-gnu-gcc \
              CXX=aarch64-linux-gnu-g++ \
              AR=aarch64-linux-gnu-ar \
              RANLIB=aarch64-linux-gnu-ranlib \
              --host=aarch64-linux-gnu \
              --build=x86_64-linux-gnu \
              --target=aarch64-linux-gnu \
              --disable-ipv6 \
              --prefix=/opt/python3 \
              --with-ensurepip=yes \
              --enable-optimizations \
              ac_cv_file__dev_ptmx=yes \
              ac_cv_file__dev_ptc=yes

无报错后，make&makeinstall即可

make -j8
sudo make install

若出现报错：subprocess.CalledProcessError: Command ‘(‘lsb_release’, ‘-a’)’ returned non-zero exit status 1.
执行以下命令后重新make install即可

sudo rm -rf /usr/bin/lsb_release
sudo make install

检查生成target端的文件类型

user@admin-machine:~/test/Python-3.7.10/target/bin$ file *
2to3:              symbolic link to 2to3-3.7
2to3-3.7:          Python script, ASCII text executable
idle3:             symbolic link to idle3.7
idle3.7:           Python script, ASCII text executable
pydoc3:            symbolic link to pydoc3.7
pydoc3.7:          a /home/user/test/Python-3.7.10/target/bin/python3.7 script, ASCII text executable
python3:           symbolic link to python3.7
python3.7:         ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=aa5fcacb8cbe414cc97371fc953eedb534db3d43, with debug_info, not stripped
python3.7-config:  symbolic link to python3.7m-config
python3.7m:        ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=aa5fcacb8cbe414cc97371fc953eedb534db3d43, with debug_info, not stripped
python3.7m-config: POSIX shell script, ASCII text executable
python3-config:    symbolic link to python3.7-config
pyvenv:            symbolic link to pyvenv-3.7
pyvenv-3.7:        a /home/user/test/Python-3.7.10/target/bin/python3.7 script, ASCII text executable

• 交叉编译python第三方库pyserial

export CROSS_COMPILE=aarch64-linux-gnu- \
export CC export CC="${CROSS_COMPILE}gcc" \
export LDSHARED="${CC} -shared" \
export LDFLAGS="-L/home/user/test/Python-3.7.10/target/lib" \
export CFLAGS="-I/home/user/test/Python-3.7.10/target/python3.7m" \
export PYTHONPATH=$PYTHONPATH:/home/user/test/Python-3.7.10/target/lib/python3.7/site-packages
sudo python3 setup.py install --prefix=/home/user/test/Python-3.7.10/target

完成后对应/home/user/test/Python-3.7.10/target/lib/python3.7/site-package下生成pyserial-3.5-py3.7.egg文件，将其解压

sudo unzip pyserial-3.5-py3.7.egg

将target文件夹打包，上传至ARM开发板上

sudo  tar zcvf rkpython37.tar.gz target/

• 在ARM开发板上解压交叉编译的rkpython37.tar.gz文件夹

sudo tar zxvf rkpython37.tar.gz /opt/python3.7
sudo rm /usr/bin/python3
sudo ln -s /opt/python3.7/bin/python3.7 /usr/bin/python3

测试python和第三方库是否正常

python3
Python 3.7.10 (default, May 18 2021, 12:33:36) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import serial
>>> serial.__version__
>>> 3.4

至此arm端python 3.7及第三方库交叉编译部署完成。

总结

在PC端交叉编译出的python版本缺少了ssl模块，导致在交叉编译h5py、numpy等第三方库时出现各种问题，后续考虑在buildroot中移植ARM版的python及其第三方库。
主要参考文章:
《python3.7移植到ARM开发板》
《ARM40之Python3.6.4移植》
《Python3.6交叉编译至ARM（含zlib+第三方库）》
《python移植三方库psutil》