[Cython] 将Python代码编译为.so文件

一、问题描述

使用Cython将Python代码文件转换为c文件，并编译为so文件¹²。

二、版本说明

1、Python

Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

2、Cython

reference³

java version "1.8.0_152"
Java(TM) SE Runtime Environment (build 1.8.0_152-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.152-b16, mixed mode)

3、Linux

LSB Version:	:core-4.1-amd64:core-4.1-noarch
Distributor ID:	CentOS
Description:	CentOS Linux release 8.1.1911 (Core) 
Release:	8.1.1911
Codename:	Core

4、Gcc

gcc version 8.3.1 20190507 (Red Hat 8.3.1-4) (GCC)

三、示例

1、Python代码准备

项目结构如下：

/test
	/m1.py
	/m2.py
/compile_test.py
/compiler.py

代码：

m1.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def name():
    print("this is m1")

m2.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def get_name():
    return "this is m2"

compile_test.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from test import m1, m2


def show_names():
    m1.name()
    print(m2.get_name())

compiler.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-


from setuptools import setup, Extension
from Cython.Build import cythonize


def c_compile(name, modules):
    """
    使用Cython进行编译
    :param str name: 项目名称
    :param list modules: [(module_name, source_file_path), (..., ...)]
    :return:
    """
    module_list = []
    for m in modules:
        module_list.append(
            Extension(name=m[0], sources=[m[1]])
        )
    setup(
        name=name,
        ext_modules=cythonize(
            module_list=module_list,
            language_level=3,  # python3
        ),
        zip_safe=False,
        install_requires=[
            'Cython',
        ]
    )


c_compile("compile-test", [
        ("compile_test", "compile_test.py"),
        ("test.m1", "test/m1.py"),
        ("test.m2", "test/m2.py"),
        ("test.__init__", "test/__init__.py")
    ])

2、执行编译

执行命令：python compiler.py build_ext --inplace，执行过程如下：

[root@gavinpan cython-compile]# python compiler.py build_ext --inplace
running build_ext
building 'compile_test' extension
creating build
creating build/temp.linux-x86_64-3.6
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c compile_test.c -o build/temp.linux-x86_64-3.6/compile_test.o
creating build/lib.linux-x86_64-3.6
gcc -pthread -shared -Wl,-z,relro -Wl,-z,now -g build/temp.linux-x86_64-3.6/compile_test.o -L/usr/lib64 -lpython3.6m -o build/lib.linux-x86_64-3.6/compile_test.cpython-36m-x86_64-linux-gnu.so
building 'test.m1' extension
creating build/temp.linux-x86_64-3.6/test
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c test/m1.c -o build/temp.linux-x86_64-3.6/test/m1.o
creating build/lib.linux-x86_64-3.6/test
gcc -pthread -shared -Wl,-z,relro -Wl,-z,now -g build/temp.linux-x86_64-3.6/test/m1.o -L/usr/lib64 -lpython3.6m -o build/lib.linux-x86_64-3.6/test/m1.cpython-36m-x86_64-linux-gnu.so
building 'test.m2' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c test/m2.c -o build/temp.linux-x86_64-3.6/test/m2.o
gcc -pthread -shared -Wl,-z,relro -Wl,-z,now -g build/temp.linux-x86_64-3.6/test/m2.o -L/usr/lib64 -lpython3.6m -o build/lib.linux-x86_64-3.6/test/m2.cpython-36m-x86_64-linux-gnu.so
building 'test.__init__' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c test/__init__.c -o build/temp.linux-x86_64-3.6/test/__init__.o
gcc -pthread -shared -Wl,-z,relro -Wl,-z,now -g build/temp.linux-x86_64-3.6/test/__init__.o -L/usr/lib64 -lpython3.6m -o build/lib.linux-x86_64-3.6/test/__init__.cpython-36m-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-3.6/compile_test.cpython-36m-x86_64-linux-gnu.so -> 
copying build/lib.linux-x86_64-3.6/test/m1.cpython-36m-x86_64-linux-gnu.so -> test
copying build/lib.linux-x86_64-3.6/test/m2.cpython-36m-x86_64-linux-gnu.so -> test
copying build/lib.linux-x86_64-3.6/test/__init__.cpython-36m-x86_64-linux-gnu.so -> test

ll查看：

3、调用测试

为便于测试，首先将.so文件归入新的文件夹中，目录结构如下：

/compile_test.cpython-36m-x86_64-linux-gnu.so
/test
	/m1.cpython-36m-x86_64-linux-gnu.so
	/m2.cpython-36m-x86_64-linux-gnu.so
	/__init__.cpython-36m-x86_64-linux-gnu.so
/runner.py

编辑测试代码:
runner.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-


import compile_test


compile_test.show_names()

执行：python runner.py，运行结果如下：

参考:

---

1. Cython-Document； ↩︎

2. Cython-With-Setuptools； ↩︎

3. Cython-PyPi； ↩︎