Python与C语言混合编程：通过distutils或setuptools实现的一个简单的C扩展

这个介绍一个扩展Python组件的例子，组件是通过C语言开发的(用python的Ｃ-API实现)， 然后通过python的distutils组件安装，例子中内容来自《python cocobook》英文版，一本非常不错的书。

目录结构

首先目录结构如下

dev
|__sample.h
|__sample.c
|__subtest01
             |__pysample.c
             |__setup.py
             |__libsample.so

源码内容

sample.h

/* sample.h */

extern int gcd(int x, int y);
extern int in_mandel(double x0, double y0, int n);
extern int divide(int a, int b, int *remainder);
extern double avg(double *a, int n);

typedef struct Point {
    double x,y;
} Point;

extern double distance(Point *p1, Point *p2);

sample.c

/* sample.c */
#include <math.h>

/* Compute the greatest common divisor */
int gcd(int x, int y) {
    int g = y;
    while (x > 0) {
        g = x;
        x = y % x;
        y = g;
    }
    return g;
}

/* Test if (x0,y0) is in the Mandelbrot set or not */
int in_mandel(double x0, double y0, int n) {
  double x=0,y=0,xtemp;
  while (n > 0) {
    xtemp = x*x - y*y + x0;
    y = 2*x*y + y0;
    x = xtemp;
    n -= 1;
    if (x*x + y*y > 4) return 0;
  }
  return 1;
}

/* Divide two numbers */
int divide(int a, int b, int *remainder) {
  int quot = a / b;
  *remainder = a % b;
  return quot;
}

/* Average values in an array */
double avg(double *a, int n) {
  int i;
  double total = 0.0;
  for (i = 0; i < n; i++) {
    total += a[i];
  }
  return total / n;
}

/* A C data structure */
typedef struct Point {
    double x,y;
} Point;

/* Function involving a C data structure */
double distance(Point *p1, Point *p2) {
   return hypot(p1->x - p2->x, p1->y - p2->y);
}

pysample.c

#include "Python.h"
#include "sample.h"

/* int gcd(int, int) */
static PyObject *py_gcd(PyObject *self, PyObject *args) {
  int x, y, result;

  if (!PyArg_ParseTuple(args,"ii", &x, &y)) {
    return NULL;
  }
  result = gcd(x,y);
  return Py_BuildValue("i", result);
}

/* int in_mandel(double, double, int) */
static PyObject *py_in_mandel(PyObject *self, PyObject *args) {
  double x0, y0;
  int n;
  int result;
  
  if (!PyArg_ParseTuple(args, "ddi", &x0, &y0, &n)) {
    return NULL;
  }
  result = in_mandel(x0,y0,n);
  return Py_BuildValue("i", result);
}

/* int divide(int, int, int *) */
static PyObject *py_divide(PyObject *self, PyObject *args) {
  int a, b, quotient, remainder;
  if (!PyArg_ParseTuple(args, "ii", &a, &b)) {
    return NULL;
  }
  quotient = divide(a,b, &remainder);
  return Py_BuildValue("(ii)", quotient, remainder);
}

/* Module method table */
static PyMethodDef SampleMethods[] = {
  {"gcd",  py_gcd, METH_VARARGS, "Greatest common divisor"},
  {"in_mandel", py_in_mandel, METH_VARARGS, "Mandelbrot test"},
  {"divide", py_divide, METH_VARARGS, "Integer division"},
  { NULL, NULL, 0, NULL}
};

/* Module structure */
static struct PyModuleDef samplemodule = {
  PyModuleDef_HEAD_INIT,
  "sample",           /* name of module */
  "A sample module",  /* Doc string (may be NULL) */
  -1,                 /* Size of per-interpreter state or -1 */
  SampleMethods       /* Method table */
};

/* Module initialization function */
PyMODINIT_FUNC
PyInit_sample(void) {
  return PyModule_Create(&samplemodule);
}

setup.py

# setup.py
from distutils.core import setup, Extension

setup(name="sample", 
      ext_modules=[
        Extension("sample",
                  ["../sample.c", "pysample.c"],
                  include_dirs = ['..'],
                  )
        ]
)

编译安装

在目录subtest01下面，直接编译安装
（注意一般情况下，大家直接使用python命令即可，我使用的是自己编译的版本python372，编译过程可参考：
https://blog.csdn.net/tanmx219/article/details/86518446）

$  python372 setup.py build_ext --inplace
此时会输出一些信息，

running build_ext
building 'sample' extension
creating build
creating build/temp.linux-x86_64-3.7-pydebug
gcc -pthread -Wno-unused-result -Wsign-compare -g -Og -Wall -fPIC -fPIC -I.. -I/usr/local/include/python3.7dm -c ../sample.c -o build/temp.linux-x86_64-3.7-pydebug/../sample.o
gcc -pthread -Wno-unused-result -Wsign-compare -g -Og -Wall -fPIC -fPIC -I.. -I/usr/local/include/python3.7dm -c pysample.c -o build/temp.linux-x86_64-3.7-pydebug/pysample.o
gcc -pthread -shared build/temp.linux-x86_64-3.7-pydebug/../sample.o build/temp.linux-x86_64-3.7-pydebug/pysample.o -L/usr/local/lib -lpython3.7dm -o /home/matthew/dev/tutorial/subtest01/sample.cpython-37dm-x86_64-linux-gnu.so

此时，如果你要在subtest01目录下启动python372，是可以直接使用sample这个组件的，如，

$ python372
Python 3.7.2 (default, Jan 18 2019, 20:12:21) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sample
>>> sample.gcd(35,42)
7
>>> sample.divide(42,8)
(5, 2)
>>> sample
<module 'sample' from '~/dev/tutorial/subtest01/sample.cpython-37dm-x86_64-linux-gnu.so'>
>>> exit()

但若要在任意目录下都能运行sample组件，则需要把组件拷贝到python的库目录下面，

$ sudo  cp  sample.cpython-37dm-x86_64-linux-gnu.so      /usr/local/lib/python3.7/lib-dynload/

测试完了就删除掉吧，没啥用

$ sudo rm -f  /usr/local/lib/python3.7/lib-dynload/sample.cpython-37dm-x86_64-linux-gnu.so

整个过程还是比较简单的。

如果你的工程项目比较大的话，相信都会综合应用一些相关的工具来实现，比如cmake，那么这些安装卸载的东西就能通过cmake脚本实现啦。

使用setuptools

本质上setuptools和distutils没有什么太大的不同，可以看作是distutils的增强版。所以如果你想使用setuptools，只要在setup.py中把distutils换成setuptools即可，下面我给出本演示中可以使用的完整的setup.py源码，编译安装命令仍然是上面那个：
｀python setup.py build_ext --inplace｀

# setup.py
from setuptools import setup, Extension

setup(name="sample", 
      ext_modules=[
        Extension("sample",
                  ["../sample.c", "pysample.c"],
                  include_dirs = ['..'],
                  )
        ]
)

如果你想把打包发布的话，使用命令｀python setup.py sdist｀会得到一个压缩包，拷贝到你需要的地方解压缩再｀python setup.py install｀就可以了，这个过程这里就不再展开详述了。

其他工具

其他工具如cython, swig我都没怎么用过，这里给个链接为参考：
https://blog.csdn.net/tanmx219/article/details/86666456

还有一个工具pybind11（python + c++11)，这个在pytorch源码third-party中有用到，源码量也比较少，参考：
https://github.com/pybind/pybind11

参考资料

https://packaging.python.org/
https://media.readthedocs.org/pdf/python-packaging-user-guide/latest/python-packaging-user-guide.pdf
https://thomasnyberg.com/cpp_extension_modules.html
https://thomasnyberg.com/what_are_extension_modules.html