举个例子,把这个函数写成 cython 的来加速
def insert_cell(ids_dict: dict, __id: str, count: int):
if __id in ids_dict:
ids_dict[__id] += count
else:
ids_dict[__id] = count
最简单的方式,就是直接扔到 pyx 文件,然后编译一下,速度提升了一倍,setup.py
脚本如下
from distutils.core import setup
from Cython.Build import cythonize
# python -m pip install cython
# python setup.py build_ext --inplace
setup(name="Cython Utils", ext_modules=cythonize("cython_utils.pyx", annotate=True))
annotation 打开的话,都是黄色,笑死
这篇文章给了一些对应关系
https://blog.csdn.net/wc781708249/article/details/80249166
于是可以简单修改一下,做一下类型转化,我实际测试了下,运行速度和上一版没多大的区别
def insert_cell(ids_dict: dict, _id: str, count: int):
cdef int __count = count
# cdef char* __id = _id
cdef dict __ids_dict = ids_dict
if _id in __ids_dict:
__ids_dict[_id] += __count
else:
__ids_dict[_id] = __count
我以为是Python的字典没有很好的转化为编译型语言的类型,于是尝试了以下方法,全部转化为C++的类型,结果更慢了hhh
博客中说这不是一个明智的选择,用C++的map去与Python的dict做对应
# distutils: language=c++
from libcpp.map cimport map
from libcpp.string cimport string
# https://stackoverflow.com/questions/32266444/using-a-dictionary-in-cython-especially-inside-nogil
def insert_cell(ids_dict: dict, _id: bytes, count: int):
cdef int __count = count
cdef string __id = _id
cdef map[string, int] __ids_dict = ids_dict
# cdef map[string, int].iterator end = __ids_dict.end()
# cdef map[string, int].iterator it = __ids_dict.begin()
if _id in ids_dict:
__ids_dict[_id] += __count
else:
__ids_dict[_id] = __count
用这个demo,有个问题:C++的string,在Python3是bytes类型,可能需要修改一下