Mac pip install python-snappy失败

在Mac上尝试安装python-snappy库时遇到错误,提示缺少snappy-c.h文件。解决方案是使用brew先安装snappy,然后再通过pip安装python-snappy。错误详细信息包括编译失败和gcc命令执行错误,提示找不到snappy-c.h头文件。
摘要由CSDN通过智能技术生成

今天在Mac本上,安装python-snappy报了如下错误,记得之前没有,在网上搜索了一圈说是缺少snappy-c.h,具体的解决方法如下(原文参考链接 具体的解决方法 感谢):

brew install snappy          # 1、先用brew 安装 snappy
pip install python-snappy    # 2、在进行安装,即可

这是我的错误提示,希望可以帮到大家:

 pip install python-snappy
Collecting python-snappy
  Using cached python-snappy-0.6.0.tar.gz (21 kB)
Building wheels for collected packages: python-snappy
  Building wheel for python-snappy (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/xyl/opt/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/d4/ykh_5r7j6214f7sts5sxj_x00000gp/T/pip-install-_n3ew0xm/python-snappy/setup.py'"'"'; __file__='"'"'/private/var/folders/d4/ykh_5r7j6214f7sts5sxj_x00000gp/T/pip-install-_n3ew0xm/python-snappy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/d4/ykh_5r7j6214f7sts5sxj_x00000gp/T/pip-wheel-uur8b72g
       cwd: /private/var/folders/d4/ykh_5r7j6214f7sts5sxj_x00000gp/T/pip-install-_n3ew0xm/python-snappy/
  Complete output (23 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.9-x86_64-3.8
  creating build/lib.macosx-10.9-x86_64-3.8/snappy
  copying snappy/snappy.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
  copying snappy/snappy_cffi.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
  copying snappy/__init__.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
  copying snappy/hadoop_snappy.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
  copying snappy/snappy_formats.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
  copying snappy/__main__.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
  copying snappy/snappy_cffi_builder.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
  running build_ext
  building 'snappy._snappy' extension
  creating build/temp.macosx-10.9-x86_64-3.8
  creating build/temp.macosx-10.9-x86_64-3.8/snappy
  gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/xyl/opt/anaconda3/include -arch x86_64 -I/Users/xyl/opt/anaconda3/include -arch x86_64 -I/Users/xyl/opt/anaconda3/include/python3.8 -c snappy/snappymodule.cc -o build/temp.macosx-10.9-x86_64-3.8/snappy/snappymodule.o
  snappy/snappymodule.cc:32:10: fatal error: 'snappy-c.h' file not found
  #include <snappy-c.h>
           ^~~~~~~~~~~~
  1 error generated.
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for python-snappy
  Running setup.py clean for python-snappy
Failed to build python-snappy
Installing collected packages: python-snappy
    Running setup.py install for python-snappy ... error
    ERROR: Command errored out with exit status 1:
     command: /Users/xyl/opt/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/d4/ykh_5r7j6214f7sts5sxj_x00000gp/T/pip-install-_n3ew0xm/python-snappy/setup.py'"'"'; __file__='"'"'/private/var/folders/d4/ykh_5r7j6214f7sts5sxj_x00000gp/T/pip-install-_n3ew0xm/python-snappy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/d4/ykh_5r7j6214f7sts5sxj_x00000gp/T/pip-record-0whs1afj/install-record.txt --single-version-externally-managed --compile --install-headers /Users/xyl/opt/anaconda3/include/python3.8/python-snappy
         cwd: /private/var/folders/d4/ykh_5r7j6214f7sts5sxj_x00000gp/T/pip-install-_n3ew0xm/python-snappy/
    Complete output (23 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.9-x86_64-3.8
    creating build/lib.macosx-10.9-x86_64-3.8/snappy
    copying snappy/snappy.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
    copying snappy/snappy_cffi.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
    copying snappy/__init__.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
    copying snappy/hadoop_snappy.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
    copying snappy/snappy_formats.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
    copying snappy/__main__.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
    copying snappy/snappy_cffi_builder.py -> build/lib.macosx-10.9-x86_64-3.8/snappy
    running build_ext
    building 'snappy._snappy' extension
    creating build/temp.macosx-10.9-x86_64-3.8
    creating build/temp.macosx-10.9-x86_64-3.8/snappy
    gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/xyl/opt/anaconda3/include -arch x86_64 -I/Users/xyl/opt/anaconda3/include -arch x86_64 -I/Users/xyl/opt/anaconda3/include/python3.8 -c snappy/snappymodule.cc -o build/temp.macosx-10.9-x86_64-3.8/snappy/snappymodule.o
    snappy/snappymodule.cc:32:10: fatal error: 'snappy-c.h' file not found
    #include <snappy-c.h>
             ^~~~~~~~~~~~
    1 error generated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/xyl/opt/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/d4/ykh_5r7j6214f7sts5sxj_x00000gp/T/pip-install-_n3ew0xm/python-snappy/setup.py'"'"'; __file__='"'"'/private/var/folders/d4/ykh_5r7j6214f7sts5sxj_x00000gp/T/pip-install-_n3ew0xm/python-snappy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/d4/ykh_5r7j6214f7sts5sxj_x00000gp/T/pip-record-0whs1afj/install-record.txt --single-version-externally-managed --compile --install-headers /Users/xyl/opt/anaconda3/include/python3.8/python-snappy Check the logs for full command output.

声明: 总结学习,有问题或不当之处,可以批评指正哦,谢谢。

要解压.snappy.parquet文件,你可以按照以下步骤进行操作: 1. 首先,确保你已经安装了python-snappy和pyarrow这两个库。你可以使用以下命令进行安装:\[1\] ``` pip install python-snappy pip install pyarrow ``` 2. 下载源码并编译打包,找到模块文件的target/datax/plugin/reader/hdfsreader.zip文件。将该文件解压到datax安装目录的${DATAX_HOME}/plugin/reader/下。请注意提前备份原有的hdfsreader插件,以防出现问题时可以回滚。\[2\] 3. 配置datax job,确保你的datax job中已经正确配置了相关的参数和路径。 4. 在代码中使用相应的库来解压.snappy.parquet文件。你可以使用python-snappy库来解压snappy压缩的数据,然后使用pyarrow库来读取和处理parquet文件。具体的代码开发步骤可以参考相关的文档和资料。\[3\] 总结起来,要解压.snappy.parquet文件,你需要安装python-snappy和pyarrow库,然后根据具体的需求进行代码开发和配置。希望这些信息对你有帮助! #### 引用[.reference_title] - *1* [成功解决RuntimeError: Decompression ‘SNAPPY‘ not available. Options: [‘GZIP‘, ‘UNCOMPRESSED‘]](https://blog.csdn.net/qq_41185868/article/details/109662915)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v91^insertT0,239^v3^insert_chatgpt"}} ] [.reference_item] - *2* *3* [Datax插件二次开发之HdfsReader支持parquet](https://blog.csdn.net/qq_34599449/article/details/122735299)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v91^insertT0,239^v3^insert_chatgpt"}} ] [.reference_item] [ .reference_list ]
评论 3
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值