ModuleNotFoundError: No module named ‘_bz2‘,UserWarning: Could not import the lzma module.问题排查

引言

在一次下载数据集的过程中遇到了很多的问题，首先用了默认版本的python2.7的pip进行了依赖包的安装，之后import出现了很多问题，之后尝试安装python3.8，官网下载包后进行解压编译，然后用了 ln -s替换了默认版本python的link，又尝试了一次import，出现了以下问题。

正文

在import desed的时候报了以下错误：

>>> import desed
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/DESED/desed/desed/__init__.py", line 1, in <module>
    from .download import (
  File "/opt/DESED/desed/desed/download.py", line 13, in <module>
    import pandas as pd
  File "/usr/lib/python3.8/lib/python3.8/site-packages/pandas/__init__.py", line 51, in <module>
    from pandas.core.api import (
  File "/usr/lib/python3.8/lib/python3.8/site-packages/pandas/core/api.py", line 31, in <module>
    from pandas.core.groupby import Grouper, NamedAgg
  File "/usr/lib/python3.8/lib/python3.8/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy
  File "/usr/lib/python3.8/lib/python3.8/site-packages/pandas/core/groupby/generic.py", line 65, in <module>
    from pandas.core.frame import DataFrame
  File "/usr/lib/python3.8/lib/python3.8/site-packages/pandas/core/frame.py", line 119, in <module>
    from pandas.core import algorithms, common as com, generic, nanops, ops
  File "/usr/lib/python3.8/lib/python3.8/site-packages/pandas/core/generic.py", line 113, in <module>
    from pandas.io.formats import format as fmt
  File "/usr/lib/python3.8/lib/python3.8/site-packages/pandas/io/formats/format.py", line 77, in <module>
    from pandas.io.common import stringify_path
  File "/usr/lib/python3.8/lib/python3.8/site-packages/pandas/io/common.py", line 3, in <module>
    import bz2
  File "/usr/lib/python3.8/lib/python3.8/bz2.py", line 19, in <module>
    from _bz2 import BZ2Compressor, BZ2Decompressor
ModuleNotFoundError: No module named '_bz2'

从最后一行可以看到意思应该是没有找到_bz2这个module，原因是在python目录下缺少了_bz2.cpython-35m-x86_64-linux-gnu.so这个文件，应该是在执行pip install -e .命令的时候把这个库文件安装在了其它版本的python目录下了（服务器上默认装了python2.7和python3.5，但是这两个python版本太低，于是又装了python3.8）。

于是直接用find查找：

xxx@ubuntu:/usr# find . -name "_bz2.*"
./lib/python3.5/lib-dynload/_bz2.cpython-35m-x86_64-linux-gnu.so

可以看到它在python3.5目录下，接下来只要把它复制到python3.8相应的目录下即可（注意：_bz2.cpython-35m-x86_64-linux-gnu.so要改为_bz2.cpython-38-x86_64-linux-gnu.so）“35m”对应python3.5版本。

mv _bz2.cpython-35m-x86_64-linux-gnu.so _bz2.cpython-38-x86_64-linux-gnu.so

还要把权限从-rw-r--r--修改为-rwxr-xr-x

chmod +x _bz2.cpython-38m-x86_64-linux-gnu.so

然后重新import，发现又出现了新的错误

/usr/lib/python3.8/lib/python3.8/site-packages/pandas/compat/__init__.py:97: UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.
  warnings.warn(msg)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/DESED/desed/desed/__init__.py", line 1, in <module>
    from .download import (
  File "/home/DESED/desed/desed/download.py", line 14, in <module>
    import youtube_dl
  File "/usr/lib/python3.8/lib/python3.8/site-packages/youtube_dl/__init__.py", line 15, in <module>
    from .options import (
  File "/usr/lib/python3.8/lib/python3.8/site-packages/youtube_dl/options.py", line 8, in <module>
    from .downloader.external import list_external_downloaders
  File "/usr/lib/python3.8/lib/python3.8/site-packages/youtube_dl/downloader/__init__.py", line 3, in <module>
    from .common import FileDownloader
  File "/usr/lib/python3.8/lib/python3.8/site-packages/youtube_dl/downloader/common.py", line 10, in <module>
    from ..utils import (
  File "/usr/lib/python3.8/lib/python3.8/site-packages/youtube_dl/utils.py", line 30, in <module>
    import ssl
  File "/usr/lib/python3.8/lib/python3.8/ssl.py", line 98, in <module>
    import _ssl             # if we can't import it, let the error propagate
ModuleNotFoundError: No module named '_ssl'

 应该是缺少了_ssl module，于是把相应的依赖包全都安装了一遍,问题解决。

python-dev libffi-dev libssl-dev build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev