sed 环境配置

ZhengXinTang

已于 2024-10-06 17:04:01 修改

阅读量171

点赞数 4

分类专栏： # 环境相关文章标签： python

于 2024-10-06 15:00:48 首次发布

本文链接：https://blog.csdn.net/chumingqian/article/details/142726832

版权

环境相关专栏收录该内容

29 篇文章 0 订阅

订阅专栏

参考项目来自这里：
https://github.com/DCASE-REPO/DESED_task/tree/master/recipes/dcase2023_task4_baseline

1. 更新自己的 conda

避免一些包在旧的conda 环境中不存在；

conda update conda

使用conda 指定安装对应版本

# CUDA 11.7
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia

然后安装 Pytorch lightning

conda install -y  ffmpeg pandas numba scipy torchmetrics  tqdm pytorch-lightning=1.9

2. 关于 pip install -e 的问题；

在执行完 pip install -e …/…/.

提示安装完成

 Running setup.py develop for desed-task
Successfully installed Cython-3.0.11 desed-task einops-0.8.0 exceptiongroup-1.2.2 iniconfig-2.0.0 lazy-dataset-0.0.15 pathlib-1.0.1 pluggy-1.5.0 psds_eval-0.5.3 pytest-8.3.3 sed_eval-0.2.1 sed_scores_eval-0.0.4 tomli-2.0.2

2.1 numpy 问题

当执行 train_sed.py 的时候，
此时会出现numpy.core.multiarray 相关问题，


anaconda3/envs/lightningGNN/lib/python3.8/site-packages/sed_scores_eval/base_modules/statistics.py", line 6, in <module>
    from sed_scores_eval.base_modules.cy_detection import onset_offset_curves
  File "sed_scores_eval/base_modules/cy_detection.pyx", line 1, in init sed_scores_eval.base_modules.cy_detection
ImportError: numpy.core.multiarray failed to import (auto-generated because you didn't call 'numpy.import_array()' after cimporting numpy; use '<void>numpy._import_array' to disable if you are certain you don't need it).


  from sed_scores_eval.base_modules.cy_medfilt import cy_medfilt
  File "sed_scores_eval/base_modules/cy_medfilt.pyx", line 1, in init sed_scores_eval.base_modules.cy_medfilt
ImportError: numpy.core.multiarray failed to import (auto-generated because you didn't call 'numpy.import_array()' after cimporting numpy; use '<void>numpy._import_array' to disable if you are certain you don't need it).

经过搜索后，给出的方案是增加 np.import_array()，但是该方法并没有结局该问题。

import numpy as np
cimport numpy as np
cimport cython
from numpy.math cimport INFINITY

# Initialize the NumPy C API
np.import_array()

然后rebuild Cython

python setup.py build_ext --inplace

该问题可能会与 numpy 版本有关系；当前在numpy 1.22.4 中就出现该情况。
笔者在另外一个环境中使用 numpy 1.24.3 没有出现该问题，但是numpy 版本的变化会影响其他库函数，故不推荐。

从而引入了下面的一种方式，即更改了setup.py 配置文件，
用来添加 extension 扩展的方式。

2.2 更改 setup 配置文件

cy_detection 导入的问题

  File "/home/anaconda3/envs/SoundDetection/lib/python3.8/site-packages/sed_scores_eval/intersection_based/intermediate_statistics.py", line 4, in <module>
    from sed_scores_eval.base_modules import statistics
  File "/home/anaconda3/envs/SoundDetection/lib/python3.8/site-packages/sed_scores_eval/base_modules/statistics.py", line 6, in <module>
    from sed_scores_eval.base_modules.cy_detection import onset_offset_curves
ModuleNotFoundError: No module named 'sed_scores_eval.base_modules.cy_detection'

对 setup.py 中配制进行修改，
原始的 setup.py 信息如下：

from setuptools import setup

setup(
    name="desed_task",
    version="0.1.1",
    description="Sound Event Detection and Separation in Domestic Environments.",
    author="DCASE Task 4 Organizers",
    author_email="romain.serizel@loria.fr",
    license="MIT",
    packages=["desed_task"],
    python_requires=">=3.8",
    install_requires=[
        "dcase_util>=0.2.16",
        "psds_eval>=0.4.0",
        "sed_eval>=0.2.1",
        "sed_scores_eval>=0.0.0",
    ],
)

修改后的配置文件如下，主要功能是，

添加对应的扩展
会在当前项目下生成 sed_scores_eval/base_modules 文件夹，执行 pip install -e 之后，
会在该文件下面生成两个 .so 共享文件，将这两个共享文件拷贝到该虚拟环境中的位置，即该路径中
~/anaconda3/envs/SoundDetection/lib/python3.8/site-packages/sed_scores_eval/base_modules

#
# from setuptools import setup, Extension
# from Cython.Build import cythonize
# import numpy as np
# import os
#
# # Ensure the output directory exists
# os.makedirs("sed_scores_eval/base_modules", exist_ok=True)
#
# extensions = [
#     Extension(
#         name="sed_scores_eval.base_modules.cy_detection",
#         sources=["/home/xxxxx/anaconda3/envs/SoundDetection/lib/python3.8/site-packages/sed_scores_eval/base_modules/cy_detection.pyx"],
#         #sources=["sed_scores_eval/base_modules/cy_detection.pyx"],
#         include_dirs=[np.get_include()]
#     ),
#
#     Extension(
#         name="sed_scores_eval.base_modules.cy_medfilt",
#         sources=["/home/xxxxx/anaconda3/envs/SoundDetection/lib/python3.8/site-packages/sed_scores_eval/base_modules/cy_medfilt.pyx"],
#         #sources=["sed_scores_eval/base_modules/cy_medfilt.pyx"],
#         include_dirs=[np.get_include()],
#     )
#
# ]
#
# setup(
#     name="desed_task",
#     version="0.1.1",
#     description="Sound Event Detection and Separation in Domestic Environments.",
#     author="DCASE Task 4 Organizers",
#     author_email="romain.serizel@loria.fr",
#     license="MIT",
#     packages=["desed_task"],
#     python_requires=">=3.8",
#     install_requires=[
#         "dcase_util>=0.2.16",
#         "psds_eval>=0.4.0",
#         "sed_eval>=0.2.1",
#         "sed_scores_eval>=0.0.0",
#     ],
#     ext_modules=cythonize(extensions),
#     zip_safe=False,
# )

3. other problem

AttributeError: 'module' object has no attribute 'ndimage'

You have to import the module:

import scipy.ndimage

pip show  numpy  cython
Name: numpy
Version: 1.22.4
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: 
License: BSD
Location: /home/respecting_god/anaconda3/envs/SoundDetection/lib/python3.8/site-packages
Requires: 
Required-by: Bottleneck, contourpy, dcase-util, desed, h5py, jams, lazy-dataset, librosa, matplotlib, mir-eval, mkl-fft, mkl-random, musdb, museval, numba, numexpr, pandas, psds-eval, pyloudnorm, pytorch-lightning, resampy, scaper, scikit-learn, scipy, sed-eval, sed_scores_eval, sox, soxr, stempeg, tensorboard, torchmetrics, torchvision
---
Name: Cython
Version: 3.0.11
Summary: The Cython compiler for writing C extensions in the Python language.
Home-page: https://cython.org/
Author: Robert Bradshaw, Stefan Behnel, Dag Seljebotn, Greg Ewing, et al.
Author-email: cython-devel@python.org
License: Apache-2.0
Location: /home/respecting_god/anaconda3/envs/SoundDetection/lib/python3.8/site-packages
Requires: 
Required-by: sed_scores_eval