【阿里魔搭】modelscope包下载安装

去月亮探险

已于 2024-03-21 16:07:28 修改

阅读量1.7k

点赞数 28

分类专栏：人工智能 Modelscope 文章标签： pycharm python pytorch github

于 2024-03-21 15:49:35 首次发布

本文链接：https://blog.csdn.net/m0_51662391/article/details/136910849

版权

人工智能同时被 2 个专栏收录

6 篇文章 1 订阅

订阅专栏

Modelscope

2 篇文章 0 订阅

订阅专栏

文章讲述了用户在尝试通过pip安装ModelScope时遇到包冲突的问题，介绍了从创建Python3.8虚拟环境、安装深度学习框架（如PyTorch或TensorFlow）、到最终采用源码安装ModelScopeNLP库的过程，以及如何处理依赖包冲突和网络问题。

摘要由CSDN通过智能技术生成

最终解决方案：使用源码安装modelscope

问题描述：pip安装包冲突

一开始的是在3.11的虚拟环境下使用命令行`pip install "modelscope[nlp]" -f [https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html](https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html)`下载，一直报错显示包冲突

并且是这种报错，根本没办法调和。

```jsx
modelscope[nlp] 1.9.0 depends on datasets<=2.13.0 and >=2.8.0
modelscope[nlp] 1.8.4 depends on datasets<=2.13.0 and >=2.8.0
modelscope[nlp] 1.8.1 depends on datasets<=2.13.0 and >=2.8.0
modelscope[nlp] 1.6.1 depends on datasets<=2.8.0 and >=2.7.0
modelscope[nlp] 1.6.0 depends on datasets<=2.8.0 and >=2.7.0
modelscope[nlp] 1.5.1 depends on datasets<=2.8.0 and >=2.7.0
modelscope[nlp] 1.4.3 depends on datasets<=2.8.0 and >=2.7.0
modelscope[nlp] 1.4.1 depends on datasets<=2.8.0 and >=2.7.0
modelscope[nlp] 1.3.1 depends on datasets<=2.8.0 and >=2.7.0
modelscope[nlp] 1.1.4 depends on datasets<=2.5.2
modelscope[nlp] 1.1.3 depends on datasets<=2.5.2
modelscope[nlp] 1.1.2 depends on datasets<=2.5.2
modelscope[nlp] 1.1.1 depends on datasets<=2.5.2
modelscope[nlp] 1.1.0 depends on datasets<=2.5.2
modelscope[nlp] 1.0.4 depends on datasets<=2.5.2
modelscope[nlp] 1.0.3 depends on datasets<=2.5.2
modelscope[nlp] 1.0.2 depends on datasets<=2.5.2
modelscope[nlp] 1.0.1 depends on datasets<=2.5.2
modelscope[nlp] 0.3.7 depends on datasets==2.1.0
modelscope[nlp] 0.3.6 depends on datasets==2.1.0
modelscope[nlp] 0.3.5 depends on datasets==2.1.0
modelscope[nlp] 0.3.4 depends on datasets==2.1.0
```

后来在官网找到环境安装的说明：[https://www.modelscope.cn/docs/环境安装](https://www.modelscope.cn/docs/%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85)

环境配置Python版本只显示了3.7和3.8，于是重新创建虚拟环境进行安装。

首先安装深度学习框架，看你的模型是需要pytorch还是tensorflow，根据官网下载对应版本

后面继续`pip install "modelscope[nlp]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html` 进行安装但是出现相同问题，于是尝试使用**通过源码安装**

完整安装过程：

以下是安装全过程：

1. 创建虚拟环境并激活

```jsx
conda create -n modelscope python=3.8
conda activate modelscope
```

2. 安装深度学习框架

Pytorch：

```jsx
pip install torch torchvision torchaudio
```

或Tensorflow：

```jsx
pip install --upgrade tensorflow==2.13.0 # 仅支持 CPU 的版本
```

注：客观上在国内使用pip安装的时候，如果默认是用海外的pypi源的话，可能因为网络问题，下载速度受限。如果有这个现象，可以考虑通过pip的"-i"命令行选项来手工配置仓库来源，例如"-i [https://pypi.tuna.tsinghua.edu.cn/simple](https://pypi.tuna.tsinghua.edu.cn/simple) " 可以将配置仓库来源使用"清华源"。例如：

```
pip3 install torch torchvision torchaudio -i https://pypi.tuna.tsinghua.edu.cn/simple
```

常见的可用源还包括 "-i [https://mirrors.bfsu.edu.cn/pypi/web/simple](https://mirrors.bfsu.edu.cn/pypi/web/simple)", "-i [https://mirrors.ustc.edu.cn/pypi/web/simple](https://mirrors.ustc.edu.cn/pypi/web/simple) "等等，可以根据自己的网络条件自行选择。

3. **ModelScope Library 安装（源码安装）**

我使用pip安装失败遂不介绍，仅介绍源码安装

1. 源码下载
a. 可以使用git下载源码

```jsx
git clone git@github.com:modelscope/modelscope.git
cd modelscope
git fetch origin master
git checkout master
```

b. 直接在github官网下载 https://github.com/modelscope/modelscope
2. 一定要先cd去源码所在的目录中先，否则会报错

可以使用命令行检查是否转到对应目录下：

```jsx
dir
type setup.py
```

3. 安装依赖：仅需体验`NLP`领域模型，执行如下命令安装依赖

```jsx
pip install ".[nlp]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
```

期间他会提示缺少某些包而报错，根据提示单独安装对应的包即可



如果有安装不成功的在后面加**`--trusted-host [pypi.mirrors.ustc.edu.cn](http://pypi.mirrors.ustc.edu.cn)`** 以避免潜在的安全警告

```jsx
pip install xxx --trusted-host pypi.mirrors.ustc.edu.cn
pip install ".[nlp]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html --trusted-host pypi.mirrors.ustc.edu.cn
```

4. 验证：安装成功后，即可使用对应领域模型进行推理，训练等操作。这里我们以NLP领域为例。安装后，可执行如下命令，运行中文分词任务，来验证安装是否正确：

```jsx
python -c "from modelscope.pipelines import pipeline;print(pipeline('word-segmentation')('今天天气不错，适合出去游玩'))"
```

```jsx
(modelscope) E:\0.University\0.dissertation\code\modelscope\modelscope>python -c "from modelscope.pipelines import pipeline;print(pipeline('word-segmentation')('今
天天气不错，适合出去游玩'))"
2024-03-21 10:23:41,578 - modelscope - INFO - PyTorch version 1.11.0+cpu Found.
2024-03-21 10:23:41,580 - modelscope - INFO - Loading ast index from C:\Users\ammy\.cache\modelscope\ast_indexer
2024-03-21 10:23:42,128 - modelscope - INFO - Loading done! Current index file version is 1.9.4, with md5 d07f5c3e9ea59df520487198f306c0b6 and a total number of 972
components indexed
2024-03-21 10:23:46,043 - modelscope - INFO - Model revision not specified, use default: master in development mode
2024-03-21 10:23:46,044 - modelscope - INFO - Development mode use revision: master
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 243k/243k [00:00<00:00, 987kB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 976/976 [00:00<00:00, 194kB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 161/161 [00:00<00:00, 40.4kB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 46.7k/46.7k [00:00<00:00, 413kB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 388M/388M [02:29<00:00, 2.71MB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9.74k/9.74k [00:00<00:00, 1.99MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30.0/30.0 [00:00<00:00, 7.54kB/s]
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 107k/107k [00:00<00:00, 659kB/s]
2024-03-21 10:26:20,450 - modelscope - INFO - initiate model from C:\Users\ammy\.cache\modelscope\hub\damo\nlp_structbert_word-segmentation_chinese-base
2024-03-21 10:26:20,450 - modelscope - INFO - initiate model from location C:\Users\ammy\.cache\modelscope\hub\damo\nlp_structbert_word-segmentation_chinese-base.
2024-03-21 10:26:20,454 - modelscope - INFO - initialize model from C:\Users\ammy\.cache\modelscope\hub\damo\nlp_structbert_word-segmentation_chinese-base
You are using a model of type bert to instantiate a model of type structbert. This is not supported for all configurations of models and can yield errors.
2024-03-21 10:26:22,373 - modelscope - WARNING - No preprocessor field found in cfg.
2024-03-21 10:26:22,373 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-03-21 10:26:22,373 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': 'C:\\Users\\amm
y\\.cache\\modelscope\\hub\\damo\\nlp_structbert_word-segmentation_chinese-base'}. trying to build by task and model information.
2024-03-21 10:26:22,392 - modelscope - INFO - cuda is not available, using cpu instead.
2024-03-21 10:26:22,397 - modelscope - WARNING - No preprocessor field found in cfg.
2024-03-21 10:26:22,397 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-03-21 10:26:22,397 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': 'C:\\Users\\amm
y\\.cache\\modelscope\\hub\\damo\\nlp_structbert_word-segmentation_chinese-base', 'sequence_length': 512}. trying to build by task and model information.
E:\anaconda\envs\modelscope\lib\site-packages\transformers\modeling_utils.py:977: FutureWarning: The `device` argument is deprecated and will be removed in v5 of Tr
ansformers.
warnings.warn(
{'output': ['今天', '天气', '不错', '，', '适合', '出去', '游玩']}

```