8G 显存玩转书生大模型 Demo

qq_42427964

已于 2024-08-30 21:16:59 修改

阅读量529

点赞数 8

文章标签： python 人工智能 pytorch

于 2024-08-29 19:49:32 首次发布

本文链接：https://blog.csdn.net/qq_42427964/article/details/141664590

版权

尝试创建新环境遇到的问题

连接开发机
在这里插入图片描述
但是报错
用conda 的命令都会报错

尝试重新安装certifi 但是会出现SSL模块有错误
尝试用pip install 也会出现ssl模块错误

pip install transformers==4.38

在这里插入图片描述

写300字小作文

(.conda) (base) root@intern-studio-50179067:~# studio-smi 
Running studio-smi by vgpu-smi
Thu Aug 29 13:56:43 2024
+------------------------------------------------------------------------------+
| VGPU-SMI 1.7.13       Driver Version: 535.54.03     CUDA Version: 12.2       |
+-------------------------------------------+----------------------------------+
| GPU  Name                Bus-Id           |        Memory-Usage     GPU-Util |
|===========================================+==================================|
|   0  NVIDIA A100-SXM...  00000000:4D:00.0 |     0MiB /  8182MiB    0% /  10% |
+-------------------------------------------+----------------------------------+
(.conda) (base) root@intern-studio-50179067:~# conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia -y
path ['/root/.conda/bin', '/root/.conda/lib/python311.zip', '/root/.conda/lib/python3.11', '/root/.conda/lib/python3.11/lib-dynload', '/root/.conda/lib/python3.11/site-packages', '/root/.conda/lib/python3.11/site-packages//certifi/']
path ['/root/.conda/bin', '/root/.conda/lib/python311.zip', '/root/.conda/lib/python3.11', '/root/.conda/lib/python3.11/lib-dynload', '/root/.conda/lib/python3.11/site-packages', '/root/.conda/lib/python3.11/site-packages//certifi/', '/root/.conda/lib/python3.11/site-packages//certifi/']
Traceback (most recent call last):
  File "/root/.conda/lib/python3.11/site-packages/conda/exception_handler.py", line 17, in __call__
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/root/.conda/lib/python3.11/site-packages/conda/cli/main.py", line 68, in main_subshell
    parser = generate_parser(add_help=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.conda/lib/python3.11/site-packages/conda/cli/conda_argparse.py", line 114, in generate_parser
    configure_parser_create(sub_parsers)
  File "/root/.conda/lib/python3.11/site-packages/conda/cli/conda_argparse.py", line 772, in configure_parser_create
    add_parser_solver(solver_mode_options)
  File "/root/.conda/lib/python3.11/site-packages/conda/cli/conda_argparse.py", line 1968, in add_parser_solver
    choices=context.plugin_manager.get_solvers(),
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.conda/lib/python3.11/site-packages/conda/base/context.py", line 492, in plugin_manager
    from ..plugins.manager import get_plugin_manager
  File "/root/.conda/lib/python3.11/site-packages/conda/plugins/__init__.py", line 28, in <module>
    from .hookspec import hookimpl  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.conda/lib/python3.11/site-packages/conda/plugins/hookspec.py", line 16, in <module>
    from .types import (
  File "/root/.conda/lib/python3.11/site-packages/conda/plugins/types.py", line 15, in <module>
    from requests.auth import AuthBase
  File "/root/.conda/lib/python3.11/site-packages/requests/__init__.py", line 147, in <module>
    from . import packages, utils
  File "/root/.conda/lib/python3.11/site-packages/requests/utils.py", line 24, in <module>
    from . import certs
  File "/root/.conda/lib/python3.11/site-packages/requests/certs.py", line 19, in <module>
    import ssl
  File "/root/.conda/lib/python3.11/ssl.py", line 100, in <module>
    import _ssl             # if we can't import it, let the error propagate
    ^^^^^^^^^^^
ImportError: libssl.so.3: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.conda/bin/conda", line 13, in <module>
    sys.exit(main())
             ^^^^^^
  File "/root/.conda/lib/python3.11/site-packages/conda/cli/main.py", line 123, in main
    return conda_exception_handler(main, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.conda/lib/python3.11/site-packages/conda/exception_handler.py", line 388, in conda_exception_handler
    return_value = exception_handler(func, *args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.conda/lib/python3.11/site-packages/conda/exception_handler.py", line 20, in __call__
    return self.handle_exception(exc_val, exc_tb)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.conda/lib/python3.11/site-packages/conda/exception_handler.py", line 51, in handle_exception
    from .exceptions import (
  File "/root/.conda/lib/python3.11/site-packages/conda/exceptions.py", line 15, in <module>
    import requests
  File "/root/.conda/lib/python3.11/site-packages/requests/__init__.py", line 147, in <module>
    from . import packages, utils
  File "/root/.conda/lib/python3.11/site-packages/requests/utils.py", line 24, in <module>
    from . import certs
  File "/root/.conda/lib/python3.11/site-packages/requests/certs.py", line 19, in <module>
    import ssl
  File "/root/.conda/lib/python3.11/ssl.py", line 100, in <module>
    import _ssl             # if we can't import it, let the error propagate
    ^^^^^^^^^^^
ImportError: libssl.so.3: cannot open shared object file: No such file or directory

出现这样的错误尝试重新安装lss 但是还是不行各种尝试。。。折腾了两天关于certifi和ssl的问题。。。。。。。。。。。。

最后方案

把远程开发机上所有的python安装包手动删除
conda也手动删除
尝试用命令：

rm -rf /root/miniconda3

然后重新安装miniconda3

终端中下载miniconda3安装程序

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

然后安装

bash Miniconda3-latest-Linux-x86_64.sh

安装后激活conda环境

source /path/to/miniconda3/bin/activate

检查conda安装

conda --version  
conda env list

PS: 如果安装时候选择了conda基础环境手动激活
则需要每次打开终端激活基础环境

eval "$(/root/miniconda3/bin/conda shell.bash hook)"

然后为shell 安装 Conda 的 shell 函数,使得你可以更轻松地管理 Conda 环境：

conda init

然后运行

conda config --set auto_activate_base True

这样每次都自动进入base环境。。。。

创建demo环境安装相关的依赖包

# 创建环境
conda create -n demo python=3.10 -y
# 激活环境
conda activate demo
# 安装 torch
conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia -y
# 安装其他依赖
pip install transformers==4.38
pip install sentencepiece==0.1.99
pip install einops==0.8.0
pip install protobuf==5.27.2
pip install accelerate==0.33.0
pip install streamlit==1.37.0

。。。。

Cli Demo 部署 InternLM2-Chat-1.8B 模型

mkdir -p /root/demo
touch /root/demo/cli_demo.py

然后运行python /root/demo/cli_demo.py

后面因为terminal开太多运行

python /root/demo/cli_demo.py

卡了！
CPU居然99%了
所以需要清理一下关下terminal
然后再运行
可以了
在这里插入图片描述
我切换到本地的vscode软件然后SSH连接试一下

克隆 Tutorial到本地

cd /root/demo
git clone https://github.com/InternLM/Tutorial.git

然后运行执行streamlit(Streamlit is an open-source Python library that makes it easy to create and share custom web apps for machine learning and data science. By using Streamlit you can quickly build and deploy powerful data applications)

cd /root/demo
streamlit run /root/demo/Tutorial/tools/streamlit_demo.py --server.address 127.0.0.1 --server.port 6006

在这里插入图片描述

需要端口映射
在本地的 PowerShell 中输入以下命令，把6006映射到本地将端口映射到本地

ssh -CNg -L 6006:127.0.0.1:6006 root@ssh.intern-ai.org.cn -p 你的 ssh 端口号

然后就成功了
在这里插入图片描述
vscode里面也可以看到端口

LMDeploy 部署 InternLM-XComposer2-VL-1.8B 模型

使用 LMDeploy 部署 InternLM-XComposer2-VL-1.8B 模型

conda activate demo
pip install lmdeploy[all]==0.5.1
pip install timm==1.0.7

在这里插入图片描述

使用 LMDeploy 启动一个与 InternLM-XComposer2-VL-1.8B 模型交互的 Gradio 服务

lmdeploy serve gradio /share/new_models/Shanghai_AI_Laboratory/internlm-xcomposer2-vl-1_8b --cache-max-entry-count 0.1

在这里插入图片描述

LMDeploy 部署 InternVL2-2B 模型

启动 InternVL2-2B 模型的 Gradio 服务

conda activate demo
lmdeploy serve gradio /share/new_models/OpenGVLab/InternVL2-2B --cache-max-entry-count 0.1