在公司的云服务器上部署chatglm3-6b，踩了太多坑，记录一下。
主要参考的是 【大模型-第一篇】在阿里云上部署ChatGLM3_使用阿里云服务器部署大模型-CSDN博客 https://blog.csdn.net/weixin_42029733/article/details/134324291
部署过程中需要特别关注的几点：
- 配置新的pip虚拟环境（很重要，否则和以前环境冲突了很麻烦）
- 部署前查看并记录磁盘空间情况（防止下载到未知位置难以清理）
- requirements.txt中包需要拆分安装，特别是pytorch包，需要官网复制命令安装
- 推荐记录所有的命令和输出内容
日志记录
个人觉得日志记录是很重要的，有时一些命令需要重复执行对比效果，或者一些命令导致副作用将来追查，或者作为流程的完整记录供以后参考，日志的命令和输出内容对后续的操作有很大的帮助。
ssh工具个人比较推荐Electerm，可以设置保存日志，是实时保存的，防止软件闪退时丢失日志。
Windterm颜色很漂亮，但是日志每次都需要手动记录，总感觉不安稳，其他工具还没测试。
查看空间使用情况
df -h查看各磁盘使用情况。输出格式如下：
Filesystem Size Used Avail Use% Mounted on
udev 30G 0 30G 0% /dev
tmpfs 5.9G 808K 5.9G 1% /run
/dev/vda3 40G 39G 0 100% /
tmpfs 30G 0 30G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 30G 0 30G 0% /sys/fs/cgroup
/dev/vda2 189M 6.1M 183M 4% /boot/efi
tmpfs 5.9G 0 5.9G 0% /run/user/0
注意有些像udev是虚拟磁盘，不能算作可用的磁盘空间。
ncdu 方便跳转各子目录查看大小情况，但是查看的记录不会输出。
du -h --max-depth=1 | sort -hr 只看当前目录，能作为输出记录下载。
顺便说一下当用git lfs下载chatglm3-6b模型时，输出显示有100%字样后停止反馈，看似程序停下了，其实还是在下载，用上面的方法可以对这个过程有所知晓。
tree以树形格式输出目录结构，可能输出较多，可以用tree -L n这样来指定只输出到第n层，如tree -L 2。
配置新的pip虚拟环境
进入一个打算放置各种包的目录，可以执行
python3 -m venv myenv #创建虚拟环境myenv
source myenv/bin/activate #激活虚拟环境myenv，这时命令行提示符前面会显示(myenv)
deactivate #退出myenv
rm -rf /path/to/your/venv/myenv #删除myenv
命令不是很复杂，但是这个思路很重要。卸载原环境重装是很笨的方法，当时没这个思路时我卸载python3.8一堆报错完了，python3.12的pip还怎么都配不好。
安装requirements.txt依赖包
直接安装依赖包 pip install -r requirements.txt，不出意外会报错。
于是分开安装，主要是torch要单独安装。
cpm_kernels
cpm_kernels>=1.0.11安装后import 报错：
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/new/workspace/glm_env/lib/python3.12/site-packages/cpm_kernels/__init__.py", line 2, in <module>
from .kernels import *
File "/new/workspace/glm_env/lib/python3.12/site-packages/cpm_kernels/kernels/__init__.py", line 1, in <module>
from .embedding import embedding_forward, embedding_backward_stage1, embedding_backward_stage2, embedding_step
File "/new/workspace/glm_env/lib/python3.12/site-packages/cpm_kernels/kernels/embedding.py", line 1, in <module>
from .base import Kernel, DevicePointer, CUDAStream, round_up
File "/new/workspace/glm_env/lib/python3.12/site-packages/cpm_kernels/kernels/base.py", line 6, in <module>
import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'
网上一些方法试了没用，好像是问了gpt，解决方法是pip install setuptools。
torch安装
torch一般从官网上选择命令复制。 https://pytorch.org/get-started/locally/
我的cuda版本是12.0，pytorch官网只有11.8和12.1版本。
安装12.1，过程没有报错。
import torch，不出意外又报错了
File "/new/workspace/glm_env/lib/python3.12/site-packages/torch/__init__.py", line 290, in <module>
from torch._C import * # noqa: F403
^^^^^^^^^^^^^^^^^^^^^^
ImportError: /new/workspace/glm_env/lib/python3.12/site-packages/torch/lib/../../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkAddData_12_1, version libnvJitLink.so.12
经过一番尝试都不行，又安装了11.8版本的pytorch，没想到居然可以运行了。
修改requirements.txt，注释掉前三个，pip install -r requirements.txt，没想到之后我没有遇到安装的报错。
chatglm3-6b运行
修改配置路径
进入目录/new/workspace/models/ChatGLM3/，修改里面文件中的配置路径。
cp basic_demo/web_demo_gradio.py basic_demo/web_demo_gradio.py.bak
cp chatglm3-6b/config.json chatglm3-6b/config.json.bak
sed -i 's#THUDM/#/new/workspace/models/ChatGLM3/#g' basic_demo/web_demo_gradio.py
sed -i 's#THUDM/#/new/workspace/models/ChatGLM3/#g' chatglm3-6b/config.json
SyntaxWarning、缺少peft
然后python basic_demo/web_demo_gradio.py
出现报错
/new/workspace/ChatGLM3/basic_demo/web_demo_gradio.py:96: SyntaxWarning: invalid escape sequence '\`'
line = line.replace("`", "\`")
Traceback (most recent call last):
File "/new/workspace/ChatGLM3/basic_demo/web_demo_gradio.py", line 26, in <module>
from peft import AutoPeftModelForCausalLM, PeftModelForCausalLM
ModuleNotFoundError: No module named 'peft'
这样修改
line = line.replace("`", "\\`")
可以消除警告SyntaxWarning。
ModuleNotFoundError直接用pip install peft解决。
gradio链接
再次运行，gradio网页可以构建，但是没有对外网提供链接。
To create a public link, set share=True in launch().
grep -o 'share' basic_demo/web_demo_gradio.py | wc -l #确认share只出现一次
tail -n 10 basic_demo/web_demo_gradio.py #显示末尾几行含有share的位置
sed -i 's/share=False/share=True/g' basic_demo/web_demo_gradio.py
再次运行，发现还要下载frpc_linux_amd64。
Could not create share link. Missing file: /new/workspace/glm_env/lib/python3.12/site-packages/gradio/frpc_linux_amd64_v0.2.
Please check your internet connection. This can happen if your antivirus software blocks the download of this file. You can install manually by following these steps:
1. Download this file: https://cdn-media.huggingface.co/frpc-gradio-0.2/frpc_linux_amd64
2. Rename the downloaded file to: frpc_linux_amd64_v0.2
3. Move the file to this location: /new/workspace/glm_env/lib/python3.12/site-packages/gradio
于是暂时没用它，改用web_demo_streamlit.py
cp basic_demo/web_demo_gradio.py basic_demo/web_demo_streamlit.py.bak
sed -i 's#THUDM/#/new/workspace/ChatGLM3/#g' basic_demo/web_demo_streamlit.py
python basic_demo/web_demo_streamlit.py
This script is a simple web demo based on Streamlit, showcasing the use of the ChatGLM3-6B model. For a more comprehensive web demo,
it is recommended to use 'composite_demo'.
Usage:
- Run the script using Streamlit: `streamlit run web_demo_streamlit.py`
- Adjust the model parameters from the sidebar.
- Enter questions in the chat input box and interact with the ChatGLM3-6B model.
Note: Ensure 'streamlit' and 'transformers' libraries are installed and the required model checkpoints are available.
streamlit run web_demo_streamlit.py
他给的链接似乎打不开。
又回到web_demo_gradio.py
下载了frpc_linux_amd64并重命名移动了。
再次运行，提示
Could not create share link. Please check your internet connection or our status page: https://status.gradio.app
按csdn上说法
cd进入frpc_linux_amd64文件的位置（应该在.conda/envs/ChatGLM2/lib/python3.11/site-packages/gradio），输入以下命令给予权限：
chmod +x frpc_linux_amd64_v0.2
这样可以在外部打开网页了。
这期间几次尝试，ctrl+C关闭太慢，用了ctrl+Z。再次运行时出现了
OSError: Cannot find empty port in range: 7870-7870. You can specify a different port by setting the GRADIO_SERVER_PORT environment variable or passing the server_port parameter to launch().
可以通过清除相应进程的pid解决。
transformers版本问题
这个问题是我最迷惑的问题，解决方法我不确定，阴差阳错跑通了，把过程记录在这里。
注意requirements.txt中要求的transformers版本是4.40.0。我确实这样安装的，之前并没有报错。
输入你好，半天没结果，查看控制台，果然又报错了。
Exception in thread Thread-8 (generate):
Traceback (most recent call last):
File "/usr/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
self.run()
File "/usr/lib/python3.12/threading.py", line 1012, in run
self._target(*self._args, **self._kwargs)
File "/new/workspace/glm_env/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/new/workspace/glm_env/lib/python3.12/site-packages/transformers/generation/utils.py", line 2024, in generate
result = self._sample(
^^^^^^^^^^^^^
File "/new/workspace/glm_env/lib/python3.12/site-packages/transformers/generation/utils.py", line 3032, in _sample
model_kwargs = self._update_model_kwargs_for_generation(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/huggingface/modules/transformers_modules/chatglm3-6b/modeling_chatglm.py", line 875, in _update_model_kwargs_for_generation
model_kwargs["past_key_values"] = self._extract_past_from_model_output(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: GenerationMixin._extract_past_from_model_output() got an unexpected keyword argument 'standardize_cache_format'
参考两个链接
TypeError: GenerationMixin._extract_past_from_model_output() got an unexpected keyword argument ‘standardize_cache_format’ · Issue #439 · THUDM/GLM-4 https://github.com/THUDM/GLM-4/issues/439
有说pip transformers==4.43.0解决，有说pip install transformers==4.40.2解决
TypeError: GenerationMixin._extract_past_from_model_output() got an unexpected keyword argument 'standardize_cache_format with transformers==4.44.0 · Issue #181 · THUDM/CogVLM2 https://github.com/THUDM/CogVLM2/issues/181
因为transformers 4.44.0 的代码有所变动。
我这里分别尝试了安装transformers 的4.43 4.44 4.42 4.40。
下面是安装后的部分输出
4.43.0：
Installing collected packages: transformers
Attempting uninstall: transformers
Found existing installation: transformers 4.44.1
Uninstalling transformers-4.44.1:
Successfully uninstalled transformers-4.44.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
vllm 0.5.4 requires transformers>=4.43.2, but you have transformers 4.43.0 which is incompatible.
Successfully installed transformers-4.43.0
4.44.0：
Installing collected packages: transformers
Attempting uninstall: transformers
Found existing installation: transformers 4.40.0
Uninstalling transformers-4.40.0:
Successfully uninstalled transformers-4.40.0
Successfully installed transformers-4.44.0
4.42.4：
Installing collected packages: transformers
Attempting uninstall: transformers
Found existing installation: transformers 4.44.0
Uninstalling transformers-4.44.0:
Successfully uninstalled transformers-4.44.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
vllm 0.5.4 requires transformers>=4.43.2, but you have transformers 4.42.4 which is incompatible.
Successfully installed transformers-4.42.4
4.40.0：
Installing collected packages: transformers
Attempting uninstall: transformers
Found existing installation: transformers 4.42.4
Uninstalling transformers-4.42.4:
Successfully uninstalled transformers-4.42.4
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
vllm 0.5.4 requires transformers>=4.43.2, but you have transformers 4.40.0 which is incompatible.
Successfully installed transformers-4.40.0
可见这里vllm最新版0.5.4与要求的transformers4.40.0版本是冲突的。所以我考虑降级vllm。
尝试降级到pip install vllm==0.4.2，正是requirements.txt中要求>=的。
(glm_env) oot@xxx: /new/workspace/ChatGLM3root@xxx:/new/workspace/ChatGLM3# pip install vllm>==
Looking in indexes: http://mirrors.cloud.aliyuncs.com/pypi/simple/
Collecting vllm==0.4.2
Downloading http://mirrors.cloud.aliyuncs.com/pypi/packages/52/86/c493b975f36d48939cc575d22813000d2c60d519283d1c61ff744fb440e8/vllm-0.4.2.tar.gz (588 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/588.8 kB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 588.8/588.8 kB 19.4 MB/s eta 0:00:00
Installing build dependencies ... / done
Getting requirements to build wheel ... - error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [21 lines of output]
Traceback (most recent call last):
File "/new/workspace/glm_env/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/new/workspace/glm_env/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/new/workspace/glm_env/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-5c973dce/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-5c973dce/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 302, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-5c973dce/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 318, in run_setup
exec(code, locals())
File "<string>", line 11, in <module>
File "/tmp/pip-build-env-5c973dce/overlay/lib/python3.12/site-packages/torch/__init__.py", line 237, in <module>
from torch._C import * # noqa: F403
^^^^^^^^^^^^^^^^^^^^^^
ImportError: /tmp/pip-build-env-5c973dce/overlay/lib/python3.12/site-packages/torch/lib/../../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkAddData_12_1, version libnvJitLink.so.12
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
看起来vllm没有降级成功，感觉各种冲突太多了，突然感觉requirements.txt里面的>=的写法也不是很靠谱了。如果能提供一个恰好等于版本号的各依赖包版本就好了。这个我把自己部署成功后得到的各版本号放在末尾。
vllm升级回去：
pip install vllm==0.5.4
Requirement already satisfied: six>=1.5 in /new/workspace/glm_env/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas->datasets->outlines<0.1,>=0.0.43->vllm==0.5.4) (1.16.0)
Installing collected packages: transformers
Attempting uninstall: transformers
Found existing installation: transformers 4.40.0
Uninstalling transformers-4.40.0:
Successfully uninstalled transformers-4.40.0
Successfully installed transformers-4.44.1
它居然自动安装了transformers-4.44.1。
尝试从更简单的程序cli_demo.py运行：
cp basic_demo/cli_demo.py basic_demo/cli_demo.py.bak
sed -i 's#THUDM/#/new/workspace/models/ChatGLM3/#g' basic_demo/cli_demo.py
python cli_demo.py
ChatGLM：Traceback (most recent call last):
File "/new/workspace/ChatGLM3/basic_demo/cli_demo.py", line 57, in <module>
main()
File "/new/workspace/ChatGLM3/basic_demo/cli_demo.py", line 43, in main
for response, history, past_key_values in model.stream_chat(tokenizer, query, history=history, top_p=1,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/new/workspace/glm_env/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 36, in generator_context
response = gen.send(None)
^^^^^^^^^^^^^^
File "/root/.cache/huggingface/modules/transformers_modules/chatglm3-6b/modeling_chatglm.py", line 1076, in stream_chat
for outputs in self.stream_generate(**inputs, past_key_values=past_key_values,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/new/workspace/glm_env/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 36, in generator_context
response = gen.send(None)
^^^^^^^^^^^^^^
File "/root/.cache/huggingface/modules/transformers_modules/chatglm3-6b/modeling_chatglm.py", line 1145, in stream_generate
logits_processor = self._get_logits_processor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/new/workspace/glm_env/lib/python3.12/site-packages/transformers/generation/utils.py", line 866, in _get_logits_processor
and generation_config._eos_token_tensor is not None
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'GenerationConfig' object has no attribute '_eos_token_tensor'
还是报错，我又随便试试，安装了transformers4.40.0。
pip install transformers==4.40.0
Looking in indexes: http://mirrors.cloud.aliyuncs.com/pypi/simple/
Collecting transformers==4.40.0
Using cached http://mirrors.cloud.aliyuncs.com/pypi/packages/09/c8/844d5518a6aeb4ffdc0cf0cae65ae13dbe5838306728c5c640b5a6e2a0c9/transformers-4.40.0-py3-none-any.whl (9.0 MB)
Requirement already satisfied: filelock in /new/workspace/glm_env/lib/python3.12/site-packages (from transformers==4.40.0) (3.15.4)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /new/workspace/glm_env/lib/python3.12/site-packages (from transformers==4.40.0) (0.24.6)
Requirement already satisfied: numpy>=1.17 in /new/workspace/glm_env/lib/python3.12/site-packages (from transformers==4.40.0) (1.26.4)
Requirement already satisfied: packaging>=20.0 in /new/workspace/glm_env/lib/python3.12/site-packages (from transformers==4.40.0) (24.1)
Requirement already satisfied: pyyaml>=5.1 in /new/workspace/glm_env/lib/python3.12/site-packages (from transformers==4.40.0) (6.0.2)
Requirement already satisfied: regex!=2019.12.17 in /new/workspace/glm_env/lib/python3.12/site-packages (from transformers==4.40.0) (2024.7.24)
Requirement already satisfied: requests in /new/workspace/glm_env/lib/python3.12/site-packages (from transformers==4.40.0) (2.32.3)
Requirement already satisfied: tokenizers<0.20,>=0.19 in /new/workspace/glm_env/lib/python3.12/site-packages (from transformers==4.40.0) (0.19.1)
Requirement already satisfied: safetensors>=0.4.1 in /new/workspace/glm_env/lib/python3.12/site-packages (from transformers==4.40.0) (0.4.4)
Requirement already satisfied: tqdm>=4.27 in /new/workspace/glm_env/lib/python3.12/site-packages (from transformers==4.40.0) (4.66.5)
Requirement already satisfied: fsspec>=2023.5.0 in /new/workspace/glm_env/lib/python3.12/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers==4.40.0) (2024.6.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /new/workspace/glm_env/lib/python3.12/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers==4.40.0) (4.12.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /new/workspace/glm_env/lib/python3.12/site-packages (from requests->transformers==4.40.0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /new/workspace/glm_env/lib/python3.12/site-packages (from requests->transformers==4.40.0) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /new/workspace/glm_env/lib/python3.12/site-packages (from requests->transformers==4.40.0) (2.2.2)
Requirement already satisfied: certifi>=2017.4.17 in /new/workspace/glm_env/lib/python3.12/site-packages (from requests->transformers==4.40.0) (2024.7.4)
Installing collected packages: transformers
Attempting uninstall: transformers
Found existing installation: transformers 4.44.1
Uninstalling transformers-4.44.1:
Successfully uninstalled transformers-4.44.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
vllm 0.5.4 requires transformers>=4.43.2, but you have transformers 4.40.0 which is incompatible.
Successfully installed transformers-4.40.0
再次尝试运行python cli_demo.py，没想到竟然成功了。
又运行python web_demo_gradio.py，居然也成功了。
看来，transformers包绕了一圈，还是用最初的版本可行。这之间的差别，除了它绕了一圈，还有vllm从0.5.4尝试降级没成功，又重新安装上0.5.4了，以及在运行web_demo_gradio.py前运行了cli_demo.py（这能有影响？？）。
至此，我在不明所以的情况，莫名其妙地成功部署了chatglm3-6b。
附本人部署后的 pip list
可能有杂项干扰，仅作参考。
Package Version
--------------------------------- ---------------
accelerate 0.33.0
aiofiles 23.2.1
aiohappyeyeballs 2.4.0
aiohttp 3.10.5
aiosignal 1.3.1
altair 5.4.0
annotated-types 0.7.0
anyio 4.4.0
arxiv 2.1.3
attrs 24.2.0
blinker 1.8.2
cachetools 5.5.0
certifi 2024.7.4
charset-normalizer 3.3.2
click 8.1.7
cloudpickle 3.0.0
cmake 3.30.2
contourpy 1.2.1
cpm-kernels 1.0.11
cycler 0.12.1
dataclasses-json 0.6.7
datasets 2.21.0
dill 0.3.8
diskcache 5.6.3
distro 1.9.0
fastapi 0.112.1
feedparser 6.0.11
ffmpy 0.4.0
filelock 3.15.4
fonttools 4.53.1
frozenlist 1.4.1
fsspec 2024.6.1
gitdb 4.0.11
GitPython 3.1.43
gradio 4.41.0
gradio_client 1.3.0
greenlet 3.0.3
h11 0.14.0
httpcore 1.0.5
httptools 0.6.1
httpx 0.27.0
huggingface-hub 0.24.6
idna 3.7
importlib_resources 6.4.3
interegular 0.3.3
Jinja2 3.1.3
jiter 0.5.0
joblib 1.4.2
jsonpatch 1.33
jsonpointer 3.0.0
jsonschema 4.23.0
jsonschema-specifications 2023.12.1
jupyter_client 8.6.2
jupyter_core 5.7.2
kiwisolver 1.4.5
langchain 0.2.14
langchain-community 0.2.12
langchain-core 0.2.33
langchain-text-splitters 0.2.2
langchainhub 0.1.21
langsmith 0.1.100
lark 1.2.2
latex2mathml 3.77.0
llvmlite 0.43.0
lm-format-enforcer 0.10.3
loguru 0.7.2
Markdown 3.7
markdown-it-py 3.0.0
MarkupSafe 2.1.5
marshmallow 3.22.0
matplotlib 3.9.2
mdtex2html 1.3.0
mdurl 0.1.2
mpmath 1.3.0
msgpack 1.0.8
multidict 6.0.5
multiprocess 0.70.16
mypy-extensions 1.0.0
narwhals 1.5.0
nest-asyncio 1.6.0
networkx 3.2.1
ninja 1.11.1.1
numba 0.60.0
numpy 1.26.4
nvidia-cublas-cu11 11.11.3.6
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu11 11.8.87
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu11 11.8.89
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu11 11.8.89
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu11 9.1.0.70
nvidia-cudnn-cu12 9.1.0.70
nvidia-cufft-cu11 10.9.0.58
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu11 10.3.0.86
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu11 11.4.1.48
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu11 11.7.5.86
nvidia-cusparse-cu12 12.1.0.106
nvidia-ml-py 12.560.30
nvidia-nccl-cu11 2.20.5
nvidia-nccl-cu12 2.20.5
nvidia-nvjitlink-cu12 12.1.105
nvidia-nvtx-cu11 11.8.86
nvidia-nvtx-cu12 12.1.105
openai 1.42.0
orjson 3.10.7
outlines 0.0.46
packaging 24.1
pandas 2.2.2
peft 0.12.0
pillow 10.2.0
pip 24.2
platformdirs 4.2.2
prometheus_client 0.20.0
prometheus-fastapi-instrumentator 7.0.0
protobuf 5.27.3
psutil 6.0.0
py-cpuinfo 9.0.0
pyairports 2.1.1
pyarrow 17.0.0
pycountry 24.6.1
pydantic 2.8.2
pydantic_core 2.20.1
pydeck 0.9.1
pydub 0.25.1
Pygments 2.18.0
pyparsing 3.1.2
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
python-multipart 0.0.9
pytz 2024.1
PyYAML 6.0.2
pyzmq 26.1.1
ray 2.34.0
referencing 0.35.1
regex 2024.7.24
requests 2.32.3
rich 13.7.1
rpds-py 0.20.0
ruff 0.6.1
safetensors 0.4.4
scikit-learn 1.5.1
scipy 1.14.1
semantic-version 2.10.0
sentence-transformers 3.0.1
sentencepiece 0.2.0
setuptools 73.0.1
sgmllib3k 1.0.0
shellingham 1.5.4
six 1.16.0
smmap 5.0.1
sniffio 1.3.1
SQLAlchemy 2.0.32
sse-starlette 2.1.3
starlette 0.38.2
streamlit 1.37.1
sympy 1.12
tenacity 8.5.0
threadpoolctl 3.5.0
tiktoken 0.7.0
timm 1.0.8
tokenizers 0.19.1
toml 0.10.2
tomlkit 0.12.0
torch 2.4.0+cu118
torchaudio 2.4.0+cu118
torchvision 0.19.0+cu118
tornado 6.4.1
tqdm 4.66.5
traitlets 5.14.3
transformers 4.40.0
triton 3.0.0
typer 0.12.4
types-requests 2.32.0.20240712
typing_extensions 4.12.2
typing-inspect 0.9.0
tzdata 2024.1
urllib3 2.2.2
uvicorn 0.30.6
uvloop 0.20.0
vllm 0.5.4
vllm-flash-attn 2.6.1
watchdog 4.0.2
watchfiles 0.23.0
websockets 12.0
xformers 0.0.27.post2
xxhash 3.5.0
yarl 1.9.4
2453
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
