详说vllm的安装

强化学习曾小健

已于 2024-07-27 12:35:09 修改

阅读量4.6k

点赞数 2

文章标签： python 开发语言

于 2024-07-25 21:16:09 首次发布

本文链接：https://blog.csdn.net/sinat_37574187/article/details/140700250

版权

如果是cu118，python版本3.8 - 3.10

vllm 0.4.0-cu118

torch==2.1.2

pip install torch==2.1.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118

# Install vLLM with CUDA 11.8.
export VLLM_VERSION=0.4.0
export PYTHON_VERSION=310
pip install https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}+cu118-cp${PYTHON_VERSION}-cp${PYTHON_VERSION}-manylinux1_x86_64.whl --extra-index-url https://download.pytorch.org/whl/cu118

如果是cu12+，python版本3.10 - 3.11

vllm 0.5.1-cu12

export VLLM_VERSION=0.5.2 # vLLM's main branch version is currently set to latest released tag
pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-${VLLM_VERSION}-cp38-abi3-manylinux1_x86_64.whl
# You can also access a specific commit
# export VLLM_COMMIT=...
# pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/${VLLM_COMMIT}/vllm-${VLLM_VERSION}-cp38-abi3-manylinux1_x86_64.whl