如题,拉取的镜像为vllm/vllm-openai:v0.6.0,运行命令如下:
docker run --runtime nvidia --gpus all \ -v ~/.cache/huggingface:/root/.cache/huggingface \ -p 8000:8000 \ --env "HUGGING_FACE_HUB_TOKEN=<secret>" \ vllm/vllm-openai <args...>
但是报错OSError: Incorrect path_or_model_id: '/models/model'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
同样的命令使用vllm的python环境运行没问题:python3 -m vllm.entrypoints.openai.api_server <args...>