NVIDIA Dynamo源码编译

Luchang-Li

已于 2025-04-22 11:45:32 修改

阅读量933

点赞数 10

分类专栏：大模型推理引擎文章标签：大模型推理引擎 dynamo

于 2025-03-20 16:20:01 首次发布

本文链接：https://blog.csdn.net/u013701860/article/details/146400219

版权

推理引擎同时被 2 个专栏收录

30 篇文章

订阅专栏

大模型

19 篇文章

订阅专栏

Ref

https://github.com/PyO3/maturin

Rust 程序设计语言

代码库：

https://github.com/ai-dynamo/dynamo

https://github.com/ai-dynamo/nixl

dynamo/container/Dockerfile.vllm

构建ai_dynamo

这里面包含的是sdk部分内容。

Install uv: uv

# curl -LsSf https://astral.sh/uv/install.sh | sh
# or 
pip install uv

在顶层目录：

uv build --wheel --out-dir /workspace/dist

构建ai_dynamo_runtime

这里面包含的是llm和runtime和它们的python binding部分内容。

参考https://github.com/ai-dynamo/dynamo/tree/main/lib/bindings/python

下载rust环境：

国内服务器最好设置环境变量，否则rustup-init.sh运行时的下载极其慢：

export RUSTUP_DIST_SERVER='https://mirrors.ustc.edu.cn/rust-static'
export RUSTUP_UPDATE_ROOT='https://mirrors.ustc.edu.cn/rust-static/rustup'

# curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
wget https://static.rust-lang.org/rustup/rustup-init.sh
./rustup-init.sh

安装依赖：

这里假设你已经有一个Python conda环境，否则可以安装miniconda设置环境，或者按照推荐使用uv环境。

pip install maturin
apt install protobuf-compiler

下载源码：

git clone --recursive https://github.com/ai-dynamo/dynamo.git
git clone --recursive https://github.com/ai-dynamo/nixl.git

在目录dynamo/lib/bindings/python运行

maturin build

即可在target/wheels/生成whl包。

或者：

uv build --wheel --out-dir /workspace/dist

构建nixl

https://github.com/ai-dynamo/nixl

https://github.com/ai-dynamo/nixl/issues/74#issuecomment-2743691272

依赖安装

apt install build-essential cmake pkg-config
pip install meson ninja pybind11

首先构建ucx

NIXL was tested with UCX version 1.18.0.

GDRCopy is available on Github and is necessary for maximum performance, but UCX and NIXL will work without it.

wget https://github.com/openucx/ucx/releases/download/v1.18.0/ucx-1.18.0.tar.gz
tar xzf ucx-1.18.0.tar.gz
cd ucx-1.18.0
./configure                          \
  --enable-shared                    \
  --disable-static                   \
  --disable-doxygen-doc              \
  --enable-optimizations             \
  --enable-cma                       \
  --enable-devel-headers             \
  --with-cuda=/usr/local/cuda         \
  --with-verbs                       \
  --with-dm                          \
  --with-gdrcopy=<gdrcopy install>   \
  --enable-mt
make -j
make -j install-strip
ldconfig

Libraries have been installed in: /usr/lib/ucx

构建nixl wheel包

mkdir build && \
meson setup build/ --prefix=/usr/local/nixl && \
cd build/ && \
ninja && \
ninja install

# Setup with custom options (example)
$ meson setup <name_of_build_dir> \
    -Dbuild_docs=true \           # Build Doxygen documentation
    -Ducx_path=/path/to/ucx \     # Custom UCX installation path
    -Dinstall_headers=true \      # Install development headers
    -Ddisable_gds_backend=false   # Enable GDS backend

cd name_of_build_dir
ninja

# ./contrib/build-wheel.sh

uv pip3 install auditwheel

uv build --wheel

export LD_LIBRARY_PATH=/usr/local/nixl/lib/x86_64-linux-gnu/plugins/:$LD_LIBRARY_PATH

# ldd --version # to check glibc version, manylinux_2_35_x86_64 means glibc 2.35
auditwheel repair dist/nixl-*-cp31*.whl --plat manylinux_2_35_x86_64

old:

在nixl源码路径，设置编译好的ucx路径

ref https://mesonbuild.com/meson-python/how-to-guides/meson-args.html

python -m pip wheel --config-settings=setup-args="-Ducx_path=/root/ucx" .
# or 
pip install build
python -m build -Csetup-args="-Ducx_path=/root/ucx" .

构建ai_dynamo_vllm

下载VLLM 0.7.2的分支，然后打patch dynamo\container\deps\vllm\vllm_v0.7.2-dynamo-kv-disagg-patch.patch，然后源码编译VLLM。

相当于直接编译VLLM，参考vllm doc和其他博客

Installation — vLLM

https://github.com/vllm-project/vllm/blob/main/.github/workflows/scripts/build.sh

https://zhuanlan.zhihu.com/p/692200573

# for cuda12.1, pytorch 2.5.1
# v0.0.28.post3 build for PyTorch 2.5.1
pip3 install -U xformers==v0.0.28.post3 --index-url https://download.pytorch.org/whl/cu121

git clone --recursive -b v0.7.2 https://github.com/vllm-project/vllm.git
git apply dynamo/container/deps/vllm/vllm_v0.7.2-dynamo-kv-disagg-patch.patch

python setup.py bdist_wheel