AI文本生成视频工具CogVideoX部署到华为昇腾NPU的详细步骤

已于 2024-11-28 11:14:10 修改
阅读量1k 收藏 6
点赞数 11
分类专栏： 开源 for Huawei 文章标签： 人工智能
于 2024-11-28 10:26:13 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_54958500/article/details/143732793
版权

CogVideoX是智谱AI开发的视频生成大模型。无需复杂的视频制作技能和工具，能够将文本描述或静态图片转化为高质量、具有视觉吸引力的动态视频。

https://github.com/THUDM/CogVideo

 一、部署到昇腾NPU

昇腾环境：

芯片类型：昇腾910B3
CANN版本：CANN 7.0.1.5
驱动版本：23.0.6
操作系统：Huawei Cloud EulerOS 2.0

1.环境搭建

conda创建python3.10环境。

conda create --name cogvideo python=3.10

 从 GitHub 拉取代码。

git clone https://github.com/THUDM/CogVideo.git
cd CogVideo

 安装依赖。

pip install -r requirements.txt 
# 安装对应pytorch版本的torch_npu
pip install torch_npu

新建测试verify_npu.py，验证NPU环境：

import torch
import torch_npu
 
# 检查 NPU 是否可用
if torch.npu.is_available():
    print("NPU is available.")
    device_id = 0
    torch.npu.set_device(device_id)

    device = 'npu:' + str(device_id)
    tensor = torch.tensor([1.0, 2.0, 3.0], device=device)
    print(tensor)
else:
    print("NPU is not available.")

输出NPU is available，NPU 环境正常。

2.下载模型

模型较大，需预留45G空间。

选择使用Git下载模型，需要先安装git-lfs。

wget https://github.com/git-lfs/git-lfs/releases/download/v3.5.1/git-lfs-linux-arm64-v3.5.1.tar.gz
tar xvf git-lfs-linux-arm64-v3.5.1.tar.gz
cd git-lfs-3.5.1
sudo ./install.sh

验证git-lfs安装：

git lfs version

下载模型：

mkdir THUDM
cd THUDM
git lfs install
git clone https://www.modelscope.cn/ZhipuAI/CogVideoX-5b.git
cd CogVideoX-5b
git lfs pull

3.在npu上运行代码

新建run_t2v_npu.py:

import torch
import torch_npu
from diffusers import CogVideoXPipeline
from diffusers.utils import export_to_video

# 检查是否有可用的 GPU
if torch.npu.is_available():
    print("NPU is available.")
    device = torch.device('npu')

    torch_dtype = torch.bfloat16  # 推荐使用 BF16
    print(f"Using data type: {torch_dtype}")

    # 创建一个随机数生成器
    generator = torch.Generator(device=device).manual_seed(42)
    print("Generator created")

    prompt = "A panda, dressed in a small, red jacket and a tiny hat, sits on a wooden stool in a serene bamboo forest. The panda's fluffy paws strum a miniature acoustic guitar, producing soft, melodic tunes. Nearby, a few other pandas gather, watching curiously and some clapping in rhythm. Sunlight filters through the tall bamboo, casting a gentle glow on the scene. The panda's face is expressive, showing concentration and joy as it plays. The background includes a small, flowing stream and vibrant green foliage, enhancing the peaceful and magical atmosphere of this unique musical performance."
    print("Prompt set")

    pipe = CogVideoXPipeline.from_pretrained(
        "THUDM/CogVideoX-5b",
        torch_dtype=torch_dtype
    )
    print("Pipeline loaded")

    # 确保模型和数据都在选定的设备上
    pipe.to(device)
    print("Model moved to device")

    video = pipe(
        prompt=prompt,
        num_videos_per_prompt=1,
        num_inference_steps=50,
        num_frames=49,
        guidance_scale=6,
        generator=generator,
    ).frames[0]
    print("Video generated")

    export_to_video(video, "output.mp4", fps=8)
    print("Video exported")

else:
    print("NPU is not available.")

运行报错：

(cogvideo) [root@devserver-314b-1 CogVideo]# python run_t2v_npu.py
Using device: npu
Using data type: torch.bfloat16
Generator created
Prompt set
Loading checkpoint shards: 100%|████████████████████████████████████████████████████| 2/2 [00:00<00:00,  4.63it/s]
Loading pipeline components...: 100%|███████████████████████████████████████████████| 5/5 [00:01<00:00,  2.52it/s]
Pipeline loaded
Model moved to device
  0%|                                                                                      | 0/50 [00:00<?, ?it/s][W1114 14:42:44.058506898 compiler_depend.ts:387] Warning: EH9999: Inner Error!
EH9999  [Init][Env]init env failed![FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]
        TraceBack (most recent call last):
        build op model failed, result = 500001[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]
 (function ExecFunc)
  0%|                                                                                      | 0/50 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/dev/shm/wangshijing/CogVideo/run_t2v_npu.py", line 31, in <module>
    video = pipe(prompt=prompt,num_videos_per_prompt=1,num_inference_steps=50,num_frames=49,guidance_scale=6,generator=generator,).frames[0]
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/pipelines/cogvideo/pipeline_cogvideox.py", line 684, in __call__
    noise_pred = self.transformer(
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/models/transformers/cogvideox_transformer_3d.py", line 473, in forward
    hidden_states, encoder_hidden_states = block(
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/models/transformers/cogvideox_transformer_3d.py", line 132, in forward
    attn_hidden_states, attn_encoder_hidden_states = self.attn1(
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/models/attention_processor.py", line 495, in forward
    return self.processor(
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/models/attention_processor.py", line 1928, in __call__
    value = attn.to_v(hidden_states)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 125, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: The Inner error is reported as above. The process exits for this inner error, and the current working operator name is Conv2D.
Since the operator is called asynchronously, the stacktrace may be inaccurate. If you want to get the accurate stacktrace, pleace set the environment variable ASCEND_LAUNCH_BLOCKING=1.
[ERROR] 2024-11-26-14:42:44 (PID:1154952, Device:0, RankID:-1) ERR00100 PTA call acl api failed
[W1114 14:42:44.066216328 compiler_depend.ts:659] Warning: 0Failed to find function aclrtSynchronizeDeviceWithTimeout (function operator())

设置环境变量获取更准确的堆栈跟踪信息：

export ASCEND_LAUNCH_BLOCKING=1

再次运行报错变为：

(cogvideo) [root@devserver-314b-1 CogVideo]# python run_t2v_npu.py
Using device: npu
Using data type: torch.bfloat16
Generator created
Prompt set
Loading checkpoint shards: 100%|████████████████████████████████████████████████████| 2/2 [00:00<00:00,  4.85it/s]
Loading pipeline components...: 100%|███████████████████████████████████████████████| 5/5 [00:01<00:00,  2.51it/s]
Pipeline loaded
Model moved to device
  0%|                                                                                      | 0/50 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/dev/shm/wangshijing/CogVideo/run_t2v_npu.py", line 31, in <module>
    video = pipe(prompt=prompt,num_videos_per_prompt=1,num_inference_steps=50,num_frames=49,guidance_scale=6,generator=generator,).frames[0]
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/pipelines/cogvideo/pipeline_cogvideox.py", line 684, in __call__
    noise_pred = self.transformer(
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/models/transformers/cogvideox_transformer_3d.py", line 446, in forward
    hidden_states = self.patch_embed(encoder_hidden_states, hidden_states)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/models/embeddings.py", line 412, in forward
    image_embeds = self.proj(image_embeds)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 554, in forward                                                                                                                 
    return self._conv_forward(input, self.weight, self.bias)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 549, in _conv_forward
    return F.conv2d(
RuntimeError: InnerRun:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:218 OPS function error: Conv2D, error code is 500001
[ERROR] 2024-11-26-14:45:31 (PID:1155878, Device:0, RankID:-1) ERR01100 OPS call acl api failed
[Error]: The internal ACL of the system is incorrect.
        Rectify the fault based on the error information in the ascend log.
EC0010: Failed to import Python module [ModuleNotFoundError: No module named 'tbe'.].
        Solution: Check that all required components are properly installed and the specified Python path matches the Python installation directory. (If the path does not match the directory, run set_env.sh in the installation package.)
        TraceBack (most recent call last):
        [GraphOpt][InitializeInner][InitTbeFunc] Failed to init tbe.[FUNC:InitializeInner][FILE:tbe_op_store_adapter.cc][LINE:1623]
        [SubGraphOpt][PreCompileOp][InitAdapter] InitializeAdapter adapter [tbe_op_adapter] failed! Ret [4294967295][FUNC:InitializeAdapter][FILE:op_store_adapter_manager.cc][LINE:85]
        [SubGraphOpt][PreCompileOp][Init] Initialize op store adapter failed, OpsStoreName[tbe-custom].[FUNC:Initialize][FILE:op_store_adapter_manager.cc][LINE:126]
        [FusionMngr][Init] Op store adapter manager init failed.[FUNC:Initialize][FILE:fusion_manager.cc][LINE:124]                                                                                                                 
        PluginManager InvokeAll failed.[FUNC:Initialize][FILE:ops_kernel_manager.cc][LINE:96]
        OpsManager initialize failed.[FUNC:InnerInitialize][FILE:gelib.cc][LINE:237]
        GELib::InnerInitialize failed.[FUNC:Initialize][FILE:gelib.cc][LINE:165]
        [Initialize][Ge]GEInitialize failed. ge result = 4294967295[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
        [Init][Compiler]Init compiler failed[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]
        [Set][Options]OpCompileProcessor init failed![FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]
        [Init][Env]init env failed![FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]
        build op model failed, result = 500001[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]
                                                                                                                  
[W1114 14:45:31.433200230 compiler_depend.ts:659] Warning: 0Failed to find function aclrtSynchronizeDeviceWithTimeout (function operator())

根据错误信息，tbe 模块未找到。尝试运行环境设置脚本。

source /usr/local/Ascend/ascend-toolkit/set_env.sh

再次运行报错变为：

(cogvideo) [root@devserver-314b-1 CogVideo]# python run_t2v_npu.py
Using device: npu
Using data type: torch.bfloat16
Generator created
Prompt set
Loading checkpoint shards: 100%|████████████████████████████████████████████████████| 2/2 [00:00<00:00,  4.74it/s]
Loading pipeline components...: 100%|███████████████████████████████████████████████| 5/5 [00:01<00:00,  2.51it/s]
Pipeline loaded
Model moved to device
  0%|                                                                                      | 0/50 [00:00<?, ?it/s]SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
  0%|                                                                                      | 0/50 [00:08<?, ?it/s]
Traceback (most recent call last):
  File "/dev/shm/wangshijing/CogVideo/run_t2v_npu.py", line 31, in <module>
    video = pipe(prompt=prompt,num_videos_per_prompt=1,num_inference_steps=50,num_frames=49,guidance_scale=6,generator=generator,).frames[0]
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/pipelines/cogvideo/pipeline_cogvideox.py", line 684, in __call__
    noise_pred = self.transformer(
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/models/transformers/cogvideox_transformer_3d.py", line 473, in forward
    hidden_states, encoder_hidden_states = block(
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/models/transformers/cogvideox_transformer_3d.py", line 132, in forward
    attn_hidden_states, attn_encoder_hidden_states = self.attn1(
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/models/attention_processor.py", line 495, in forward
    return self.processor(
  File "/root/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/models/attention_processor.py", line 1950, in __call__
    hidden_states = F.scaled_dot_product_attention(
RuntimeError: aclnnFusedInferAttentionScoreV2 or aclnnFusedInferAttentionScoreV2GetWorkspaceSize not in libopapi.so, or libopapi.sonot found.
[ERROR] 2024-11-26-14:55:15 (PID:1158036, Device:0, RankID:-1) ERR01004 OPS invalid pointer
[W1114 14:55:15.526441221 compiler_depend.ts:659] Warning: 0Failed to find function aclrtSynchronizeDeviceWithTimeout (function operator())

F.scaled_dot_product_attention 操作时发生的内部错误， aclnnFusedInferAttentionScoreV2 或 aclnnFusedInferAttentionScoreV2GetWorkspaceSize 函数未在 libopapi.so 中找到。

查看libopapi.so 的依赖库是否正确加载：

ldd /usr/local/Ascend/ascend-toolkit/7.0.1.5/aarch64-linux/lib64/lib opapi.so

检查 libopapi.so 的符号表：

nm -gC /usr/local/Ascend/ascend-toolkit/7.0.1.5/aarch64-linux/lib64/libopapi.so | grep aclnnFusedInferAttentionScoreV2

显示函数不存在：

nm: /usr/local/Ascend/ascend-toolkit/7.0.1.5/aarch64-linux/lib64/libopapi.so: no symbols

查询到相关项目：cann-ops-adv: cann-ops-adv，是基于昇腾硬件的融合算子库（adv表示advanced）。

通过Issues提问得知该算子库当前只支持torch 2.1(稳定版本) 和 torch 2.3(beta版)。改为安装torch 2.1版本依赖。

pip install torch==2.1.0 torch_npu==2.1.0 torchvision==0.16.0

运行后显示内存不足。

改成使用NPU设备1，减少生成视频的帧数，降低模型精度：

import torch
import torch_npu
from diffusers import CogVideoXPipeline
from diffusers.utils import export_to_video
                                                                                                                 
# 检查是否有可用的 GPU
if torch.npu.is_available():
    print("NPU is available.")
    device_id = 1
    torch.npu.set_device(device_id)
                                                                                                                 
    # 创建一个 NPU 张量
    device = torch.device('npu:1')  # 指定使用 npu:1
                                                                                                                 
    # torch_dtype = torch.bfloat16  # 推荐使用 BF16
    torch_dtype = torch.float16  # 降低精度
    print(f"Using data type: {torch_dtype}")
                                                                                                                 
    # 创建一个随机数生成器
    generator = torch.Generator(device=device).manual_seed(42)
    print("Generator created")                                                                                                                 
    prompt = "A panda, dressed in a small, red jacket and a tiny hat, sits on a wooden stool in a serene bamboo f
orest. The panda's fluffy paws strum a miniature acoustic guitar, producing soft, melodic tunes. Nearby, a few ot
her pandas gather, watching curiously and some clapping in rhythm. Sunlight filters through the tall bamboo, cast
ing a gentle glow on the scene. The panda's face is expressive, showing concentration and joy as it plays. The ba
ckground includes a small, flowing stream and vibrant green foliage, enhancing the peaceful and magical atmospher
e of this unique musical performance."
    print("Prompt set")
                                                                                                                 
    pipe = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b",torch_dtype=torch_dtype)
    print("Pipeline loaded")
                                                                                                                 
    # 确保模型和数据都在选定的设备上
    pipe.to(device)
    print("Model moved to device")
    print(f"Current NPU device: {torch.npu.current_device()}")
                                                                                                                 
    video = pipe(prompt=prompt,num_videos_per_prompt=1,num_inference_steps=50,num_frames=24,guidance_scale=6,gene
rator=generator,).frames[0]
    print("Video generated")
                                                                                                                 
    export_to_video(video, "output.mp4", fps=8)
    print("Video exported")
                                                                                                                 
else:
    print("NPU is not available.")

模型运行过程中出现警告：

SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

通常是由于使用了不兼容的 Python 版本或编译选项导致的，新建python 3.8的环境即可。

生成的视频：

基于昇腾NPU的YOLOv8部署
如果想成为中心，那么就到中心去吧。
04-02 355
由于模型检测结果是用内存连续的一维数组进行表示的，因此，需要根据yolov8输出尺寸的实际含义，来访问需要的数组内存地址来获取需要的值。根据资料显示，yolov8模型不另外对置信度进行预测， 而是采用类别里面最大的概率作为置信度的值，8400是yolov8模型各尺度输出特征图叠加之后的结果，一般推理不需要处理。即可，参考onnx输出尺寸[1,84,8400]中的84-4 = 80 , 比如自定义的模型输出为[1,26,8400]，则 size_t classNum = 22 （26-4）.
LLMs之NPU之Ascend之PyTorch：基于华为昇腾NPU设备实现PyTorch模型迁移和训练之单机多卡+混合精度训练手动迁移—导入支持库、参数配置、设备映射、进程管理、模型迁移、数据加载
头部AI社区如有邀博主AI主题演讲请私信—心比天高，仗剑走天涯，保持热爱，奔赴向梦想！低调，专注，谦虚，自律，反思，成长，还算比较正能量的博主，公益免费传播…内心特别想在AI界做出一些可以推进历史进程影响力的技术(兴趣使然，有点小情怀，也有点使命感呀
11-12 1365
​ LLMs之NPU之Ascend之PyTorch：基于华为昇腾NPU设备实现PyTorch模型迁移和训练之单机多卡+混合精度训练手动迁移—导入支持库、参数配置、设备映射、进程管理、模型迁移、数据加载、训练过程等多方面调整 目录 大模型软硬配置—对比英伟达GPU和华为NPU—适配方案细节 基于华为昇腾NPU设备实现PyTorch模型迁移和训练之单机多卡+混合精度训练手动迁移—导入支持库、参数配置、设备映射、进程管理、模型迁移、数据加载、训练过程等多方面调整 大模型软硬配置—对比英伟达GPU和华为
CogVideoX-5B的安装与使用教程
gitblog_02238的博客
12-12 1159
CogVideoX-5B的安装与使用教程 CogVideoX-5b 项目地址: https://gitcode.com/hf_mirrors/ai-gitcode/CogVideoX-5b ...
丹摩|智谱清影-CogVideoX-2b-部署与使用
记录学习路上的一些拙见
11-25 1076
丹摩|智谱清影-CogVideoX-2b-部署与使用
DeepSeek本地部署 ：AI视频管理平台工程师手把手教你如何操作
tt120326的博客
02-25 1191
DeepSeek本地部署 ：AI视频管理平台工程师手把手教你如何操作
CogVideoX 初探与部署实践：从基础操作到视频生成
menetholl的博客
09-23 1637
在当今人工智能的发展浪潮中，生成式模型不断推陈出新，尤其是视频生成领域取得了突破性的进展。2023年8月6日，智谱AI发布了一款开源视频生成模型——CogVideoX，该模型具备强大的视频生成能力，为视频创作和自动生成领域提供了全新的思路。本篇博客将带领大家从创建实例、配置环境到生成视频，全方位了解如何在丹摩服务器上部署并使用CogVideoX模型。CogVideoX 是一款基于3D变分自编码器的视频生成模型。
国产版Sora复现——智谱AI开源CogVideoX-2b 本地部署复现实践教程
热门推荐
白洞，白色的明天在等着我们
08-06 1万+
本篇博客将详细介绍，使用丹摩服务器部署和初步使用CogVideoX的实践流程。智谱AI在8月6日宣布了一个令人兴奋的消息：他们将开源视频生成模型CogVideoX。目前，其提示词上限为226 个 token，视频长度为 6 秒，帧率为 8 帧 / 秒，视频分辨率为 720*480，而这仅仅是初代，性能更强参数量更大的模型正在路上。先看两个效果（均为个人本次实践复现得出）：CogVideoX生成视频1CogVideo生成视频2首先简单介绍下原理，CogVideoX的核心在于它的3D变分自编码器。
昇腾设备torch_npu推理配置
qq_65562738的博客
02-05 2602
在昇腾 Ascend 310B1 NPU 上基于 PyTorch 进行推理时，通过torch_npu替换原有 GPU/CUDA 操作。
mediapipe转onnx，并使用onnx在华为升腾NPU的910A上运行
qq_57467780的博客
09-29 2005
mediapipe是使用tensorflow框架运行的模型，我想把它放到华为的npu上运行，这就需要使用一些模型转换的方法，我首先想到的是编译npu版的计算图，但是这个对于我来说太难了，而且教程太零碎，最终无法成功。第二个方法是转成onnx格式，我查了资料，华为的框架也支持onnx，所以这个简单的方法可以实现。
使用华为Ascend NPU加速LangChain：深入探讨和实践指南
ppoojjj的博客
08-22 1353
Ascend NPU是华为设计的专用AI处理器，旨在加速深度学习和机器学习任务。它具有高性能、低功耗的特点，特别适合大规模AI模型的训练和部署。本文介绍了如何在LangChain中使用华为Ascend NPU来加速嵌入计算和模型训练。Ascend NPU为AI应用提供了强大的硬件加速能力，特别适合处理大规模语言模型和复杂的NLP任务。华为Ascend官方文档LangChain文档PyTorch NPU教程。
[深入探索华为Ascend NPU与LangChain的集成：性能与实用性解析]
stjklkjhgffxw的博客
11-27 470
通过将华为Ascend NPU与LangChain结合，开发者可以大幅度提升AI应用的计算效率。然而，环境配置与网络访问仍然是部署过程中的挑战。Ascend NPU官方文档LangChain框架指南。
ubuntu20.04 昇腾310p 开发环境部署
u010053962的博客
09-02 3727
GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 5.4.0-26-generic" # Ubuntu, with Linux 5.4.0-26-generic由上一个查出的。指定Python版本编包方式，以Python3.8为例，其他Python版本请使用 --python=3.9或--python3.10。5、安装CANN及pytorch环境（注意这里的版本和上面下载驱动的版本对应。注意：这里要求cmake的版本>=3.18.0。
RuntimeError: An attempt has been made to start a new process before the current process has finishe
方脸大公鸡的博客
07-26 3281
RuntimeError: An attempt has been made to start a new process before the current process has finshe...（解决方案））
AI视频生成本地搭建图文详细过程
张小三的博客
07-25 3325
在自己的电脑上搭建一个AI，主要用于通过自己输入的关键字来自动生成视频文案，把视频文案自动生成视频，再自动给视频配音配字幕。
【EZ9999】RuntimeError: call aclnnRsubs failed
懒惰是科技进步的原始动力
03-12 2517
detail:EZ9999: Inner Error! EZ9999 Cannot parse json for config file [/usr/local/Ascend/nnae/latest/opp/built-in/op_impl/ai_core/tbe//kernel/config/ascend910/sub.json].
解决 torch.embedding 的 IndexError: index out of range in self
hjt374200573的博客
04-14 1169
修改为CPU运行查看错误原因发现 torch.embedding 报错 index out of range in self这是因为torch.nn.Embedding（num_embeddings,embedding_dim）中的num_embeddings，合法范围在[0,num_embeddings-1]内。
AI黑科技！CogVideo：文本秒变视频，创作从此颠覆想象！
2401_83063795的博客
03-09 1887
CogVideo：创意即现，影像随形！让想象瞬间成影，多维场景视频即刻生成！ - 精选真开源，释放新价值。
AI视频自动生成本地部署，完美实现“剪映”图文成片功能
2403_88996352的博客
01-22 4221
AI视频自动生成本地部署，完美实现“剪映”图文成片功能
智谱清影开源！
强化学习曾小健
08-06 1446
CogVideoX是智谱清影团队开发的一系列视频生成模型，它们能够根据文本提示生成视频内容。最新开源的模型以其强大的生成能力和较低的资源需求，为视频生成领域带来了新的活力。智谱清影的开源行动不仅为视频生成领域带来了新的技术突破，也为全球的技术社区提供了宝贵的资源和学习机会。视频生成将迎来群雄争霸的“乱纪元”，无法预估的黑天鹅事件正不断上演，生态将更加繁荣、发展更加迅猛。智谱也将成为视频生成的引领者，随着更强大的模型和工具的不断推出，我们有理由相信，视频内容创作将迎来一个全新的时代。
X86服务器+ 华为昇腾NPU 怎么部署deepseek
最新发布
02-26
### 部署DeepSeek于X86服务器与华为昇腾NPU #### 准备工作环境 为了确保能够在X86服务器以及华为昇腾NPU上顺利部署DeepSeek，需先确认当前的操作系统版本和支持架构。对于X86服务器而言，通常采用的是Linux操作系统，并且支持x86_64架构。通过命令`uname -m && cat /etc/*release`可以获取到系统的具体信息[^3]。 #### 安装必要的依赖库 针对X86服务器，在安装任何应用之前，应该确保已经安装了所有必需的软件包和工具链。特别是当涉及到神经网络推理时，可能还需要额外配置CUDA、cuDNN等相关组件来充分利用GPU加速性能；而对于华为昇腾NPU，则需要特定驱动程序的支持以便能够调用其强大的NN计算资源[^1]。 #### 获取并加载预训练模型 无论是哪种硬件平台，都需要事先准备好用于执行任务的具体AI模型文件。这些模型通常是经过大量数据集训练得到的最佳参数集合。对于想要运行的任务类型（比如图像识别），可以从官方渠道或者其他开源项目中找到合适的预训练权重文件。 #### 构建适合目标平台的应用容器 考虑到不同体系结构之间的差异性，建议使用Docker作为跨平台开发测试的理想解决方案之一。特别是在ARM架构下，可以通过指定--platform选项拉取适用于该架构的基础镜像来进行后续操作，例如： ```bash docker pull ghcr.io/open-webui/open-webui:main --platform=linux/aarch64 ``` 这一步骤同样适用于准备要在昇腾设备上使用的应用程序环境[^2]。 #### 编写启动脚本实现自动化流程控制 最后就是编写相应的Shell或Python脚本来简化整个过程中的复杂度，使得每次重新部署变得简单快捷。这里提供了一个简单的例子展示如何设置环境变量指向正确的路径位置： ```shell export ASCEND_HOME=/usr/local/Ascend # 假设这是昇腾SDK所在目录 source ${ASCEND_HOME}/toolkit/setup_env.sh python3 deepseek_app.py ``` 以上即是在X86服务器及华为昇腾NPU之上成功部署DeepSeek所需经历的主要环节概述。
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值