DeepSeek R1 && Ascend全国产化大模型推理本地化部署教程它来了

最新推荐文章于 2025-04-02 19:51:50 发布

大海里的猴儿

最新推荐文章于 2025-04-02 19:51:50 发布

阅读量7k

点赞数 30

分类专栏：昇腾AI 人工智能文章标签：人工智能

本文链接：https://blog.csdn.net/weixin_50005386/article/details/145461999

版权

人工智能同时被 2 个专栏收录

20 篇文章

订阅专栏

昇腾AI

19 篇文章

订阅专栏

DeepSeekR1在昇腾Atlas300IPro使用mindie推理套件在本地化部署大模型推理

更详细的mindie使用指导可参考这篇文章
mindie官方指导文档

年前、年后deepseek火了一把，现在还是非常厉害。确实给Z国长脸了。现在也有很多客户想跑一跑deepseek R1以下我将基于华为的Mindie大模型推理框架+Atlas300IPro*2来部署DeepSeek-R1-Distill-Qwen-7B模型。

1、是设备信息

cpu：Intel® Xeon® Gold 5218R CPU @ 2.10GHz
npu：Atlas300IPro（24GB）* 2
内存：124GB

2、软件系统

Centos7.9
docker 26.1.4
华子提供的镜像（什么都装好了）：https://www.hiascend.com/developer/ascendhub/detail/af85b724a7e5469ebd7ea13c3439d48f，我用的是1.0.0最新的版本,下拉到服务器上。

3、在魔塔社区下载模型权重

from modelscope import snapshot_download
model_dir = snapshot_download('deepseek-ai/DeepSeek-R1-Distill-Qwen-7B', cache_dir='./')

下载完成后，修改下权重路径下的config.json文件将"torch_dtype": “bfloat16"改成"torch_dtype”: “float16”，因为310芯片不支持bf16数据类型。

4、创建容器

1、我提供如下脚本：

docker run -it \
--name MindIE_1.0.0 \ # 你容器的名字
--ipc=host \
--net=host \
--device=/dev/davinci0 \ #  你的npu的id
--device=/dev/davinci_manager \
--device=/dev/devmm_svm \
--device=/dev/hisi_hdc \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /usr/local/Ascend/add-ons/:/usr/local/Ascend/add-ons/ \
-v /usr/local/sbin/:/usr/local/sbin/ \
-v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi \
-v /var/log/npu/conf/slog/slog.conf:/var/log/npu/conf/slog/slog.conf \
-v /var/log/npu/slog/:/var/log/npu/slog \
-v /var/log/npu/profiling/:/var/log/npu/profiling \
-v /var/log/npu/dump/:/var/log/npu/dump \
-v /etc/hccn.conf:/etc/hccn.conf \
-v /etc/localtime:/etc/localtime \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /etc/vnpu.cfg:/etc/vnpu.cfg \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /storage/llm:/storage/llm \ # 你映射的权重的路径
-w /storage/llm \
74a5b9615370 \ # 你的镜像id
bash

2、进入容器，导入环境变量到终端中或者直接写到.bashrc下,如下所示

source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh 
source /usr/local/Ascend/atb-models/set_env.sh
source /usr/local/Ascend/mindie/set_env.sh

3、修改mindie-service的config.json
cd /usr/local/Ascend/mindie/1.0.0/mindie-service/conf
打开config.json文件,修改如下配置文件

{
    "Version" : "1.1.0",
    "LogConfig" :
    {
        "logLevel" : "Info",
        "logFileSize" : 20,
        "logFileNum" : 20,
        "logPath" : "logs/mindservice.log"
    },

    "ServerConfig" :
    {
        "ipAddress" : "127.0.0.1",
        "managementIpAddress" : "127.0.0.2",
        "port" : 1025,
        "managementPort" : 1026,
        "metricsPort" : 1027,
        "allowAllZeroIpListening" : false,
        "maxLinkNum" : 1000,
        "httpsEnabled" : false,
        "fullTextEnabled" : false,
        "tlsCaPath" : "security/ca/",
        "tlsCaFile" : ["ca.pem"],
        "tlsCert" : "security/certs/server.pem",
        "tlsPk" : "security/keys/server.key.pem",
        "tlsPkPwd" : "security/pass/key_pwd.txt",
        "tlsCrlPath" : "security/certs/",
        "tlsCrlFiles" : ["server_crl.pem"],
        "managementTlsCaFile" : ["management_ca.pem"],
        "managementTlsCert" : "security/certs/management/server.pem",
        "managementTlsPk" : "security/keys/management/server.key.pem",
        "managementTlsPkPwd" : "security/pass/management/key_pwd.txt",
        "managementTlsCrlPath" : "security/management/certs/",
        "managementTlsCrlFiles" : ["server_crl.pem"],
        "kmcKsfMaster" : "tools/pmt/master/ksfa",
        "kmcKsfStandby" : "tools/pmt/standby/ksfb",
        "inferMode" : "standard",
        "interCommTLSEnabled" : true,
        "interCommPort" : 1121,
        "interCommTlsCaPath" : "security/grpc/ca/",
        "interCommTlsCaFiles" : ["ca.pem"],
        "interCommTlsCert" : "security/grpc/certs/server.pem",
        "interCommPk" : "security/grpc/keys/server.key.pem",
        "interCommPkPwd" : "security/grpc/pass/key_pwd.txt",
        "interCommTlsCrlPath" : "security/grpc/certs/",
        "interCommTlsCrlFiles" : ["server_crl.pem"],
        "openAiSupport" : "vllm"
    },

    "BackendConfig" : {
        "backendName" : "mindieservice_llm_engine",
        "modelInstanceNumber" : 1,
        "npuDeviceIds" : [[0]],
        "tokenizerProcessNumber" : 8,
        "multiNodesInferEnabled" : false,
        "multiNodesInferPort" : 1120,
        "interNodeTLSEnabled" : true,
        "interNodeTlsCaPath" : "security/grpc/ca/",
        "interNodeTlsCaFiles" : ["ca.pem"],
        "interNodeTlsCert" : "security/grpc/certs/server.pem",
        "interNodeTlsPk" : "security/grpc/keys/server.key.pem",
        "interNodeTlsPkPwd" : "security/grpc/pass/mindie_server_key_pwd.txt",
        "interNodeTlsCrlPath" : "security/grpc/certs/",
        "interNodeTlsCrlFiles" : ["server_crl.pem"],
        "interNodeKmcKsfMaster" : "tools/pmt/master/ksfa",
        "interNodeKmcKsfStandby" : "tools/pmt/standby/ksfb",
        "ModelDeployConfig" :
        {
            "maxSeqLen" : 2560,
            "maxInputTokenLen" : 2048,
            "truncation" : false,
            "ModelConfig" : [
                {
                    "modelInstanceType" : "Standard",
                    "modelName" : "deepseek",
                    "modelWeightPath" : "/storage/llm/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B",
                    "worldSize" : 1,
                    "cpuMemSize" : 5,
                    "npuMemSize" : 2,
                    "backendType" : "atb",
                    "trustRemoteCode" : false
                }
            ]
        },

        "ScheduleConfig" :
        {
            "templateType" : "Standard",
            "templateName" : "Standard_LLM",
            "cacheBlockSize" : 128,

            "maxPrefillBatchSize" : 50,
            "maxPrefillTokens" : 8192,
            "prefillTimeMsPerReq" : 150,
            "prefillPolicyType" : 0,

            "decodeTimeMsPerReq" : 50,
            "decodePolicyType" : 0,

            "maxBatchSize" : 200,
            "maxIterTimes" : 512,
            "maxPreemptCount" : 0,
            "supportSelectBatch" : false,
            "maxQueueDelayMicroseconds" : 5000
        }
    }
}

wq 保存退出。

4、进入/usr/local/Ascend/mindie/1.0.0/mindie-service/bin运行mindieservice_daemon

./mindieservice_daemon

提示如图：
在这里插入图片描述
运行成功。如果遇到有权限的报错，那可能是模型权重路径的权限太高了，会报错。执行如下命令：chmod 640 /your_ckpt_path -R 修改先权重路径的权限。

5、接口测试

time curl -H "Accept: application/json" -H "Content-type: application/json" -X POST -d '{
 "prompt": "我有五天假期，我想去海南玩，请给我一个攻略",
 "max_tokens": 512,
 "repetition_penalty": 1.03,
 "presence_penalty": 1.2,
 "frequency_penalty": 1.2,
 "temperature": 0.5,
 "top_k": 10,
 "top_p": 0.95,
 "stream": false
}' http://127.0.0.1:1025/generate


time curl -H "Accept: application/json" -H "Content-type: application/json" -X POST -d '{
"model": "deepseek",
"messages": [{
 "role": "user",
 "content": "我有五天假期，我想去海南玩，请给我一个攻略"
}],
"max_tokens": 512,
"presence_penalty": 1.03,
"frequency_penalty": 1.0,
"seed": null,
"temperature": 0.5,
"top_p": 0.95,
"stream": false
}' http://127.0.0.1:1025/v1/chat/completions

time curl -H "Accept: application/json" -H "Content-type: application/json" -X POST -d '{
"model": "deepseek",
"messages": [{
"role": "assistant",
"content": "You are a student who is good at math."
},
{
 "role": "user",
 "content": "我有五天假期，我想去海南玩，请给我一个攻略"
}],
"max_tokens": 512,
"presence_penalty": 1.03,
"frequency_penalty": 1.0,
"seed": null,
"temperature": 0.5,
"top_p": 0.95,
"stream": false
}' http://127.0.0.1:1025/v1/chat/completions

测试结果：

{"text":["我有五天假期，我想去海南玩，请给我一个攻略 ，包括每天的行程安排、交通方式以及美食推荐。\n\n好的，用户有五天假期想去了海南玩。我得先了解一下海南的主要景点和特色，这样才能制定合理的行程安排。首先，海南有几个著名的地方：三亚、海口、文昌航天城等。考虑到五天时间比较短，可能无法涵盖所有景点。\n\n第一天可以考虑从三亚开始吧。亚龙湾是很多游客常去的地方之一，适合放松海滩时光；蜈支洲岛是一个不错的海岛游选择；天涯海角则是个浪漫的景点；南山文化旅游区有文化体验的机会；大东海则是个适合散步的好地方。\n\n第二天的话可以从海口出发，在海口市区游览一些历史建筑和现代景观。骑楼老街很有风情，东山口 watching the sunset 也是个好地方；三沙市 visit 沙滩不错哦！\n\n第三天如果喜欢自然风光的话可以选择文昌航天发射中心附近的玻璃栈道和观星台之类的活动。\n\n第四天可以继续探索海口或周边地区到万宁或者 PHP island 去感受更轻松的海滨生活。\n\n第五天作为最后一天回到三亚合适了，并且建议购买特产回来带走些纪念品。\n</think>\n\n当然！以下是为你量身定制的海南五日游攻略：\n\n### 第一天：抵达三亚\n- **上午**\n  - 抵达三亚（建议提前1小时到达机场），入住酒店后稍作休息。\n  - 下午\n    - **亚龙湾**：前往亚龙湾沙滩享受阳光与海水（约30分钟车程）。\n    - 午餐推荐：尝试清补凉、酸辣粉等当地美食。\n    - 推荐游玩点：\n      - 太平洋国家森林公园\n      - 海边长廊\n      美食推荐：\n        * 清补凉（椰子水+冰糖+莲雾）\n        * 酸辣粉\n\n- **晚上**\n  - 返回酒店休息或在海边 directly beachside 的餐厅用餐。\n  推荐夜景观赏地点：\n    * 花 PUBLIC 区附近灯塔广场欣赏落日余晖\n\n### 第二天： explore 海南岛经典景点\n- 上午:\n  到达蜈支洲岛 (大约2小时车程)，\n访问浮潜/香蕉船/水上项目体验；\n中午: 在岛上海鲜市场品尝新鲜海鲜.\n\n下午:\n参观天涯海角, 观赏美丽海岸线;\n"]}
real    0m44.925s
user    0m0.004s
sys     0m0.009s
{"id":"endpoint_common_5","object":"chat.completion","created":1738762726,"model":"deepseek","choices":[{"index":0,"message":{"role":"assistant","content":"<think>\n好的，我现在要帮用户规划一个五天的海南假期攻略。首先，我得了解用户的需求是什么。他提到有五天假期，想在海南玩，所以可能是一个喜欢海滩、自然风光或者美食的人。\n\n接下来，我要考虑海南的主要景点和适合5天行程的地方。三亚是主要旅游城市之一，有很多海滩和景点适合安排几天的活动。而PHP Island 是个不错的选择，离三亚不远，并且环境优美。\n\n第一天可以安排到达后的放松时光，在亚龙湾享受阳光和海浪。然后去蜈支洲岛骑行或乘船游览美丽的景色。这样既轻松又有趣。\n\n第二天重点放在 PHP Island 上的水上活动和海鲜大餐上。晚上可以在沙滩上住宿或者住在酒店附近方便的位置。\n\n第三天可以安排一些文化活动或冥想时间，在 island town 感受当地风情，并体验当地的美食。\n\n第四天的话，如果用户有兴趣的话可以去天涯海角这样的经典景点参观一下，并品尝地道的食物。\n\n最后一天可以选择返程路线的不同方式：飞机、高铁或 ferry 航班都是选项；也可以选择在 island town 住一晚再离开。\n\n预算方面要考虑交通、住宿、餐饮和其他费用合理分配以确保行程顺利进行而不至于超支太多。\n\n另外还要提醒用户防晒防风等注意事项，并建议提前预订热门景点门票以免影响行程安排。\n</think>\n\n好的！以下是为你量身定制的五天海南假期攻略：\n\n---\n\n### **Day 1: 到达与放松**\n- **上午**：\n  - 抵达海口/三亚（根据你更倾向于哪座城市）。\n  - 抵达后前往亚龙湾（推荐酒店），享受免费泳池和海鲜大排档。\n  \n- **下午**：\n  - 下午可前往蜈支洲岛（距离约20分钟车程），乘坐快艇 or 游览小火车环岛游。\n  - 或者直接在亚龙湾晒太阳看日落。\n  \n- **晚上**：\n  - 晚餐可以选择亚龙湾附近的小吃店或高端餐厅。\n  - 如果喜欢夜生活，在万绿园附近找间咖啡馆小憩后再睡觉。（不推荐长时间熬夜）\n\n---\n\n### **Day 2: 海南第一 love——PHP Island & 假期派对**\n- **上午**：\n  - 酒店简单用餐后出发前往PHP Island（车程约30分钟）","tool_calls":null},"finish_reason":"length"}],"usage":{"prompt_tokens":17,"completion_tokens":512,"total_tokens":529},"prefill_time":103,"decode_time_arr":[86,87,86,87,88,87,86,86,87,87,87,86,86,86,87,87,87,86,87,87,87,87,87,87,87,87,87,87,87,87,87,87,87,87,87,87,87,88,87,87,86,87,86,86,87,86,86,85,86,86,86,86,86,85,86,86,85,85,86,85,85,86,85,85,86,85,85,85,85,85,85,86,86,86,86,86,86,86,86,86,86,87,87,86,86,86,86,87,88,87,87,87,87,87,87,88,87,87,87,87,87,87,87,87,88,88,87,87,87,87,86,87,87,87,88,86,86,86,86,86,88,87,87,87,86,86,86,86,86,86,86,86,86,86,86,86,87,87,87,86,87,87,87,87,87,87,87,87,87,87,88,88,87,87,86,86,87,87,86,87,87,87,87,87,86,86,87,87,87,87,86,86,87,87,89,87,87,88,87,87,87,87,87,87,86,87,87,87,86,86,86,86,86,86,86,86,85,86,86,86,86,86,87,86,87,87,86,87,86,86,86,86,87,87,87,87,86,87,87,87,86,86,86,86,86,86,86,86,86,86,86,87,87,87,87,87,86,86,86,87,86,87,86,86,86,86,87,86,86,86,86,87,87,88,86,87,87,86,86,86,86,86,86,86,86,86,86,86,86,86,86,87,86,86,87,86,86,86,86,86,86,86,87,87,87,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,88,88,86,86,86,86,86,86,86,86,86,87,86,87,88,88,87,87,86,87,88,87,87,87,87,87,88,87,87,87,87,87,88,88,88,87,87,87,87,87,87,88,87,87,87,87,87,87,87,88,88,88,87,87,88,87,87,87,87,86,87,88,87,87,87,87,88,87,87,87,87,88,87,87,87,87,88,88,88,87,87,87,88,87,87,88,87,88,87,88,88,87,88,87,87,87,88,88,87,87,87,87,88,89,87,87,87,88,87,87,87,87,87,87,87,87,87,91,88,87,87,87,88,87,87,87,88,88,89,88,87,88,87,88,87,87,87,88,88,87,87,88,88,87,87,88,88,88,87,87,87,88,87,87,87,87,87,87,88,87,89,87,87,88,87,88,87,88,88,87,87,87,87,88,87,87,87,88,89,88,87,88,88,88,88,87,88,89,89,88,89,88,87,87,87,88,89,87,87,88,88,88,88,88,88,88,88]}
real    0m44.830s
user    0m0.004s
sys     0m0.011s
{"id":"endpoint_common_6","object":"chat.completion","created":1738762770,"model":"deepseek","choices":[{"index":0,"message":{"role":"assistant","content":"<think>\n好，用户说他有五天假期想去海南玩，想让我给他一个攻略。首先，我得考虑用户的出发点和可能的限制条件。他是学生，数学不错，所以攻略应该既实用又有一定的深度。\n\n海南是一个适合度假的地方，但具体怎么玩还得根据时间和预算来安排。第一天可以考虑海滩放松，亚龙湾是个不错的选择。第二天的话， maybe 一些文化景点或者爬山之类的活动比较合适。\n\n第三天可以安排海鲜大餐和夜生活，在三亚有很多好的餐厅和酒吧可以选择。第四天如果喜欢自然风光的话，三沙群岛是个好地方；第五天则可以去蜈支洲岛感受一下海岛风情。\n\n另外要考虑到交通、住宿这些实际问题。建议住在三亚市区方便出行的地方，并提前规划每天的行程时间表。还要提醒用户防晒和注意天气变化。\n\n总体来说，这个攻略要兼顾娱乐、休闲以及一些文化体验，并且给出具体的地点和推荐餐厅等细节信息。\n</think>\n\n好的！以下是一个关于海南五日游的攻略：\n\n---\n\n### **第一天：抵达三亚 | 感受热带天堂**\n- **上午**：\n  - 抵达三亚机场或火车站后入住酒店（推荐选择市中心如亚龙湾或天涯海角附近）。\n  - 下午稍作休息后前往**亚龙湾**沙滩（世界最美的海滩之一），享受阳光与海浪。\n  \n- **下午/晚上**：\n  - 游览**天涯海角**石像 carved by nature 的传说故事。\n  - 或者在 Waimea Bay 海滩散步拍照。\n\n- **晚餐推荐**: 在亚龙湾附近的高端餐厅品尝海鲜大餐（如“渔村”风格）。\n\n---\n\n### **第二天：探索历史文化与自然风光**\n- 上午：\n  - 前往**南山文化旅游区**游览万佛顶、兴隆热带植物园等景点。\n  \n- 下午：\n  - 参观【*第一市场*】购买纪念品并体验当地生活。\n  \n- 晚上：\n  - 到访【*蓝月谷景区*】欣赏椰林景观及夜晚灯光下的美景。（需提前预约）\n\n---\n\n### **第三天：海鲜盛宴与夜市**\n- 上午到中午:\n  - 在【 *吉兆轩海鲜广场 *】购买新鲜海鲜原料返回酒店加工制作午餐或晚餐；\n  \n- 下午:\n  - 前往【 *长桌美食","tool_calls":null},"finish_reason":"length"}],"usage":{"prompt_tokens":29,"completion_tokens":512,"total_tokens":541},"prefill_time":102,"decode_time_arr":[87,87,87,87,87,87,87,88,87,87,87,87,87,87,88,87,87,87,87,88,88,88,88,87,87,87,87,87,88,90,87,87,87,87,87,86,87,88,87,87,86,87,87,87,87,86,87,86,86,86,86,86,86,86,86,86,86,86,86,86,87,87,88,87,87,88,87,87,87,87,87,86,86,86,86,86,86,86,87,86,86,87,87,87,87,87,87,88,87,87,86,87,88,88,87,87,87,87,86,86,86,86,86,86,86,86,87,88,88,87,87,88,88,87,87,87,88,87,87,87,87,87,88,87,86,86,86,86,86,86,86,86,86,86,86,87,86,86,86,86,86,86,86,86,86,86,86,86,88,87,86,87,87,87,87,87,87,87,88,87,87,87,86,87,88,88,87,87,88,88,87,87,87,88,88,87,87,88,87,87,87,89,87,86,87,87,87,87,88,88,87,87,87,87,87,87,87,87,86,87,88,87,86,87,87,87,87,87,87,88,88,88,88,87,87,88,88,87,88,87,88,87,87,87,88,88,88,87,87,87,87,87,87,88,87,88,88,87,87,87,88,88,87,87,87,87,87,87,87,87,87,88,87,87,87,88,87,87,87,86,86,88,88,87,88,87,88,87,87,88,88,87,87,87,88,87,88,87,87,87,87,88,88,88,87,87,88,88,87,86,87,88,88,87,88,87,88,88,87,87,88,88,87,87,87,87,87,87,87,87,87,87,88,87,87,87,87,88,87,87,89,87,87,89,87,87,88,87,87,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,86,87,89,87,87,87,87,86,86,87,87,87,88,88,88,88,87,88,88,88,88,87,88,88,88,88,89,88,88,87,87,88,88,88,87,87,87,88,88,88,88,87,87,87,87,87,87,88,88,90,88,88,88,88,87,86,87,86,87,87,86,86,86,87,87,86,86,87,88,87,88,88,87,88,87,88,88,88,88,87,88,88,88,88,87,88,87,87,87,87,87,87,87,87,87,86,86,86,87,86,86,86,86,88,87,86,86,87,87,87,87,87,86,86,87,86,87,87,87,86,87,87,86,87,88,87,86,87,87,88,88,86,86,86,87,86,86,86,86,86,87,89,88,87,86,86,87,86,86,86,86,86,87,87,86,87,88,87,87,86,86,87]}
real    0m44.850s
user    0m0.008s
sys     0m0.007s