在项目里快速运行史上最强开源大模型「Llama 3.1」

7月份，Meta正式发布迄今为止最强大的开源模型——Llama 3.1，包含了405B、70B和8B三个版本的模型。

作为开发者，如何在项目里快速运行LLaMa3.1大模型？

今天，我们结合英智未来自己的LLM推理API平台，给大家介绍一套免费实践流程。

免费使用地址：https://cognihub.baystoneai.com

方式一：不使用流式传输，直接让大模型返回最终结果

请求：

# sh
export API_BASE_URL="https://api.baystoneai.com/cognihub/service/66b97dd5fb19cc7397384f42"export API_KEY="Sy0FY5eH1jOaKtcfVvIxN1zsgRWED4IgfoFA"
curl -X POST $API_BASE_URL/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" -d '{    "model": "llama3.1",    "stream": false,    "messages": [        {"role": "system", "content": "You are a helpful assistant."},        {"role": "user", "content": "英智人工智能智汇API是什么？"}    ]}'

应答：

{  "id": "chatcmpl-754",  "object": "chat.completion",  "created": 1723433933,  "model": "llama3.1",  "system_fingerprint": "fp_ollama",  "choices": [    {      "index": 0,      "message": {        "role": "assistant",        "content": "英智人工智能智汇API（Application Programming Interface）是指一种开放的开发接口，让第三方服务或应用能够访问和调用英智的人工智能技术。通过这些API，开发者可以整合英智的人工智能功能，如自然语言处理、图像识别和情绪分析，来提高自己的产品或服务。\n\n例如，一家移动应用开发商可能会使用英智的语音识别API，让用户能够通过语音说话给应用进行交互。或者，一家教育平台可能会利用英智的情绪分析API，对学生的情绪变化进行实时监测和反馈等应用。\n\n通过API接口，第三方开发者可以使用英智的人工智能能力，为更加强大和有趣的应用或服务提供支持。这也促进了应用和服务之间的快速创新与集成。"      },      "finish_reason": "stop"    }  ],  "usage": {    "prompt_tokens": 31,    "completion_tokens": 186,    "total_tokens": 217  }}

方式二：设置stream（流式传输）

请求：

# sh
export API_BASE_URL="https://api.baystoneai.com/cognihub/service/66b97dd5fb19cc7397384f42"export API_KEY="Sy0FY5eH1jOaKtcfVvIxN1zsgRWED4IgfoFA"
curl -X POST $API_BASE_URL/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" -d '{    "model": "llama3.1",    "stream": true,    "messages": [        {"role": "system", "content": "You are a helpful assistant."},        {"role": "user", "content": "英智人工智能智汇API是什么？"}    ]}'

应答：​​​​​​​

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"英"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"人"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"工"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智能"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"汇"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"API"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"是一"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"种"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"可以"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"为"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"网站"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"、"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"应用"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"程序"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"和"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"其他"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"设备"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"提供"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"话"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"语"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"建议"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"、"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"文"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"本"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"分析"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"等"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"服务"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"的"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"软件"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"接"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"口"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"。"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop"}]}
data: [DONE]

下面，给大家介绍一下「英智LLM推理API平台」，以及平台具有的优势。

英智LLM推理API平台

英智LLM推理API是一种基于人工智能大语言模型的推理API服务，它可以帮助开发者简单灵活地使用市面上的所有人工智能大语言模型，为企业定制专属的智能化解决方案。

目前已支持包括Llama 3.1、Mistral Large 2、Qwen 2等在内的主流开源大模型的最新版本，同时兼容OpenAI API。

平台优势

• 免费注册：申请服务的免费套餐后可获取专属您的“API_KEY”，即可接入本服务来免费使用API。

• 在线使用：支持在线使用开源大模型。

• 兼容能力强：提供API调用方式，与OpenAI API接口兼容(只要支持OpenAI API的应用，都可以使用)。

• 推理速度快：基于NVIDIA高性能GPU算力，推理速度更快。

• 自然语言处理：通过对大量文本数据进行分析和理解，实现对自然语言的智能处理，如情感分析、文本分类、关键词提取等。

• 语义理解：通过对自然语言的深入理解，实现对用户意图的准确识别，从而提供更精准的信息服务。

• 智能问答：通过对大量知识库的学习和理解，实现对用户提问的智能回答，提高用户满意度。

• 文本生成：通过对大量文本数据的学习，实现对特定主题或场景的文本生成，如新闻稿、产品描述等。

• 语音识别与合成：通过对语音数据的分析，实现对语音的识别和合成，提高人机交互的自然度和便捷性。

• 图像识别与处理：通过对图像数据的分析和理解，实现对图像内容的识别和处理，如人脸识别、物体检测等。

免费使用，请访问：https://cognihub.baystoneai.com

加“英智未来”好友，免费体验更多AI产品

---