【LocalAI】（11）：本地使用CPU运行LocalAI，一次运行5个能力，embedding模型，qwen-1.5-05b模型，生成图模型，语音转文字模型，文字转语音（只支持英文）

fly-iot

已于 2024-05-18 17:41:47 修改

阅读量970

点赞数 8

分类专栏： ai技术 chatgpt 大模型文章标签： embedding localai

于 2024-05-18 16:57:49 首次发布

本文链接：https://blog.csdn.net/freewebsys/article/details/138744350

版权

大模型同时被 3 个专栏收录

74 篇文章

订阅专栏

chatgpt

30 篇文章

订阅专栏

ai技术

19 篇文章

订阅专栏

1，LocalAI最新版本2.15.0发布

视频介绍地址

【LocalAI】（11）：本地使用CPU运行LocalAI，一次运行5个能力，embedding模型，qwen-1.5-05b模型，生成图模型，语音转文字互转

https://github.com/mudler/LocalAI/releases
在这里插入图片描述

docker pull localai/localai:latest-aio-cpu
latest-aio-cpu: Pulling from localai/localai
a8b1c5f80c2d: Pull complete 
76d649c52fc7: Pull complete 
0847d5ebb6fc: Pull complete 
6173b0bab578: Pull complete 
ca3860c4efab: Pull complete 
68fbe7d9ff84: Pull complete 
6e8396086688: Pull complete 
4f4fb700ef54: Pull complete 
24c1678944bc: Pull complete 
a086218fd406: Pull complete 
9ed5489da9cf: Pull complete 
bc41cf56e9e2: Pull complete 
8e96fba646af: Pull complete 
207df61f8993: Downloading [==>                                                ]  75.01MB/1.385GB
bca14863df35: Download complete 
920a01d0483f: Download complete 
06c6b26848bd: Download complete 
bc5c781d2ccd: Download complete 
620e046baa21: Download complete 
5137158bcb2f: Download complete 
e4204c3596cc: Download complete 


$ docker images
REPOSITORY        TAG              IMAGE ID       CREATED       SIZE
localai/localai   latest-aio-cpu   5d94a5bfe990   6 days ago    6.51GB

2，本地运行镜像

# 下载配置文件
git clone https://gitee.com/fly-llm/localai-run-llm.git

# 开启日志：
docker run -p 8080:8080 -e DEBUG=true --name local-ai -it \
-v `pwd`/aio:/aio -v `pwd`/models:/build/models localai/localai:latest-aio-cpu

3，可以图片生成

在这里插入图片描述

curl http://localhost:8080/v1/images/generations -H "Content-Type: application/json" -d '{
  "prompt": "floating hair, portrait, ((loli)), ((one girl)), cute face, hidden hands, asymmetrical bangs, beautiful detailed eyes, eye shadow, hair ornament, ribbons, bowties, buttons, pleated skirt, (((masterpiece))), ((best quality)), colorful|((part of the head)), ((((mutated hands and fingers)))), deformed, blurry, bad anatomy, disfigured, poorly drawn face, mutation, mutated, extra limb, ugly, poorly drawn hands, missing limb, blurry, floating limbs, disconnected limbs, malformed hands, blur, out of focus, long neck, long body, Octane renderer, lowres, bad anatomy, bad hands, text",
  "size": "256x256"
}'

5，vl的模型下载

11:14PM DBG Checking "bakllava.gguf" exists and matches SHA
11:14PM INF Downloading "https://hf-mirror.com/mys/ggml_bakllava-1/resolve/main/ggml-model-q4_k.gguf"
11:15PM INF Downloading /build/models/bakllava.gguf.partial: 8.0 KiB/4.1 GiB (0.00%) ETA: 2071h0m35.001986616s
11:15PM INF Downloading /build/models/bakllava.gguf.partial: 2.7 MiB/4.1 GiB (0.06%) ETA: 8h10m21.308472958s
11:15PM INF Downloading /build/models/bakllava.gguf.partial: 7.2 MiB/4.1 GiB (0.17%) ETA: 3h52m10.105726885s
11:15PM INF Downloading /build/models/bakllava.gguf.partial: 11.6 MiB/4.1 GiB (0.28%) ETA: 2h52m54.723827272s

但是界面中运行不了

6，文字转语音

在这里插入图片描述

直接生成文件下载：
在这里插入图片描述

也可以使用命令行执行：

curl http://localhost:8080/tts -H "Content-Type: application/json" -d '{
    "model":"tts-1",
    "input": "Hi, this is a test."
}'  --output ./test.mp3

# 即可下载到本地
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 61287  100 61228  100    59   185k    182 --:--:-- --:--:-- --:--:--  185k

7，聊天模型，qwen0.5b，使用cpu运行速度特别快

在这里插入图片描述

curl -X 'POST' 'http://0.0.0.0:8080/v1/chat/completions' \
-H 'Content-Type: application/json' -d '{
    "model": "qwen-1.5-0.5b-chat",
    "messages": [
        {
            "role": "user",
            "content": "北京景点?"
        }
    ],
    "temperature": 1
}'

8，进行embedding 测试

curl -X 'POST' http://0.0.0.0:8080/v1/embeddings \
 -H "Content-Type: application/json" \
 -d '{
  "input": "测试ebmeddings",
  "model": "text-embedding-ada-002"
}'