Xinference：大模型部署与分布式推理框架（五）语音模型、自定义模型、异常

Code1994

于 2024-09-28 09:15:00 发布

阅读量271

点赞数 6

文章标签：分布式人工智能大模型 AI大模型语音模型大模型部署 AI

本文链接：https://blog.csdn.net/Code1994/article/details/142593440

版权

五、语音模型

1）部署

语音模型是指用于处理和分析语音数据的机器学习或深度学习模型。这些模型旨在理解语音信号，执行各种任务，如语音识别、语音合成、说话人识别等。

使用 Xinference 将音频转换为文本或将文本转换为音频。

部署完成，查看运行的模型

2）使用API

Audio API提供了三种与音频交互的方法：

API端点	OpenAI兼容端点	描述
Transcription API	/v1/audio/transcriptions	转录终端将音频转录为输入语言
Translation API	/v1/audio/translations	翻译端点将音频转换为英文
Speech API	/v1/audio/speech	转录终端将音频转录为输入语言

可以通过 cURL、OpenAI Client 或者 Xinference 的 Python 客户端来尝试 Transcription API：

1.转录

curl -X 'POST' \
  'http://<XINFERENCE_HOST>:<XINFERENCE_PORT>/v1/audio/transcriptions' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "<MODEL_UID>",
    "file": "<audio bytes>",
  }'

import openai

client = openai.Client(
    api_key="cannot be empty",
    base_url="http://<XINFERENCE_HOST>:<XINFERENCE_PORT>/v1"
)
with open("speech.mp3", "rb") as audio_file:
    client.audio.transcriptions.create(
        model=<MODEL_UID>,
        file=audio_file,
    )

from xinference.client import Client

client = Client("http://<XINFERENCE_HOST>:<XINFERENCE_PORT>")

model = client.get_model("<MODEL_UID>")
with open("speech.mp3", "rb") as audio_file:
    model.transcriptions(audio=audio_file.read())

2.翻译

curl -X 'POST' \
  'http://<XINFERENCE_HOST>:<XINFERENCE_PORT>/v1/audio/translations' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "<MODEL_UID>",
    "file": "<audio bytes>",
  }'

import openai

client = openai.Client(
    api_key="cannot be empty",
    base_url="http://<XINFERENCE_HOST>:<XINFERENCE_PORT>/v1"
)
with open("speech.mp3", "rb") as audio_file:
    client.audio.translations.create(
        model=<MODEL_UID>,
        file=audio_file,
    )

from xinference.client import Client

client = Client("http://<XINFERENCE_HOST>:<XINFERENCE_PORT>")

model = client.get_model("<MODEL_UID>")
with open("speech.mp3", "rb") as audio_file:
    model.translations(audio=audio_file.read())

3.语音

curl -X 'POST' \
  'http://<XINFERENCE_HOST>:<XINFERENCE_PORT>/v1/audio/speech' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "<MODEL_UID>",
    "text": "<The text to generate audio for>",
    "voice": "echo",
    "stream": True,
  }'

import openai

client = openai.Client(
    api_key="cannot be empty",
    base_url="http://<XINFERENCE_HOST>:<XINFERENCE_PORT>/v1"
)
client.audio.speech.create(
    model=<MODEL_UID>,
    input=<The text to generate audio for>,
    voice="echo",
)

from xinference.client import Client

client = Client("http://<XINFERENCE_HOST>:<XINFERENCE_PORT>")

model = client.get_model("<MODEL_UID>")
model.speech(
    input=<The text to generate audio for>,
    voice="echo",
    stream: True,
)

六、自定义模型

Xinference 提供了一种灵活而全面的方式来集成、管理和应用自定义模型。

五、异常

1、异常1

在执行过程中，出现安装llama-cpp-python时，出现以下问题：

Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [76 lines of output]
      *** scikit-build-core 0.9.8 using CMake 3.22.1 (wheel)
      *** Configuring CMake...
      loading initial cache file /tmp/tmp0pe3_qsj/build/CMakeInit.txt
      -- The C compiler identification is GNU 11.4.0
      -- The CXX compiler identification is GNU 11.4.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/gcc - skipped


*** CMake build failed
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

访问：llama-cpp-python项目

目前llama-cpp-python最新版本v0.2.82-cu123，根据系统版本、python版本选择下载

wget https://github.com/abetlen/llama-cpp-python/releases/download/v0.2.82-cu124/llama_cpp_python-0.2.82-cp310-cp310-linux_x86_64.whl

执行以下命令安装

pip install llama_cpp_python-0.2.82-cp310-cp310-linux_x86_64.whl

2、异常2

在执行过程中，如果出现安装chatglm.cpp相关异常，执行如下操作解决。

访问：chatglm.cpp项目

目前 chatglm.cpp最新版本v0.4.0，根据系统版本、python版本选择下载

wget https://github.com/li-plus/chatglm.cpp/releases/download/v0.4.0/chatglm_cpp-0.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

执行以下命令安装

pip install chatglm_cpp-0.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

3、异常3

在LLM模型对话界面出现异常提示：

Error
[address=0.0.0.0:34387, pid=354238] GenerationMixin._get_logits_warper() missing 1 required positional argument: 'device'

后台运行异常提示：

Exception: [address=0.0.0.0:34387, pid=354238] GenerationMixin._get_logits_warper() missing 1 required positional argument: 'device'

参阅GitHub项目的issues，对transformers进行降级

(xinference) root@master:~# pip list | grep  transformers
sentence-transformers             3.0.1
transformers                      4.42.4
transformers-stream-generator     0.0.5
(xinference) root@master:~# pip install 'transformers==4.41.2'