snowboy+sherpa-onnx+Rasa实现聊天机器人【语音助手】

最新推荐文章于 2024-05-14 16:14:45 发布

殷长庆

最新推荐文章于 2024-05-14 16:14:45 发布

阅读量1.7k

点赞数

分类专栏：语音助手文章标签：机器人人工智能语音助手

本文链接：https://blog.csdn.net/anshichuxuezhe/article/details/132159565

版权

语音助手专栏收录该内容

4 篇文章 2 订阅

订阅专栏

背景

本系列主要目标初步完成一款智能音箱的基础功能，包括语音唤醒、语音识别(语音转文字)、处理用户请求（比如查天气等，主要通过rasa自己定义意图实现）、语音合成(文字转语音)功能。

rasa主要在项目中完成接收语音识别内容，识别用户意图，根据意图执行相应动作，并将结果返回给调用端。

本文用到的一些安装包在snowboy那一篇的必要条件中已经完成了部分构建，rasa的api调用部分会把相关代码写到snowboy项目中，处理完用户操作之后会打印响应信息。

语音唤醒文章地址：

snowboy 自定义唤醒词实现语音唤醒【语音助手】_殷长庆的博客-CSDN博客

sherpa-onnx文章地址：

snowboy+新一代kaldi（k2-fsa）sherpa-onnx实现离线语音识别【语音助手】

参考文章

rasa安装（强烈建议按官网的步骤安装）：

Installing Rasa Open Source

rasa官方简单教程

Rasa Playground

rasa实例

https://github.com/RasaHQ/rasa/tree/main/examples/

rasa参考教程

对话机器人 Rasa 中文系列教程 - AI - 大象笔记

实践

下载安装Rasa

pip3 install -U pip

pip3 install rasa

安装完成之后可以执行rasa init建一个新项目

rasa init

init会创建一个文件夹，起名叫rasa就行

接下来rasa部分的代码放在了github中需要的可以下载

GitHub - Ycqing/rasa: This is a chat robot for beginners to learn the rasa API

调试Rasa

让它能听懂人话

添加依赖

apt install rustc && apt install cargo

rustc --version

export PATH="$HOME/.cargo/bin:$PATH"

安装结巴（如果用jieba提取tokens）

pip3 install jieba
pip3 install transformers

安装ducking（如果需要提取句子中的数）

docker run -d -p 8000:8000 rasa/duckling

安装语言模型

可能用到的其他安装包

torch尽量安装上，因为语音合成也要用到它，下面的语句二选一即可，只根下载速度有关

pip3 install torch torchvision torchaudio

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu118

语言模型也不是必要安装的，python 安装jieba之后也能解决大部分分词功能，不过这里我还是选择使用spacy，下载了中文的模型

pip3 install rasa[spacy]

# 下面这俩安装起来可能很慢，_sm,_md,_lg模型小中大;_trf用transformer训练的

# python3 -m spacy download en_core_web_md

# python3 -m spacy download zh_core_web_sm

注意：python -m spacy download zh_core_web_sm 这个命令由于网络问题可能下载非常慢，导致安装失败，我们可以直接去官网下载，我下载的是lg版本的模型，相对来说文件比较大，以下两个连接均可下载，不管是gz文件还是whl文件，下载下来用pip安装一下就好了。

Chinese · spaCy Models Documentation

Release zh_core_web_lg-3.6.0 · explosion/spacy-models · GitHub

pip install zh_core_web_lg-3.6.0.tar.gz

至此准备工作差不多了，接下来改文件内容，训练模型

训练Rasa

下面是主要配置项，其他的可以看github项目里的配置，因为有的字典太长了，只能在这里显示一小部分，只做说明作用。

config.yml

这套配置让rasa能读懂中英文、提前句子中的数值，需要ducking正常运行着

recipe: default.v1
assistant_id: mybot_default
language: zh
pipeline:
- name: "SpacyNLP"
  model: "zh_core_web_lg"
- name: "SpacyTokenizer"
- name: "SpacyFeaturizer"
- name: "RegexFeaturizer"
  use_word_boundaries: False
- name: RegexEntityExtractor
  case_sensitive: false
  use_lookup_tables: true
  use_word_boundaries: False
  use_regexes: True
- name: EntitySynonymMapper
- name: DucklingEntityExtractor
  url: http://localhost:8000
  dimensions:
    - number
- name: "SpacyEntityExtractor"
- name: "DIETClassifier"
  epochs: 100
policies:
  - name: RulePolicy

credentials.yml

让rasa允许http访问

# This file contains the credentials for the voice & chat platforms
# which your bot is using.
# https://rasa.com/docs/core/connectors/

rest:
#  # you don't need to provide anything here - this channel doesn't
#  # require any credentials

rasa:
  url: "http://localhost:5005"

domain.yml

本文主要在官方示例formbot的基础上对intent、entity、slot、action、form部分添加了天气相关的配置

version: "3.1"
intents:
  - request_restaurant:
      use_entities: []
  - chitchat:
      use_entities: []
  - inform
  - affirm
  - deny
  - stop
  - thankyou
  - greet
  - bot_challenge
  - weather_city
  - weather

entities:
  - cuisine
  - number
  - feedback
  - seating
  - city

slots:
  cuisine:
    type: text
    influence_conversation: false
    mappings:
    - type: from_entity
      entity: cuisine
  num_people:
    type: float
    influence_conversation: false
    mappings:
    - type: from_entity
      entity: number
      intent: [inform, request_restaurant]
  outdoor_seating:
    type: text
    influence_conversation: false
    mappings:
    - type: from_entity
      entity: seating
    - type: from_intent
      intent: affirm
      value: true
      conditions:
      - active_loop: restaurant_form
        requested_slot: outdoor_seating
    - type: from_intent
      intent: deny
      value: false
      conditions:
      - active_loop: restaurant_form
        requested_slot: outdoor_seating
  preferences:
    type: text
    influence_conversation: false
    mappings:
    - type: from_intent
      intent: deny
      value: no additional preferences
      conditions:
      - active_loop: restaurant_form
        requested_slot: preferences
    - type: from_text
      not_intent: deny
      conditions:
      - active_loop: restaurant_form
        requested_slot: preferences
  feedback:
    type: text
    influence_conversation: false
    mappings:
    - type: from_entity
      entity: feedback
    - type: from_text
      conditions:
      - active_loop: restaurant_form
        requested_slot: feedback
  city:
    type: text
    influence_conversation: false
    mappings:
    - type: from_entity
      entity: city
      value: ip
      intent: weather_city
    - type: from_intent
      value: ip
      intent: weather

responses:
  utter_ask_cuisine:
    - text: "What cuisine?"
  utter_ask_num_people:
    - text: "How many people?"
  utter_ask_outdoor_seating:
    - text: "Do you want to sit outside?"
  utter_ask_preferences:
    - text: "Please provide additional preferences"
  utter_ask_feedback:
    - text: "Please give your feedback on your experience so far"
  utter_submit:
  - text: "All done!"
  utter_slots_values:
    - text: "I am going to run a restaurant search using the following parameters:\n
             - cuisine: {cuisine}\n
             - num_people: {num_people}\n
             - outdoor_seating: {outdoor_seating}\n
             - preferences: {preferences}\n
             - feedback: {feedback}"
  utter_noworries:
    - text: "You are welcome :)"
  utter_chitchat:
    - text: "chitchat"
  utter_ask_continue:
    - text: "Do you want to continue?"
  utter_wrong_cuisine:
    - text: "Cuisine type is not in the database, please try again"
  utter_wrong_num_people:
    - text: "Number of people should be a positive integer, please try again"
  utter_wrong_outdoor_seating:
    - text: "Could not convert input to boolean value, please try again"
  utter_default:
    - text: "Sorry, I didn't understand you, please try input something else"
  utter_greet:
    - text: "Hello! I am restaurant search assistant! How can I help?"
  utter_iamabot:
    - text: "I am a bot, powered by Rasa."
  utter_restart:
    - text: "restarted"

actions:
- validate_restaurant_form
- validate_weather_form

forms:
  restaurant_form:
    ignored_intents:
    - chitchat
    required_slots:
      - cuisine
      - num_people
      - outdoor_seating
      - preferences
      - feedback
  weather_form:
    ignored_intents:
    - chitchat
    required_slots:
      - city

session_config:
  session_expiration_time: 60  # value in minutes
  carry_over_slots_to_new_session: true

endpoints.yml

action_endpoint:
    url: http://localhost:5055/webhook

actions.py

首先去心知天气注册账号，主要用他们的免费api，一分钟请求20次天气。

心知天气 - 高精度气象数据 - 天气数据API接口 - 行业气象解决方案

天气查询由ValidateWeatherForm处理

from typing import Dict, Text, Any, List, Union

from rasa_sdk import Tracker
from rasa_sdk.executor import CollectingDispatcher
from rasa_sdk.forms import FormValidationAction

import ssl
from urllib import request, parse
import json

class ValidateRestaurantForm(FormValidationAction):
    """Example of a form validation action."""

    def name(self) -> Text:
        return "validate_restaurant_form"

    @staticmethod
    def cuisine_db() -> List[Text]:
        """Database of supported cuisines."""

        return [
            "caribbean",
            "chinese",
            "french",
            "greek",
            "indian",
            "italian",
            "mexican",
        ]

    @staticmethod
    def is_int(string: Text) -> bool:
        """Check if a string is an integer."""

        try:
            int(string)
            return True
        except ValueError:
            return False

    def validate_cuisine(
        self,
        value: Text,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: Dict[Text, Any],
    ) -> Dict[Text, Any]:
        """Validate cuisine value."""
        print(value)

        if value.lower() in self.cuisine_db():
            # validation succeeded, set the value of the "cuisine" slot to value
            return {"cuisine": value}
        else:
            dispatcher.utter_message(response="utter_wrong_cuisine")
            # validation failed, set this slot to None, meaning the
            # user will be asked for the slot again
            return {"cuisine": None}

    def validate_num_people(
        self,
        value: Text,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: Dict[Text, Any],
    ) -> Dict[Text, Any]:
        """Validate num_people value."""

        if self.is_int(value) and int(value) > 0:
            return {"num_people": value}
        else:
            dispatcher.utter_message(response="utter_wrong_num_people")
            # validation failed, set slot to None
            return {"num_people": None}

    def validate_outdoor_seating(
        self,
        value: Text,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: Dict[Text, Any],
    ) -> Dict[Text, Any]:
        """Validate outdoor_seating value."""

        if isinstance(value, str):
            if "out" in value:
                # convert "out..." to True
                return {"outdoor_seating": True}
            elif "in" in value:
                # convert "in..." to False
                return {"outdoor_seating": False}
            else:
                dispatcher.utter_message(response="utter_wrong_outdoor_seating")
                # validation failed, set slot to None
                return {"outdoor_seating": None}

        else:
            # affirm/deny was picked up as True/False by the from_intent mapping
            return {"outdoor_seating": value}


class ValidateWeatherForm(FormValidationAction):
    """Example of a form validation action."""

    def name(self) -> Text:
        return "validate_weather_form"

    @staticmethod
    def getLocation(plocation):
        location = plocation if len(plocation)>0 else 'ip' # 默认根据当前IP查询，所查询的位置，可以使用城市拼音、v3 ID、经纬度等
        return location

    @staticmethod
    def fetchWeather(location):
        params = parse.urlencode({
            'key': 'SiJLwLBuHlDyGpYeA',  # API key
            'location': location,
            'language': 'zh-Hans', # 查询结果的返回语言
            'unit': 'c'  # 单位
        })
        ssl._create_default_https_context = ssl._create_unverified_context
        gcontext = ssl._create_unverified_context()
        req = request.Request('{api}?{params}'.format(api='https://api.seniverse.com/v3/weather/now.json', params=params))# API URL，可替换为其他 URL
        response = request.urlopen(req, context=gcontext).read().decode('UTF-8')
        print(response)
        data = json.loads(response)
        city = data["results"][0]["location"]["name"]
        weather = data["results"][0]["now"]["text"]
        temperature = data["results"][0]["now"]["temperature"]
        return city + "今天" + weather + " 温度 " + temperature + "摄氏度"
    def validate_city(
        self,
        value: Text,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: Dict[Text, Any],
    ) -> Dict[Text, Any]:
        """Validate cuisine value."""
        location = self.getLocation(value)
        try:
            print(location)
            weather = self.fetchWeather(location)
            dispatcher.utter_message(text=weather)
        except:
            dispatcher.utter_message(text="未查询到"+value+"天气")
        return {"city": None}

city.yml

开始使用 👋 · 心知科技

主要是中国的市县区数据，全部的数据可以在心知天气官网下载，处理成yml形式

version: "3.1"
nlu:
- lookup: city
  examples: |
    - 北京
    - 东城
    - 西城
    - 朝阳
    - 丰台
    - 石景山

nlu.yml

尽量列举用户可能输入的句子，方便rasa学习

version: "3.1"
nlu:
- intent: greet
  examples: |
    - Hi
    - Hey
    - Hi bot
    - Hey bot
    - Hello
    - Good morning
    - hi again
    - hi folks
    - hi Mister
    - hi pal!
    - hi there
    - greetings
    - hello everybody
    - hello is anybody there
    - hello robot
    - hallo
    - heeey
    - hi hi
    - hey
    - hey hey
    - hello there
    - hi
    - hello
    - yo
    - hola
    - hi?
    - hey bot!
    - hello friend

- intent: request_restaurant
  examples: |
    - im looking for a restaurant
    - can i get [swedish](cuisine) food in any area
    - a restaurant that serves [caribbean](cuisine) food
    - id like a restaurant
    - im looking for a restaurant that serves [mediterranean](cuisine) food
    - can i find a restaurant that serves [chinese](cuisine)
    - i am looking for any place that serves [indonesian](cuisine) food for three
    - i need to find a restaurant
    - uh im looking for a restaurant that serves [kosher](cuisine) food
    - uh can i find a restaurant and it should serve [brazilian](cuisine) food
    - im looking for a restaurant serving [italian](cuisine) food
    - restaurant please
    - i'd like to book a table for two with [spanish](cuisine) cuisine
    - i need a table for 4
    - book me a table for three at the [italian](cuisine) restaurant
    - can you please book a table for 5?
    - I would like to book a table for 2
    - looking for a table at the [mexican](cuisine) restaurant for five
    - find me a table for 7 people
    - Can I get a table for four at the place which server [greek](cuisine) food?

- intent: affirm
  examples: |
    - yeah a cheap restaurant serving international food
    - correct
    - ye
    - uh yes
    - let's do it
    - yeah
    - uh yes
    - um yes
    - yes knocking
    - that's correct
    - yes yes
    - right
    - yea
    - yes
    - yes right
    - yes and i dont care
    - right on
    - i love that

- intent: deny
  examples: |
    - no
    - no new selection
    - no thanks
    - no thank you
    - uh no
    - breath no
    - do you have something else
    - no this does not work for me

- intent: inform
  examples: |
    - [afghan](cuisine) food
    - how bout [asian oriental](cuisine)
    - what about [indian](cuisine) food
    - uh how about [turkish](cuisine) type of food
    - um [english](cuisine)
    - im looking for [tuscan](cuisine) food
    - id like [moroccan](cuisine) food
    - [seafood](cuisine)
    - [french](cuisine) food
    - serves [british](cuisine) food
    - id like [canapes](cuisine)
    - serving [jamaican](cuisine) food
    - um what about [italian](cuisine) food
    - im looking for [corsica](cuisine) food
    - im looking for [world](cuisine) food
    -  serves [french](cuisine) food
    - how about [indian](cuisine) food
    - can i get [chinese](cuisine) food
    - [irish](cuisine) food
    - [english](cuisine) food
    - [spanish](cuisine) food
    - how bout one that serves [portuguese](cuisine) food and is cheap
    - [german](cuisine)
    - [korean](cuisine) food
    - im looking for [romanian](cuisine) food
    -  serves [canapes](cuisine) food
    - [gastropub](cuisine)
    - i want [french](cuisine) food
    - how about [modern european](cuisine) type of food
    - it should serve [scandinavian](cuisine) food
    - how [european](cuisine)
    - how about [european](cuisine) food
    - serves [traditional](cuisine) food
    - [indonesian](cuisine) food
    - [modern european](cuisine)
    - serves [brazilian](cuisine)
    - i would like [modern european](cuisine) food
    - looking for [lebanese](cuisine) food
    - [portuguese](cuisine)
    - [european](cuisine)
    - i want [polish](cuisine) food
    - id like [thai](cuisine)
    - i want to find [moroccan](cuisine) food
    - [afghan](cuisine)
    - [scottish](cuisine) food
    - how about [vietnamese](cuisine)
    - hi im looking for [mexican](cuisine) food
    - how about [indian](cuisine) type of food
    - [polynesian](cuisine) food
    - [mexican](cuisine)
    - instead could it be for four people
    - any [japanese](cuisine) food
    - what about [thai](cuisine) food
    - how about [asian oriental](cuisine) food
    - im looking for [japanese](cuisine) food
    - im looking for [belgian](cuisine) food
    - im looking for [turkish](cuisine) food
    - serving [corsica](cuisine) food
    - serving [gastro pub](cuisine:gastropub)
    - is there [british](cuisine) food
    - [world](cuisine) food
    - im looking for something serves [japanese](cuisine) food
    - id like a [greek](cuisine)
    - im looking for [malaysian](cuisine) food
    - i want to find [world](cuisine) food
    - serves [pan asian](cuisine:asian) food
    - looking for [afghan](cuisine) food
    - that serves [portuguese](cuisine) food
    - [asian oriental](cuisine:asian) food
    - [russian](cuisine) food
    - [corsica](cuisine)
    - [asian oriental](cuisine:asian)
    - serving [basque](cuisine) food
    - how about [italian](cuisine)
    - looking for [spanish](cuisine) food in the center of town
    - it should serve [gastropub](cuisine) food
    - [welsh](cuisine) food
    - i want [vegetarian](cuisine) food
    - im looking for [swedish](cuisine) food
    - um how about [chinese](cuisine) food
    - [world](cuisine) food
    - can i have a [seafood](cuisine) please
    - how about [italian](cuisine) food
    - how about [korean](cuisine)
    - [corsica](cuisine) food
    - [scandinavian](cuisine)
    - [vegetarian](cuisine) food
    - what about [italian](cuisine)
    - how about [portuguese](cuisine) food
    - serving [french](cuisine) food
    - [tuscan](cuisine) food
    - how about uh [gastropub](cuisine)
    - im looking for [creative](cuisine) food
    - im looking for [malaysian](cuisine) food
    - im looking for [unusual](cuisine) food
    - [danish](cuisine) food
    - how about [spanish](cuisine) food
    - im looking for [vietnamese](cuisine) food
    - [spanish](cuisine)
    - a restaurant serving [romanian](cuisine) food
    - im looking for [lebanese](cuisine) food
    - [italian](cuisine) food
    - a restaurant with [afghan](cuisine) food
    - im looking for [traditional](cuisine) food
    - uh i want [cantonese](cuisine) food
    - im looking for [thai](cuisine)
    - i want to seat [outside](seating)
    - i want to seat [inside](seating)
    - i want to seat [outdoor](seating)
    - i want to seat [indoor](seating)
    - let's go [inside](seating)
    - [inside](seating)
    - [outdoor](seating)
    - prefer sitting [indoors](seating)
    - I would like to seat [inside](seating) please
    - I prefer sitting [outside](seating)
    - my feedback is [good](feedback)
    - my feedback is [great](feedback)
    - it was [terrible](feedback)
    - i consider it [success](feedback)
    - you are [awful](feedback)
    - for ten people
    - 2 people
    - for three people
    - just one person
    - book for seven people
    - 2 please
    - nine people

- intent: thankyou
  examples: |
    - um thank you good bye
    - okay cool uh good bye thank you
    - okay thank you good bye
    - you rock
    - and thats all thank you and good bye
    - thank you and good bye
    - sorry about my mistakes thank you good bye
    - noise thank you good bye
    - thank you goodbye noise
    - okay thank you goodbye
    - uh thank you good bye
    - thank you goodbye
    - thank you goodbye noise thank you goodbye
    - breath thank you goodbye
    - thank you
    - okay thank you
    - thanks goodbye
    - ah thank you goodbye
    - thank you noise
    - thank you good bye
    - breath thank you very much goodbye
    - thanks
    - noise thank you goodbye
    - unintelligible thank you goodbye
    - uh okay thank you good bye
    - thank you bye
    - um okay thank you good bye

- intent: chitchat
  examples: |
    - can you share your boss with me?
    - i want to get to know your owner
    - i want to know the company which designed you
    - i want to know the company which generated you
    - i want to know the company which invented you
    - i want to know who invented you
    - May I ask who invented you?
    - please tell me the company who created you
    - please tell me who created you
    - tell me more about your creators
    - tell me more about your founders
    - Ahoy matey how are you?
    - are you alright
    - are you having a good day
    - Are you ok?
    - are you okay
    - Do you feel good?
    - how are things going
    - how are things with you?
    - How are things?
    - how are you
    - how are you doing
    - how are you doing this morning
    - how are you feeling
    - how are you today
    - How are you?
    - How is the weather today?
    - What's the weather like?
    - How is the weather?
    - What is the weather at your place?
    - Do you have good weather?
    - Is it raining?
    - What's it like out there?
    - Is it hot or cold?
    - Beautiful day, isn't it?
    - What's the weather forecast?
    - Is it quite breezy outside?

- intent: stop
  examples: |
    - ok then you cant help me
    - that was shit, you're not helping
    - you can't help me
    - you can't help me with what i need
    - i guess you can't help me then
    - ok i guess you can't help me
    - that's not what i want
    - ok, but that doesnt help me
    - this is leading to nothing
    - this conversation is not really helpful
    - you cannot help me with what I want
    - I think you cant help me
    - hm i don't think you can do what i want
    - stop
    - stop go back
    - do you get anything?
    - and you call yourself bot company? pff
    - and that's it?
    - nothing else?

- intent: bot_challenge
  examples: |
    - are you a bot?
    - are you a human?
    - am I talking to a bot?
    - am I talking to a human?

- intent: weather
  examples: |
    - 天气
    - 天气情况
    - 天气状况
    - 今天空气怎么样？
    - 今天多少度？
    - 今天气温？
    - 今天天气怎么样？
    - 今天温度
    - 今天天气
    - 今天下雨吗？
    - 今天冷吗？
    - 今天热吗？
    - 今天需要带伞吗？

- intent: weather_city
  examples: |
    - [北京](city)天气情况
    - [上海](city)今天天气情况？
    - [深圳](city)今天温度
    - [杭州](city)今天下雨吗？
    - [青岛](city)今天冷吗？
    - [乌鲁木齐](city)今天热吗？
    - [呼和浩特](city)今天需要带伞吗？
    - [北京](city)市天气情况
    - [上海](city)市今天天气情况？
    - [深圳](city)市今天温度
    - [杭州](city)市今天下雨吗？
    - [青岛](city)市今天冷吗？
    - [乌鲁木齐](city)市今天热吗？
    - [呼和浩特](city)市今天需要带伞吗？
    - 现在[青岛](city)市空气怎么样？
    - 现在[青州](city)市多少度？
    - 现在[潍坊](city)市气温？
    - 现在[苏州](city)市天气怎么样？
    - 现在[滨州](city)市温度
    - 现在[淄博](city)市天气
    - 现在[洛阳](city)市下雨吗？
    - 现在[武汉](city)市冷吗？
    - 现在[重庆](city)市热吗？
    - 现在[成都](city)市需要带伞吗？

rules.yml

rasa识别到weather意图，会调用weather的action

version: "3.1"
rules:
  - rule: Greet user
    steps:
      - intent: greet
      - action: utter_greet

  - rule: Thank you
    steps:
      - intent: thankyou
      - action: utter_noworries

  - rule: Bot challenge
    steps:
      - intent: bot_challenge
      - action: utter_iamabot

  - rule: Chitchat
    steps:
      - intent: chitchat
      - action: utter_chitchat

  - rule: activate restaurant form
    steps:
      - intent: request_restaurant
      - action: restaurant_form
      - active_loop: restaurant_form

  - rule: submit form
    condition:
    - active_loop: restaurant_form
    steps:
      - action: restaurant_form
      - active_loop: null
      - action: utter_submit
      - action: utter_slots_values

  - rule: ask weather with city
    steps:
      - intent: weather_city
      - action: weather_form

  - rule: ask weather with city
    steps:
      - intent: weather
      - action: weather_form

stories.yml

version: "3.1"
stories:
  - story: stop form + continue
    steps:
      - intent: request_restaurant
      - action: restaurant_form
      - active_loop: restaurant_form
      - intent: stop
      - action: utter_ask_continue
      - intent: affirm
      - action: restaurant_form
      - active_loop: null
      - action: utter_submit
      - action: utter_slots_values

  - story: stop form + stop
    steps:
      - intent: request_restaurant
      - action: restaurant_form
      - active_loop: restaurant_form
      - intent: stop
      - action: utter_ask_continue
      - intent: deny
      - action: action_deactivate_loop
      - active_loop: null

配置部分至此完成了，下面就是训练

训练

训练如不报错会在models下生成一个模型文件，本项目模型文件大小差不多在20M左右

cd /home/test/rasa/

rasa train

运行action

nohup rasa run actions > runaction.log &

测试rasa

rasa shell

等待启动完成之后输入“天气”，看能不能出现当前城市的天气情况

运行rasa

nohup rasa run --enable-api --auth-token thisismysecret > runserver.log &

enable-api 允许http访问rasa服务，auth-token指定一个密码

集成snowboy

rasabot.py

建一个python文件，调用rasa api，获取响应结果

cd /home/test/snowboy/examples/Python3/

touch rasabot.py

vim rasabot.py

编辑以下内容

from urllib import request, parse
import json
headers = {'content-type': 'application/json'}

# 问问rasa
# text 询问的内容
# return rasa响应结果
def ask(text):
  try:
    data = json.dumps({'sender':'snowboy','message':text})
    req = request.Request(url='http://localhost:5005/webhooks/rest/webhook?token=thisismysecret', data=bytes(data, 'utf8'), headers=headers)
    response = request.urlopen(req).read().decode('utf-8')
    resp = json.loads(response)
    return resp[0]['text']
  except:
    return '你帮我翻译翻译什么叫做：'+text

保存之后修改demo.py(在上一篇的基础上稍加改动)，把离线识别的结果传入ask方法，接收rasa响应结果

demo.py

vim demo.py

import snowboydecoder
import signal
import os
import offlinedecode
import rasabot

interrupted = False

def signal_handler(signal, frame):
    global interrupted
    interrupted = True

def interrupt_callback():
    global interrupted
    return interrupted


# 初始化语音识别
offlinedecode.init()

# 唤醒词模型文件
model = '../../model/hotword.pmdl'

# capture SIGINT signal, e.g., Ctrl+C
signal.signal(signal.SIGINT, signal_handler)

detector = snowboydecoder.HotwordDetector(model, sensitivity=0.5)
print('Listening... Press Ctrl+C to exit')

# 录音之后的回调
# fname 音频文件路径
def audio_recorder_callback(fname):
    text = offlinedecode.asr(fname)
    # 打印识别内容
    print(text)
    # 问问rasa
    resp = rasabot.ask(text)
    print(resp)
    # 删除录音文件
    if isinstance(fname, str) and os.path.exists(fname):
        if os.path.isfile(fname):
            os.remove(fname)


# main loop
detector.start(detected_callback=snowboydecoder.play_audio_file,
               audio_recorder_callback=audio_recorder_callback,
               interrupt_check=interrupt_callback,
               sleep_time=0.03)

detector.terminate()

编辑完成保存，然后测试是否有识别成功

测试集成效果

cd /home/test/snowboy/examples/Python3/
 
python demo.py

成功之后会打印识别内容，然后删除本地录音文件。

殷长庆

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
snowboy+sherpa-onnx+Rasa实现聊天机器人【语音助手】

本系列主要目标初步完成一款智能音箱的基础功能，包括语音唤醒、语音识别(语音转文字)、处理用户请求（比如查天气等，主要通过rasa自己定义意图实现）、语音合成(文字转语音)功能。rasa主要在项目中完成接收语音识别内容，识别用户意图，根据意图执行相应动作，并将结果返回给调用端。本文用到的一些安装包在snowboy那一篇的必要条件中已经完成了部分构建，rasa的api调用部分会把相关代码写到snowboy项目中，处理完用户操作之后会打印响应信息。
复制链接

扫一扫