智能客服搭建(3) - MRCP Server 与 FreeSWITCH 对接

小爱玄策

已于 2023-03-15 19:22:32 修改

阅读量4.7k

点赞数 4

分类专栏：呼叫中心文章标签： linux

于 2021-07-31 20:33:28 首次发布

本文链接：https://blog.csdn.net/initiallht/article/details/119281023

版权

呼叫中心专栏收录该内容

6 篇文章 30 订阅

订阅专栏

文章目录

1. 写在前面
2. MRCP Server 配置与测试
- 2.1 MRCP Server 配置
- 2.2 MRCP Server 测试
3. FreeSwitch 配置
4. 效果测试

1. 写在前面

上一篇文章中，对MRCP Server的开发进行了介绍，下面讲解如何通过配置让FreeSWITCH能够与MRCP Server进行对接，获得电话呼叫的语音识别结果。

2. MRCP Server 配置与测试

2.1 MRCP Server 配置

cd /usr/local/unimrcp/conf
vim unimrcpserver.xml

以下xml只展示了需要进行调整的部分，将*.so的名称添加到配置文件中enable="true"

<!-- Factory of plugins (MRCP engines) -->
<plugin-factory>
  <engine id="Demo-Synth-1" name="demosynth" enable="false"/>
  <engine id="XFyun-Synth-1" name="xfyunsynth" enable="true"/>

  <engine id="Demo-Recog-1" name="demorecog" enable="false"/>
  <!--engine id="XFyun-Recog-1" name="xfyunrecog" enable="true"/-->
  <engine id="Xxx-Recog-1" name="XxxRecog" enable="true"/>

  <engine id="Demo-Verifier-1" name="demoverifier" enable="true"/>
  <engine id="Recorder-1" name="mrcprecorder" enable="true"/>
</plugin-factory>

2.2 MRCP Server 测试

完成配置后，启动UniMRCP Server

cd /usr/local/unimrcp/bin
./unimrcpserver -o 3

运行结果如下图：

启动UniMRCP Client，输入run recog进行测试。

cd /usr/local/unimrcp/bin
./unimrcpclient
>help
usage:

- run [app_name] [profile_name] (run demo application)
       app_name is one of 'synth', 'recog', 'bypass', 'discover'
       profile_name is one of 'uni2', 'uni1', ...

       examples: 
           run synth
           run recog
           run synth uni1
           run recog uni1

- loglevel [level] (set loglevel, one of 0,1...7)

- quit, exit

3. FreeSwitch 配置

3.1. 配置 mrcp profile

创建新的 mrcp服务连接的 配置文件 xiaoi-mrcp-v2.xml

cd /usr/local/freeswitch/conf/mrcp_profiles
vim xiaoi-mrcp-v2.xml

<include>
  <!-- UniMRCP Server MRCPv2 -->
  <!-- 后面我们使用该配置文件，均使用 name 作为唯一标识，而不是文件名 -->
  <profile name="xiaoi-mrcp-v2" version="2">
    <!-- MRCP 服务器地址 -->
    <param name="server-ip" value="192.168.160.2"/>
    <!-- MRCP SIP 端口号 -->
    <param name="server-port" value="8060"/>
    <param name="resource-location" value=""/>

    <!-- FreeSWITCH IP、端口以及 SIP 传输方式 -->
    <param name="client-ip" value="192.168.160.58"/>
    <!-- 注意，当配置多个mrcp server的时候，该端口不能冲突，否则可能不可用 -->
    <param name="client-port" value="5089"/>
    <param name="sip-transport" value="udp"/>

    <param name="speechsynth" value="speechsynthesizer"/>
    <param name="speechrecog" value="speechrecognizer"/>
    <!--param name="rtp-ext-ip" value="auto"/-->
    <param name="rtp-ip" value="192.168.160.58"/>
    <param name="rtp-port-min" value="4000"/>
    <param name="rtp-port-max" value="5000"/>
    <param name="codecs" value="PCMU PCMA L16/96/8000"/>

    <!-- Add any default MRCP params for SPEAK requests here -->
    <synthparams>
    </synthparams>

    <!-- Add any default MRCP params for RECOGNIZE requests here -->
    <recogparams>
      <!--param name="start-input-timers" value="false"/-->
    </recogparams>
  </profile>
</include>

3.2. 配置 mrcp conf

配置/usr/local/freeswitch/conf/autoload_configs/unimrcp.conf.xml文件：

cd /usr/local/freeswitch/conf/autoload_configs
vim unimrcp.conf.xml

配置文件如下，al-unimrcpv2为默认的，进行注释，下面配置中，只配置了xiaoi-mrcp-v2，注意，该名字与1配置的xiaoi-mrcp-v2.xml中profile name名称一致。

<configuration name="unimrcp.conf" description="UniMRCP Client">
  <settings>

    <!--al-unimrcpv2 UniMRCP profile to use for TTS -->
    <param name="default-tts-profile" value="al-unimrcpv2"/>
    <!-- <param name="default-tts-profile" value="xiaoi-mrcp-v2"/> -->
    
    <!-- UniMRCP profile to use for ASR -->
    <!-- <param name="default-asr-profile" value="al-unimrcpv2"/> -->
    <param name="default-asr-profile" value="xiaoi-mrcp-v2"/>
    
    <!-- UniMRCP logging level to appear in freeswitch.log.  Options are:
         EMERGENCY|ALERT|CRITICAL|ERROR|WARNING|NOTICE|INFO|DEBUG -->
    <param name="log-level" value="DEBUG"/>
    <!-- Enable events for profile creation, open, and close -->
    <param name="enable-profile-events" value="false"/>

    <param name="max-connection-count" value="100"/>
    <param name="offer-new-connection" value="1"/>
    <param name="request-timeout" value="3000"/>
  </settings>

  <profiles>
    <X-PRE-PROCESS cmd="include" data="../mrcp_profiles/*.xml"/>
  </profiles>

</configuration>

注意，完成配置后，不能直接 F6 reloadxml 需要重启FreeSWITCH才生效

systemctl restart freeswitch

3.3. 配置拨号任务

在/usr/local/freeswitch/conf/dialplan/default.xml里新增如下配置：

<extension name="unimrcp">
    <condition field="destination_number" expression="^5001$">
   	    <action application="answer"/>
        <action application="lua" data="asr.lua"/>
    </condition>
</extension>

lua脚本存放地址及内容，可根据需要调整：

cd /usr/local/freeswitch/scripts
vim asr.lua

session:answer()

--freeswitch.consoleLog("INFO", "Called extension is '".. argv[1]"'\n")
welcome = "ivr/ivr-welcome_to_freeswitch.wav"
menu = "ivr/ivr-this_ivr_will_let_you_test_features.wav"
--
grammar = "hello"
no_input_timeout = 80000
recognition_timeout = 80000
confidence_threshold = 0.2
--
session:streamFile(welcome)
--freeswitch.consoleLog("INFO", "Prompt file is \n")
tryagain = 1
 while (tryagain == 1) do
--
       session:execute("play_and_detect_speech",menu .. "detect:unimrcp {start-input-timers=false,no-input-timeout=" .. no_input_timeout .. ",recognition-timeout=" .. recognition_timeout .. "}" .. grammar)
       xml = session:getVariable('detect_speech_result')
--
       if (xml == nil) then
               freeswitch.consoleLog("CRIT","Result is 'nil'\n")
               tryagain = 0
       else
               freeswitch.consoleLog("CRIT","Result is '" .. xml .. "'\n")
               tryagain = 0
    end
end
session:hangup()

从上面的lua脚本能看出，会加载grammar语法文件，在/usr/local/freeswitch/grammar目录新增hello.gram语法文件，可以为空语法文件须满足语音识别语法规范1.0标准（简称 SRGS1.0），该语法文件 ASR 引擎在进行识别时可以使用。

<!-- grammar  hello.gram -->
<?xml version="1.0" encoding="utf-8" ?>
<grammar version="1.0" xml:lang="zh-cn" root="digit" tag- format="semantics/1.0" mode="continuous" xmlns="http://www.w3.org/2001/06/grammar">
  <!-- 这些都是必不可少的,做一个假配置-->
  <rule id="digit">
    <one-of>
     <item>one</item>
     <item>two</item>
    </one-of>
  </rule>
</grammar>

4. 效果测试

完成以上配置后，使用sip软电话，拨打配置的号码5001，在fs端fs_cli查看log

2021-05-19 14:43:37.981898 [DEBUG] apt_task.c:265 () Signal Message to [MRCP Client] [0x7f6500023ec0;2;3]
2021-05-19 14:43:37.981898 [DEBUG] apt_poller_task.c:251 () Wait for Messages [MRCPv2ConnectionAgent]
2021-05-19 14:43:37.981898 [DEBUG] apt_task.c:337 () Process Message [MRCP Client] [0x7f6500023ec0;2;3]
2021-05-19 14:43:37.981898 [INFO] mrcp_client_session.c:516 (ASR-13) Raise App MRCP Event ASR-13 <ea2c920254a0462d>
2021-05-19 14:43:37.981898 [DEBUG] mod_unimrcp.c:3675 (ASR-13) RECOGNITION COMPLETE, Completion-Cause: 000
2021-05-19 14:43:37.981898 [DEBUG] mod_unimrcp.c:3684 (ASR-13) Recognition result is not null-terminated.  Appending null terminator.
2021-05-19 14:43:37.981898 [DEBUG] mod_unimrcp.c:2733 (ASR-13) ASR adding result headers
2021-05-19 14:43:37.981898 [DEBUG] mod_unimrcp.c:2628 (ASR-13) result:

<?xml version="1.0"?>
<result>
  <interpretation confidence="99">
    <instance>
       <engineName>Xxx</engineName>
       <engineStartTime></engineStartTime>
       <result>喂喂喂讲话.</result>
       <beginTime></beginTime>
       <endTime></endTime>
    </instance>
    <input mode="speech">喂喂喂讲话.</input>
  </interpretation>
</result>

后续开发中，使用mod_event_socket的内连模式，对fs事件进行监听，在play_and_detect_speech 应用执行完成的事件CHANNEL_EXECUTE_COMPLETE中就能获取到语音识别结果。

socket 连接命令与监听命令如下：

nc 127.0.0.1 8021
auth ClueCon
event plain CHANNEL_EXECUTE_COMPLETE

监听事件的结果如下（便于查看，进行了URL解码，省略了该事件的部分细节内容）

Content-Length: 9403
Content-Type: text/event-plain

Event-Name: CHANNEL_EXECUTE_COMPLETE
Core-UUID: 6f535b68-8025-4727-a210-70c2278501f9
FreeSWITCH-Hostname: freeswitch-seat
FreeSWITCH-Switchname: freeswitch-seat
FreeSWITCH-IPv4: 192.168.160.58
FreeSWITCH-IPv6: ::1
......
variable_current_application: play_and_detect_speech
variable_playback_last_offset_pos: 29760
variable_playback_seconds: 3
variable_playback_ms: 3970
variable_playback_samples: 31765
variable_detect_speech_result: <?xml version="1.0"?>
<result>
  <interpretation confidence="99">
    <instance>
       <engineName>Xxx</engineName>
       <engineStartTime></engineStartTime>
       <result>喂喂喂讲话.</result>
       <beginTime></beginTime>
       <endTime></endTime>
    </instance>
    <input mode="speech">喂喂喂讲话.</input>
  </interpretation>
</result>
variable_current_application_response: DONE
Application: play_and_detect_speech
Application-Data: ivr/ivr-this_ivr_will_let_you_test_features.wavdetect:unimrcp {start-input-timers=false,no-input-timeout=80000,recognition-timeout=80000}hello
Application-Response: DONE
Application-UUID: 7f56ddc1-2d8f-4a2c-aae1-a236749f8133

技术参考资料
https://github.com/wangkaisine/mrcp-plugin-with-freeswitch

小爱玄策

关注

4
点赞
踩
10

收藏

觉得还不错? 一键收藏
打赏
4
评论
智能客服搭建(3) - MRCP Server 与 FreeSWITCH 对接

文章目录1. 写在前面2. MRCP Server 配置与测试2.1 MRCP Server 配置2.2 MRCP Server 测试3. FreeSwitch 配置3.1. 配置 mrcp profile3.2. 配置 mrcp conf3.3. 配置拨号任务4. 效果测试1. 写在前面上一篇文章中，对MRCP Server的开发进行了介绍，下面讲解如何通过配置让FreeSWITCH能够与MRCP Server进行对接，获得电话呼叫的语音识别结果。2. MRCP Server 配置与测试2.1
复制链接

扫一扫