树莓派4 使用 SnowBoy 搭建热词唤醒

环境配置

查看收音设备

arecord -l

查看输出设备

aplay -l

更新系统

sudo apt-get update
sudo apt-get upgrade

编辑 ~/.asoundrc 指定对应的设备

pcm.!default {
  type asym
   playback.pcm {
     type plug
     slave.pcm "hw:0,0"
   }
   capture.pcm {
     type plug
     slave.pcm "hw:1,0"
   }
}

安装 PortAudio

sudo apt-get install python-pyaudio python3-pyaudio sox

安装 Python 绑定

pip install pyaudio

编译 SnowBoy 新版
官网有树莓派 1、2、3、Zero 的预编译包，不知道 4 能不能用，我尝试自己编译一个试试

sudo apt-get install autotools-dev automake libpcre3 libpcre3-dev libatlas-base-dev byacc
./autogen
./configure
make
sudo make install

训练热词

模型训练可以通过网页录制上传，也可以自己录制编写代码训练。

网页录制上传请访问官网：https://snowboy.kitt.ai/

点击 Create Hotword 就可以了，按照提示来。

我没有选择这种模式，因为我树莓派没有接显示屏不方便使用网页，如果用电脑怕环境不一样，采集样本有区别，主要的是自己想写代码折腾一下：

录制训练样本数据：

arecord bb1.wav

或者，这里根据麦克风指定了波特率

rec -r 16000 b1.wav

编写程序本地训练：

import sys
import base64
import requests




def get_wave(fname):
    with open(fname, 'rb') as infile:
        return base64.b64encode(infile.read())


# api 地址
endpoint = "https://snowboy.kitt.ai/api/v1/train/"




############# 修改参数 #############
# api 令牌，官网注册申请，必须
token = ""
# 热词，如果未知必须
hotword_name = "笨笨"
# 语言
language = "zh"
# 年龄段 0_9, 10_19, 20_29, 30_39, 40_49, 50_59, 60+
age_group = "40_49"
# 性别 F = female M = Male
gender = "M"
# 麦克类型
microphone = "macbook microphone"
############### END OF MODIFY ##################


if __name__ == "__main__":
    try:
        # 获取运行参数，三个训练音频和一个输出模型
        [_, wav1, wav2, wav3, out] = sys.argv
    except ValueError:
        print("Usage: %s wave_file1 wave_file2 wave_file3 out_model_name" % sys.argv[0])
        sys.exit()
    # 训练参数结构
    data = {
        "name": hotword_name,
        "language": language,
        "age_group": age_group,
        "gender": gender,
        "microphone": microphone,
        "token": token,
        # 训练数据是三个 wav 格式的录音音频
        "voice_samples": [
            {"wave": get_wave(wav1)},
            {"wave": get_wave(wav2)},
            {"wave": get_wave(wav3)}
        ]
    }
    # 提交训练数据
    response = requests.post(endpoint, json=data)
    if response.ok:
        # 训练成功，保存模型到指定文件
        with open(out, "wb") as outfile:
            outfile.write(response.content)
        print ("Saved model to '%s'." % out)
    else:
        print ("Request failed.")
        print (response.text)

训练样本生成模型：

python3 training.py b1.wav b2.wav b3.wav model.umdl

编写测试程序

from . import snowboydecoder
import sys
import signal
import wave
import os
import pyaudio
import time


"""
使用说明：
In [1]: import snowboydecoder


In [2]: def detected_callback():
   ....:     print "hotword detected"
   ....:


In [3]: detector = snowboydecoder.HotwordDetector("resources/snowboy.umdl", sensitivity=0.5, audio_gain=1)


In [4]: detector.start(detected_callback)
"""

TOP_DIR = os.path.dirname(os.path.abspath(__file__))

RESOURCE_FILE = os.path.join(TOP_DIR, "resources/common.res")
DETECT_DING = os.path.join(TOP_DIR, "resources/imhere.wav")
DETECT_DONG = os.path.join(TOP_DIR, "resources/dong.wav")

interrupted = False

def play_audio_file(fname):
    """Simple callback function to play a wave file. By default it plays
    a Ding sound.
    :param str fname: wave file name
    :return: None
    """
    ding_wav = wave.open(fname, 'rb')
    ding_data = ding_wav.readframes(ding_wav.getnframes())
    audio = pyaudio.PyAudio()
    stream_out = audio.open(
        format=audio.get_format_from_width(ding_wav.getsampwidth()),
        channels=ding_wav.getnchannels(),
        rate=44100, # ding_wav.getframerate(), 
	input=False, output=True)
    stream_out.start_stream()
    stream_out.write(ding_data)
    time.sleep(0.2)
    stream_out.stop_stream()
    stream_out.close()
    audio.terminate()

def detected_callback():
    print ("hotword detected")
    play_audio_file(DETECT_DING)

# 响应中断信号，设置中断变量为 True
def signal_handler(signal, frame):
    global interrupted
    interrupted = True
# 返回中断状态
def interrupt_callback():
    global interrupted
    return interrupted

# 输入参数处理
if len(sys.argv) == 1:
    print("Error: need to specify model name")
    print("Usage: python demo.py your.model")
    sys.exit(-1)
# 模型文件
model = sys.argv[1]

# 初始化中断信号处理
signal.signal(signal.SIGINT, signal_handler)

# 实例化热词监测
detector = snowboydecoder.HotwordDetector(model, sensitivity=0.5)
print('Listening... Press Ctrl+C to exit')

# 启动实例，开始热词监测
detector.start(detected_callback=detected_callback,
               interrupt_check=interrupt_callback,
               sleep_time=0.03)

detector.terminate()

因为麦克风的采样率问题，需要修改 snowboydecoder.py 的采样率，大约在115 行

self.stream_in = self.audio.open(
    input=True, output=False,
    format=self.audio.get_format_from_width(
        self.detector.BitsPerSample() / 8),
    channels=self.detector.NumChannels(),
    rate=44100, #self.detector.SampleRate(),
    frames_per_buffer=2048,
    stream_callback=audio_callback)

功能演示

需要注意的是 python3 的相对引用问题

python demo.py model.umdl
python3 -m Python3.demo Python3/model.umdl

python3 -m Python3.demo5 Python3/model.umdl

参考资料

pocketsphinx snowboy

• 唤醒词的科普研究 —— https://www.leiphone.com/news/201801/6a8FrmvHBf2Zmika.html?uniqueCode=9E2L1pCktaUeHu6G

https://github.com/wanleg/snowboyPi

https://pimylifeup.com/raspberry-pi-snowboy/

https://snowboy.kitt.ai/docspartials/docs/index.html#access-microphone

http://docs.kitt.ai/snowboy/

视频讲解

使用树莓派 4 和 SnowBoy 实现热词唤醒