unimrcp 实现阿里云的plugin

最新推荐文章于 2024-07-24 05:00:00 发布
最新推荐文章于 2024-07-24 05:00:00 发布
阅读量1.7k 收藏 10
点赞数 3
分类专栏： # MRCP 文章标签： unimrcp 阿里云 plugin ASR
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/yetyongjin/article/details/120743158
版权

1. 环境说明

unimrcp版本V1.7.0 

阿里SDK版本：V3.X

操作系统：CentOS7

GCC版本：V4.8.5，注意，使用高版本的GCC可能会有SDK编译兼容问题

    

2. 修改configure.ac

        首先进入unimrcp-1.7.0源码所有的目录。

        因为 unimrcp 使用 automake 进行源码编译管理，所以除了添加源代码，我们还需要进行相应配置添加。首先编辑 configure.ac 文件，添加一些内容。可以参考demorecog的内容编辑一份

打开configure.ac添加模块的宏定义（搜索demorecog，参照其内容）：

dnl Ali recognizer plugin.     
UNI_PLUGIN_ENABLED(alirecog)   

AM_CONDITIONAL([ALIRECOG_PLUGIN],[test "${enable_alirecog_plugiec}" = "yes"])

在AC_CONFIG_FILES里添加一行：

plugins/demo-recog/Makefile

在最后的Echo内容里添加一行：

echo Ali recognizer plugin......... : $enable_alirecog_plugin

3. 新增源代码及目录

        在 plugins目录下，新建 ali-recog 目录，并在该目录下新建 src 目录，可以将plugins/demo-recog/src/目录下的 demo_recog_engine.c 拷贝到src目录下改名为 ali_recog_engine.c，并将源代码中的所有“demo”替换为“ali”，当然，也可以自己写一遍。

3.1 Makefile.am

        把demo-recog目录下的Makefile.am拷贝过来，打开编辑，把里面的所有“demo”替换为“ali”。

AM_CPPFLAGS                = $(UNIMRCP_PLUGIN_INCLUDES)

plugin_LTLIBRARIES         = alirecog.la

alirecog_la_SOURCES       = src/ali_recog_engine.c
alirecog_la_LDFLAGS       = $(UNIMRCP_PLUGIN_OPTS)                                                                                                                                                              

include $(top_srcdir)/build/rules/uniplugin.am

目录文件结构：

3.2 plugins 目录下的 Makefile.am

修改plugins 目录下的 Makefile.am 文件，新增如下内容：

if ALIRECOG_PLUGIN
SUBDIRS               += ali-recog
endif

这里if后面的内容就是之前config.ac里这一行定义的内容：

AM_CONDITIONAL([ALIRECOG_PLUGIN],[test "${enable_alirecog_plugiec}" = "yes"])

3.3 重新编译

# cd unimrcp-1.7.0/
# ./bootstrap
# ./configure
# make
# make install

看一上编译结果：

ls /usr/local/unimrcp/plugin/

4. 导入阿里的SDK

        到阿里云官网下载SDK，可以是编译好的版本，也可以是GIT源码。这里直接下载编译好的二进制包，解压放在ali-recog 目录下

        因为阿里提供的SDK是C++版本的，为了方便，把实现代码也改为C++：

# mv ali_recog_engine.c ali_recog_engine.cpp

把原来的引用头文件加上extern "C" ：

编辑ali-recog目录下的Makefile.am，添加SDK的头文件路径和库文件路径：

从阿里SDK的demo里导入代码修改：

主要修改Channel初始化，读取帧处理，结束识别的几个点，具体的就是下面代码中有中文注释的地方

/*
 * Copyright 2008-2015 Arsen Chaloyan
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *	 http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

/* 
 * Mandatory rules concerning plugin implementation.
 * 1. Each plugin MUST implement a plugin/engine creator function
 *	with the exact signature and name (the main entry point)
 *		MRCP_PLUGIN_DECLARE(mrcp_engine_t*) mrcp_plugin_create(apr_pool_t *pool)
 * 2. Each plugin MUST declare its version number
 *		MRCP_PLUGIN_VERSION_DECLARE
 * 3. One and only one response MUST be sent back to the received request.
 * 4. Methods (callbacks) of the MRCP engine channel MUST not block.
 *   (asynchronous response can be sent from the context of other thread)
 * 5. Methods (callbacks) of the MPF engine stream MUST not block.
 */

extern "C" {
#include "mrcp_recog_engine.h"
#include "mpf_activity_detector.h"
#include "apt_consumer_task.h"
#include "apt_log.h"
}  // extern C

#define RECOG_ENGINE_TASK_NAME "ali Recog Engine"

#include <string.h>
#include <unistd.h>
#include <pthread.h>
#include <stdlib.h>
#include <ctime>
#include <string>
#include <iostream>
#include <vector>
#include <fstream>
#include <sys/time.h>
#include "nlsClient.h"
#include "nlsEvent.h"
#include "speechRecognizerRequest.h"
#include "nlsCommonSdk/Token.h"

#define FRAME_SIZE 3200
#define SAMPLE_RATE 16000

using namespace AlibabaNlsCommon;
using AlibabaNls::NlsClient;
using AlibabaNls::NlsEvent;
using AlibabaNls::LogDebug;
using AlibabaNls::LogInfo;
using AlibabaNls::LogError;
using AlibabaNls::SpeechRecognizerRequest;

typedef struct ali_recog_engine_t ali_recog_engine_t;
typedef struct ali_recog_channel_t ali_recog_channel_t;
typedef struct ali_recog_msg_t ali_recog_msg_t;


// Globally maintain a service authentication token and its corresponding validity time stamp
// Before each call to the service, first determine whether the token has expired
// If it has expired, a token is regenerated based on the AccessKey ID and AccessKey Secret, and the global token and its validity timestamp are updated.
// Note: Do not regenerate a new token before calling the service, just regenerate it when the token is about to expire. All concurrent services can share a token.
std::string g_appkey = "";   //阿里的APP KEY
std::string g_akId = "";     // 阿里的access ID
std::string g_akSecret = ""; // 阿里的access secret
std::string g_token = "";
long g_expireTime = -1;

struct timeval tv;
struct timeval tv1;

// 阿里SDK相关函数声明
int generateToken(std::string akId, std::string akSecret, std::string* token, long* expireTime);
void OnRecognitionStarted(NlsEvent* cbEvent, void* recog_channel);
void OnRecognitionResultChanged(NlsEvent* cbEvent, void* recog_channel);
void OnRecognitionCompleted(NlsEvent* cbEvent, void* recog_channel);
void OnRecognitionTaskFailed(NlsEvent* cbEvent, void* recog_channel);
void OnRecognitionChannelClosed(NlsEvent* cbEvent, void* recog_channel);


/** Declaration of recognizer engine methods */
static apt_bool_t ali_recog_engine_destroy(mrcp_engine_t *engine);
static apt_bool_t ali_recog_engine_open(mrcp_engine_t *engine);
static apt_bool_t ali_recog_engine_close(mrcp_engine_t *engine);
static mrcp_engine_channel_t* ali_recog_engine_channel_create(mrcp_engine_t *engine, apr_pool_t *pool);

static const struct mrcp_engine_method_vtable_t engine_vtable = {
	ali_recog_engine_destroy,
	ali_recog_engine_open,
	ali_recog_engine_close,
	ali_recog_engine_channel_create
};


/** Declaration of recognizer channel methods */
static apt_bool_t ali_recog_channel_destroy(mrcp_engine_channel_t *channel);
static apt_bool_t ali_recog_channel_open(mrcp_engine_channel_t *channel);
static apt_bool_t ali_recog_channel_close(mrcp_engine_channel_t *channel);
static apt_bool_t ali_recog_channel_request_process(mrcp_engine_channel_t *channel, mrcp_message_t *request);

static const struct mrcp_engine_channel_method_vtable_t channel_vtable = {
	ali_recog_channel_destroy,
	ali_recog_channel_open,
	ali_recog_channel_close,
	ali_recog_channel_request_process
};

/** Declaration of recognizer audio stream methods */
static apt_bool_t ali_recog_stream_destroy(mpf_audio_stream_t *stream);
static apt_bool_t ali_recog_stream_open(mpf_audio_stream_t *stream, mpf_codec_t *codec);
static apt_bool_t ali_recog_stream_close(mpf_audio_stream_t *stream);
static apt_bool_t ali_recog_stream_write(mpf_audio_stream_t *stream, const mpf_frame_t *frame);

static const mpf_audio_stream_vtable_t audio_stream_vtable = {
	ali_recog_stream_destroy,
	NULL,
	NULL,
	NULL,
	ali_recog_stream_open,
	ali_recog_stream_close,
	ali_recog_stream_write,
	NULL
};

/** Declaration of ali recognizer engine */
struct ali_recog_engine_t {
	apt_consumer_task_t	*task;
};

/** Declaration of ali recognizer channel */
struct ali_recog_channel_t {
	/** Back pointer to engine */
	ali_recog_engine_t	 *ali_engine;
	/** Engine channel base */
	mrcp_engine_channel_t   *channel;

	/** Active (in-progress) recognition request */
	mrcp_message_t		  *recog_request;
	/** Pending stop response */
	mrcp_message_t		  *stop_response;
	/** Indicates whether input timers are started */
	apt_bool_t			   timers_started;
	/** Voice activity detector */
	mpf_activity_detector_t *detector;
	/** File to write utterance to */
	FILE					*audio_out;

	/** Ali SpeechRecognizerRequest */
	SpeechRecognizerRequest *ali_request;   //阿里SDK的属性
	/** Ali Recognizer Result */
	const char			  *result;          //阿里SDK返回结果存储在这
};

typedef enum {
	ali_RECOG_MSG_OPEN_CHANNEL,
	ali_RECOG_MSG_CLOSE_CHANNEL,
	ali_RECOG_MSG_REQUEST_PROCESS
} ali_recog_msg_type_e;

/** Declaration of ali recognizer task message */
struct ali_recog_msg_t {
	ali_recog_msg_type_e  type;
	mrcp_engine_channel_t *channel; 
	mrcp_message_t		*request;
};

static apt_bool_t ali_recog_msg_signal(ali_recog_msg_type_e type, mrcp_engine_channel_t *channel, mrcp_message_t *request);
static apt_bool_t ali_recog_msg_process(apt_task_t *task, apt_task_msg_t *msg);

/** Declare this macro to set plugin version */
MRCP_PLUGIN_VERSION_DECLARE

/**
 * Declare this macro to use log routine of the server, plugin is loaded from.
 * Enable/add the corresponding entry in logger.xml to set a cutsom log source priority.
 *	<source name="RECOG-PLUGIN" priority="DEBUG" masking="NONE"/>
 */
MRCP_PLUGIN_LOG_SOURCE_IMPLEMENT(RECOG_PLUGIN,"RECOG-PLUGIN")

/** Use custom log source mark */
#define RECOG_LOG_MARK   APT_LOG_MARK_DECLARE(RECOG_PLUGIN)

/** Create ali recognizer engine */
MRCP_PLUGIN_DECLARE(mrcp_engine_t*) mrcp_plugin_create(apr_pool_t *pool)
{
	ali_recog_engine_t *ali_engine = (ali_recog_engine_t *)apr_palloc(pool,sizeof(ali_recog_engine_t));
	apt_task_t *task;
	apt_task_vtable_t *vtable;
	apt_task_msg_pool_t *msg_pool;

	msg_pool = apt_task_msg_pool_create_dynamic(sizeof(ali_recog_msg_t),pool);
	ali_engine->task = apt_consumer_task_create(ali_engine,msg_pool,pool);
	if(!ali_engine->task) {
		return NULL;
	}
	task = apt_consumer_task_base_get(ali_engine->task);
	apt_task_name_set(task,RECOG_ENGINE_TASK_NAME);
	vtable = apt_task_vtable_get(task);
	if(vtable) {
		vtable->process_msg = ali_recog_msg_process;
	}

	/* create engine base */
	return mrcp_engine_create(
				MRCP_RECOGNIZER_RESOURCE,  /* MRCP resource identifier */
				ali_engine,			   /* object to associate */
				&engine_vtable,			/* virtual methods table of engine */
				pool);					 /* pool to allocate memory from */
}

/** Destroy recognizer engine */
static apt_bool_t ali_recog_engine_destroy(mrcp_engine_t *engine)
{
	ali_recog_engine_t *ali_engine = (ali_recog_engine_t *)engine->obj;
	if(ali_engine->task) {
		apt_task_t *task = apt_consumer_task_base_get(ali_engine->task);
		apt_task_destroy(task);
		ali_engine->task = NULL;
	}
	return TRUE;
}

/** Open recognizer engine */
static apt_bool_t ali_recog_engine_open(mrcp_engine_t *engine)
{
	ali_recog_engine_t *ali_engine = (ali_recog_engine_t *)engine->obj;
	if(ali_engine->task) {
		apt_task_t *task = apt_consumer_task_base_get(ali_engine->task);
		apt_task_start(task);
	}

    // 初始化阿里引擎
	// Set Ali logger
	if (-1 == NlsClient::getInstance()->setLogConfig("log-recognizer", LogDebug, 400)) {
		printf("set log failed.\n");
	   
		return mrcp_engine_open_respond(engine,FALSE);
	}

	// Generate a new token
	if (-1 == generateToken(g_akId, g_akSecret, &g_token, &g_expireTime)) {
		return mrcp_engine_open_respond(engine,FALSE);
	}

	// Start Ali Worker threak
	NlsClient::getInstance()->startWorkThread(4);

	return mrcp_engine_open_respond(engine,TRUE);
}

/** Close recognizer engine */
static apt_bool_t ali_recog_engine_close(mrcp_engine_t *engine)
{
	ali_recog_engine_t *ali_engine = (ali_recog_engine_t *)engine->obj;
	if(ali_engine->task) {
		apt_task_t *task = apt_consumer_task_base_get(ali_engine->task);
		apt_task_terminate(task,TRUE);
	}

	NlsClient::releaseInstance();
	return mrcp_engine_close_respond(engine);
}

static mrcp_engine_channel_t* ali_recog_engine_channel_create(mrcp_engine_t *engine, apr_pool_t *pool)
{
	mpf_stream_capabilities_t *capabilities;
	mpf_termination_t *termination; 

	/* create ali recog channel */
	ali_recog_channel_t *recog_channel = (ali_recog_channel_t *)apr_palloc(pool,sizeof(ali_recog_channel_t));
	recog_channel->ali_engine = (ali_recog_engine_t *)engine->obj;
	recog_channel->recog_request = NULL;
	recog_channel->stop_response = NULL;
	recog_channel->detector = mpf_activity_detector_create(pool);
	recog_channel->audio_out = NULL;
	recog_channel->ali_request = NULL;	
	
	capabilities = mpf_sink_stream_capabilities_create(pool);
	mpf_codec_capabilities_add(
			&capabilities->codecs,
			MPF_SAMPLE_RATE_8000 | MPF_SAMPLE_RATE_16000,
			"LPCM");

	/* create media termination */
	termination = mrcp_engine_audio_termination_create(
			recog_channel,		/* object to associate */
			&audio_stream_vtable, /* virtual methods table of audio stream */
			capabilities,		 /* stream capabilities */
			pool);				/* pool to allocate memory from */

	/* create engine channel base */
	recog_channel->channel = mrcp_engine_channel_create(
			engine,			   /* engine */
			&channel_vtable,	  /* virtual methods table of engine channel */
			recog_channel,		/* object to associate */
			termination,		  /* associated media termination */
			pool);				/* pool to allocate memory from */

	return recog_channel->channel;
}

/** Destroy engine channel */
static apt_bool_t ali_recog_channel_destroy(mrcp_engine_channel_t *channel)
{
	/* nothing to destrtoy */
	return TRUE;
}

/** Open engine channel (asynchronous response MUST be sent)*/
static apt_bool_t ali_recog_channel_open(mrcp_engine_channel_t *channel)
{
	if(channel->attribs) {
		/* process attributes */
		const apr_array_header_t *header = apr_table_elts(channel->attribs);
		apr_table_entry_t *entry = (apr_table_entry_t *)header->elts;
		int i;
		for(i=0; i<header->nelts; i++) {
			apt_log(RECOG_LOG_MARK,APT_PRIO_INFO,"Attrib name [%s] value [%s]",entry[i].key,entry[i].val);
		}
	}

	return ali_recog_msg_signal(ali_RECOG_MSG_OPEN_CHANNEL,channel,NULL);
}

/** Close engine channel (asynchronous response MUST be sent)*/
static apt_bool_t ali_recog_channel_close(mrcp_engine_channel_t *channel)
{
	return ali_recog_msg_signal(ali_RECOG_MSG_CLOSE_CHANNEL,channel,NULL);
}

/** Process MRCP channel request (asynchronous response MUST be sent)*/
static apt_bool_t ali_recog_channel_request_process(mrcp_engine_channel_t *channel, mrcp_message_t *request)
{
	return ali_recog_msg_signal(ali_RECOG_MSG_REQUEST_PROCESS,channel,request);
}

/** Process RECOGNIZE request */
static apt_bool_t ali_recog_channel_recognize(mrcp_engine_channel_t *channel, mrcp_message_t *request, mrcp_message_t *response)
{
	/* process RECOGNIZE request */
	mrcp_recog_header_t *recog_header;
	ali_recog_channel_t *recog_channel = (ali_recog_channel_t *)channel->method_obj;
	const mpf_codec_descriptor_t *descriptor = mrcp_engine_sink_stream_codec_get(channel);

	if(!descriptor) {
		apt_log(RECOG_LOG_MARK,APT_PRIO_WARNING,"Failed to Get Codec Descriptor " APT_SIDRES_FMT, MRCP_MESSAGE_SIDRES(request));
		response->start_line.status_code = MRCP_STATUS_CODE_METHOD_FAILED;
		return FALSE;
	}

	recog_channel->timers_started = TRUE;

	/* get recognizer header */
	recog_header = (mrcp_recog_header_t *)mrcp_resource_header_get(request);
	if(recog_header) {
		if(mrcp_resource_header_property_check(request,RECOGNIZER_HEADER_START_INPUT_TIMERS) == TRUE) {
			recog_channel->timers_started = recog_header->start_input_timers;
		}
		if(mrcp_resource_header_property_check(request,RECOGNIZER_HEADER_NO_INPUT_TIMEOUT) == TRUE) {
			mpf_activity_detector_noinput_timeout_set(recog_channel->detector,recog_header->no_input_timeout);
		}
		if(mrcp_resource_header_property_check(request,RECOGNIZER_HEADER_SPEECH_COMPLETE_TIMEOUT) == TRUE) {
			mpf_activity_detector_silence_timeout_set(recog_channel->detector,recog_header->speech_complete_timeout);
		}
	}

	if(!recog_channel->audio_out) {
		const apt_dir_layout_t *dir_layout = channel->engine->dir_layout;
		char *file_name = apr_psprintf(channel->pool,"utter-%dkHz-%s.pcm",
							descriptor->sampling_rate/1000,
							request->channel_id.session_id.buf);
		char *file_path = apt_vardir_filepath_get(dir_layout,file_name,channel->pool);
		if(file_path) {
			apt_log(RECOG_LOG_MARK,APT_PRIO_INFO,"Open Utterance Output File [%s] for Writing",file_path);
			recog_channel->audio_out = fopen(file_path,"wb");
			if(!recog_channel->audio_out) {
				apt_log(RECOG_LOG_MARK,APT_PRIO_WARNING,"Failed to Open Utterance Output File [%s] for Writing",file_path);
			}
		}
	}

	SpeechRecognizerRequest *ali_request = NlsClient::getInstance()->createRecognizerRequest();
	recog_channel->ali_request = ali_request;

    // 初始化阿里SDK的request
	if (ali_request == NULL) {
		printf("createRecognizerRequest failed\n");
		return FALSE;
	}
	ali_request->setOnRecognitionStarted(OnRecognitionStarted, recog_channel);
	ali_request->setOnTaskFailed(OnRecognitionTaskFailed, recog_channel);
	ali_request->setOnChannelClosed(OnRecognitionChannelClosed, recog_channel);
	ali_request->setOnRecognitionResultChanged(OnRecognitionResultChanged, recog_channel);
	ali_request->setOnRecognitionCompleted(OnRecognitionCompleted, recog_channel);
	ali_request->setAppKey(g_appkey.c_str());
	ali_request->setFormat("pcm");
	ali_request->setSampleRate(SAMPLE_RATE);
	ali_request->setIntermediateResult(true);
	ali_request->setPunctuationPrediction(true);
	ali_request->setInverseTextNormalization(true);
	//ali_request->setEnableVoiceDetection(true); 
	//ali_request->setMaxStartSilence(5000);
	//ali_request->setMaxEndSilence(800);
	//ali_request->setCustomizationId("TestId_123");
	//ali_request->setVocabularyId("TestId_456");
	//ali_request->setPayloadParam("{\"vad_model\": \"farfield\"}");
	ali_request->setToken(g_token.c_str()); // 设置账号校验token。必填参数。

	if (ali_request->start() < 0) {
		printf("start() failed. may be can not connect server. please check network or firewalld\n");
		NlsClient::getInstance()->releaseRecognizerRequest(recog_channel->ali_request); 
		return FALSE;
	}

	response->start_line.request_state = MRCP_REQUEST_STATE_INPROGRESS;
	/* send asynchronous response */
	mrcp_engine_channel_message_send(channel,response);
	recog_channel->recog_request = request;
	return TRUE;
}

/** Process STOP request */
static apt_bool_t ali_recog_channel_stop(mrcp_engine_channel_t *channel, mrcp_message_t *request, mrcp_message_t *response)
{
	/* process STOP request */
	ali_recog_channel_t *recog_channel = (ali_recog_channel_t *)channel->method_obj;
	/* store STOP request, make sure there is no more activity and only then send the response */
	recog_channel->stop_response = response;
	return TRUE;
}

/** Process START-INPUT-TIMERS request */
static apt_bool_t ali_recog_channel_timers_start(mrcp_engine_channel_t *channel, mrcp_message_t *request, mrcp_message_t *response)
{
	ali_recog_channel_t *recog_channel = (ali_recog_channel_t *)channel->method_obj;
	recog_channel->timers_started = TRUE;
	return mrcp_engine_channel_message_send(channel,response);
}

/** Dispatch MRCP request */
static apt_bool_t ali_recog_channel_request_dispatch(mrcp_engine_channel_t *channel, mrcp_message_t *request)
{
	apt_bool_t processed = FALSE;
	mrcp_message_t *response = mrcp_response_create(request,request->pool);
	switch(request->start_line.method_id) {
		case RECOGNIZER_SET_PARAMS:
			break;
		case RECOGNIZER_GET_PARAMS:
			break;
		case RECOGNIZER_DEFINE_GRAMMAR:
			break;
		case RECOGNIZER_RECOGNIZE:
			processed = ali_recog_channel_recognize(channel,request,response);
			break;
		case RECOGNIZER_GET_RESULT:
			break;
		case RECOGNIZER_START_INPUT_TIMERS:
			processed = ali_recog_channel_timers_start(channel,request,response);
			break;
		case RECOGNIZER_STOP:
			processed = ali_recog_channel_stop(channel,request,response);
			break;
		default:
			break;
	}
	if(processed == FALSE) {
		/* send asynchronous response for not handled request */
		mrcp_engine_channel_message_send(channel,response);
	}
	return TRUE;
}

/** Callback is called from MPF engine context to destroy any additional data associated with audio stream */
static apt_bool_t ali_recog_stream_destroy(mpf_audio_stream_t *stream)
{
	return TRUE;
}

/** Callback is called from MPF engine context to perform any action before open */
static apt_bool_t ali_recog_stream_open(mpf_audio_stream_t *stream, mpf_codec_t *codec)
{
	return TRUE;
}

/** Callback is called from MPF engine context to perform any action after close */
static apt_bool_t ali_recog_stream_close(mpf_audio_stream_t *stream)
{
	return TRUE;
}

/* Raise ali START-OF-INPUT event */
static apt_bool_t ali_recog_start_of_input(ali_recog_channel_t *recog_channel)
{
	/* create START-OF-INPUT event */
	mrcp_message_t *message = mrcp_event_create(
						recog_channel->recog_request,
						RECOGNIZER_START_OF_INPUT,
						recog_channel->recog_request->pool);
	if(!message) {
		return FALSE;
	}

	/* set request state */
	message->start_line.request_state = MRCP_REQUEST_STATE_INPROGRESS;
	/* send asynch event */
	return mrcp_engine_channel_message_send(recog_channel->channel,message);
}

/* Load ali recognition result */
static apt_bool_t ali_recog_result_load(ali_recog_channel_t *recog_channel, mrcp_message_t *message)
{
	mrcp_engine_channel_t *channel = recog_channel->channel;
	const apt_dir_layout_t *dir_layout = channel->engine->dir_layout;
	char *file_path = apt_datadir_filepath_get(dir_layout,"result.xml",message->pool);
	if(!file_path) {
		return FALSE;
	}
	
	/* read the ali result from result */
	mrcp_generic_header_t *generic_header;

	apt_string_assign_n(&message->body,recog_channel->result,strlen(recog_channel->result),message->pool);

	/* get/allocate generic header */
	generic_header = mrcp_generic_header_prepare(message);
	if(generic_header) {
		/* set content types */
		apt_string_assign(&generic_header->content_type,"application/x-nlsml",message->pool);
		mrcp_generic_header_property_add(message,GENERIC_HEADER_CONTENT_TYPE);
	}

	return TRUE;
}

/* Raise ali RECOGNITION-COMPLETE event */
static apt_bool_t ali_recog_recognition_complete(ali_recog_channel_t *recog_channel, mrcp_recog_completion_cause_e cause)
{
	mrcp_recog_header_t *recog_header;
	/* create RECOGNITION-COMPLETE event */
	mrcp_message_t *message = mrcp_event_create(
						recog_channel->recog_request,
						RECOGNIZER_RECOGNITION_COMPLETE,
						recog_channel->recog_request->pool);
	if(!message) {
		return FALSE;
	}

	/* get/allocate recognizer header */
	recog_header = (mrcp_recog_header_t *)mrcp_resource_header_prepare(message);
	if(recog_header) {
		/* set completion cause */
		recog_header->completion_cause = cause;
		mrcp_resource_header_property_add(message,RECOGNIZER_HEADER_COMPLETION_CAUSE);
	}
	/* set request state */
	message->start_line.request_state = MRCP_REQUEST_STATE_COMPLETE;

	if(cause == RECOGNIZER_COMPLETION_CAUSE_SUCCESS) {
		ali_recog_result_load(recog_channel,message);
	}

	recog_channel->recog_request = NULL;
	/* send asynch event */
	return mrcp_engine_channel_message_send(recog_channel->channel,message);
}

/** Callback is called from MPF engine context to write/send new frame */
static apt_bool_t ali_recog_stream_write(mpf_audio_stream_t *stream, const mpf_frame_t *frame)
{
	ali_recog_channel_t *recog_channel = (ali_recog_channel_t *)stream->obj;
	if(recog_channel->stop_response) {
		/* send asynchronous response to STOP request */
		mrcp_engine_channel_message_send(recog_channel->channel,recog_channel->stop_response);
		recog_channel->stop_response = NULL;
		recog_channel->recog_request = NULL;
		return TRUE;
	}

    // 接收的数据帧转给阿里SDK
	if(recog_channel->recog_request) {
		mpf_detector_event_e det_event = mpf_activity_detector_process(recog_channel->detector,frame);
		switch(det_event) {
			case MPF_DETECTOR_EVENT_ACTIVITY:
				apt_log(RECOG_LOG_MARK,APT_PRIO_INFO,"Detected Voice Activity " APT_SIDRES_FMT,
					MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
				ali_recog_start_of_input(recog_channel);
				break;
			case MPF_DETECTOR_EVENT_INACTIVITY:
				apt_log(RECOG_LOG_MARK,APT_PRIO_INFO,"Detected Voice Inactivity " APT_SIDRES_FMT,
					MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
				//ali_recog_recognition_complete(recog_channel,RECOGNIZER_COMPLETION_CAUSE_SUCCESS);
				if(recog_channel->ali_request) {
					recog_channel->ali_request->stop();
				}
				break;
			case MPF_DETECTOR_EVENT_NOINPUT:
				apt_log(RECOG_LOG_MARK,APT_PRIO_INFO,"Detected Noinput " APT_SIDRES_FMT,
					MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
				if(recog_channel->timers_started == TRUE) {
					//ali_recog_recognition_complete(recog_channel,RECOGNIZER_COMPLETION_CAUSE_NO_INPUT_TIMEOUT);
					if(recog_channel->ali_request) {
						recog_channel->ali_request->stop();
					}
				}
				break;
			default:
				break;
		}

		if(recog_channel->recog_request) {
			if((frame->type & MEDIA_FRAME_TYPE_EVENT) == MEDIA_FRAME_TYPE_EVENT) {
				if(frame->marker == MPF_MARKER_START_OF_EVENT) {
					apt_log(RECOG_LOG_MARK,APT_PRIO_INFO,"Detected Start of Event " APT_SIDRES_FMT " id:%d",
						MRCP_MESSAGE_SIDRES(recog_channel->recog_request),
						frame->event_frame.event_id);
				}
				else if(frame->marker == MPF_MARKER_END_OF_EVENT) {
					apt_log(RECOG_LOG_MARK,APT_PRIO_INFO,"Detected End of Event " APT_SIDRES_FMT " id:%d duration:%d ts",
						MRCP_MESSAGE_SIDRES(recog_channel->recog_request),
						frame->event_frame.event_id,
						frame->event_frame.duration);
				}
			}
		}

		if(recog_channel->audio_out) {
			fwrite(frame->codec_frame.buffer,1,frame->codec_frame.size,recog_channel->audio_out);
		}

		if(recog_channel->ali_request) {
			recog_channel->ali_request->sendAudio((const uint8_t *)frame->codec_frame.buffer, frame->codec_frame.size);
		}
	}
	return TRUE;
}

static apt_bool_t ali_recog_msg_signal(ali_recog_msg_type_e type, mrcp_engine_channel_t *channel, mrcp_message_t *request)
{
	apt_bool_t status = FALSE;
	ali_recog_channel_t *ali_channel = (ali_recog_channel_t *)channel->method_obj;
	ali_recog_engine_t *ali_engine = ali_channel->ali_engine;
	apt_task_t *task = apt_consumer_task_base_get(ali_engine->task);
	apt_task_msg_t *msg = apt_task_msg_get(task);
	if(msg) {
		ali_recog_msg_t *ali_msg;
		msg->type = TASK_MSG_USER;
		ali_msg = (ali_recog_msg_t*) msg->data;

		ali_msg->type = type;
		ali_msg->channel = channel;
		ali_msg->request = request;
		status = apt_task_msg_signal(task,msg);
	}
	return status;
}

static apt_bool_t ali_recog_msg_process(apt_task_t *task, apt_task_msg_t *msg)
{
	ali_recog_msg_t *ali_msg = (ali_recog_msg_t*)msg->data;
	switch(ali_msg->type) {
		case ali_RECOG_MSG_OPEN_CHANNEL:
			/* open channel and send asynch response */
			mrcp_engine_channel_open_respond(ali_msg->channel,TRUE);
			break;
		case ali_RECOG_MSG_CLOSE_CHANNEL:
		{
			/* close channel, make sure there is no activity and send asynch response */
			ali_recog_channel_t *recog_channel = (ali_recog_channel_t *)ali_msg->channel->method_obj;
			if(recog_channel->audio_out) {
				fclose(recog_channel->audio_out);
				recog_channel->audio_out = NULL;
			}

			mrcp_engine_channel_close_respond(ali_msg->channel);
			break;
		}
		case ali_RECOG_MSG_REQUEST_PROCESS:
			ali_recog_channel_request_dispatch(ali_msg->channel,ali_msg->request);
			break;
		default:
			break;
	}
	return TRUE;
}

、、
// Regenerate a new token base no  AccessKey ID and AccessKey Secrt, and get validity timestamp.
// All concurrent service can share a token, just Regenerate before expire.
int generateToken(std::string akId, std::string akSecret, std::string* token, long* expireTime) {
	NlsToken nlsTokenRequest;
	nlsTokenRequest.setAccessKeyId(akId.c_str());
	nlsTokenRequest.setKeySecret(akSecret.c_str());

	if (-1 == nlsTokenRequest.applyNlsToken()) {
		// Get failure reson.
		printf("generateToken Failed: %s\n", nlsTokenRequest.getErrorMsg());
		return -1;
	}

	*token = nlsTokenRequest.getToken();
	*expireTime = nlsTokenRequest.getExpireTime();
	return 0;
}

void OnRecognitionStarted(NlsEvent* cbEvent, void* recog_channel) {
	ali_recog_channel_t* tmp_chan = (ali_recog_channel_t*)recog_channel;
	printf("OnRecognitionStarted: status code=%d, task id=%s\n", cbEvent->getStatusCode(), cbEvent->getTaskId());
}

// 阿里的SDK回调实现
void OnRecognitionResultChanged(NlsEvent* cbEvent, void* recog_channel) {
	ali_recog_channel_t* tmp_chan = (ali_recog_channel_t*)recog_channel;
	printf("OnRecognitionResultChanged: status code=%d, task id=%s, result=%s\n", cbEvent->getStatusCode(), cbEvent->getTaskId(), cbEvent->getResult());
	tmp_chan->result = cbEvent->getResult();	
}

void OnRecognitionCompleted(NlsEvent* cbEvent, void* recog_channel) {
	ali_recog_channel_t* tmp_chan = (ali_recog_channel_t*)recog_channel;
	printf("OnRecognitionCompleted: status code=%d, task id=%s, result=%s\n", cbEvent->getStatusCode(), cbEvent->getTaskId(), cbEvent->getResult());
	tmp_chan->result = cbEvent->getResult();	
	ali_recog_recognition_complete(tmp_chan, RECOGNIZER_COMPLETION_CAUSE_SUCCESS);
}

void OnRecognitionTaskFailed(NlsEvent* cbEvent, void* recog_channel) {
	ali_recog_channel_t* tmp_chan = (ali_recog_channel_t*)recog_channel;
	printf("OnRecognitionTaskFailed: status code=%d, task id=%s, error message=%s\n", cbEvent->getStatusCode(), cbEvent->getTaskId(), cbEvent->getErrorMessage());
}

void OnRecognitionChannelClosed(NlsEvent* cbEvent, void* recog_channel) {
	ali_recog_channel_t* tmp_chan = (ali_recog_channel_t*)recog_channel;
	printf("OnRecognitionChannelClosed: response=%s\n", cbEvent->getAllResponse());
}

运行测试

./unimrcpserver

打一通电话：

FS输出：

Server端输出：

云梦谭
关注
  • 3
    点赞
  • 踩
  • 10
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
开源代码走读--unimrcp
HAH--M的博客
04-11 754
当unimrcp作为录音仪使用时，其rtp数据流在client端得发送流程及server端的解析流程
Gradle配置阿里云代理 init.gradle文件
04-24
Gradle 配置阿里云，配置国内代理，配置HTTPS，替换maven central，替换google，替换gradle plugin，替换jcenter，全部项目可使用，放入%GRADLE_HOME%\init.d文件夹中即可
【Linux】Centos7安装部署unimrcp，搭建MRCP服务器
姜太小白的博客
05-09 747
Centos7安装部署unimrcp，搭建MRCP服务器
编译unimrcp-vosk-plugin
fs交流的博客
12-18 500
编译unimrcp-vosk-plugin
阿里云实现发送短信（Java实例教程）
SuiHao
10-13 7666
短信发送是电信运营商提供的服务，需要访问对应的接口，不同运营商提供的接口地址肯定不一样，如果直接访问这些接口就需要判断收信息的手机号属于哪个运营商，关键在于这些接口不对个人开放，还要考虑调用短信服务的费用问题。因此目前调用短信业务都是使用第三方企业的短信服务，他们与运营商合作，封装了短信接口，调用方法，而且费用相对便宜。第一步：参考API，在【API接口】中已经给出了Java代码怎么调用该服务的接口。第三方的短信服务有很多，其中阿里云也提供了短信服务。第二步：选择购买的短信服务。
Java实现操作阿里云OSS云存储详解，含配置和完整代码
一火的专栏
11-07 6617
Java实现操作阿里云OSS云存储详解，含配置和完整代码
Gradle配置阿里云仓库
Mr_su_9726的博客
09-23 3226
配置阿里云仓库
【SpringBoot+MP】阿里云短信测试服务操作流程、用户手机验证码登录实现
懒羊羊的博客
10-30 3595
前面是站在后台员工的角度，现在站在消费者的角度，当客户从移动端选择通过验证码登录APP，需要调用云服务器的短信功能（厂商短信服务API），后端要做的就是生成验证码并发送，并针对请求中客户收到的验证码与后端发送的进行比对，确保一致后成功登录
阿里云NAS存储部署简介
haiziccc的专栏
12-02 1924
阿里云上申请NAS存储
阿里云服务器部署（保姆级）
热门推荐
qq_45660133的博客
03-08 2万+
一、前后端不分离SpringBoot项目部署 1、配置阿里云服务器 1.1、购买服务器 阿里云官网 https://www.aliyun.com/ 注册阿里云账号（自行注册） 阿里云购买教程 https://yq.aliyun.com/articles/699313 公网IP 私网IP 1.2、购买域名 进入阿里云官网 https://www.aliyun.com/，点击 “域名” 搜索你想要的域名进行购买； 1.3、备案 进入阿里云官网 https://www.aliyun.com/> 进入“控
基于阿里云语音合成（TTS）C/C++ Sdk2.0编写的unimrcp中间件
qq1779062842的博客
01-11 1834
支持实时流推送，无需生成录音文件再由freeswitch播放 支持录音 暂时不支持 pause 及 resume （有空再更新支持，或者有兴趣的大佬也可以直接拿源码改） 暂时不支持录音文件的格式，只支持原始数据pcm（偷懒，或者有兴趣的大佬也可以直接拿源码改） 测试： 测试平台：Centos_6.9_x64 freeswitch： 播放请求： unimrcpserver: 生成的录音文件： ...
阿里云yum源安装包
12-26
阿里云Yum源安装包是针对RHEL7和CentOS7系统进行操作的重要工具，它提供了从阿里云服务器上高效、稳定地下载和更新软件包的功能。这些安装包旨在替换默认的Red Hat或CentOS官方Yum源，以利用阿里云的高速网络服务，...
阿里云OSS官方eclipse插件包
07-10
阿里云对象存储服务（Object Storage Service，简称OSS），是阿里云提供的海量、安全、低成本、高可靠的云存储服务。为了方便开发者在Eclipse集成开发环境中便捷地使用OSS服务，阿里云官方推出了OSS Eclipse插件。这...
阿里云RDS官方eclipse插件包
07-10
阿里云RDS官方eclipse插件包是一款专为开发者设计的工具，旨在简化与阿里云关系型数据库服务（Relational Database Service，简称RDS）的交互过程，提高开发和调试的效率。这款插件适用于使用Eclipse集成开发环境的...
阿里云ECS官方eclipse插件包
07-10
2. **content.jar** 和 **artifacts.jar**：这两个文件是插件的核心组件，包含了一系列的类和资源，用于实现与阿里云ECS服务的交互。它们包含了必要的API调用、数据模型和UI元素，使得开发者能在Eclipse内部无缝地...
Elastic 及阿里云 AI 搜索 Tech Day 将于 7 月 27 日在上海举办
Elastic 中国社区官方博客
07-22 1242
参加 Elastic 原厂与阿里云联合举办的 Generative AI 技术交流分享日。借助 The Elastic Search AI Platform， 使用开放且灵活的企业解决方案，以前所未有的速度获得搜索最相关的结果。借助强大搜索功能和生成式 AI 的合力，加速商业转化和业务成果。
分析性能提升40%，阿里云Hologres流量场景最佳实践
科技很有意思
07-23 992
文章探讨了流量数据分析在互联网时代对企业决策的重要性，指出流量数据源于用户在APP、网站等平台的操作，经数据埋点采集、清洗和存储，成为实时数据仓库的基础。
通过mqtt.fx连接阿里云
最新发布
jingling122的博客
07-24 454
通过mqtt.fx连接阿里云，需要完成以下步骤。请注意，以下步骤基于当前可用的信息和常见的操作流程，但实际操作时可能因阿里云平台的更新而有所变化。
spring-boot-maven-plugin 阿里云报红
07-15
### 回答1： Spring Boot Maven插件是一个用于构建和运行Spring Boot应用程序的插件，它可以简化应用程序的开发和部署过程。阿里云报红可能是由于一些配置或依赖问题引起的。以下是几种可能的解决方法： 1. 版本问题：检查Spring Boot Maven插件的版本与阿里云的其他依赖项的版本是否兼容。可以尝试升级或降级插件版本，以解决冲突。 2. 依赖问题：请确保所有的依赖项都正确地添加到项目的pom.xml文件中，并且没有重复或冲突的依赖项。可以使用`mvn dependency:tree`命令来查看项目的依赖树，以确定是否存在任何问题。 3. 代理设置：如果您的机器上有代理服务器，请确保正确配置了Maven的代理。在您的settings.xml文件中添加或更新代理配置，以便Maven可以正确连接到阿里云。 4. 配置问题：检查Spring Boot Maven插件的配置是否正确。特别是，确保已正确配置插件的元数据URL，以便从阿里云下载依赖项。 5. 本地缓存问题：有时候，Maven的本地缓存可能会出现问题。您可以尝试删除本地.m2目录中的缓存文件夹，然后重新构建项目，以重新下载依赖项。 希望以上方法能够帮助您解决Spring Boot Maven插件与阿里云冲突的问题。如果问题仍然存在，请尝试搜索和查找更多的解决方法或向阿里云支持团队寻求帮助。 ### 回答2： spring-boot-maven-plugin 是一个用于构建和打包 Spring Boot 应用程序的 Maven 插件。当在使用该插件时，有时候在阿里云的环境下会出现报红现象。 造成在阿里云环境下 spring-boot-maven-plugin 报红的原因可能有以下几种： 1. 镜像配置问题：阿里云的镜像源可能无法正确解析插件的依赖关系。在 Maven 的 settings.xml 文件中，可以尝试将镜像源配置为阿里云提供的镜像，以确保插件依赖的正确下载。 2. 版本兼容性问题：有时候阿里云的环境使用的 Maven 版本与插件的版本不兼容，导致报红。可以尝试升级或降级插件的版本，以解决版本兼容性问题。 3. 依赖冲突问题：阿里云的环境下可能存在其他依赖与 spring-boot-maven-plugin 产生冲突，导致报红。可以检查 pom.xml 文件中的依赖关系，尝试解决冲突问题，例如通过 exclusions 排除冲突的依赖。 解决 spring-boot-maven-plugin 在阿里云报红的方法如下： 1. 检查并更换镜像源：在 Maven 的 settings.xml 文件中，将镜像源配置为阿里云提供的镜像。例如，可以使用下面的镜像配置： ```xml <mirror> <id>aliyun</id> <name>Aliyun Maven Mirror</name> <url>https://maven.aliyun.com/repository/public/</url> <mirrorOf>central</mirrorOf> </mirror> ``` 2. 升级或降级插件版本：根据阿里云的环境，尝试升级或降级 spring-boot-maven-plugin 的版本。可以通过在 pom.xml 文件中指定插件的版本来实现。 3. 解决依赖冲突：检查并解决 pom.xml 文件中的依赖冲突问题。可以通过 exclusions 排除冲突的依赖，或者调整依赖的版本。 综上所述，当在阿里云环境下 spring-boot-maven-plugin 报红时，可以尝试更换镜像源、升级或降级插件版本，以及解决依赖冲突来解决该问题。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值