1. 上层接口设置
音频的相关参数需要在创建audioTrack的时候就去设置,在iOS端(Android类似设置的方式如下:
NSDictionary *mandatoryConstraints = @{};
NSDictionary *optionalConstraints = @{ @"googEchoCancellation" : openEc?kRTCMediaConstraintsValueTrue:kRTCMediaConstraintsValueFalse,
@"googAutoGainControl" : openGc?kRTCMediaConstraintsValueTrue:kRTCMediaConstraintsValueFalse,
@"googNoiseSuppression" :(!openNoise)?kRTCMediaConstraintsValueTrue:kRTCMediaConstraintsValueFalse,};
// ,@"ios_force_software_aec_HACK"
RTCMediaConstraints *constraints = [[RTCMediaConstraints alloc] initWithMandatoryConstraints:mandatoryConstraints
optionalConstraints:optionalConstraints];
RTCMediaConstraints *constraints = [self defaultMediaAudioConstraintsWithOpenEc:_openEc openGC:_openGc noise:_openNs];
RTCAudioSource *source = [_factory audioSourceWithConstraints:constraints];
上面三个参数一次是
- googEchoCancellation 是AEC也就是回声消除
- googAutoGainControl 是AGC 也就是增益控制
- googNoiseSuppression 也就是NS 噪声控制
其中ios
提供了强制开启AEC的参数ios_force_software_aec_HACK
,但是在参数转换的时候并没有效果,如何让这个参数有效在下节分析;
当然在iOS端上面的参数都有没有效果的,这个在后面分析设置apm参数的时候再说为啥;
Android按照上面设置是可以生效,也就是可以作为apm的参数使用;
1.1 参数转换
在iOS端设置完次参数后会在RTCPeerConnectionFactory的audioSourceWithConstraints方法中转换,实现如下:
cricket::AudioOptions options;
CopyConstraintsIntoAudioOptions(nativeConstraints.get(), &options);
可以看到这里参数转换的核心调用是CopyConstraintsIntoAudioOptions方法,这个方法在media_constaints.cc中,实现如下:
void CopyConstraintsIntoAudioOptions(const MediaConstraints* constraints,
cricket::AudioOptions* options) {
if (!constraints) {
return;
}
ConstraintToOptional<bool>(constraints,
MediaConstraints::kGoogEchoCancellation,
&options->echo_cancellation);
ConstraintToOptional<bool>(constraints,
。。。省略
ConstraintToOptional<bool>(constraints, MediaConstraints::kAutoGainControl,
&options->auto_gain_control);
ConstraintToOptional<bool>(constraints,
。。。省略
MediaConstraints::kNoiseSuppression,
&options->noise_suppression);
ConstraintToOptional<bool>(constraints,
。。。省略
#if defined(WEBRTC_IOS)
// // hosten 修改参数
// ConstraintToOptional<bool>(constraints, MediaConstraints::kiOSForceSoftwareAecHACK,
// &options->ios_force_software_aec_HACK);
#endif
if (options->audio_network_adaptor_config) {
options->audio_network_adaptor = true;
}
// end
}
在上述代码中可以看到相关的参数转换成了AudioOptions
,赋值给出参options;
在这里上节说的ios_force_software_aec_HACK这个参数如果想生效需要吧按照注释中修改参数那里的方式添加,其中的kiOSForceSoftwareAecHACK,定义如下:
#if defined(WEBRTC_IOS)
const char MediaConstraints::kiOSForceSoftwareAecHACK[] = "googiOSForceSoftwareAecHACK";
#endif
按照以上方式设置就能让此参数生效;
1.2 参数传递
在audioSourceWithConstraints:方法中转换参数成AudioOptions
,后接着调用Factory的CreateAudioSource方法:
rtc::scoped_refptr<AudioSourceInterface>
PeerConnectionFactory::CreateAudioSource(const cricket::AudioOptions& options) {
rtc::scoped_refptr<LocalAudioSource> source(
LocalAudioSource::Create(&options));
return source;
}
这个里面主要是调用LocalAudioSource的Create方法去创建AudioSourceInterface对象;最终options保存在options_
属性中;调用栈如下:
2. AudioOption的传递流程
通过在代码中断点我们可以定位到这个参数首次的使用是在AudioRtpSender的SetSend
中调用,如下:
void AudioRtpSender::SetSend() {
...省略
cricket::AudioOptions options;
#if !defined(WEBRTC_CHROMIUM_BUILD) && !defined(WEBRTC_WEBKIT_BUILD)
if (track_->enabled() && audio_track()->GetSource() &&
!audio_track()->GetSource()->remote()) {
options = audio_track()->GetSource()->options();
}
#endif
bool track_enabled = track_->enabled();
bool success = worker_thread_->Invoke<bool>(RTC_FROM_HERE, [&] {
return voice_media_channel()->SetAudioSend(ssrc_, track_enabled, &options,
sink_adapter_.get());
});
}
可以看到这里从audio_track获取到souce,上一节最后AudioOptions
参数最终保存在Source的options_
变量;
接着切换到工作线程调用channal的SetAudioSend方法这里的调试堆栈如下:
在work线程中调用堆栈如下:
其中WebRtcVoiceEngine的ApplyOptions中最终使用这个参数,参数如下:
这里和我们上面的OC层接口配置的参数一致;
还有其他调用ApplyOptions
的流程这里暂时不做分析,不是主要的流程,后续如果有时间补充;
3. apm参数配置
最终的参数设置是在ApplyOptions方法中,这个方法比较的多且分平台处理,这里分解说明如下:
3.1 AEC的处理
分平台的处理,源码如下:
#if defined(WEBRTC_IOS)
if (options.ios_force_software_aec_HACK &&
*options.ios_force_software_aec_HACK) {
// EC may be forced on for a device known to have non-functioning platform
// AEC.
options.echo_cancellation = true;
options.extended_filter_aec = true;
RTC_LOG(LS_WARNING)
<< "Force software AEC on iOS. May conflict with platform AEC.";
} else {
// On iOS, VPIO provides built-in EC.
options.echo_cancellation = false;
options.extended_filter_aec = false;
RTC_LOG(LS_INFO) << "Always disable AEC on iOS. Use built-in instead.";
}
#elif defined(WEBRTC_ANDROID)
ec_mode = webrtc::kEcAecm;
options.extended_filter_aec = false;
#endif
// Delay Agnostic AEC automatically turns on EC if not set except on iOS
// where the feature is not supported.
bool use_delay_agnostic_aec = false;
#if !defined(WEBRTC_IOS)
if (options.delay_agnostic_aec) {
use_delay_agnostic_aec = *options.delay_agnostic_aec;
if (use_delay_agnostic_aec) {
options.echo_cancellation = true;
options.extended_filter_aec = true;
ec_mode = webrtc::kEcConference;
}
}
#endif
对于ios如果启用了ios_force_software_aec_HACK
,这里就会设置echo_cancellation为true也就是启用软件的AEC,否则ios这里是直接设置成false,也就是使用VOIP自带的AEC;
对于Android 等其他平台如果开启了delay_agnostic_aec
,这里会这直接把echo_cancellation设置为true;
3.2 NS降噪处理
降噪这里也是按照不同平台有不通的处理:
#if defined(WEBRTC_IOS)
//options.noise_suppression = false;
options.typing_detection = false;
options.experimental_ns = false;
RTC_LOG(LS_INFO) << "Always disable NS on iOS. Use built-in instead.";
#elif defined(WEBRTC_ANDROID)
options.typing_detection = false;
options.experimental_ns = false;
#endif
// Set and adjust gain control options.
#if defined(WEBRTC_IOS)
options.auto_gain_control = false;
options.experimental_agc = false;
RTC_LOG(LS_INFO) << "Always disable AGC on iOS. Use built-in instead.";
#elif defined(WEBRTC_ANDROID)
options.experimental_agc = false;
#endif
对于ios 这里的auto_gain_control
默认是设置成false,如果需要启用外部参数控制这里需要注释掉;
4. 各平台ADM接口调用
3A算法在各个的平台都有实现,webrtc这里通过调用ADM的相关方法判断平台是不是支持响应的算法从而是否启用webrtc的3A,其实现如下:
if (options.echo_cancellation) {
// in combination with Open SL ES audio.
const bool built_in_aec = adm()->BuiltInAECIsAvailable();
if (built_in_aec) {
const bool enable_built_in_aec =
*options.echo_cancellation && !use_delay_agnostic_aec;
if (adm()->EnableBuiltInAEC(enable_built_in_aec) == 0 &&
enable_built_in_aec) {
options.echo_cancellation = false;
RTC_LOG(LS_INFO)
<< "Disabling EC since built-in EC will be used instead";
}
}
webrtc::apm_helpers::SetEcStatus(apm(), *options.echo_cancellation,
ec_mode);
}
if (options.auto_gain_control) {
bool built_in_agc_avaliable = adm()->BuiltInAGCIsAvailable();
if (built_in_agc_avaliable) {
if (adm()->EnableBuiltInAGC(*options.auto_gain_control) == 0 &&
*options.auto_gain_control) {
options.auto_gain_control = false;
RTC_LOG(LS_INFO)
<< "Disabling AGC since built-in AGC will be used instead";
}
}
}
if (options.noise_suppression) {
if (adm()->BuiltInNSIsAvailable()) {
bool builtin_ns = *options.noise_suppression;
if (adm()->EnableBuiltInNS(builtin_ns) == 0 && builtin_ns) {
options.noise_suppression = false;
RTC_LOG(LS_INFO)
<< "Disabling NS since built-in NS will be used instead";
}
}
webrtc::apm_helpers::SetNsStatus(apm(), *options.noise_suppression);
}
需要说明的是ios的adm是没有实现BuiltInAECIsAvailable,BuiltInAGCIsAvailable等方法的,所以这里都是返回false;
对于其他平台都是先调用对应的buildin方法判断平台是否开启了对应的算法,如果开启了就调用对应的enable关闭;
后续的ApplyConfig参数设置,这里就不做介绍感兴趣的参看源码;