从member->session->channel 拿 audio数据,做重采样处理,switch_buffer_write 写入member->audio_buffer, 数据会在conference当前会议线程来mux。
会议线程里做混音:从audio_buffer读取数据,通过switch_buffer_write将混音数据写入每个成员的omember->mux_buffer
混音后的数据在从member/mux_buffer取出,switch_core_session_write_frame 写回到member channel。
音频处理过程:
音频获取和重采样-》混音音频输入-》混音-》混音输出-》写入到member
整个过程中比较重要的三个buffer:
在函数conference_member_setup_media中分配的内存:
=>switch_buffer_create_dynamic(&member->resample_buffer //重采样buffer
=>switch_buffer_create_dynamic(&member->audio_buffer //输入buffer
=>switch_buffer_create_dynamic(&member->mux_buffer //输出buffer
1.音频获取和重采样:
输出线程中启动【input线程】:conference_loop_launch_input(member, switch_core_session_get_pool(member->session));
conference_loop_input是一个线程,conference_loop_output是一个循环。
从member->session->channel 拿 audio数据,做重采样处理
=》conference_loop_launch_input
=>conference_loop_input
=>switch_core_session_read_frame
=>switch_resample_process
=>switch_buffer_write(member->audio_buffer, tmp_frame.data, tmp_frame.datalen);
2.混音音频输入:
conference_thread_run
=》ok = switch_buffer_write(omember->mux_buffer, write_frame, bytes);
switch_buffer_write 写入member->audio_buffer, 数据会在conference当前会议线程来mux
3.混音:
conference_thread_run
将已知正在产生音频的每个成员的音频复制到主帧中(mux?):
for (omember = conference->members; omember; omember = omember->next) {
conference->member_loop_count++;
if (!(conference_utils_member_test_flag(omember, MFLAG_RUNNING) && conference_utils_member_test_flag(omember, MFLAG_HAS_AUDIO))) {
continue;
}
bptr = (int16_t *) omember->frame;
for (x = 0; x < omember->read / 2; x++) {
main_frame[x] += (int32_t) bptr[x];
}
}
接下来对各个member具备的能力进行分别处理:
conference_utils_member_test_flag(imember, MFLAG_HAS_AUDIO)
switch_test_flag(rel, RFLAG_CAN_SPEAK)
switch_test_flag(rel, RFLAG_CAN_HEAR)
//为了保持采样样本的精确性,主帧采用的是32位,这里做了32位转换成16位?
switch_normalize_to_16bit(z);
write_frame[x] = (int16_t) z;
conference_thread_run线程中通过read每个member的audio_buffer来判断该member是否具备audio能力:
switch_buffer_read(imember->audio_buffer, imember->frame, bytes)
conference_utils_member_set_flag_locked(imember, MFLAG_HAS_AUDIO);
ready++;
后面会根据ready的判断做出不同的操作
如果具备的话:
从audio_buffer读取数据,通过switch_buffer_write将混音数据写入每个成员的omember->mux_buffer
如果不具备的话:
ok = switch_buffer_write(omember->mux_buffer, write_frame, bytes);
4.混音音频输出
混音后的数据在从member/mux_buffer取出
ok = switch_buffer_write(omember->mux_buffer, write_frame, bytes);
5.写回到member channel
conference_loop_output
=》switch_core_session_write_frame