【x264】编码模块(x264_encoder_encode)简单分析
- 1.编码帧函数(x264_encoder_encode)
- 1.1 拷贝一帧并移动到buffer中(x264_frame_pop_unused)
- 1.2 拷贝一帧送入队列用于确定帧类型(x264_lookahead_put_frame)
- 1.3 从lookahead中获取帧(x264_lookahead_get_frames)
- 1.4 获取要编码的帧(x264_frame_shift)
- 1.5 根据帧的类型初始化数据依赖关系
- 1.6 构建双向的参考帧列表(reference_build_list)
- 1.7 写入码流文件(x264_sps_write,x264_pps_write等)
- 1.8 开始进行码率控制(x264_ratecontrol_start)
- 1.9 创建帧头信息(slice_init)
- 1.10 帧编码(slices_write -> slice_write)
- 1.11 处理额外信息且更新编码器状态(encoder_frame_end)
- 2.小结
参考:
雷霄骅博士,x264源代码简单分析:x264_slice_write()
流程分析:
【x264】x264编码主流程简单分析
1.编码帧函数(x264_encoder_encode)
x264_encoder_encode是x264编码器的核心函数,实现了编码一个视频帧的功能。它的工作流程为:
- 拷贝一帧并移动到buffer中(x264_frame_pop_unused)
- 拷贝一帧送入队列用于确定帧类型(x264_lookahead_put_frame)
- 从lookahead获取帧(x264_lookahead_get_frames)
- 获取要编码的帧(x264_frame_shift)
- 根据帧的类型初始化数据依赖关系,如I帧,P帧,B帧
- 构建双向的参考帧列表(reference_build_list)
- 写入码流文件(x264_sps_write,x264_pps_write等)
- 开始进行码率控制(x264_ratecontrol_start)
- 创建帧的头信息(slice_init)
- 帧编码(slices_write -> slice_write)
- 处理额外信息且更新编码器状态(encoder_frame_end)
1.1 拷贝一帧并移动到buffer中(x264_frame_pop_unused)
该函数主要的功能是从h中拷贝一帧出来,移动到buffer中,并且初始化这个帧之中的信息。h是encoder的不可视化句柄,横贯上下文
x264_frame_t *x264_frame_pop_unused( x264_t *h, int b_fdec )
{
x264_frame_t *frame;
// 如果这一帧没有使用,则会取出一帧,否则new一帧
if( h->frames.unused[b_fdec][0] )
frame = x264_frame_pop( h->frames.unused[b_fdec] );
else
frame = frame_new( h, b_fdec );
if( !frame )
return NULL;
// 将帧中的信息初始化
frame->b_last_minigop_bframe = 0;
frame->i_reference_count = 1;
frame->b_intra_calculated = 0;
frame->b_scenecut = 1;
frame->b_keyframe = 0;
frame->b_corrupt = 0;
frame->i_slice_count = h->param.b_sliced_threads ? h->param.i_threads : 1;
memset( frame->weight, 0, sizeof(frame->weight) );
memset( frame->f_weighted_cost_delta, 0, sizeof(frame->f_weighted_cost_delta) );
return frame;
}
1.2 拷贝一帧送入队列用于确定帧类型(x264_lookahead_put_frame)
将前面取出的帧送入到lookahead队列中,并确认帧的类型。在x264编码器中,Lookahead队列中存储的帧信息是当前帧的前向参考帧,以及由其后继的参考帧组成的预测帧序列。这个预测帧序列在编码过程中用来评估当前帧的最优编码方式。通过预测帧序列,x264编码器可以提前看到未来的参考帧,从而更准确地决定当前帧的最优编码方式。
Lookahead队列的作用是提高视频编码的质量和效率。通过预测未来的参考帧,可以更准确地选择最优的编码方式,从而达到更好的视频质量。此外,Lookahead队列还可以利用多线程处理技术,在不影响编码速度的情况下提高编码效率。
void x264_lookahead_put_frame( x264_t *h, x264_frame_t *frame )
{
if( h->param.i_sync_lookahead )
x264_sync_frame_list_push( &h->lookahead->ifbuf, frame ); // 进行预测,将frame推入输入帧列表
else
x264_sync_frame_list_push( &h->lookahead->next, frame ); // 不进行预测,将frame推入next列表(下一帧)
}
1.3 从lookahead中获取帧(x264_lookahead_get_frames)
该函数分析lookahead中的帧信息,如果存在lookahead线程,则在这里获取帧;如果不存在lookahead线程,这里会决定slice type信息
void x264_lookahead_get_frames( x264_t *h )
{
if( h->param.i_sync_lookahead )
{ /* We have a lookahead thread, so get frames from there */
x264_pthread_mutex_lock( &h->lookahead->ofbuf.mutex );
while( !h->lookahead->ofbuf.i_size && h->lookahead->b_thread_active )
x264_pthread_cond_wait( &h->lookahead->ofbuf.cv_fill, &h->lookahead->ofbuf.mutex );
lookahead_encoder_shift( h );
x264_pthread_mutex_unlock( &h->lookahead->ofbuf.mutex );
}
else
{ /* We are not running a lookahead thread, so perform all the slicetype decide on the fly */
if( h->frames.current[0] || !h->lookahead->next.i_size )
return;
x264_slicetype_decide( h );
lookahead_update_last_nonb( h, h->lookahead->next.list[0] );
int shift_frames = h->lookahead->next.list[0]->i_bframes + 1;
lookahead_shift( &h->lookahead->ofbuf, &h->lookahead->next, shift_frames );
/* For MB-tree and VBV lookahead, we have to perform propagation analysis on I-frames too. */
if( h->lookahead->b_analyse_keyframe && IS_X264_TYPE_I( h->lookahead->last_nonb->i_type ) )
x264_slicetype_analyse( h, shift_frames );
lookahead_encoder_shift( h );
}
}
1.4 获取要编码的帧(x264_frame_shift)
从frame_list中获取一帧
x264_frame_t *x264_frame_shift( x264_frame_t **list )
{
x264_frame_t *frame = list[0];
int i;
for( i = 0; list[i]; i++ )
list[i] = list[i+1];
assert(frame);
return frame;
}
1.5 根据帧的类型初始化数据依赖关系
判断帧的类型是IDR、I、P或者是B帧,并根据其类型进行参数配置
if( h->fenc->i_type == X264_TYPE_IDR )
{
/* reset ref pictures */
i_nal_type = NAL_SLICE_IDR;
i_nal_ref_idc = NAL_PRIORITY_HIGHEST;
h->sh.i_type = SLICE_TYPE_I;
reference_reset( h );
h->frames.i_poc_last_open_gop = -1;
}
else if( h->fenc->i_type == X264_TYPE_I )
{
i_nal_type = NAL_SLICE;
i_nal_ref_idc = NAL_PRIORITY_HIGH; /* Not completely true but for now it is (as all I/P are kept as ref)*/
h->sh.i_type = SLICE_TYPE_I;
reference_hierarchy_reset( h );
if( h->param.b_open_gop )
h->frames.i_poc_last_open_gop = h->fenc->b_keyframe ? h->fenc->i_poc : -1;
}
else if( h->fenc->i_type == X264_TYPE_P )
{
i_nal_type = NAL_SLICE;
i_nal_ref_idc = NAL_PRIORITY_HIGH; /* Not completely true but for now it is (as all I/P are kept as ref)*/
h->sh.i_type = SLICE_TYPE_P;
reference_hierarchy_reset( h );
h->frames.i_poc_last_open_gop = -1;
}
else if( h->fenc->i_type == X264_TYPE_BREF )
{
i_nal_type = NAL_SLICE;
i_nal_ref_idc = h->param.i_bframe_pyramid == X264_B_PYRAMID_STRICT ? NAL_PRIORITY_LOW : NAL_PRIORITY_HIGH;
h->sh.i_type = SLICE_TYPE_B;
reference_hierarchy_reset( h );
}
else /* B frame */
{
i_nal_type = NAL_SLICE;
i_nal_ref_idc = NAL_PRIORITY_DISPOSABLE;
h->sh.i_type = SLICE_TYPE_B;
}
1.6 构建双向的参考帧列表(reference_build_list)
根据前面判断的帧的类型,可以确定参考帧的情况;对于IDR帧,没有参考帧;对于P帧,有前向参考帧;对于B帧,有双向参考帧。构建的参考帧列表有二维,第一个维度0记录的是前向参考帧的idx,第二个维度1记录的是后向参考帧的idx。
函数主要工作流程为:
- 将参考帧存储到队列中
- 根据与当前帧的距离进行排序
- 检查我们是否选择了与标准默认值不同的引用列表顺序
static inline void reference_build_list( x264_t *h, int i_poc )
{
int b_ok;
/* build ref list 0/1 */
h->mb.pic.i_fref[0] = h->i_ref[0] = 0;
h->mb.pic.i_fref[1] = h->i_ref[1] = 0;
if( h->sh.i_type == SLICE_TYPE_I )
return;
// ----- 1.将参考帧存储到队列中 ----- //
for( int i = 0; h->frames.reference[i]; i++ )
{
if( h->frames.reference[i]->b_corrupt )
continue;
if( h->frames.reference[i]->i_poc < i_poc )
h->fref[0][h->i_ref[0]++] = h->frames.reference[i]; // 存入前向参考队列
else if( h->frames.reference[i]->i_poc > i_poc )
h->fref[1][h->i_ref[1]++] = h->frames.reference[i]; // 存入后向参考队列
}
if( h->sh.i_mmco_remove_from_end )
{
/* Order ref0 for MMCO remove */
do
{
b_ok = 1;
for( int i = 0; i < h->i_ref[0] - 1; i++ )
{
if( h->fref[0][i]->i_frame < h->fref[0][i+1]->i_frame )
{
XCHG( x264_frame_t*, h->fref[0][i], h->fref[0][i+1] );
b_ok = 0;
break;
}
}
} while( !b_ok );
for( int i = h->i_ref[0]-1; i >= h->i_ref[0] - h->sh.i_mmco_remove_from_end; i-- )
{
int diff = h->i_frame_num - h->fref[0][i]->i_frame_num;
h->sh.mmco[h->sh.i_mmco_command_count].i_poc = h->fref[0][i]->i_poc;
h->sh.mmco[h->sh.i_mmco_command_count++].i_difference_of_pic_nums = diff;
}
}
// ----- 2.根据与当前帧的距离排序参考列表 ----- //
/* Order reference lists by distance from the current frame. */
for( int list = 0; list < 2; list++ )
{
h->fref_nearest[list] = h->fref[list][0];
do
{
b_ok = 1;
for( int i = 0; i < h->i_ref[list] - 1; i++ )
{
if( list ? h->fref[list][i+1]->i_poc < h->fref_nearest[list]->i_poc
: h->fref[list][i+1]->i_poc > h->fref_nearest[list]->i_poc )
h->fref_nearest[list] = h->fref[list][i+1];
if( reference_distance( h, h->fref[list][i] ) > reference_distance( h, h->fref[list][i+1] ) )
{
XCHG( x264_frame_t*, h->fref[list][i], h->fref[list][i+1] );
b_ok = 0;
break;
}
}
} while( !b_ok );
}
// ----- 3.检查我们是否选择了与标准默认值不同的引用列表顺序 ----- //
reference_check_reorder( h );
h->i_ref[1] = X264_MIN( h->i_ref[1], h->frames.i_max_ref1 );
h->i_ref[0] = X264_MIN( h->i_ref[0], h->frames.i_max_ref0 );
h->i_ref[0] = X264_MIN( h->i_ref[0], h->param.i_frame_reference ); // if reconfig() has lowered the limit
/* For Blu-ray compliance, don't reference frames outside of the minigop. */
if( IS_X264_TYPE_B( h->fenc->i_type ) && h->param.b_bluray_compat )
h->i_ref[0] = X264_MIN( h->i_ref[0], IS_X264_TYPE_B( h->fref[0][0]->i_type ) + 1 );
/* add duplicates */
if( h->fenc->i_type == X264_TYPE_P )
{
int idx = -1;
if( h->param.analyse.i_weighted_pred >= X264_WEIGHTP_SIMPLE )
{
x264_weight_t w[3];
w[1].weightfn = w[2].weightfn = NULL;
if( h->param.rc.b_stat_read )
x264_ratecontrol_set_weights( h, h->fenc ); // 码率控制设置参数
if( !h->fenc->weight[0][0].weightfn )
{
h->fenc->weight[0][0].i_denom = 0;
SET_WEIGHT( w[0], 1, 1, 0, -1 );
idx = weighted_reference_duplicate( h, 0, w );
}
else
{
if( h->fenc->weight[0][0].i_scale == 1<<h->fenc->weight[0][0].i_denom )
{
SET_WEIGHT( h->fenc->weight[0][0], 1, 1, 0, h->fenc->weight[0][0].i_offset );
}
weighted_reference_duplicate( h, 0, x264_weight_none );
if( h->fenc->weight[0][0].i_offset > -128 )
{
w[0] = h->fenc->weight[0][0];
w[0].i_offset--;
h->mc.weight_cache( h, &w[0] );
idx = weighted_reference_duplicate( h, 0, w );
}
}
}
h->mb.ref_blind_dupe = idx;
}
assert( h->i_ref[0] + h->i_ref[1] <= X264_REF_MAX );
h->mb.pic.i_fref[0] = h->i_ref[0];
h->mb.pic.i_fref[1] = h->i_ref[1];
}
1.7 写入码流文件(x264_sps_write,x264_pps_write等)
如果帧的类型为I帧,则还需要写入SPS和PPS的信息。
if( h->fenc->b_keyframe )
{
/* Write SPS and PPS */
if( h->param.b_repeat_headers )
{
/* generate sequence parameters */
nal_start( h, NAL_SPS, NAL_PRIORITY_HIGHEST );
x264_sps_write( &h->out.bs, h->sps ); // 写入SPS信息
if( nal_end( h ) )
return -1;
/* Pad AUD/SPS to 256 bytes like Panasonic */
if( h->param.i_avcintra_class )
h->out.nal[h->out.i_nal-1].i_padding = 256 - bs_pos( &h->out.bs ) / 8 - 2*NALU_OVERHEAD;
overhead += h->out.nal[h->out.i_nal-1].i_payload + h->out.nal[h->out.i_nal-1].i_padding + NALU_OVERHEAD;
/* generate picture parameters */
nal_start( h, NAL_PPS, NAL_PRIORITY_HIGHEST );
x264_pps_write( &h->out.bs, h->sps, h->pps ); // 写入PPS信息
if( nal_end( h ) )
return -1;
if( h->param.i_avcintra_class )
{
int total_len = 256;
/* Sony XAVC uses an oversized PPS instead of SEI padding */
if( h->param.i_avcintra_flavor == X264_AVCINTRA_FLAVOR_SONY )
total_len += h->param.i_height >= 1080 ? 18*512 : 10*512;
h->out.nal[h->out.i_nal-1].i_padding = total_len - h->out.nal[h->out.i_nal-1].i_payload - NALU_OVERHEAD;
}
overhead += h->out.nal[h->out.i_nal-1].i_payload + h->out.nal[h->out.i_nal-1].i_padding + NALU_OVERHEAD;
}
/* when frame threading is used, buffering period sei is written in encoder_frame_end */
if( h->i_thread_frames == 1 && h->sps->vui.b_nal_hrd_parameters_present )
{
x264_hrd_fullness( h );
nal_start( h, NAL_SEI, NAL_PRIORITY_DISPOSABLE );
x264_sei_buffering_period_write( h, &h->out.bs );
if( nal_end( h ) )
return -1;
overhead += h->out.nal[h->out.i_nal-1].i_payload + SEI_OVERHEAD;
}
}
1.8 开始进行码率控制(x264_ratecontrol_start)
开始进行码率控制,为当前的帧选择一个QP。主要的工作流程为:
- 检查vbv模式
- 检查abr模式
- 检查2pass模式
- 检查CQP模式
- 将qp进行clip操作,固定在特定范围内
- 更新pqp
码率控制是编码器当中一个重要的技术思想,使用另外的章节记录
void x264_ratecontrol_start( x264_t *h, int i_force_qp, int overhead )
{
x264_ratecontrol_t *rc = h->rc;
ratecontrol_entry_t *rce = NULL;
x264_zone_t *zone = get_zone( h, h->fenc->i_frame );
float q;
x264_emms();
if( h->param.rc.b_stat_read )
{
int frame = h->fenc->i_frame;
assert( frame >= 0 && frame < rc->num_entries );
rce = rc->rce = &rc->entry[frame];
if( h->sh.i_type == SLICE_TYPE_B
&& h->param.analyse.i_direct_mv_pred == X264_DIRECT_PRED_AUTO )
{
h->sh.b_direct_spatial_mv_pred = ( rce->direct_mode == 's' );
h->mb.b_direct_auto_read = ( rce->direct_mode == 's' || rce->direct_mode == 't' );
}
}
// ----- 1.检查是否使用vbv模式 ----- //
// 在VBV模式下,x264允许每个MB有不同的QP值,其他模式下,整帧的QP值是固定的
if( rc->b_vbv )
{
memset( h->fdec->i_row_bits, 0, h->mb.i_mb_height * sizeof(int) );
memset( h->fdec->f_row_qp, 0, h->mb.i_mb_height * sizeof(float) );
memset( h->fdec->f_row_qscale, 0, h->mb.i_mb_height * sizeof(float) );
rc->row_pred = rc->row_preds[h->sh.i_type];
rc->buffer_rate = h->fenc->i_cpb_duration * rc->vbv_max_rate * h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale;
update_vbv_plan( h, overhead );
const x264_level_t *l = x264_levels;
while( l->level_idc != 0 && l->level_idc != h->param.i_level_idc )
l++;
int mincr = l->mincr;
if( h->param.b_bluray_compat )
mincr = 4;
/* Profiles above High don't require minCR, so just set the maximum to a large value. */
if( h->sps->i_profile_idc > PROFILE_HIGH )
rc->frame_size_maximum = 1e9;
else
{
/* The spec has a bizarre special case for the first frame. */
if( h->i_frame == 0 )
{
//384 * ( Max( PicSizeInMbs, fR * MaxMBPS ) + MaxMBPS * ( tr( 0 ) - tr,n( 0 ) ) ) / MinCR
double fr = 1. / (h->param.i_level_idc >= 60 ? 300 : 172);
int pic_size_in_mbs = h->mb.i_mb_width * h->mb.i_mb_height;
rc->frame_size_maximum = 384 * BIT_DEPTH * X264_MAX( pic_size_in_mbs, fr*l->mbps ) / mincr;
}
else
{
//384 * MaxMBPS * ( tr( n ) - tr( n - 1 ) ) / MinCR
rc->frame_size_maximum = 384 * BIT_DEPTH * ((double)h->fenc->i_cpb_duration * h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale) * l->mbps / mincr;
}
}
}
if( h->sh.i_type != SLICE_TYPE_B )
rc->bframes = h->fenc->i_bframes;
// ----- 2.检查是否是abr模式 ----- //
if( rc->b_abr )
{
q = qscale2qp( rate_estimate_qscale( h ) );
}
else if( rc->b_2pass ) // ----- 3.检查是否是2pass模式 ----- //
{
rce->new_qscale = rate_estimate_qscale( h );
q = qscale2qp( rce->new_qscale );
}
else /* CQP */ // ----- 4.CQP模式,QP为固定值 ----- //
{
if( h->sh.i_type == SLICE_TYPE_B && h->fdec->b_kept_as_ref )
q = ( rc->qp_constant[ SLICE_TYPE_B ] + rc->qp_constant[ SLICE_TYPE_P ] ) / 2;
else
q = rc->qp_constant[ h->sh.i_type ];
if( zone )
{
if( zone->b_force_qp )
q += zone->i_qp - rc->qp_constant[SLICE_TYPE_P];
else
q -= 6*log2f( zone->f_bitrate_factor );
}
}
if( i_force_qp != X264_QP_AUTO )
q = i_force_qp - 1;
// ---- 5.将qp进行clip操作,固定在特定范围内 ----- //
q = x264_clip3f( q, h->param.rc.i_qp_min, h->param.rc.i_qp_max );
rc->qpa_rc = rc->qpa_rc_prev =
rc->qpa_aq = rc->qpa_aq_prev = 0;
h->fdec->f_qp_avg_rc =
h->fdec->f_qp_avg_aq =
rc->qpm = q;
if( rce )
rce->new_qp = q;
// ----- 6.更新pqp ----- //
accum_p_qp_update( h, rc->qpm );
if( h->sh.i_type != SLICE_TYPE_B )
rc->last_non_b_pict_type = h->sh.i_type;
}
1.9 创建帧头信息(slice_init)
创建帧的头部信息,如果是IDR帧,slice的头信息还会包括i_idr_pic_id信息。slice的头信息当中,包括了SPS、PPS、frame_num、qp和delta_qp等等信息。函数主要的工作流程为:
- 检查是否IDR帧
- 初始化slice头部信息(根据是否是IDR帧有所区别)
- 利用slice的信息初始化mb
static inline void slice_init( x264_t *h, int i_nal_type, int i_global_qp )
{
/* ------------------------ Create slice header ----------------------- */
// ----- 1.检查是否是IDR帧 ----- //
if( i_nal_type == NAL_SLICE_IDR )
{ // ----- 2.初始化slice头部信息 ----- //
slice_header_init( h, &h->sh, h->sps, h->pps, h->i_idr_pic_id, h->i_frame_num, i_global_qp );
/* alternate id */
if( h->param.i_avcintra_class )
{
switch( h->i_idr_pic_id )
{
case 5:
h->i_idr_pic_id = 3;
break;
case 3:
h->i_idr_pic_id = 4;
break;
case 4:
default:
h->i_idr_pic_id = 5;
break;
}
}
else
h->i_idr_pic_id ^= 1;
}
else
{
slice_header_init( h, &h->sh, h->sps, h->pps, -1, h->i_frame_num, i_global_qp );
h->sh.i_num_ref_idx_l0_active = h->i_ref[0] <= 0 ? 1 : h->i_ref[0];
h->sh.i_num_ref_idx_l1_active = h->i_ref[1] <= 0 ? 1 : h->i_ref[1];
if( h->sh.i_num_ref_idx_l0_active != h->pps->i_num_ref_idx_l0_default_active ||
(h->sh.i_type == SLICE_TYPE_B && h->sh.i_num_ref_idx_l1_active != h->pps->i_num_ref_idx_l1_default_active) )
{
h->sh.b_num_ref_idx_override = 1;
}
}
if( h->fenc->i_type == X264_TYPE_BREF && h->param.b_bluray_compat && h->sh.i_mmco_command_count )
{
h->b_sh_backup = 1;
h->sh_backup = h->sh;
}
h->fdec->i_frame_num = h->sh.i_frame_num;
if( h->sps->i_poc_type == 0 )
{
h->sh.i_poc = h->fdec->i_poc;
if( PARAM_INTERLACED )
{
h->sh.i_delta_poc_bottom = h->param.b_tff ? 1 : -1;
h->sh.i_poc += h->sh.i_delta_poc_bottom == -1;
}
else
h->sh.i_delta_poc_bottom = 0;
h->fdec->i_delta_poc[0] = h->sh.i_delta_poc_bottom == -1;
h->fdec->i_delta_poc[1] = h->sh.i_delta_poc_bottom == 1;
}
else
{
/* Nothing to do ? */
}
// ----- 3.将slice的信息拷贝给mb ----- //
x264_macroblock_slice_init( h );
}
1.10 帧编码(slices_write -> slice_write)
帧编码函数是对一帧进行分析和编码的核心函数,在进行一些参数的配置之后,直接调用了slice_write进行编码。主要工作流程为:
- 将analyser中的配置参数初始化给mb
- 将QP设置为片中的第一个QP,以获得更准确的CABAC初始化
- 初始化cabac
- 循环编码slice当中的每个mb
- 分析mb(预测)
- 编码mb
- 收集mb的信息进行更新
- 是否进行deblock
- 如果使用多线程,则告知主线程编码一个slice完成
static intptr_t slice_write( x264_t *h )
{
int i_skip;
int mb_xy, i_mb_x, i_mb_y;
/* NALUs other than the first use a 3-byte startcode.
* Add one extra byte for the rbsp, and one more for the final CABAC putbyte.
* Then add an extra 5 bytes just in case, to account for random NAL escapes and
* other inaccuracies. */
int overhead_guess = (NALU_OVERHEAD - (h->param.b_annexb && h->out.i_nal)) + 1 + h->param.b_cabac + 5;
int slice_max_size = h->param.i_slice_max_size > 0 ? (h->param.i_slice_max_size-overhead_guess)*8 : 0;
int back_up_bitstream_cavlc = !h->param.b_cabac && h->sps->i_profile_idc < PROFILE_HIGH;
int back_up_bitstream = slice_max_size || back_up_bitstream_cavlc;
int starting_bits = bs_pos(&h->out.bs);
int b_deblock = h->sh.i_disable_deblocking_filter_idc != 1;
int b_hpel = h->fdec->b_kept_as_ref;
int orig_last_mb = h->sh.i_last_mb;
int thread_last_mb = h->i_threadslice_end * h->mb.i_mb_width - 1;
uint8_t *last_emu_check;
#define BS_BAK_SLICE_MAX_SIZE 0
#define BS_BAK_CAVLC_OVERFLOW 1
#define BS_BAK_SLICE_MIN_MBS 2
#define BS_BAK_ROW_VBV 3
x264_bs_bak_t bs_bak[4];
b_deblock &= b_hpel || h->param.b_full_recon || h->param.psz_dump_yuv;
bs_realign( &h->out.bs );
/* Slice */
nal_start( h, h->i_nal_type, h->i_nal_ref_idc );
h->out.nal[h->out.i_nal].i_first_mb = h->sh.i_first_mb;
// ----- 1.将analyser中的配置参数初始化给mb ----- //
/* Slice header */
x264_macroblock_thread_init( h );
// ----- 2.将QP设置为片中的第一个QP,以获得更准确的CABAC初始化 ----- //
/* Set the QP equal to the first QP in the slice for more accurate CABAC initialization. */
h->mb.i_mb_xy = h->sh.i_first_mb;
h->sh.i_qp = x264_ratecontrol_mb_qp( h );
h->sh.i_qp = SPEC_QP( h->sh.i_qp );
h->sh.i_qp_delta = h->sh.i_qp - h->pps->i_pic_init_qp;
slice_header_write( &h->out.bs, &h->sh, h->i_nal_ref_idc );
// ----- 3.初始化cabac ----- //
if( h->param.b_cabac )
{
/* alignment needed */
bs_align_1( &h->out.bs );
/* init cabac */
x264_cabac_context_init( h, &h->cabac, h->sh.i_type, x264_clip3( h->sh.i_qp-QP_BD_OFFSET, 0, 51 ), h->sh.i_cabac_init_idc );
x264_cabac_encode_init ( &h->cabac, h->out.bs.p, h->out.bs.p_end );
last_emu_check = h->cabac.p;
}
else
last_emu_check = h->out.bs.p;
h->mb.i_last_qp = h->sh.i_qp;
h->mb.i_last_dqp = 0;
h->mb.field_decoding_flag = 0;
i_mb_y = h->sh.i_first_mb / h->mb.i_mb_width;
i_mb_x = h->sh.i_first_mb % h->mb.i_mb_width;
i_skip = 0;
// ----- 4.循环编码slice当中的每个mb ----- //
while( 1 )
{
mb_xy = i_mb_x + i_mb_y * h->mb.i_mb_width;
int mb_spos = bs_pos(&h->out.bs) + x264_cabac_pos(&h->cabac);
if( i_mb_x == 0 )
{
if( bitstream_check_buffer( h ) )
return -1;
if( !(i_mb_y & SLICE_MBAFF) && h->param.rc.i_vbv_buffer_size )
bitstream_backup( h, &bs_bak[BS_BAK_ROW_VBV], i_skip, 1 );
if( !h->mb.b_reencode_mb )
fdec_filter_row( h, i_mb_y, 0 );
}
if( back_up_bitstream )
{
if( back_up_bitstream_cavlc )
bitstream_backup( h, &bs_bak[BS_BAK_CAVLC_OVERFLOW], i_skip, 0 );
if( slice_max_size && !(i_mb_y & SLICE_MBAFF) )
{
bitstream_backup( h, &bs_bak[BS_BAK_SLICE_MAX_SIZE], i_skip, 0 );
if( (thread_last_mb+1-mb_xy) == h->param.i_slice_min_mbs )
bitstream_backup( h, &bs_bak[BS_BAK_SLICE_MIN_MBS], i_skip, 0 );
}
}
if( PARAM_INTERLACED )
{
if( h->mb.b_adaptive_mbaff )
{
if( !(i_mb_y&1) )
{
/* FIXME: VSAD is fast but fairly poor at choosing the best interlace type. */
h->mb.b_interlaced = x264_field_vsad( h, i_mb_x, i_mb_y );
memcpy( &h->zigzagf, MB_INTERLACED ? &h->zigzagf_interlaced : &h->zigzagf_progressive, sizeof(h->zigzagf) );
if( !MB_INTERLACED && (i_mb_y+2) == h->mb.i_mb_height )
x264_expand_border_mbpair( h, i_mb_x, i_mb_y );
}
}
h->mb.field[mb_xy] = MB_INTERLACED;
}
/* load cache */
if( SLICE_MBAFF )
x264_macroblock_cache_load_interlaced( h, i_mb_x, i_mb_y );
else
x264_macroblock_cache_load_progressive( h, i_mb_x, i_mb_y );
// ----- 5.分析mb(预测) ----- //
x264_macroblock_analyse( h );
/* encode this macroblock -> be careful it can change the mb type to P_SKIP if needed */
reencode:
// ----- 6.编码mb ----- //
x264_macroblock_encode( h );
if( h->param.b_cabac ) // 按照CABAC方式写入
{
if( mb_xy > h->sh.i_first_mb && !(SLICE_MBAFF && (i_mb_y&1)) )
x264_cabac_encode_terminal( &h->cabac );
if( IS_SKIP( h->mb.i_type ) )
x264_cabac_mb_skip( h, 1 );
else
{
if( h->sh.i_type != SLICE_TYPE_I )
x264_cabac_mb_skip( h, 0 );
x264_macroblock_write_cabac( h, &h->cabac );
}
}
else // 按照CAVLC方式写入
{
if( IS_SKIP( h->mb.i_type ) )
i_skip++;
else
{
if( h->sh.i_type != SLICE_TYPE_I )
{
bs_write_ue( &h->out.bs, i_skip ); /* skip run */
i_skip = 0;
}
x264_macroblock_write_cavlc( h );
/* If there was a CAVLC level code overflow, try again at a higher QP. */
if( h->mb.b_overflow )
{
h->mb.i_chroma_qp = h->chroma_qp_table[++h->mb.i_qp];
h->mb.i_skip_intra = 0;
h->mb.b_skip_mc = 0;
h->mb.b_overflow = 0;
bitstream_restore( h, &bs_bak[BS_BAK_CAVLC_OVERFLOW], &i_skip, 0 );
goto reencode;
}
}
}
int total_bits = bs_pos(&h->out.bs) + x264_cabac_pos(&h->cabac);
int mb_size = total_bits - mb_spos;
if( slice_max_size && (!SLICE_MBAFF || (i_mb_y&1)) )
{
/* Count the skip run, just in case. */
if( !h->param.b_cabac )
total_bits += bs_size_ue_big( i_skip );
/* Check for escape bytes. */
uint8_t *end = h->param.b_cabac ? h->cabac.p : h->out.bs.p;
for( ; last_emu_check < end - 2; last_emu_check++ )
if( last_emu_check[0] == 0 && last_emu_check[1] == 0 && last_emu_check[2] <= 3 )
{
slice_max_size -= 8;
last_emu_check++;
}
/* We'll just re-encode this last macroblock if we go over the max slice size. */
if( total_bits - starting_bits > slice_max_size && !h->mb.b_reencode_mb )
{
if( !x264_frame_new_slice( h, h->fdec ) )
{
/* Handle the most obnoxious slice-min-mbs edge case: we need to end the slice
* because it's gone over the maximum size, but doing so would violate slice-min-mbs.
* If possible, roll back to the last checkpoint and try again.
* We could try raising QP, but that would break in the case where a slice spans multiple
* rows, which the re-encoding infrastructure can't currently handle. */
if( mb_xy <= thread_last_mb && (thread_last_mb+1-mb_xy) < h->param.i_slice_min_mbs )
{
if( thread_last_mb-h->param.i_slice_min_mbs < h->sh.i_first_mb+h->param.i_slice_min_mbs )
{
x264_log( h, X264_LOG_WARNING, "slice-max-size violated (frame %d, cause: slice-min-mbs)\n", h->i_frame );
slice_max_size = 0;
goto cont;
}
bitstream_restore( h, &bs_bak[BS_BAK_SLICE_MIN_MBS], &i_skip, 0 );
h->mb.b_reencode_mb = 1;
h->sh.i_last_mb = thread_last_mb-h->param.i_slice_min_mbs;
break;
}
if( mb_xy-SLICE_MBAFF*h->mb.i_mb_stride != h->sh.i_first_mb )
{
bitstream_restore( h, &bs_bak[BS_BAK_SLICE_MAX_SIZE], &i_skip, 0 );
h->mb.b_reencode_mb = 1;
if( SLICE_MBAFF )
{
// set to bottom of previous mbpair
if( i_mb_x )
h->sh.i_last_mb = mb_xy-1+h->mb.i_mb_stride*(!(i_mb_y&1));
else
h->sh.i_last_mb = (i_mb_y-2+!(i_mb_y&1))*h->mb.i_mb_stride + h->mb.i_mb_width - 1;
}
else
h->sh.i_last_mb = mb_xy-1;
break;
}
else
h->sh.i_last_mb = mb_xy;
}
else
slice_max_size = 0;
}
}
cont:
h->mb.b_reencode_mb = 0;
/* save cache */
x264_macroblock_cache_save( h );
if( x264_ratecontrol_mb( h, mb_size ) < 0 )
{
bitstream_restore( h, &bs_bak[BS_BAK_ROW_VBV], &i_skip, 1 );
h->mb.b_reencode_mb = 1;
i_mb_x = 0;
i_mb_y = i_mb_y - SLICE_MBAFF;
h->mb.i_mb_prev_xy = i_mb_y * h->mb.i_mb_stride - 1;
h->sh.i_last_mb = orig_last_mb;
continue;
}
// ----- 7.收集mb的信息进行更新 ----- //
/* accumulate mb stats */
h->stat.frame.i_mb_count[h->mb.i_type]++;
int b_intra = IS_INTRA( h->mb.i_type );
int b_skip = IS_SKIP( h->mb.i_type );
if( h->param.i_log_level >= X264_LOG_INFO || h->param.rc.b_stat_write )
{
if( !b_intra && !b_skip && !IS_DIRECT( h->mb.i_type ) )
{
if( h->mb.i_partition != D_8x8 )
h->stat.frame.i_mb_partition[h->mb.i_partition] += 4;
else
for( int i = 0; i < 4; i++ )
h->stat.frame.i_mb_partition[h->mb.i_sub_partition[i]] ++;
if( h->param.i_frame_reference > 1 )
for( int i_list = 0; i_list <= (h->sh.i_type == SLICE_TYPE_B); i_list++ )
for( int i = 0; i < 4; i++ )
{
int i_ref = h->mb.cache.ref[i_list][ x264_scan8[4*i] ];
if( i_ref >= 0 )
h->stat.frame.i_mb_count_ref[i_list][i_ref] ++;
}
}
}
if( h->param.i_log_level >= X264_LOG_INFO )
{
if( h->mb.i_cbp_luma | h->mb.i_cbp_chroma )
{
if( CHROMA444 )
{
for( int i = 0; i < 4; i++ )
if( h->mb.i_cbp_luma & (1 << i) )
for( int p = 0; p < 3; p++ )
{
int s8 = i*4+p*16;
int nnz8x8 = M16( &h->mb.cache.non_zero_count[x264_scan8[s8]+0] )
| M16( &h->mb.cache.non_zero_count[x264_scan8[s8]+8] );
h->stat.frame.i_mb_cbp[!b_intra + p*2] += !!nnz8x8;
}
}
else
{
int cbpsum = (h->mb.i_cbp_luma&1) + ((h->mb.i_cbp_luma>>1)&1)
+ ((h->mb.i_cbp_luma>>2)&1) + (h->mb.i_cbp_luma>>3);
h->stat.frame.i_mb_cbp[!b_intra + 0] += cbpsum;
h->stat.frame.i_mb_cbp[!b_intra + 2] += !!h->mb.i_cbp_chroma;
h->stat.frame.i_mb_cbp[!b_intra + 4] += h->mb.i_cbp_chroma >> 1;
}
}
if( h->mb.i_cbp_luma && !b_intra )
{
h->stat.frame.i_mb_count_8x8dct[0] ++;
h->stat.frame.i_mb_count_8x8dct[1] += h->mb.b_transform_8x8;
}
if( b_intra && h->mb.i_type != I_PCM )
{
if( h->mb.i_type == I_16x16 )
h->stat.frame.i_mb_pred_mode[0][h->mb.i_intra16x16_pred_mode]++;
else if( h->mb.i_type == I_8x8 )
for( int i = 0; i < 16; i += 4 )
h->stat.frame.i_mb_pred_mode[1][h->mb.cache.intra4x4_pred_mode[x264_scan8[i]]]++;
else //if( h->mb.i_type == I_4x4 )
for( int i = 0; i < 16; i++ )
h->stat.frame.i_mb_pred_mode[2][h->mb.cache.intra4x4_pred_mode[x264_scan8[i]]]++;
h->stat.frame.i_mb_pred_mode[3][x264_mb_chroma_pred_mode_fix[h->mb.i_chroma_pred_mode]]++;
}
h->stat.frame.i_mb_field[b_intra?0:b_skip?2:1] += MB_INTERLACED;
}
// ----- 8.是否进行deblock ----- //
/* calculate deblock strength values (actual deblocking is done per-row along with hpel) */
if( b_deblock )
x264_macroblock_deblock_strength( h );
if( mb_xy == h->sh.i_last_mb )
break;
if( SLICE_MBAFF )
{
i_mb_x += i_mb_y & 1;
i_mb_y ^= i_mb_x < h->mb.i_mb_width;
}
else
i_mb_x++;
if( i_mb_x == h->mb.i_mb_width )
{
i_mb_y++;
i_mb_x = 0;
}
}
if( h->sh.i_last_mb < h->sh.i_first_mb )
return 0;
h->out.nal[h->out.i_nal].i_last_mb = h->sh.i_last_mb;
if( h->param.b_cabac )
{
x264_cabac_encode_flush( h, &h->cabac );
h->out.bs.p = h->cabac.p;
}
else
{
if( i_skip > 0 )
bs_write_ue( &h->out.bs, i_skip ); /* last skip run */
/* rbsp_slice_trailing_bits */
bs_rbsp_trailing( &h->out.bs );
bs_flush( &h->out.bs );
}
if( nal_end( h ) )
return -1;
if( h->sh.i_last_mb == (h->i_threadslice_end * h->mb.i_mb_width - 1) )
{
h->stat.frame.i_misc_bits = bs_pos( &h->out.bs )
+ (h->out.i_nal*NALU_OVERHEAD * 8)
- h->stat.frame.i_tex_bits
- h->stat.frame.i_mv_bits;
fdec_filter_row( h, h->i_threadslice_end, 0 );
// ----- 9.多线程情况下,告知主线程,使用broadcast ----- //
if( h->param.b_sliced_threads )
{
/* Tell the main thread we're done. */
x264_threadslice_cond_broadcast( h, 1 );
/* Do hpel now */
for( int mb_y = h->i_threadslice_start; mb_y <= h->i_threadslice_end; mb_y++ )
fdec_filter_row( h, mb_y, 1 );
x264_threadslice_cond_broadcast( h, 2 );
/* Do the first row of hpel, now that the previous slice is done */
if( h->i_thread_idx > 0 )
{
x264_threadslice_cond_wait( h->thread[h->i_thread_idx-1], 2 );
fdec_filter_row( h, h->i_threadslice_start + (1 << SLICE_MBAFF), 2 );
}
}
/* Free mb info after the last thread's done using it */
if( h->fdec->mb_info_free && (!h->param.b_sliced_threads || h->i_thread_idx == (h->param.i_threads-1)) )
{
h->fdec->mb_info_free( h->fdec->mb_info );
h->fdec->mb_info = NULL;
h->fdec->mb_info_free = NULL;
}
}
return 0;
}
1.11 处理额外信息且更新编码器状态(encoder_frame_end)
该函数位于编码一帧结束之后,用于处理编码之后的信息,更新编码器状态。主要的工作流程为:
1. List item
static int encoder_frame_end( x264_t *h, x264_t *thread_current,
x264_nal_t **pp_nal, int *pi_nal,
x264_picture_t *pic_out )
{
char psz_message[80];
if( !h->param.b_sliced_threads && h->b_thread_active )
{
h->b_thread_active = 0;
if( (intptr_t)x264_threadpool_wait( h->threadpool, h ) )
return -1;
}
if( !h->out.i_nal )
{
pic_out->i_type = X264_TYPE_AUTO;
return 0;
}
x264_emms();
/* generate buffering period sei and insert it into place */
if( h->i_thread_frames > 1 && h->fenc->b_keyframe && h->sps->vui.b_nal_hrd_parameters_present )
{
x264_hrd_fullness( h );
nal_start( h, NAL_SEI, NAL_PRIORITY_DISPOSABLE );
x264_sei_buffering_period_write( h, &h->out.bs );
if( nal_end( h ) )
return -1;
/* buffering period sei must follow AUD, SPS and PPS and precede all other SEIs */
int idx = 0;
while( h->out.nal[idx].i_type == NAL_AUD ||
h->out.nal[idx].i_type == NAL_SPS ||
h->out.nal[idx].i_type == NAL_PPS )
idx++;
x264_nal_t nal_tmp = h->out.nal[h->out.i_nal-1];
memmove( &h->out.nal[idx+1], &h->out.nal[idx], (h->out.i_nal-idx-1)*sizeof(x264_nal_t) );
h->out.nal[idx] = nal_tmp;
}
int frame_size = encoder_encapsulate_nals( h, 0 );
if( frame_size < 0 )
return -1;
// ----- 1.设置输出图片格式 ----- //
/* Set output picture properties */
pic_out->i_type = h->fenc->i_type;
pic_out->b_keyframe = h->fenc->b_keyframe;
pic_out->i_pic_struct = h->fenc->i_pic_struct;
pic_out->i_pts = h->fdec->i_pts;
pic_out->i_dts = h->fdec->i_dts;
if( pic_out->i_pts < pic_out->i_dts )
x264_log( h, X264_LOG_WARNING, "invalid DTS: PTS is less than DTS\n" );
pic_out->opaque = h->fenc->opaque;
pic_out->img.i_csp = h->fdec->i_csp;
#if HIGH_BIT_DEPTH
pic_out->img.i_csp |= X264_CSP_HIGH_DEPTH;
#endif
pic_out->img.i_plane = h->fdec->i_plane;
for( int i = 0; i < pic_out->img.i_plane; i++ )
{
pic_out->img.i_stride[i] = h->fdec->i_stride[i] * SIZEOF_PIXEL;
pic_out->img.plane[i] = (uint8_t*)h->fdec->plane[i];
}
x264_frame_push_unused( thread_current, h->fenc );
// ----- 2.更新编码器状态 ----- //
/* ---------------------- Update encoder state ------------------------- */
/* update rc */
int filler = 0;
if( x264_ratecontrol_end( h, frame_size * 8, &filler ) < 0 )
return -1;
pic_out->hrd_timing = h->fenc->hrd_timing;
pic_out->prop.f_crf_avg = h->fdec->f_crf_avg;
/* Filler in AVC-Intra mode is written as zero bytes to the last slice
* We don't know the size of the last slice until encapsulation so we add filler to the encapsulated NAL */
if( h->param.i_avcintra_class )
{
if( check_encapsulated_buffer( h, h->thread[0], h->out.i_nal, frame_size, (int64_t)frame_size + filler ) < 0 )
return -1;
x264_nal_t *nal = &h->out.nal[h->out.i_nal-1];
memset( nal->p_payload + nal->i_payload, 0, filler );
nal->i_payload += filler;
nal->i_padding = filler;
frame_size += filler;
/* Fix up the size header for mp4/etc */
if( !h->param.b_annexb )
{
/* Size doesn't include the size of the header we're writing now. */
uint8_t *nal_data = nal->p_payload;
int chunk_size = nal->i_payload - 4;
nal_data[0] = chunk_size >> 24;
nal_data[1] = chunk_size >> 16;
nal_data[2] = chunk_size >> 8;
nal_data[3] = chunk_size >> 0;
}
}
else
{
while( filler > 0 )
{
int f, overhead = FILLER_OVERHEAD - h->param.b_annexb;
if( h->param.i_slice_max_size && filler > h->param.i_slice_max_size )
{
int next_size = filler - h->param.i_slice_max_size;
int overflow = X264_MAX( overhead - next_size, 0 );
f = h->param.i_slice_max_size - overhead - overflow;
}
else
f = X264_MAX( 0, filler - overhead );
if( bitstream_check_buffer_filler( h, f ) )
return -1;
nal_start( h, NAL_FILLER, NAL_PRIORITY_DISPOSABLE );
x264_filler_write( h, &h->out.bs, f );
if( nal_end( h ) )
return -1;
int total_size = encoder_encapsulate_nals( h, h->out.i_nal-1 );
if( total_size < 0 )
return -1;
frame_size += total_size;
filler -= total_size;
}
}
/* End bitstream, set output */
*pi_nal = h->out.i_nal;
*pp_nal = h->out.nal;
h->out.i_nal = 0;
x264_noise_reduction_update( h );
// ----- 3.计算/输出编码信息 ----- //
/* ---------------------- Compute/Print statistics --------------------- */
thread_sync_stat( h, h->thread[0] );
/* Slice stat */
h->stat.i_frame_count[h->sh.i_type]++;
h->stat.i_frame_size[h->sh.i_type] += frame_size;
h->stat.f_frame_qp[h->sh.i_type] += h->fdec->f_qp_avg_aq;
// ...
psz_message[0] = '\0';
double dur = h->fenc->f_duration;
h->stat.f_frame_duration[h->sh.i_type] += dur;
if( h->param.analyse.b_psnr ) // 计算psnr
{
int64_t ssd[3] =
{
h->stat.frame.i_ssd[0],
h->stat.frame.i_ssd[1],
h->stat.frame.i_ssd[2],
};
int luma_size = h->param.i_width * h->param.i_height;
int chroma_size = CHROMA_SIZE( luma_size );
pic_out->prop.f_psnr[0] = calc_psnr( ssd[0], luma_size );
pic_out->prop.f_psnr[1] = calc_psnr( ssd[1], chroma_size );
pic_out->prop.f_psnr[2] = calc_psnr( ssd[2], chroma_size );
pic_out->prop.f_psnr_avg = calc_psnr( ssd[0] + ssd[1] + ssd[2], luma_size + chroma_size*2 );
h->stat.f_ssd_global[h->sh.i_type] += dur * (ssd[0] + ssd[1] + ssd[2]);
h->stat.f_psnr_average[h->sh.i_type] += dur * pic_out->prop.f_psnr_avg;
h->stat.f_psnr_mean_y[h->sh.i_type] += dur * pic_out->prop.f_psnr[0];
h->stat.f_psnr_mean_u[h->sh.i_type] += dur * pic_out->prop.f_psnr[1];
h->stat.f_psnr_mean_v[h->sh.i_type] += dur * pic_out->prop.f_psnr[2];
snprintf( psz_message, 80, " PSNR Y:%5.2f U:%5.2f V:%5.2f", pic_out->prop.f_psnr[0],
pic_out->prop.f_psnr[1],
pic_out->prop.f_psnr[2] );
}
if( h->param.analyse.b_ssim ) // 计算ssim
{
pic_out->prop.f_ssim = h->stat.frame.f_ssim / h->stat.frame.i_ssim_cnt;
h->stat.f_ssim_mean_y[h->sh.i_type] += pic_out->prop.f_ssim * dur;
int msg_len = strlen(psz_message);
snprintf( psz_message + msg_len, 80 - msg_len, " SSIM Y:%.5f", pic_out->prop.f_ssim );
}
psz_message[79] = '\0';
...
}
2.小结
x264_encoder_encode中实现了实际编码一帧的功能,其中涉及到了编码器中诸多方面的内容,大体可以归类如下:
- 码流存储的方式
- 码率控制的实现
- 预测模块(帧内预测,帧间预测)
- 实际编码(DCT变换,Quant量化)
- 熵编码(CABAC,CAVLC)
- 滤波模块
在视频编码当中,工作的大致流程是,输入一帧,经过mb块划分之后,对每一个mb进行参考帧预测(帧内或帧间)获得残差,对残差进行实际编码(变换量化)获得系数,系数通过熵编码进行编码并写入码流。滤波模块应用于重建帧之后,为了降低重建帧和原始帧之间的差异,对重建帧进行滤波处理,将滤波之后的重建帧作为下一帧的参考帧。
此外,还需要了解码流存储的方式和结构,比如不同的信息使用几个比特进行存储,存储在哪个位置,使用的是算术编码还是哥伦布编码,这些都值得思考和探索。
CSDN : https://blog.csdn.net/weixin_42877471
Github : https://github.com/DoFulangChen