predInterSearch函数在xCheckRDCostInter中被调用,用来获取inter_ME模式下的最优inter_ME运动信息。predInterSearch函数是inter_ME模式的最主要实现函数。
函数首先对单向预测,分别对L0和L1的所有帧进行AMVP、ME,得到各个参考帧下的各种最优的运动信息;
再进行双向预测,双向预测用到了单向时得到的各帧的各种运动信息,通过4次迭代,找到最优的双向运动信息;
对比前向、后向和双向预测的cost,取最优的模式,将运动信息存入pu;
如果cu不是2Nx2N,则需要合并运动信息;
最后统一进行一次MC,得到pred像素信息;
最后再设置加权预测的参数。
Void InterSearch::predInterSearch(CodingUnit& cu, Partitioner& partitioner)
{
CodingStructure& cs = *cu.cs;
AMVPInfo amvp[2]; //存储一帧经过ME后的最优amvp信息。AMVP选择的运动信息为ME起始点
Mv cMvSrchRngLT;
Mv cMvSrchRngRB;
Mv cMvZero;
Mv cMv[2]; //两个单向预测的最优mv
Mv cMvBi[2]; //双向预测的最优mv
Mv cMvTemp[2][33];
Int iNumPredDir = cs.slice->isInterP() ? 1 : 2; //前向/后向预测
Mv cMvPred[2][33]; //存储AMVP得到的最优mvp
Mv cMvPredBi[2][33]; //双向预测时,ME的起点mv
Int aaiMvpIdxBi[2][33];
Int aaiMvpIdx[2][33]; //2表示前向后向预测,33表示L0、L1参考列表中最多33帧
Int aaiMvpNum[2][33]; //AMVP后得到的最优MvpIdx和MvpNum
AMVPInfo aacAMVPInfo[2][33]; //存储每一帧的最优AMVP信息
Int iRefIdx[2]={0,0}; //If un-initialized, may cause SEGV in bi-directional prediction iterative stage. 最优mv的refIdx
Int iRefIdxBi[2];
UInt uiMbBits[3] = {1, 1, 0};
UInt uiLastMode = 0;
Int iRefStart, iRefEnd;
Int bestBiPRefIdxL1 = 0;
Int bestBiPMvpL1 = 0;
Distortion biPDistTemp = std::numeric_limits<Distortion>::max();
MergeCtx mergeCtx;
// Loop over Prediction Units
CHECK(!cu.firstPU, "CU does not contain any PUs");
UInt puIdx = 0;
#if HEVC_USE_PART_SIZE
UInt numPUs = CU::getNumPUs(cu);
for( auto &pu : CU::traversePUs( cu ) )
#else
auto &pu = *cu.firstPU; //xCheckRDCostInter函数中添加的pu
#endif
{
// motion estimation only evaluates luma component
m_maxCompIDToPred = MAX_NUM_COMPONENT;
// m_maxCompIDToPred = COMPONENT_Y;
CHECK(pu.cu != &cu, "PU is contained in another CU");
PU::spanMotionInfo( pu );
Distortion uiCost[2] = { std::numeric_limits<Distortion>::max(), std::numeric_limits<Distortion>::max() };
Distortion uiCostBi = std::numeric_limits<Distortion>::max();
Distortion uiCostTemp;
UInt uiBits[3];
UInt uiBitsTemp;
Distortion bestBiPDist = std::numeric_limits<Distortion>::max();
Distortion uiCostTempL0[MAX_NUM_REF];
for (Int iNumRef=0; iNumRef < MAX_NUM_REF; iNumRef++)
{
uiCostTempL0[iNumRef] = std::numeric_limits<Distortion>::max(); //失真都设为最大值
}
UInt uiBitsTempL0[MAX_NUM_REF];
Mv mvValidList1;
Int refIdxValidList1 = 0;
UInt bitsValidList1 = MAX_UINT;
Distortion costValidList1 = std::numeric_limits<Distortion>::max();
PelUnitBuf origBuf = pu.cs->getOrgBuf( pu ); //orig像素信息
xGetBlkBits( cu.partSize, cs.slice->isInterP(), puIdx, uiLastMode, uiMbBits ); //统计bit数
m_pcRdCost->selectMotionLambda( cu.transQuantBypass );
Bool bFastSkipBi = false;
if( auto slsCtrl = dynamic_cast< SaveLoadEncInfoCtrl* >( m_modeCtrl ) )
{
bFastSkipBi = ( LOAD_ENC_INFO == slsCtrl->getSaveLoadTag( pu ) && 3 != slsCtrl->getSaveLoadInterDir( pu ) );
}
#if AMP_MRG
Bool bTestNormalMC = true;
if ( bUseMRG && cu.lumaSize().width > 8 && numPUs == 2 )
{
bTestNormalMC = false;
}
if( bTestNormalMC )
{
#endif
// Uni-directional prediction
for ( Int iRefList = 0; iRefList < iNumPredDir; iRefList++ ) //前向/后向,单向预测inter_ME
{
RefPicList eRefPicList = ( iRefList ? REF_PIC_LIST_1 : REF_PIC_LIST_0 );
for ( Int iRefIdxTemp = 0; iRefIdxTemp < cs.slice->getNumRefIdx(eRefPicList); iRefIdxTemp++ ) //L0或L1中各参考帧
{
uiBitsTemp = uiMbBits[iRefList]; //统计bit信息
if ( cs.slice->getNumRefIdx(eRefPicList) > 1 )
{
uiBitsTemp += iRefIdxTemp+1;
if ( iRefIdxTemp == cs.slice->getNumRefIdx(eRefPicList)-1 )
{
uiBitsTemp--;
}
}
//AMVP,获取当前pu周围的最优运动信息,作为ME起点
xEstimateMvPredAMVP( pu, origBuf, eRefPicList, iRefIdxTemp, cMvPred[iRefList][iRefIdxTemp], amvp[eRefPicList], false, &biPDistTemp);
aaiMvpIdx[iRefList][iRefIdxTemp] = pu.mvpIdx[eRefPicList]; //每一帧通过AMVP获取的最优mvp
aaiMvpNum[iRefList][iRefIdxTemp] = pu.mvpNum[eRefPicList];
if(cs.slice->getMvdL1ZeroFlag() && iRefList==1 && biPDistTemp < bestBiPDist) //广义B帧时,L1信息直接复制L0
{
bestBiPDist = biPDistTemp;
bestBiPMvpL1 = aaiMvpIdx[iRefList][iRefIdxTemp];
bestBiPRefIdxL1 = iRefIdxTemp;
}
uiBitsTemp += m_auiMVPIdxCost[aaiMvpIdx[iRefList][iRefIdxTemp]][AMVP_MAX_NUM_CANDS]; //统计bit信息
if ( m_pcEncCfg->getFastMEForGenBLowDelayEnabled() && iRefList == 1 ) // list 1
{
if ( cs.slice->getList1IdxToList0Idx( iRefIdxTemp ) >= 0 ) //如果使用广义B帧,则list1直接复制list0的信息
{
cMvTemp[1][iRefIdxTemp] = cMvTemp[0][cs.slice->getList1IdxToList0Idx( iRefIdxTemp )];
uiCostTemp = uiCostTempL0[cs.slice->getList1IdxToList0Idx( iRefIdxTemp )];
/*first subtract the bit-rate part of the cost of the other list*/
uiCostTemp -= m_pcRdCost->getCost( uiBitsTempL0[cs.slice->getList1IdxToList0Idx( iRefIdxTemp )] );
/*correct the bit-rate part of the current ref*/
m_pcRdCost->setPredictor ( cMvPred[iRefList][iRefIdxTemp] );
uiBitsTemp += m_pcRdCost->getBitsOfVectorWithPredictor( cMvTemp[1][iRefIdxTemp].getHor(), cMvTemp[1][iRefIdxTemp].getVer() );
/*calculate the correct cost*/
uiCostTemp += m_pcRdCost->getCost( uiBitsTemp );
}
else
{
xMotionEstimation( pu, origBuf, eRefPicList, cMvPred[iRefList][iRefIdxTemp], iRefIdxTemp, cMvTemp[iRefList][iRefIdxTemp], aaiMvpIdx[iRefList][iRefIdxTemp], uiBitsTemp, uiCostTemp, amvp[eRefPicList] );
}
}
else
{ //运动估计,获取MVP和MVD
xMotionEstimation( pu, origBuf, eRefPicList, cMvPred[iRefList][iRefIdxTemp], iRefIdxTemp, cMvTemp[iRefList][iRefIdxTemp], aaiMvpIdx[iRefList][iRefIdxTemp], uiBitsTemp, uiCostTemp, amvp[eRefPicList] );
}
//保存最优AMVP信息,并检查是否是最优MVP
xCopyAMVPInfo( &amvp[eRefPicList], &aacAMVPInfo[iRefList][iRefIdxTemp]); // must always be done ( also when AMVP_MODE = AM_NONE )
xCheckBestMVP( eRefPicList, cMvTemp[iRefList][iRefIdxTemp], cMvPred[iRefList][iRefIdxTemp], aaiMvpIdx[iRefList][iRefIdxTemp], amvp[eRefPicList], uiBitsTemp, uiCostTemp );
//更新记录最优的运动信息
if ( iRefList == 0 )
{
uiCostTempL0[iRefIdxTemp] = uiCostTemp;
uiBitsTempL0[iRefIdxTemp] = uiBitsTemp;
}
if ( uiCostTemp < uiCost[iRefList] )
{
uiCost[iRefList] = uiCostTemp;
uiBits[iRefList] = uiBitsTemp; // storing for bi-prediction
// set motion
cMv [iRefList] = cMvTemp[iRefList][iRefIdxTemp];
iRefIdx[iRefList] = iRefIdxTemp;
}
if ( iRefList == 1 && uiCostTemp < costValidList1 && cs.slice->getList1IdxToList0Idx( iRefIdxTemp ) < 0 )
{
costValidList1 = uiCostTemp;
bitsValidList1 = uiBitsTemp;
// set motion
mvValidList1 = cMvTemp[iRefList][iRefIdxTemp];
refIdxValidList1 = iRefIdxTemp;
}
}
}
// Bi-predictive Motion estimation //双向预测
if ( (cs.slice->isInterB()) && ( PU::isBipredRestriction(pu) == false) && !bFastSkipBi )
{
cMvBi[0] = cMv[0]; //用单向预测得到的运动信息初始化
cMvBi[1] = cMv[1];
iRefIdxBi[0] = iRefIdx[0];
iRefIdxBi[1] = iRefIdx[1];
::memcpy( cMvPredBi, cMvPred, sizeof( cMvPred ) );
::memcpy( aaiMvpIdxBi, aaiMvpIdx, sizeof( aaiMvpIdx ) );
UInt uiMotBits[2];
//L1为空,则使用广义B帧
if(cs.slice->getMvdL1ZeroFlag()) //广义B帧,需要对后向参考预测块进行运动补偿,在运动补偿之后重新进行运动估计,找到最优MV
{
xCopyAMVPInfo(&aacAMVPInfo[1][bestBiPRefIdxL1], &amvp[REF_PIC_LIST_1]);
aaiMvpIdxBi[1][bestBiPRefIdxL1] = bestBiPMvpL1;
cMvPredBi [1][bestBiPRefIdxL1] = amvp[REF_PIC_LIST_1].mvCand[bestBiPMvpL1];
cMvBi [1] = cMvPredBi[1][bestBiPRefIdxL1];
iRefIdxBi[1] = bestBiPRefIdxL1;
pu.mv [REF_PIC_LIST_1] = cMvBi[1];
pu.refIdx[REF_PIC_LIST_1] = iRefIdxBi[1];
pu.mvpIdx[REF_PIC_LIST_1] = bestBiPMvpL1;
PelUnitBuf predBufTmp = m_tmpPredStorage[REF_PIC_LIST_1].getBuf( UnitAreaRelative(cu, pu) );
motionCompensation( pu, predBufTmp, REF_PIC_LIST_1 ); //MC
uiMotBits[0] = uiBits[0] - uiMbBits[0];
uiMotBits[1] = uiMbBits[1];
if ( cs.slice->getNumRefIdx(REF_PIC_LIST_1) > 1 )
{
uiMotBits[1] += bestBiPRefIdxL1 + 1;
if ( bestBiPRefIdxL1 == cs.slice->getNumRefIdx(REF_PIC_LIST_1)-1 )
{
uiMotBits[1]--;
}
}
uiMotBits[1] += m_auiMVPIdxCost[aaiMvpIdxBi[1][bestBiPRefIdxL1]][AMVP_MAX_NUM_CANDS];
uiBits[2] = uiMbBits[2] + uiMotBits[0] + uiMotBits[1];
cMvTemp[1][bestBiPRefIdxL1] = cMvBi[1];
}
else
{
uiMotBits[0] = uiBits[0] - uiMbBits[0];
uiMotBits[1] = uiBits[1] - uiMbBits[1];
uiBits[2] = uiMbBits[2] + uiMotBits[0] + uiMotBits[1];
}
// 4-times iteration (default)
Int iNumIter = 4;
// fast encoder setting: only one iteration
if ( m_pcEncCfg->getFastInterSearchMode()==FASTINTERSEARCH_MODE1 || m_pcEncCfg->getFastInterSearchMode()==FASTINTERSEARCH_MODE2 || cs.slice->getMvdL1ZeroFlag() )
{
iNumIter = 1;
}
for ( Int iIter = 0; iIter < iNumIter; iIter++ ) //loop L0和L1分别进行两次全帧搜索找最优mv
{
Int iRefList = iIter % 2;
if ( m_pcEncCfg->getFastInterSearchMode()==FASTINTERSEARCH_MODE1 || m_pcEncCfg->getFastInterSearchMode()==FASTINTERSEARCH_MODE2 )
{
if( uiCost[0] <= uiCost[1] )
{
iRefList = 1;
}
else
{
iRefList = 0;
}
}
else if ( iIter == 0 )
{
iRefList = 0;
}
if ( iIter == 0 && !cs.slice->getMvdL1ZeroFlag()) //第一次迭代时进行MC
{
pu.mv [1 - iRefList] = cMv [1 - iRefList];
pu.refIdx[1 - iRefList] = iRefIdx[1 - iRefList];
PelUnitBuf predBufTmp = m_tmpPredStorage[1 - iRefList].getBuf( UnitAreaRelative(cu, pu) );
motionCompensation( pu, predBufTmp, RefPicList(1 - iRefList) );
}
RefPicList eRefPicList = ( iRefList ? REF_PIC_LIST_1 : REF_PIC_LIST_0 );
if(cs.slice->getMvdL1ZeroFlag())
{
iRefList = 0;
eRefPicList = REF_PIC_LIST_0;
}
Bool bChanged = false;
iRefStart = 0;
iRefEnd = cs.slice->getNumRefIdx(eRefPicList)-1; //起始参考帧和最后参考帧
for ( Int iRefIdxTemp = iRefStart; iRefIdxTemp <= iRefEnd; iRefIdxTemp++ ) //对于单向的所有帧循环
{ //对于L0和L1分别找最优
uiBitsTemp = uiMbBits[2] + uiMotBits[1-iRefList]; //统计bit信息
if ( cs.slice->getNumRefIdx(eRefPicList) > 1 )
{
uiBitsTemp += iRefIdxTemp+1;
if ( iRefIdxTemp == cs.slice->getNumRefIdx(eRefPicList)-1 )
{
uiBitsTemp--;
}
}
uiBitsTemp += m_auiMVPIdxCost[aaiMvpIdxBi[iRefList][iRefIdxTemp]][AMVP_MAX_NUM_CANDS];
// call ME
xCopyAMVPInfo(&aacAMVPInfo[iRefList][iRefIdxTemp], &amvp[eRefPicList] ); //copy AMVP信息作为ME起点
//ME,得到mv和mvd
xMotionEstimation ( pu, origBuf, eRefPicList, cMvPredBi[iRefList][iRefIdxTemp], iRefIdxTemp, cMvTemp[iRefList][iRefIdxTemp], aaiMvpIdxBi[iRefList][iRefIdxTemp], uiBitsTemp, uiCostTemp, amvp[eRefPicList], true );
xCheckBestMVP( eRefPicList, cMvTemp[iRefList][iRefIdxTemp], cMvPredBi[iRefList][iRefIdxTemp], aaiMvpIdxBi[iRefList][iRefIdxTemp], amvp[eRefPicList], uiBitsTemp, uiCostTemp);
if ( uiCostTemp < uiCostBi ) //更新存储最优信息
{
bChanged = true;
cMvBi[iRefList] = cMvTemp[iRefList][iRefIdxTemp];
iRefIdxBi[iRefList] = iRefIdxTemp;
uiCostBi = uiCostTemp;
uiMotBits[iRefList] = uiBitsTemp - uiMbBits[2] - uiMotBits[1-iRefList];
uiBits[2] = uiBitsTemp;
if(iNumIter!=1) //4次循环时,即不是fastSearch时,重新运动补偿
{
// Set motion
pu.mv [eRefPicList] = cMvBi [iRefList];
pu.refIdx[eRefPicList] = iRefIdxBi[iRefList];
PelUnitBuf predBufTmp = m_tmpPredStorage[iRefList].getBuf( UnitAreaRelative(cu, pu) );
motionCompensation( pu, predBufTmp, eRefPicList ); //MC获得pred像素
}
}
} // for loop-iRefIdxTemp
if ( !bChanged ) //说明前面的单向所有帧的循环,获得的最优运动信息没变
{
if ( uiCostBi <= uiCost[0] && uiCostBi <= uiCost[1] ) //Bi的cost已经小于两个单向预测,说明Bi运动信息最优,直接结束Bi的处理
{
xCopyAMVPInfo(&aacAMVPInfo[0][iRefIdxBi[0]], &amvp[REF_PIC_LIST_0]);
xCheckBestMVP( REF_PIC_LIST_0, cMvBi[0], cMvPredBi[0][iRefIdxBi[0]], aaiMvpIdxBi[0][iRefIdxBi[0]], amvp[eRefPicList], uiBits[2], uiCostBi);
if(!cs.slice->getMvdL1ZeroFlag())
{
xCopyAMVPInfo(&aacAMVPInfo[1][iRefIdxBi[1]], &amvp[REF_PIC_LIST_1]);
xCheckBestMVP( REF_PIC_LIST_1, cMvBi[1], cMvPredBi[1][iRefIdxBi[1]], aaiMvpIdxBi[1][iRefIdxBi[1]], amvp[eRefPicList], uiBits[2], uiCostBi);
}
}
break;
}
} // for loop-iter
} // if (B_SLICE)
#if AMP_MRG
} //end if bTestNormalMC
#endif
// Clear Motion Field //清空pu中的运动信息
pu.mv [REF_PIC_LIST_0] = Mv();
pu.mv [REF_PIC_LIST_1] = Mv(); //mv、mvd、refIdx、mvpIdx、mvpNum
pu.mvd [REF_PIC_LIST_0] = cMvZero;
pu.mvd [REF_PIC_LIST_1] = cMvZero;
pu.refIdx[REF_PIC_LIST_0] = NOT_VALID;
pu.refIdx[REF_PIC_LIST_1] = NOT_VALID;
pu.mvpIdx[REF_PIC_LIST_0] = NOT_VALID;
pu.mvpIdx[REF_PIC_LIST_1] = NOT_VALID;
pu.mvpNum[REF_PIC_LIST_0] = NOT_VALID;
pu.mvpNum[REF_PIC_LIST_1] = NOT_VALID;
UInt uiMEBits = 0;
// Set Motion Field
cMv [1] = mvValidList1;
iRefIdx[1] = refIdxValidList1;
uiBits [1] = bitsValidList1;
uiCost [1] = costValidList1;
#if AMP_MRG
if (bTestNormalMC)
{
#endif //根据cost选择最优的inter_ME模式和运动信息
if ( uiCostBi <= uiCost[0] && uiCostBi <= uiCost[1]) //双向预测的cost比两个单项预测cost都小,则选双向预测
{
uiLastMode = 2;
pu.mv [REF_PIC_LIST_0] = cMvBi[0];
pu.mv [REF_PIC_LIST_1] = cMvBi[1];
pu.mvd [REF_PIC_LIST_0] = cMvBi[0] - cMvPredBi[0][iRefIdxBi[0]];
pu.mvd [REF_PIC_LIST_1] = cMvBi[1] - cMvPredBi[1][iRefIdxBi[1]];
pu.refIdx[REF_PIC_LIST_0] = iRefIdxBi[0];
pu.refIdx[REF_PIC_LIST_1] = iRefIdxBi[1];
pu.mvpIdx[REF_PIC_LIST_0] = aaiMvpIdxBi[0][iRefIdxBi[0]];
pu.mvpIdx[REF_PIC_LIST_1] = aaiMvpIdxBi[1][iRefIdxBi[1]];
pu.mvpNum[REF_PIC_LIST_0] = aaiMvpNum[0][iRefIdxBi[0]];
pu.mvpNum[REF_PIC_LIST_1] = aaiMvpNum[1][iRefIdxBi[1]];
pu.interDir = 3;
uiMEBits = uiBits[2];
}
else if ( uiCost[0] <= uiCost[1] ) //前向预测
{
uiLastMode = 0;
pu.mv [REF_PIC_LIST_0] = cMv[0];
pu.mvd [REF_PIC_LIST_0] = cMv[0] - cMvPred[0][iRefIdx[0]];
pu.refIdx[REF_PIC_LIST_0] = iRefIdx[0];
pu.mvpIdx[REF_PIC_LIST_0] = aaiMvpIdx[0][iRefIdx[0]];
pu.mvpNum[REF_PIC_LIST_0] = aaiMvpNum[0][iRefIdx[0]];
pu.interDir = 1;
uiMEBits = uiBits[0];
}
else //后向预测
{
uiLastMode = 1;
pu.mv [REF_PIC_LIST_1] = cMv[1];
pu.mvd [REF_PIC_LIST_1] = cMv[1] - cMvPred[1][iRefIdx[1]];
pu.refIdx[REF_PIC_LIST_1] = iRefIdx[1];
pu.mvpIdx[REF_PIC_LIST_1] = aaiMvpIdx[1][iRefIdx[1]];
pu.mvpNum[REF_PIC_LIST_1] = aaiMvpNum[1][iRefIdx[1]];
pu.interDir = 2;
uiMEBits = uiBits[1];
}
#if AMP_MRG
} // end if bTestNormalMC
#endif
if ( cu.partSize != SIZE_2Nx2N ) //VTM中默认的cu.partSize为SIZE_2Nx2N
{ //对于非2Nx2N的分块,计算并合并运动估计代价
UInt uiMRGIndex = 0;
#if AMP_MRG
// calculate ME cost
Distortion uiMEError = std::numeric_limits<Distortion>::max();
Distortion uiMECost = std::numeric_limits<Distortion>::max();
if( bTestNormalMC )
{
uiMEError = xGetInterPredictionError( pu, origBuf );
uiMECost = uiMEError + m_pcRdCost->getCost( uiMEBits );
}
#else
// calculate ME cost
Distortion uiMEError = xGetInterPredictionError( pu, origBuf );
Distortion uiMECost = uiMEError + m_pcRdCost->getCost( uiMEBits );
#endif
// save ME result.
InterPredictionData savedPU = pu;
// find Merge result
Distortion uiMRGCost = std::numeric_limits<Distortion>::max();
pu.initData();
xMergeEstimation( pu, origBuf, puIdx, uiMRGIndex, uiMRGCost, mergeCtx ); //合并估计信息
if( uiMRGCost < uiMECost )
{
// set Merge result
mergeCtx.setMergeInfo( pu, uiMRGIndex );
}
else
{
pu = savedPU;
}
}
CHECK( !( !cu.cs->pcv->only2Nx2N || cu.partSize == SIZE_2Nx2N ), "Unexpected part size for QTBT." );
m_maxCompIDToPred = MAX_NUM_COMPONENT;
{
PU::spanMotionInfo( pu, mergeCtx );
}
// MC
PelUnitBuf predBuf = pu.cs->getPredBuf(pu);
motionCompensation( pu, predBuf, REF_PIC_LIST_X ); //MC,用得到的最优运动信息,得到pred像素
puIdx++;
}
setWpScalingDistParam( -1, REF_PIC_LIST_X, cu.cs->slice ); //加权预测参数
return;
}