VVC IBC 提案机器翻译 O1170 Bitstream conformance with a virtual IBC buffer concept

Abstract

During discussion of IBC conformance, it is agreed that it is better targeting a bitstream constraint without imposing a mandatory check at the decoder and a simple solution is preferred to define the IBC bitstream constrains. This document presents a simple design of bitstream constrains based on a virtual IBC buffer concept.
在讨论IBC一致性时，大家一致认为最好是针对位流约束，而不是强制检查解码器，并且最好使用简单的解决方案来定义IBC位流约束。本文提出了一个基于虚拟IBC缓冲区概念的位流约束的简单设计。

1.Introduction

During the discussion of IBC related proposals of decoder checks and bitstream constrain. It is asserted that it may not be desired to move IBC block vector check to decoder and prefer bitstream constrains at encoder. However, the current IBC bitstream constrains in the current draft seems complicated, which makes an encoder easy to generate illegal bitstream; conformance test difficult to be perform and decoder difficult to detect non-conforming bitstreams.
在讨论IBC相关解码器检查和位流约束的建议时。有人断言，可能不希望将IBC块向量检查移到解码器中，而希望在编码器中使用位流约束。但是当前草案中对IBC码流的约束比较复杂，使得编码器容易产生非法码流;一致性测试难以执行，译码器难以检测不一致的比特流。

This proposal presents a simple design, which is straightforward and clean in concept. The scheme shares some common elements of related proposals at this meeting, e.g., JVET-O0073, JVET-O0127 and JVET-O0248, JVET-O0482 and does not change the reference area of the current design.
本方案设计简单，概念简单明了。本方案与本次会议的相关提案有一些共同之处，如:JVET-O0073、JVET-O0127、JVET-O0248、JVET-O0482，并不改变当前设计的参考范围。

2 Proposed scheme

A virtual buffer concept is introduced to help describing the reference region for IBC prediction mode. For CTU size being ctbSize, we denote wIbcBuf = 128*128/ctbSize and define a virtual IBC buffer, ibcBuf, with width being wIbcBuf and height being ctbSize. Thus,
引入虚拟缓冲区概念来帮助描述IBC预测模式的参考区域。对于CTU大小为ctbSize，我们表示wIbcBuf = 128*128/ctbSize，并定义一个虚拟IBC缓冲区IbcBuf，宽度为wIbcBuf，高度为ctbSize。因此，

– For CTU size being 128x128, the size of ibcBuf is also 128x128.
– For CTU size being 64x64, the size of ibcBuf is 256x64.
– For CTU size being 32x32, the size of ibcBuf is 512x32.
–对于128x128的CTU尺寸，ibcBuf的尺寸也是128x128。
–对于64x64的CTU尺寸，ibcBuf的尺寸为256x64。
–对于32x32的CTU尺寸，ibcBuf的尺寸为512x32。

It is noted that VPDU width and height are min(ctbSize, 64). We denote Wv = min(ctbSize, 64)
请注意，VPDU的宽度和高度是 $min(ctbSize,64)$。我们表示 $W_v=min(ctbSize,64)$。

The virtual IBC buffer, ibcBuf is maintained as follows.
虚拟IBC缓冲区ibcBuf的维护如下。

1)At the beginning of decoding each CTU row, refresh the whole ibcBuf with value (-1).
1)在解码每个CTU行的开始，用值(-1)刷新整个ibcBuf。

2)At the beginning of decoding a VPDU (xVPDU, yVPDU) relative to the top-left corner of the picture, set the ibcBuf[ x ][ y ] = -1, with
x = xVPDU%wIbcBuf, …, xVPDU% wIbcBuf + Wv – 1;
y = yVPDU%ctbSize, …, yVPDU%ctbSize + Wv – 1.

在开始解码图片左上角的VPDU (xVPDU，yVPDU)时，设置ibcBuf[ x ][ y ] = -1，同时：

$$x=xVPDU\%wIbcBuf,\dots ,xVPDU\%wIbcBuf+Wv–1$$

$$y=yVPDU\%ctbSize,\dots ,yVPDU\%ctbSize+Wv–1$$

3)After decoding a CU contains (x, y) relative to the top-left of the picture, set
ibcBuf[ x % wIbcBuf ][ y % ctbSize ] = recSample[ x ][ y ]

解码一个包含相对于图片左上角(x，y)的CU后，设置
$ibcBuf[x\%wIbcBuf][y\%ctbSize]=recSample[x][y]$

So a bitstream constrain can be simply described as
因此比特流约束可以简单地描述为

It is a requirement of bitstream comformance that for a bv, ibcBuf[ (x + bv[0])% wIbcBuf] [ (y + bv[1]) % ctbSize ] shall not be equal to -1.
比特流一致性的要求方面：对一个bv来说，$ibcBuf[(x+bv[0])\%wIbcBuf][(y+bv[1])\%ctbSize]$不应该等于-1

With the concept of IBC reference buffer, it also simplifies the text for the decoding process by avoid reference to the inter interpolation and motion compensation process, including subblock process.
利用IBC参考缓冲器的概念，它还通过避免参考帧间插值和运动补偿过程(包括子块过程)来简化解码过程的文本。

3 BD-rate performance

The proposed scheme is compatible to the VTM-5.0, where bitstreams generated by VTM-5.0 should be all conforming bitstreams. So, there is no BD-rate change.
提议的方案与VTM-5.0兼容，其中由VTM-5.0生成的比特流应该都是一致的比特流。所以，没有BD-rate变化。

4 Conclusions

In this document, by introducing a virtual IBC buffer concept, bitstream constrains can be represented in a straightforward and simple way. It also significantly simplifies IBC decoding process in the text.
在本文中，通过引入虚拟IBC缓冲器的概念，可以以简单明了的方式表示比特流约束。这也大大简化了文本中的IBC解码过程。

Thus, we recommend adoption this proposal into the next version of VVC draft.
因此，我们建议在下一版VVC草案中采纳这一建议。

5 Changes to VVC working draft

The changes, marked in yellow, are based on JVET-N1001_v10.
以黄色标记的变更基于JVET-N1001_v10。（Draft 5）

（省略一堆对draft的修改）

看Draft开头的记录显示，Draft 6接受了修改。

来看看代码吧

这三个函数分别对应上面算法描述中关于IBCBuffer维护的三句话。

InterPrediction::resetIBCBuffer

1)At the beginning of decoding each CTU row, refresh the whole ibcBuf with value (-1).
1)在解码每个CTU行的开始，用值(-1)刷新整个ibcBuf。

//DecSlice.cpp
//decpmressSlice
//这个ctuXPosInCtus == tileXPosInCtus就代表着一个CTU行的开始，行的开始要把IBCBuffer清零
    if ((cs.slice->getSliceType() != I_SLICE || cs.sps->getIBCFlag()) && ctuXPosInCtus == tileXPosInCtus)
    {
      cs.motionLut.lut.resize(0);
      cs.motionLut.lutIbc.resize(0);
      cs.resetIBCBuffer = true; //重置IBCBuffer的标志
    }
    m_pcCuDecoder->decompressCtu( cs, ctuArea );

//DecCu.cpp
void DecCu::decompressCtu( CodingStructure& cs, const UnitArea& ctuArea )
{
  if (cs.resetIBCBuffer)
  {
    m_pcInterPred->resetIBCBuffer(cs.pcv->chrFormat, cs.slice->getSPS()->getMaxCUHeight());
    cs.resetIBCBuffer = false;
  }
}

void InterPrediction::resetIBCBuffer(const ChromaFormat chromaFormatIDC, const int ctuSize)
{
  const UnitArea area = UnitArea(chromaFormatIDC, Area(0, 0, m_IBCBufferWidth, ctuSize));
  m_IBCBuffer.getBuf(area).fill(-1);
}

InterPrediction::resetVPDUforIBC

resetVPDUforIBC这个函数就是算法描述中这个图的实现。

2)At the beginning of decoding a VPDU (xVPDU, yVPDU) relative to the top-left corner of the picture, set the ibcBuf[ x ][ y ] = -1, with
x = xVPDU%wIbcBuf, …, xVPDU% wIbcBuf + Wv – 1;
y = yVPDU%ctbSize, …, yVPDU%ctbSize + Wv – 1.
在开始解码图片左上角的VPDU (xVPDU, yVPDU)时，设置ibcBuf[ x ][ y ] = -1，同时：
$$x=xVPDU\%wIbcBuf,\dots ,xVPDU\%wIbcBuf+Wv–1$$
$$y=yVPDU\%ctbSize,\dots ,yVPDU\%ctbSize+Wv–1$$

这两个公式指的就是图中画着X的区域，VPDU (xVPDU,yVPDU)代表的是图中Curr当前区域中的某个cu的坐标，xVPDU%wIbcBuf就实现了坐标的转换，把一个全局的坐标转换到IBCBuffer里的坐标。wIbcBuf一般是128，就是图中一个CTU，Wv一般是64，就是图中一个块的宽度。

VTM6.0好理解一点

        const int vSize = cs.slice->getSPS()->getMaxCUHeight() > 64 ? 64 : cs.slice->getSPS()->getMaxCUHeight();
        if((currCU.Y().x % vSize) == 0 && (currCU.Y().y % vSize) == 0)
        {
          m_pcInterPred->resetVPDUforIBC(cs.pcv->chrFormat, cs.slice->getSPS()->getMaxCUHeight(), vSize, currCU.Y().x, currCU.Y().y);
        }

VTM10.0

        const int vSize = cs.slice->getSPS()->getMaxCUHeight() > 64 ? 64 : cs.slice->getSPS()->getMaxCUHeight();
        if((currCU.Y().x % vSize) == 0 && (currCU.Y().y % vSize) == 0)
        {
          for(int x = currCU.Y().x; x < currCU.Y().x + currCU.Y().width; x += vSize)
          {
            for(int y = currCU.Y().y; y < currCU.Y().y + currCU.Y().height; y += vSize)
            {
              m_pcInterPred->resetVPDUforIBC(cs.pcv->chrFormat, cs.slice->getSPS()->getMaxCUHeight(), vSize, x + g_IBCBufferSize / cs.slice->getSPS()->getMaxCUHeight() / 2, y);
            }
          }
        }
//g_IBCBufferSize == 32768 == 128*128*2
//相比较VTM6.0，不仅仅使用x和y，而且还尝试使用x+vSize和y+vSize，应该是为了防止漏掉那些width和height大于vSize的情况

void InterPrediction::resetVPDUforIBC(const ChromaFormat chromaFormatIDC, const int ctuSize, const int vSize, const int xPos, const int yPos)
{
  const UnitArea area = UnitArea(chromaFormatIDC, Area(xPos & (m_IBCBufferWidth - 1), yPos & (ctuSize - 1), vSize, vSize));
  m_IBCBuffer.getBuf(area).fill(-1);
}

InterPrediction::xFillIBCBuffer

3)After decoding a CU contains (x, y) relative to the top-left of the picture, set
ibcBuf[ x % wIbcBuf ][ y % ctbSize ] = recSample[ x ][ y ]
解码一个包含相对于图片左上角(x，y)的CU后，设置
$ibcBuf[x\%wIbcBuf][y\%ctbSize]=recSample[x][y]$

void DecCu::decompressCtu( CodingStructure& cs, const UnitArea& ctuArea )
{
      m_pcInterPred->xFillIBCBuffer(currCU);
}

void InterPrediction::xFillIBCBuffer(CodingUnit &cu)
{
  for (auto &currPU : CU::traverseTUs(cu))
  {
    for (const CompArea &area : currPU.blocks)
    {
      if (!area.valid())
      {
        continue;
      }

      const unsigned int lcuWidth = cu.cs->slice->getSPS()->getMaxCUWidth();
      const int shiftSampleHor = ::getComponentScaleX(area.compID, cu.chromaFormat);
      const int shiftSampleVer = ::getComponentScaleY(area.compID, cu.chromaFormat);
      const int ctuSizeLog2Ver = floorLog2(lcuWidth) - shiftSampleVer;
      const int pux = area.x & ((m_IBCBufferWidth >> shiftSampleHor) - 1);
      const int puy = area.y & (( 1 << ctuSizeLog2Ver ) - 1);
      const CompArea dstArea = CompArea(area.compID, cu.chromaFormat, Position(pux, puy), Size(area.width, area.height));
      CPelBuf srcBuf = cu.cs->getRecoBuf(area);
      PelBuf dstBuf = m_IBCBuffer.getBuf(dstArea);

      dstBuf.copyFrom(srcBuf);
    }
  }
}

DecCu::decompressCtu

完整地看一下decompressCtu

//VTM10.0
void DecCu::decompressCtu( CodingStructure& cs, const UnitArea& ctuArea )
{

  const int maxNumChannelType = cs.pcv->chrFormat != CHROMA_400 && CS::isDualITree( cs ) ? 2 : 1;

  if (cs.resetIBCBuffer)
  {
    m_pcInterPred->resetIBCBuffer(cs.pcv->chrFormat, cs.slice->getSPS()->getMaxCUHeight());
    cs.resetIBCBuffer = false;
  }
  for( int ch = 0; ch < maxNumChannelType; ch++ )
  {
    const ChannelType chType = ChannelType( ch );
    Position prevTmpPos;
    prevTmpPos.x = -1; prevTmpPos.y = -1;

    for( auto &currCU : cs.traverseCUs( CS::getArea( cs, ctuArea, chType ), chType ) )
    {
      if(currCU.Y().valid())
      {
        const int vSize = cs.slice->getSPS()->getMaxCUHeight() > 64 ? 64 : cs.slice->getSPS()->getMaxCUHeight();
        if((currCU.Y().x % vSize) == 0 && (currCU.Y().y % vSize) == 0)
        {
          for(int x = currCU.Y().x; x < currCU.Y().x + currCU.Y().width; x += vSize)
          {
            for(int y = currCU.Y().y; y < currCU.Y().y + currCU.Y().height; y += vSize)
            {
              m_pcInterPred->resetVPDUforIBC(cs.pcv->chrFormat, cs.slice->getSPS()->getMaxCUHeight(), vSize, x + g_IBCBufferSize / cs.slice->getSPS()->getMaxCUHeight() / 2, y);
            }
          }
        }
      }
      if (currCU.predMode != MODE_INTRA && currCU.predMode != MODE_PLT && currCU.Y().valid())
      {
        xDeriveCUMV(currCU);
      }
      switch( currCU.predMode )
      {
      case MODE_INTER:
      case MODE_IBC:
        xReconInter( currCU );
        break;
      case MODE_PLT:
      case MODE_INTRA:
        xReconIntraQT( currCU );
        break;
      default:
        THROW( "Invalid prediction mode" );
        break;
      }

      m_pcInterPred->xFillIBCBuffer(currCU);

      DTRACE_BLOCK_REC( cs.picture->getRecoBuf( currCU ), currCU, currCU.predMode );
    }
  }
}

编码端约束

把以前的PU::isBlockVectorValid函数改名为InterSearch::searchBv

#if JVET_O1170_IBC_VIRTUAL_BUFFER
        bool validCand = searchBv(pu, cuPelX, cuPelY, roiWidth, roiHeight, picWidth, picHeight, xPred, yPred, lcuWidth);
#else
        bool validCand = PU::isBlockVectorValid(pu, cuPelX, cuPelY, roiWidth, roiHeight, picWidth, picHeight, 0, 0, xPred, yPred, lcuWidth);
#endif

source/Lib/EncoderLib/InterSearch.cpp · VTM-6.0 · jvet / VVCSoftware_VTM · GitLab

#if JVET_O1170_IBC_VIRTUAL_BUFFER
bool InterSearch::searchBv(PredictionUnit& pu, int xPos, int yPos, int width, int height, int picWidth, int picHeight, int xBv, int yBv, int ctuSize)
{
  const int ctuSizeLog2 = g_aucLog2[ctuSize];

  int refRightX = xPos + xBv + width - 1;
  int refBottomY = yPos + yBv + height - 1;

  int refLeftX = xPos + xBv;
  int refTopY = yPos + yBv;

  if ((xPos + xBv) < 0)
  {
    return false;
  }
  if (refRightX >= picWidth)
  {
    return false;
  }

  if ((yPos + yBv) < 0)
  {
    return false;
  }
  if (refBottomY >= picHeight)
  {
    return false;
  }
  if ((xBv + width) > 0 && (yBv + height) > 0)
  {
    return false;
  }

  // Don't search the above CTU row
  if (refTopY >> ctuSizeLog2 < yPos >> ctuSizeLog2)
    return false;

  // Don't search the below CTU row
  if (refBottomY >> ctuSizeLog2 > yPos >> ctuSizeLog2)
  {
    return false;
  }

  // in the same CTU line
  int numLeftCTUs = (1 << ((7 - ctuSizeLog2) << 1)) - ((ctuSizeLog2 < 7) ? 1 : 0);
  if ((refRightX >> ctuSizeLog2 <= xPos >> ctuSizeLog2) && (refLeftX >> ctuSizeLog2 >= (xPos >> ctuSizeLog2) - numLeftCTUs))
  {

    // in the same CTU, or left CTU
    // if part of ref block is in the left CTU, some area can be referred from the not-yet updated local CTU buffer
    if (((refLeftX >> ctuSizeLog2) == ((xPos >> ctuSizeLog2) - 1)) && (ctuSizeLog2 == 7))
    {
      // ref block's collocated block in current CTU
      const Position refPosCol = pu.Y().topLeft().offset(xBv + ctuSize, yBv);
      int offset64x = (refPosCol.x >> (ctuSizeLog2 - 1)) << (ctuSizeLog2 - 1);
      int offset64y = (refPosCol.y >> (ctuSizeLog2 - 1)) << (ctuSizeLog2 - 1);
      const Position refPosCol64x64 = {offset64x, offset64y};
      if (pu.cs->isDecomp(refPosCol64x64, toChannelType(COMPONENT_Y)))
        return false;
      if (refPosCol64x64 == pu.Y().topLeft())
        return false;
    }
  }
  else
    return false;

  // in the same CTU, or valid area from left CTU. Check if the reference block is already coded
  const Position refPosLT = pu.Y().topLeft().offset(xBv, yBv);
  const Position refPosBR = pu.Y().bottomRight().offset(xBv, yBv);
  const ChannelType      chType = toChannelType(COMPONENT_Y);
  if (!pu.cs->isDecomp(refPosBR, chType))
    return false;
  if (!pu.cs->isDecomp(refPosLT, chType))
    return false;
  return true;
}
#endif

解码端约束

source/Lib/CommonLib/InterPrediction.cpp · VTM-6.0 · jvet / VVCSoftware_VTM · GitLab

#if JVET_O1170_IBC_VIRTUAL_BUFFER
void InterPrediction::xFillIBCBuffer(CodingUnit &cu)
{
  for (auto &currPU : CU::traverseTUs(cu))
  {
    for (const CompArea &area : currPU.blocks)
    {
      if (!area.valid())
        continue;

      const unsigned int lcuWidth = cu.cs->slice->getSPS()->getMaxCUWidth();
      const int shiftSample = ::getComponentScaleX(area.compID, cu.chromaFormat);
      const int ctuSizeLog2 = g_aucLog2[lcuWidth] - shiftSample;
      const int pux = area.x & ((m_IBCBufferWidth >> shiftSample) - 1);
      const int puy = area.y & (( 1 << ctuSizeLog2 ) - 1);
      const CompArea dstArea = CompArea(area.compID, cu.chromaFormat, Position(pux, puy), Size(area.width, area.height));
      CPelBuf srcBuf = cu.cs->getRecoBuf(area);
      PelBuf dstBuf = m_IBCBuffer.getBuf(dstArea);

      dstBuf.copyFrom(srcBuf);
    }
  }
}

void InterPrediction::xIntraBlockCopy(PredictionUnit &pu, PelUnitBuf &predBuf, const ComponentID compID)
{
  const unsigned int lcuWidth = pu.cs->slice->getSPS()->getMaxCUWidth();
  int shiftSample = ::getComponentScaleX(compID, pu.chromaFormat);
  const int ctuSizeLog2 = g_aucLog2[lcuWidth] - shiftSample;
  pu.bv = pu.mv[REF_PIC_LIST_0];
  pu.bv.changePrecision(MV_PRECISION_INTERNAL, MV_PRECISION_INT);
  int refx, refy;
  if (compID == COMPONENT_Y)
  {
    refx = pu.Y().x + pu.bv.hor;
    refy = pu.Y().y + pu.bv.ver;
  }
  else
  {//Cb or Cr
    refx = pu.Cb().x + (pu.bv.hor >> shiftSample);
    refy = pu.Cb().y + (pu.bv.ver >> shiftSample);
  }
  refx &= ((m_IBCBufferWidth >> shiftSample) - 1);
  refy &= ((1 << ctuSizeLog2) - 1);

  if (refx + predBuf.bufs[compID].width <= (m_IBCBufferWidth >> shiftSample))
  {
    const CompArea srcArea = CompArea(compID, pu.chromaFormat, Position(refx, refy), Size(predBuf.bufs[compID].width, predBuf.bufs[compID].height));
    const CPelBuf refBuf = m_IBCBuffer.getBuf(srcArea);
    predBuf.bufs[compID].copyFrom(refBuf);
  }
  else
  {//wrap around
    int width = (m_IBCBufferWidth >> shiftSample) - refx;
    CompArea srcArea = CompArea(compID, pu.chromaFormat, Position(refx, refy), Size(width, predBuf.bufs[compID].height));
    CPelBuf srcBuf = m_IBCBuffer.getBuf(srcArea);
    PelBuf dstBuf = PelBuf(predBuf.bufs[compID].bufAt(Position(0, 0)), predBuf.bufs[compID].stride, Size(width, predBuf.bufs[compID].height));
    dstBuf.copyFrom(srcBuf);

    width = refx + predBuf.bufs[compID].width - (m_IBCBufferWidth >> shiftSample);
    srcArea = CompArea(compID, pu.chromaFormat, Position(0, refy), Size(width, predBuf.bufs[compID].height));
    srcBuf = m_IBCBuffer.getBuf(srcArea);
    dstBuf = PelBuf(predBuf.bufs[compID].bufAt(Position((m_IBCBufferWidth >> shiftSample) - refx, 0)), predBuf.bufs[compID].stride, Size(width, predBuf.bufs[compID].height));
    dstBuf.copyFrom(srcBuf);
  }
}

#if JVET_O1170_CHECK_BV_AT_DECODER
void InterPrediction::resetIBCBuffer(const ChromaFormat chromaFormatIDC, const int ctuSize)
{
  const UnitArea area = UnitArea(chromaFormatIDC, Area(0, 0, m_IBCBufferWidth, ctuSize));
  m_IBCBuffer.getBuf(area).fill(-1);
}

void InterPrediction::resetVPDUforIBC(const ChromaFormat chromaFormatIDC, const int ctuSize, const int vSize, const int xPos, const int yPos)
{
  const UnitArea area = UnitArea(chromaFormatIDC, Area(xPos & (m_IBCBufferWidth - 1), yPos & (ctuSize - 1), vSize, vSize));
  m_IBCBuffer.getBuf(area).fill(-1);
}

bool InterPrediction::isLumaBvValid(const int ctuSize, const int xCb, const int yCb, const int width, const int height, const int xBv, const int yBv)
{
  if(((yCb + yBv) & (ctuSize - 1)) + height > ctuSize)
  {
    return false;
  }
  int refTLx = xCb + xBv;
  int refTLy = (yCb + yBv) & (ctuSize - 1);
  PelBuf buf = m_IBCBuffer.Y();
  for(int x = 0; x < width; x += 4)
  {
    for(int y = 0; y < height; y += 4)
    {
      if(buf.at((x + refTLx) & (m_IBCBufferWidth - 1), y + refTLy) == -1) return false;
      if(buf.at((x + 3 + refTLx) & (m_IBCBufferWidth - 1), y + refTLy) == -1) return false;
      if(buf.at((x + refTLx) & (m_IBCBufferWidth - 1), y + 3 + refTLy) == -1) return false;
      if(buf.at((x + 3 + refTLx) & (m_IBCBufferWidth - 1), y + 3 + refTLy) == -1) return false;
    }
  }
  return true;
}
#endif
#endif