在JM8.6解码端提取DCT系数和残差并模拟验证itrans函数的功能

最新推荐文章于 2023-04-26 10:37:30 发布

涛歌依旧

最新推荐文章于 2023-04-26 10:37:30 发布

阅读量1w

点赞数 4

分类专栏： S1: C/C++ s6: 视频研究

本文链接：https://blog.csdn.net/stpeace/article/details/8125377

版权

S1: C/C++ 同时被 2 个专栏收录

972 篇文章 329 订阅

订阅专栏

s6: 视频研究

69 篇文章 18 订阅

订阅专栏

如何在JM8.6的解码端提取DCT系数呢？自然而然会想到的问题是：DCT系数从哪里来，要到哪里去，所以，要提取DCT系数，可以看它是哪里产生的，也可以看它要到哪里去，然后在任意一条路上"截击"它，必然能找到DCT系数. 下面找DCT系数的思路是要看DCT系数到哪里去. 很显然，在解码端，DCT系数的下一个动作是进行反DCT变换，所以自然而言找到反DCT变换函数，找了一下，发现itans函数正是反DCT变换的函数.

先把JM8.6中的itrans函数的原型列出来：

void itrans(struct img_par *img, 
			int ioff,
            int joff,
            int i0,
            int j0);

仍然以foreman视频第一帧第一宏块第一个4*4块为例：

在itrans函数中，我们分别打印出img->cof和img->mpr,得到的结果分别为：

2304    -3840   -2816   -1600
960     -1200    320        0
768     -320     -512       320
0         0           0            0

128     128     128     128
128     128     128     128
128     128     128     128
128     128     128     128

利用H.264visa分析得：

====================== Y Data ======================    （解码端的DCT系数）
+------------------------+------------------------+------------------------+------------------------+
| 2304,-3840,-2816,-1600,| 768, 640, -256, 640,| 1280, 320, 256, -640,| 768, -320, -768,    0,|
| 960,-1200, 320,    0,|    0,    0, 320,    0,| -640, -800,    0,    0,| 960, -800, 320,    0,|
| 768, -320, -512, 320,| 512, -640,    0, -320,| -768, 320, -512, 320,| 768,    0, 256,    0,|
|    0,    0,    0,    0,|    0,    0,    0,    0,|    0, 400,    0,    0,| -320,    0,    0,    0,|
+------------------------+------------------------+------------------------+------------------------+
|-1024,-5120,-1792, -640,| 2560, -640, -256,    0,| 512,    0,    0,    0,|    0,    0,    0,    0,|
|    0, 1200, -640, -400,| -320, 400,    0,    0,| 640,    0,    0,    0,| 320,    0,    0,    0,|
| 512,    0, -256,    0,|    0,    0,    0,    0,|    0,    0,    0,    0,|    0,    0,    0,    0,|
| 320,    0,    0,    0,| -320,    0,    0,    0,|    0,    0,    0,    0,|    0,    0,    0,    0,|
+------------------------+------------------------+------------------------+------------------------+
|    0, 320,    0,    0,|    0,    0,    0,    0,| -768, 640,    0,    0,| 512,    0, -256,    0,|
|    0,    0,    0,    0,|    0,    0,    0,    0,| 640, -800,    0,    0,| -640, -400, 320,    0,|
|    0,    0,    0,    0,|    0,    0,    0,    0,| -256, 320,    0,    0,|    0, 320,    0,    0,|
|    0,    0,    0,    0,|    0,    0,    0,    0,| 320, -400,    0,    0,| -320,    0,    0,    0,|
+------------------------+------------------------+------------------------+------------------------+
| -512, 640, -256,    0,|    0,    0,    0,    0,| 1024, -320, -256,    0,| 1024, -320,    0,    0,|
| 960,-1200, 320,    0,| 960, -800, 320,    0,|-1280, 800,    0, 400,| -640, 400,    0,    0,|
| -512, 320,    0,    0,|    0,    0, -256,    0,|    0,    0, -256,    0,| 512, 640,    0,    0,|
|    0,    0,    0,    0,| -320,    0,    0,    0,| -640, 800,    0,    0,|    0,    0,    0,    0,|
+------------------------+------------------------+------------------------+------------------------+

====================== Y Data ====================== （预测值）
+----------------+----------------+----------------+----------------+
|128,128,128,128,|238,238,238,238,|255,255,255,255,|245,228,210,192,|
|128,128,128,128,|205,205,205,205,|180,180,180,180,|210,192,174,170,|
|128,128,128,128,|167,167,167,167,|137,137,137,137,|174,170,166,166,|
|128,128,128,128,|160,160,160,160,|169,169,169,169,|166,166,166,166,|
+----------------+----------------+----------------+----------------+
|161,161,161,161,|179,189,182,169,|201,201,201,201,|219,219,219,219,|
|161,161,161,161,|184,185,175,169,|214,214,214,214,|227,227,227,227,|
|161,161,161,161,|189,182,169,169,|216,216,216,216,|219,219,219,219,|
|161,161,161,161,|185,175,169,169,|229,229,229,229,|227,227,227,227,|
+----------------+----------------+----------------+----------------+
| 28,127,218,229,|224,224,224,224,|227,227,228,227,|224,224,224,224,|
| 28,127,218,229,|224,224,224,224,|224,225,227,227,|214,214,214,214,|
| 28,127,218,229,|224,224,224,224,|224,224,224,225,|212,212,212,212,|
| 28,127,218,229,|224,224,224,224,|224,224,224,224,|165,165,165,165,|
+----------------+----------------+----------------+----------------+
| 33,130,216,224,|231,223,215,194,|204,182,160,142,|127,133,139,147,|
| 33,130,216,224,|215,194,173,162,|160,142,125,120,|139,147,154,165,|
| 33,130,216,224,|173,162,150,150,|125,120,116,116,|154,165,176,176,|
| 33,130,216,224,|150,150,150,150,|116,116,116,116,|176,176,176,176,|
+----------------+----------------+----------------+----------------+

可见，img->cof存的正是解码端的DCT系数，而img->mpr存得正是预测值.

在之前的博文中已经介绍了根据公式生硬地实现反DCT的过程，并给出了相应的代码。在JM8.6解码器中虽然遵照了同样的公式，但是实现方式却有所不同，采用了蝶形运算法，结果相同，但效果更好。可以打这样一个简单的比方，根据公式直接实现类似于DFT, 而采用蝶形运算则类似于FFT.

（题外话：分析问题，解决问题要抓住问题的本质和关键. 在此，不得不承认，蝶形算法跟我所研究的方向关系不大，所以不需要具体管蝶形运算是如何实现的，我要的是运算的结果; 同理，运动估计在H.264中是最重要的，但是依然与我的研究方向关系不大，所以，不需要清楚地知道运动矢量到底是怎么估计出来的，我需要知道的是，估计出来的运动矢量在哪里，怎么提取.）

下面来模拟itrans函数的功能，同时提取残差，代码如下：

#include<iostream>
#define DQ_BITS  6
#define DQ_ROUND 32
#define BLOCK_SIZE 4
using namespace std;

// 解码端的DCT系数
int cof[BLOCK_SIZE][BLOCK_SIZE] =
{
	2304, -3840, -2816, -1600,
	960,  -1200, 320,   0,
	768,  -320,  -512,  320,
	0,    0,     0,     0
};

// 预测值
int mpr[BLOCK_SIZE][BLOCK_SIZE] =
{
	128, 128, 128, 128,
	128, 128, 128, 128,
	128, 128, 128, 128,
	128, 128, 128, 128
};

int m5[BLOCK_SIZE]; //中间变量
int m6[BLOCK_SIZE]; //中间变量
int m7[BLOCK_SIZE][BLOCK_SIZE]; // 进行了[0, 255]化的值
int myMat[BLOCK_SIZE][BLOCK_SIZE]; // 未进行[0, 255]化的值

int min(int x, int y)
{
	return x < y ? x : y;
}

int max(int x, int y)
{
	return x > y ? x : y;
}

//矩阵显示
void matrixShow(int a[][BLOCK_SIZE])
{
    int i, j;
    cout << "*****************************" << endl;
    for(i = 0; i < BLOCK_SIZE; i++)
	{
        for(j = 0; j < BLOCK_SIZE; j++)
        {
            cout << a[i][j] << "\t";
        }
		cout << endl;
	}

    cout << "*****************************" << endl << endl;
}

//矩阵相减
void  matrixSubtract(int a[][BLOCK_SIZE],int b[][BLOCK_SIZE])
{
    int i, j;
    for(i = 0;i < BLOCK_SIZE; i++)
	{
        for(j = 0; j < BLOCK_SIZE; j++)
        {
            a[i][j] -= b[i][j];
		}
	}
}

//反DCT变换
void itrans()
{
  int i, j, i1, j1;

  //蝶形算法（不用蝶形算法也可以，但要慢一些）

  for (j = 0; j < BLOCK_SIZE; j++)
  {
    for (i = 0; i < BLOCK_SIZE; i++)
        m5[i] = cof[i][j];
   
    m6[0] = m5[0] + m5[2];
    m6[1] = m5[0] - m5[2];
    m6[2] = (m5[1] >> 1) - m5[3];
    m6[3] = m5[1] + (m5[3] >> 1);

    for (i = 0; i < 2; i++)
    {
      i1 = 3 - i;

      m7[i][j] = m6[i] + m6[i1];
      m7[i1][j] = m6[i] - m6[i1];
    }

  }// end  for (j = 0; j < BLOCK_SIZE; j++)

  for (i = 0; i < BLOCK_SIZE; i++)
  {
    for (j = 0; j < BLOCK_SIZE; j++)
      m5[j] = m7[i][j];

    m6[0] = m5[0] + m5[2];
    m6[1] = m5[0] - m5[2];
    m6[2] = (m5[1] >> 1) - m5[3];
    m6[3] = m5[1] + (m5[3] >> 1);

    for (j = 0; j < 2; j++)
    {
      j1 = 3 - j;
	  
	  myMat[i][j] = (m6[j] + m6[j1] + (mpr[i][j] << DQ_BITS) + DQ_ROUND) >> DQ_BITS;
	  myMat[i][j1] = (m6[j] - m6[j1] + (mpr[i][j1] << DQ_BITS) + DQ_ROUND) >> DQ_BITS;
      
	  m7[i][j] = max(0, min(255, (m6[j] + m6[j1] + (mpr[i][j] << DQ_BITS) + DQ_ROUND) >> DQ_BITS));
      m7[i][j1] = max(0, min(255, (m6[j] - m6[j1] + (mpr[i][j1] << DQ_BITS) + DQ_ROUND) >> DQ_BITS));
    }
	
  }// end for (i = 0; i < BLOCK_SIZE; i++)

}

int main()
{
	cout << "解码端的DCT矩阵为:" << endl;
	matrixShow(cof);

	cout << "预测矩阵为:" << endl;
	matrixShow(mpr);

	itrans();

	cout <<"未进行[0, 255]化的矩阵为:" << endl;
	matrixShow(myMat);

	cout <<"进行了[0, 255]化的矩阵为:" << endl;
	matrixShow(m7); 

	//下面要用myMat, 而不能用m7
	cout <<"残差矩阵为:" << endl;
    matrixSubtract(myMat, mpr); 
	matrixShow(myMat);
	
	return 0;
}

解码端的DCT矩阵为:
*****************************
2304    -3840   -2816   -1600
960     -1200   320     0
768     -320    -512    320
0       0       0       0
*****************************

预测矩阵为:
*****************************
128     128     128     128
128     128     128     128
128     128     128     128
128     128     128     128
*****************************

未进行[0, 255]化的矩阵为:
*****************************
50      216     260     238
47      191     195     205
45      190     176     167
48      215     221     160
*****************************

进行了[0, 255]化的矩阵为:
*****************************
50      216     255     238
47      191     195     205
45      190     176     167
48      215     221     160
*****************************

残差矩阵为:
*****************************
-78     88      132     110
-81     63      67      77
-83     62      48      39
-80     87      93      32
*****************************

而用H.264visa观察的结果为：

====================== Y Data ====================== （进行了[0, 255]化的值）

+----------------+----------------+----------------+----------------+
| 50,216,255,238,|255,252,255,255,|247,255,252,251,|255,255,255,243,|
| 47,191,195,205,|235,206,216,180,|199,204,186,239,|198,212,206,180,|
| 45,190,176,167,|192,173,183,137,|191,183,141,181,|144,177,171,134,|
| 48,215,221,160,|174,184,194,169,|200,206,176,166,|172,194,187,157,|
+----------------+----------------+----------------+----------------+
| 44,183,212,183,|204,224,220,201,|219,219,219,219,|224,224,224,224,|
| 26,139,183,180,|216,228,225,214,|227,227,227,227,|230,230,230,230,|
| 30,123,199,216,|209,217,217,216,|219,219,219,219,|217,217,217,217,|
| 28,127,218,229,|212,218,229,229,|227,227,227,227,|222,222,222,222,|
+----------------+----------------+----------------+----------------+
| 33,130,216,224,|224,224,224,224,|223,223,225,224,|219,218,219,222,|
| 33,130,216,224,|224,224,224,224,|221,220,217,214,|212,219,228,229,|
| 33,130,216,224,|224,224,224,224,|221,219,214,212,|212,226,227,215,|
| 33,130,216,224,|224,224,224,224,|226,211,180,165,|188,200,189,165,|
+----------------+----------------+----------------+----------------+
| 29,126,216,228,|232,228,233,220,|204,182,158,108,|152,153,147,150,|
| 35,137,227,234,|228,197,182,187,|166,149,150,146,|130,144,163,180,|
| 33,136,208,196,|168,151,133,133,|141,140,128,132,|149,169,198,207,|
| 27,125,177,150,|141,153,140,116,|122,159,171,176,|209,209,211,211,|
+----------------+----------------+----------------+----------------+

====================== Y Data ====================== （残差值）
+------------------------+------------------------+------------------------+------------------------+
| -78,   88, 132, 110,|   24,   14,   24,   19,|   -8,    7,   -3,   -4,|   16,   31,   48,   51,|
| -81,   63,   67,   77,|   30,    1,   11, -25,|   19,   24,    6,   59,| -12,   20,   32,   10,|
| -83,   62,   48,   39,|   25,    6,   16, -30,|   54,   46,    4,   44,| -30,    7,    5, -32,|
| -80,   87,   93,   32,|   14,   24,   34,    9,|   31,   37,    7,   -3,|    6,   28,   21,   -9,|
+------------------------+------------------------+------------------------+------------------------+
| -117,   22,   51,   22,|   25,   35,   38,   32,|   18,   18,   18,   18,|    5,    5,    5,    5,|
| -135, -22,   22,   19,|   32,   43,   50,   45,|   13,   13,   13,   13,|    3,    3,    3,    3,|
| -131, -38,   38,   55,|   20,   35,   48,   47,|    3,    3,    3,    3,|   -2,   -2,   -2,   -2,|
| -133, -34,   57,   68,|   27,   43,   60,   60,|   -2,   -2,   -2,   -2,|   -5,   -5,   -5,   -5,|
+------------------------+------------------------+------------------------+------------------------+
|    5,    3,   -2,   -5,|    0,    0,    0,    0,|   -4,   -4,   -3,   -3,|   -5,   -6,   -5,   -2,|
|    5,    3,   -2,   -5,|    0,    0,    0,    0,|   -3,   -5, -10, -13,|   -2,    5,   14,   15,|
|    5,    3,   -2,   -5,|    0,    0,    0,    0,|   -3,   -5, -10, -13,|    0,   14,   15,    3,|
|    5,    3,   -2,   -5,|    0,    0,    0,    0,|    2, -13, -44, -59,|   23,   35,   24,    0,|
+------------------------+------------------------+------------------------+------------------------+
|   -4,   -4,    0,    4,|    1,    5,   18,   26,|    0,    0,   -2, -34,|   25,   20,    8,    3,|
|    2,    7,   11,   10,|   13,    3,    9,   25,|    6,    7,   25,   26,|   -9,   -3,    9,   15,|
|    0,    6,   -8, -28,|   -5, -11, -17, -17,|   16,   20,   12,   16,|   -5,    4,   22,   31,|
|   -6,   -5, -39, -74,|   -9,    3, -10, -34,|    6,   43,   55,   60,|   33,   33,   35,   35,|
+------------------------+------------------------+------------------------+------------------------+

数据高度一致，结果不言而喻.

最后值得一提的是，在JM8.6的解码器中，共有3处调用到了itrans函数，其中有一处我目前还没有发现程序执行时候真正调用到它. 另外两处分别是进行亮度反DCT变化和色度反DCT变换，具体是这样的：对于一个宏块而言，先进行16次亮度y的变换，再进行4次色度u的变换，再进行4次色度v的变换. 所以，如果只需要提取亮度y的相关数据时，可以在itrans函数里面提取数据时屏蔽掉色度数据，这样就更好.之前我的实验把色度数据也提取出来了，然后又用matlab把色度数据分割掉，这无疑多了两次不必要的操作.