数据压缩原理实验5 JPEG原理分析及JPEG解码器的调试
班级:17广电工方向三
姓名:利满雯
学号:201710413068
一、实验原理
1. JPEG编码原理(解码是编码的逆过程)
(1)图像预处理
将输入图像分成若干个8×8的小块。在此过程中需对图像的宽和高进行剪裁剪,使其都为8的倍数,不足的部分复制与其最近的像素值。目的是方便进行8×8 DCT变换。
(2)零偏置(Level Offset)
对于灰度级是2n的像素,通过减去2n-1,将无符号的整数值变成有符号数。例如对于n=8,即0255的值域,通过减去128,转换为值域在-128127之间的值。这样做的目的是使像素的绝对值出现3位10进制的概率大大减少。
(3)8×8 DCT变换
对每个单独的彩色图像分量,把整个分量图像分成8×8的图像块,再以8×8的图像块为一个单位进行量化和编码处理。由于DCT变换是能量守恒变换,即DCT变换前后图像的能量不变,因此通过DCT变换可以实现能量集中和去相关,有利于提高编码的效率。同时由于人眼对低频敏感,高频不敏感,经DCT变化后,多数低频信息集中在左上角,且数值较大,相关性较小,只需对左上角的值进行细量化,能在保证图像质量的同时提高压缩比。
(4)量化(中平型均匀量化器)
量化步距按照系数所在的位置和颜色分量来确定。Y分量和左上角位置的系数步距小,U、V分量和右下角位置的分量步距大。因为人眼对亮度信号比对色差信号更敏感,因此使用了两种量化表:亮度量化值和色差量化值。根据人眼的对低频敏感而对高频不太敏感视觉特性对低频分量采取较细的量化,对高频分量采取较粗的量化。
(5)编码
a. DC系数的差分编码
8x8图像块经过DCT变换后,得到的DC直流系数有两个特点:数值大、相邻8×8图像块的DC系数值变化不大。根据这个特点,JPEG对DC系数进行差分脉冲调制编码(DPCM)(其中,相邻DC系数之差不在进行量化),对相邻图像块之间的量化DC系数之差DIFF进行Huffman编码。对DIFF用Huffman编码:码字由两部分组成:类别ID和类内索引。通过下表判断DIFF在哪一个范围内,记下类别ID,作为码字的第一部分,DIFF的真值作为码字的第二部分。例:DC=8,上一DC=5,则DIFF=8-5=3 类别ID=2,类内索引=3,则码流=10011。
b.AC系数的游程编码
由于经DCT变换后,系数大多数集中在左上角,即低频分量区,因此采用Z 字形扫描按频率的高低顺序读出,可以出现很多连零的机会。尤其在最后,如果都是零,给出EOB (End of Block)即可。
之字形扫描扫描后二维数据变成成一维数据,定义(run,level)结构来对此一维数据编码。其中,run表示非0数据前0的个数,level表示非0数据的值。run的最大值为15个,用RRRR表示;level的编码类似DC系数,先分为16个类别,用4位SSSS表示类别号,再查类内索引。对(RRRR, SSSS)联合用Huffman编码,对类内索引采用定长码编码。即交流huffman码字的前四位表示0的个数,后四位表示该交流分量数值的二进制位数,也即接下来需要读入的位数。
2. JPEG文件格式
(1)Segment的组织形式
JPEG 在文件中以 Segment 的形式组织,它具有以下特点:
a. 均以 0xFF 开始,后跟1byte的Marker和2byte的Segment length(包含表示 Length 本身所占用的 2 byte,不含“0xFF”+“Marker”所占用的2byte);
b. 采用 Motorola 序(相对于Intel 序),即保存时高位在前,低位在后;
c. Data 部分中,0xFF 后若为 0x00,则跳过此字节不予处理。
(2)JPEG 的 Segment Marker
(3)文件格式示例
①FFD8:SOI, Start of Image,图像开始,所有的JPEG文件都必须以SOI开始。
②FFE0:Application0,应用程序保留标记0
③FFDB:DQT,Define Quantization Table,定义量化表
④FFC0: SOF0 , Start of Frame, 基线离散余弦变换
⑤FFC4: DHT, Define Huffman Table,定义 Huffman 树表
⑥FFDA: SOS, Start of Scan,扫描开始
⑦熵编码数据
⑧FFD9:EOI,End of Image,图像结束,JPEG 文件必须以 EOI 结束
二、JPEG的解码流程
(1)读取文件
(2)解析 Segment Marker
解析 SOI
解析 APP0
检查标识“JFIF”及版本
得到一些参数
解析 DQT
得到量化表长度(可能包含多张量化表)
得到量化表的精度
得到及检查量化表的序号(只能是 0-3)
得到量化表内容(64 个数据)
解析 SOF0
得到每个 sample 的比特数、长宽、颜色分量数
得到每个颜色分量的ID、水平采样因子、垂直采样因子、使用的量化表序号(与DQT中序号对应)
解析 DHT
得到 Huffman 表的类型(AC、DC)、序号
依据数据重建 Huffman 表
解析 SOS
得到解析每个颜色分量的 DC、AC 值所使用的 Huffman 表序号(与 DHT 中序号对应)
(3)依据每个分量的水平、垂直采样因子计算 MCU 的大小,并得到每个MCU 中8x8宏块的个数
对每个宏块进行 Huffman 解码,得到 DCT 系数
对每个宏块的 DCT 系数进行 IDCT,得到 Y、Cb、Cr
遇到 Segment Marker RST 时,清空之前的 DC DCT 系数
(4)对每个 MCU 解码(依照各分量水平、垂直采样因子对 MCU 中每个分量宏块解 码)
(5)解析到 EOI,解码结束
(6)将 Y、Cb、Cr 转化为需要的色彩空间并保存。
三、关键代码分析及实验任务
tinyjpeg.h
#include <stdio.h>
#ifndef __JPEGDEC_H__
#define __JPEGDEC_H__
#ifdef __cplusplus
extern "C" {
#endif
// 添加
static const unsigned char zigzag[64];
struct jdec_private;
/* Flags that can be set by any applications */
#define TINYJPEG_FLAGS_MJPEG_TABLE (1<<1)
/* Format accepted in outout */
enum tinyjpeg_fmt {
TINYJPEG_FMT_GREY = 1,
TINYJPEG_FMT_BGR24,
TINYJPEG_FMT_RGB24,
TINYJPEG_FMT_YUV420P,
};
FILE *p_trace;//add by nxn
static char error_string[256];//add by nxn
struct jdec_private *tinyjpeg_init(void);
void tinyjpeg_free(struct jdec_private *priv);
int tinyjpeg_parse_header(struct jdec_private *priv, const unsigned char *buf, unsigned int size);
int tinyjpeg_decode(struct jdec_private *priv, int pixel_format);
const char *tinyjpeg_get_errorstring(struct jdec_private *priv);
void tinyjpeg_get_size(struct jdec_private *priv, unsigned int *width, unsigned int *height);
int tinyjpeg_get_components(struct jdec_private *priv, unsigned char **components);
int tinyjpeg_set_components(struct jdec_private *priv, unsigned char **components, unsigned int ncomponents);
int tinyjpeg_set_flags(struct jdec_private *priv, int flags);
static void outdcac(struct jdec_private *priv, FILE *DC_image, FILE *AC_image);
static void getdcac(struct jdec_private *priv);
#ifdef __cplusplus
}
#endif
#endif
#define snprintf _snprintf//add by nxn
// 通过定义TRACE的值为0或1来打开和关闭TRACE
#define TRACE 1//add by nxn
#define TRACEFILE "trace_jpeg.txt"//add by nxn
loadjpeg.c
main函数:打开输入输出文件,解析输出格式
int main(int argc, char *argv[])
{
// 初始化输出文件格式为YUV420P
int output_format = TINYJPEG_FMT_YUV420P;
char *output_filename, *input_filename;
clock_t start_time, finish_time;
unsigned int duration;
int current_argument;
int benchmark_mode = 0;
// 通过在tinyjpeg.h文件中设置TRACE的值为1以及#if TRACE … #endif打开打开TRACE文件,TRACE是用来跟踪和记录解码过程,存放解码过程中得到的数据
#if TRACE
p_trace=fopen(TRACEFILE,"w");
if (p_trace==NULL)
{
printf("trace file open error!");
}
#endif
if (argc < 3)
usage();
current_argument = 1;
while (1)
{
if (strcmp(argv[current_argument], "--benchmark")==0)
benchmark_mode = 1;
else
break;
current_argument++;
}
if (argc < current_argument+2)
usage();
input_filename = argv[current_argument];
if (strcmp(argv[current_argument+1],"yuv420p")==0)
output_format = TINYJPEG_FMT_YUV420P;
else if (strcmp(argv[current_argument+1],"rgb24")==0)
output_format = TINYJPEG_FMT_RGB24;
else if (strcmp(argv[current_argument+1],"bgr24")==0)
output_format = TINYJPEG_FMT_BGR24;
else if (strcmp(argv[current_argument+1],"grey")==0)
output_format = TINYJPEG_FMT_GREY;
else
exitmessage("Bad format: need to be one of yuv420p, rgb24, bgr24, grey\n");
output_filename = argv[current_argument+2];
start_time = clock();
if (benchmark_mode)
load_multiple_times(input_filename, output_filename, output_format);
else
convert_one_image(input_filename, output_filename, output_format);
finish_time = clock();
duration = finish_time - start_time;
snprintf(error_string, sizeof(error_string),"Decoding finished in %u ticks\n", duration);
#if TRACE
fclose(p_trace);
#endif
return 0;
}
convert_one_image:加载一幅jpeg图像,解压缩后存出结果
int convert_one_image(const char *infilename, const char *outfilename, int output_format)
{
// 定义文件指针
FILE *fp;
// 保存文件大小
unsigned int length_of_file;
// 保存图像宽高
unsigned int width, height;
// 用来存储JPEG文件中的数据的缓冲区
unsigned char *buf;
struct jdec_private *jdec;
unsigned char *components[3];
fp = fopen(infilename, "rb");
if (fp == NULL)
exitmessage("Cannot open filename\n");
// 获取文件大小
length_of_file = filesize(fp);
buf = (unsigned char *)malloc(length_of_file + 4);
if (buf == NULL)
exitmessage("Not enough memory for loading file\n");
// 将文件中的数据读入缓存
fread(buf, length_of_file, 1, fp);
fclose(fp);
// 解压缩
// 首先初始化结构体变量
jdec = tinyjpeg_init();
if (jdec == NULL)
exitmessage("Not enough memory to alloc the structure need for decompressing\n");
// 解析 Segment Marker
if (tinyjpeg_parse_header(jdec, buf, length_of_file)<0)
exitmessage(tinyjpeg_get_errorstring(jdec));
/* Get the size of the image */
tinyjpeg_get_size(jdec, &width, &height);
snprintf(error_string, sizeof(error_string),"Decoding JPEG image...\n");
// 文件解码
if (tinyjpeg_decode(jdec, output_format) < 0)
exitmessage(tinyjpeg_get_errorstring(jdec));
/*
* Get address for each plane (not only max 3 planes is supported), and
* depending of the output mode, only some components will be filled
* RGB: 1 plane, YUV420P: 3 planes, GREY: 1 plane
*/
tinyjpeg_get_components(jdec, components);
/* Save it */
// 按要求输出文件
switch (output_format)
{
case TINYJPEG_FMT_RGB24:
case TINYJPEG_FMT_BGR24:
write_tga(outfilename, output_format, width, height, components);
break;
case TINYJPEG_FMT_YUV420P:
write_yuv(outfilename, width, height, components);
break;
case TINYJPEG_FMT_GREY:
write_pgm(outfilename, width, height, components);
break;
}
tinyjpeg_free(jdec);
/* else called just free(jdec); */
free(buf);
return 0;
}
任务一:将输出文件保存为可供YUVViewer观看的YUV文件
将write_yuv函数体内y、u、v三个文件单独输出改成合并输出到一个yuv文件
static void write_yuv(const char *filename, int width, int height, unsigned char **components)
{
FILE *F;
char temp[1024];
snprintf(temp, 1024, "%s.yuv", filename);
F = fopen(temp, "wb");
fwrite(components[0], width, height, F);
fwrite(components[1], width*height / 4, 1, F);
fwrite(components[2], width*height / 4, 1, F);
fclose(F);
}
tinyjpeg-internal.h
任务二:理解三个结构体的设计目的
// Huffman码表结构体
struct huffman_table
{
/* Fast look up table, using HUFFMAN_HASH_NBITS bits we can have directly the symbol,
* if the symbol is <0, then we need to look into the tree table */
//快速查找到权值对应的码字
short int lookup[HUFFMAN_HASH_SIZE];
/* code size: give the number of bits of a symbol is encoded */
//码长对应的权值
unsigned char code_size[HUFFMAN_HASH_SIZE];
/* some place to store value that is not encoded in the lookup table
* FIXME: Calculate if 256 value is enough to store all values
*/
uint16_t slowtable[16-HUFFMAN_HASH_NBITS][256];
};
// 分量结构体:代表当前正在处理的某个分量的8x8块
struct component
{
unsigned int Hfactor;//水平采样因子
unsigned int Vfactor;//垂直采样因子
float *Q_table; /* Pointer to the quantisation table to use */
struct huffman_table *AC_table;//指向该宏块交流系数的Huffman码表
struct huffman_table *DC_table;//指向该宏块直交流系数的Huffman码表
short int previous_DC; /* Previous DC coefficient *///前一个块的DC系数
short int DCT[64]; /* DCT coef *///该块的DCT系数,其中DCT[0]为该块直流,其他为交流
#if SANITY_CHECK
unsigned int cid;
#endif
};
// 文件解码信息结构体
struct jdec_private
{
/* Public variables */
//分别指向YUV分量结构体的指针数组
uint8_t *components[COMPONENTS];
// 图像的宽高
unsigned int width, height; /* Size of the image */
unsigned int flags;
/* Private variables */
// 文件流的开始和结束
const unsigned char *stream_begin, *stream_end;
// 文件流的长度
unsigned int stream_length;
// 指向那个当前文件流
const unsigned char *stream; /* Pointer to the current stream */
unsigned int reservoir, nbits_in_reservoir;
struct component component_infos[COMPONENTS];
// 对YUV进行量化的量化表
float Q_tables[COMPONENTS][64]; /* quantization tables */
// DC系数的huffman码表
struct huffman_table HTDC[HUFFMAN_TABLES]; /* DC huffman tables */
// AC系数的huffman码表
struct huffman_table HTAC[HUFFMAN_TABLES]; /* AC huffman tables */
int default_huffman_table_initialized;
int restart_interval;
int restarts_to_go; /* MCUs left in this restart interval */
int last_rst_marker_seen; /* Rst marker is incremented each time */
/* Temp space used after the IDCT to store each components */
// 临时存储反DCT之后存三个分量的数组
uint8_t Y[64*4], Cr[64], Cb[64];
jmp_buf jump_state;
/* Internal Pointer use for colorspace conversion, do not modify it !!! */
uint8_t *plane[COMPONENTS];
// 添加:用于存放DC系数DCT[0]和某个AC值DCT[1]的数组
int *dc,*ac;
unsigned char *outdc,*outac;
};
tinyjpeg.c
tinyjpeg_parse_header函数:解析 Segment Marker
int tinyjpeg_parse_header(struct jdec_private *priv, const unsigned char *buf, unsigned int size)
{
int ret;
/* Identify the file */
// JPEG 文件必须以 0xFFD8(SOI)起始
if ((buf[0] != 0xFF) || (buf[1] != SOI))
snprintf(error_string, sizeof(error_string),"Not a JPG file ?\n");
// 如果是0xFFD8,则将文件流的开始向后移两个字节
priv->stream_begin = buf+2;
// 将文件的长度也减小两个字节
priv->stream_length = size-2;
// 定位到文件的最后
priv->stream_end = priv->stream_begin + priv->stream_length;
// 调用parse_JFIF函数解析文件头
ret = parse_JFIF(priv, priv->stream_begin);
return ret;
}
static int parse_JFIF(struct jdec_private *priv, const unsigned char *stream)
{
int chuck_len;
int marker;
int sos_marker_found = 0;
int dht_marker_found = 0;
const unsigned char *next_chunck;
/* Parse marker */
// 循环一直读到扫描开始,即编码数据块
while (!sos_marker_found)
{
if (*stream++ != 0xff)
goto bogus_jpeg_format;
/* Skip any padding ff byte (this is normal) */
while (*stream == 0xff)
stream++;
marker = *stream++;
chuck_len = be16_to_cpu(stream);
next_chunck = stream + chuck_len;
switch (marker)
{
case SOF:
if (parse_SOF(priv, stream) < 0)
return -1;
break;
case DQT:
if (parse_DQT(priv, stream) < 0)
return -1;
break;
case SOS:
if (parse_SOS(priv, stream) < 0)
return -1;
sos_marker_found = 1;
break;
case DHT:
if (parse_DHT(priv, stream) < 0)
return -1;
dht_marker_found = 1;
break;
case DRI:
if (parse_DRI(priv, stream) < 0)
return -1;
break;
default:
#if TRACE
fprintf(p_trace,"> Unknown marker %2.2x\n", marker);
fflush(p_trace);
#endif
break;
}
stream = next_chunck;// 跳到下一个数据块,再判断
}
if (!dht_marker_found) {
#if TRACE
fprintf(p_trace,"No Huffman table loaded, using the default one\n");
fflush(p_trace);
#endif
build_default_huffman_tables(priv);
}
#ifdef SANITY_CHECK
if ( (priv->component_infos[cY].Hfactor < priv->component_infos[cCb].Hfactor)
|| (priv->component_infos[cY].Hfactor < priv->component_infos[cCr].Hfactor))
snprintf(error_string, sizeof(error_string),"Horizontal sampling factor for Y should be greater than horitontal sampling factor for Cb or Cr\n");
if ( (priv->component_infos[cY].Vfactor < priv->component_infos[cCb].Vfactor)
|| (priv->component_infos[cY].Vfactor < priv->component_infos[cCr].Vfactor))
snprintf(error_string, sizeof(error_string),"Vertical sampling factor for Y should be greater than vertical sampling factor for Cb or Cr\n");
if ( (priv->component_infos[cCb].Hfactor!=1)
|| (priv->component_infos[cCr].Hfactor!=1)
|| (priv->component_infos[cCb].Vfactor!=1)
|| (priv->component_infos[cCr].Vfactor!=1))
snprintf(error_string, sizeof(error_string),"Sampling other than 1x1 for Cr and Cb is not supported");
#endif
return 0;
bogus_jpeg_format:
#if TRACE
fprintf(p_trace,"Bogus jpeg format\n");
fflush(p_trace);
#endif
return -1;
}
parse_DQT函数:解析DQT,得到量化表长度(可能包含多张量化表)、量化表的精度,得到及检查量化表的序号(只能是 0-3),得到量化表内容(64 个数据)
任务三:以txt文件输出所有的量化矩阵
// 解析DQT
static int parse_DQT(struct jdec_private *priv, const unsigned char *stream)
{
int qi;
//int count = 0;
// 定义指向量化表的指针
float *table;
// 指向量化表结束的位置
const unsigned char *dqt_block_end;
#if TRACE
fprintf(p_trace,"> DQT marker\n");
fflush(p_trace);
#endif
// 得到量化表长度(可能包含多张量化表)
dqt_block_end = stream + be16_to_cpu(stream);
stream += 2; /* Skip length 跳过两个字节的存储长度*/
// 检查是否还有表
while (stream < dqt_block_end)
{
qi = *stream++;
#if SANITY_CHECK
// 得到量化表的精度(高四位),有两个可选值,0:8 位;1:16
// 如果是00,代表量化精度为8bit,即用一个字节表示一个量化系数
if (qi>>4)
snprintf(error_string, sizeof(error_string),"16 bits quantization table is not supported\n");
// 得到量化表序号(低四位),其取值范围为 0~3
// 最多可能有四张量化表(亮度量化表,色度量化表,可能会有 R、G、B 各一张量化表)
if (qi>4)
snprintf(error_string, sizeof(error_string),"No more 4 quantization table is supported (got %d)\n", qi);
#endif
table = priv->Q_tables[qi];
// 得到量化表内容
// 数据:64 字节的量化系数,(一个字节对应一个量化系数,对于 8*8 的宏块来说,共 8*8=64 个量化系数)
build_quantization_table(table, stream);
//count++;
/* 添加:以之字形扫描顺序输出量化矩阵*/
/* a:以“追加”方式打开文件
如果文件不存在,那么创建一个新文件
如果文件存在,那么将写入的数据追加到文件的末尾(文件原有的内容保留)*/
FILE *fp = fopen("table.txt", "a");
if (fp == NULL)
{
printf("Fail to open file!\n");
exit(0); //退出程序(结束程序)
}
fprintf(fp, "量化表\n");
for (int i = 0; i < 64; i++)
{
if (i != 0 && i % 8 == 0)
fprintf(fp, "\n");
fprintf(fp, "%f\t ", table[i]);
}
printf(fp, "\n");
stream += 64;
}
#if TRACE
fprintf(p_trace,"< DQT marker\n");
fflush(p_trace);
#endif
return 0;
}
static void build_quantization_table(float *qtable, const unsigned char *ref_table)
{
/* Taken from libjpeg. Copyright Independent JPEG Group's LLM idct.
* For float AA&N IDCT method, divisors are equal to quantization
* coefficients scaled by scalefactor[row]*scalefactor[col], where
* scalefactor[0] = 1
* scalefactor[k] = cos(k*PI/16) * sqrt(2) for k=1..7
* We apply a further scale factor of 8.
* What's actually stored is 1/divisor so that the inner loop can
* use a multiplication rather than a division.
*/
int i, j;
static const double aanscalefactor[8] = {
1.0, 1.387039845, 1.306562965, 1.175875602,
1.0, 0.785694958, 0.541196100, 0.275899379
};
const unsigned char *zz = zigzag;
// 以 zig-zag 序存储
for (i=0; i<8; i++) {
for (j=0; j<8; j++) {
*qtable++ = ref_table[*zz++] * aanscalefactor[i] * aanscalefactor[j];
}
}
}
// 之字形扫描顺序
static const unsigned char zigzag[64] =
{
0, 1, 5, 6, 14, 15, 27, 28,
2, 4, 7, 13, 16, 26, 29, 42,
3, 8, 12, 17, 25, 30, 41, 43,
9, 11, 18, 24, 31, 40, 44, 53,
10, 19, 23, 32, 39, 45, 52, 54,
20, 22, 33, 38, 46, 51, 55, 60,
21, 34, 37, 47, 50, 56, 59, 61,
35, 36, 48, 49, 57, 58, 62, 63
};
parse_SOF函数:解析SOF,得到每个 sample 的比特数、长宽、颜色分量数, 每个颜色分量的 ID、水平采样因子、垂直采样因子、使用的量化表序号(与 DQT 中序号对应)
static int parse_SOF(struct jdec_private *priv, const unsigned char *stream)
{
int i, width, height, nr_components, cid, sampling_factor;
int Q_table;
struct component *c;
#if TRACE
fprintf(p_trace,"> SOF marker\n");
fflush(p_trace);
#endif
print_SOF(stream);
// 得到当前图片的高度
height = be16_to_cpu(stream+3);
// 得到当前图片的宽度
width = be16_to_cpu(stream+5);
// 得到颜色分量数(JFIF中使用YCrCb,故这里颜色分量数恒为3)
nr_components = stream[7];
#if SANITY_CHECK
if (stream[2] != 8)
snprintf(error_string, sizeof(error_string),"Precision other than 8 is not supported\n");
if (width>JPEG_MAX_WIDTH || height>JPEG_MAX_HEIGHT)
snprintf(error_string, sizeof(error_string),"Width and Height (%dx%d) seems suspicious\n", width, height);
if (nr_components != 3)
snprintf(error_string, sizeof(error_string),"We only support YUV images\n");
if (height%16)
snprintf(error_string, sizeof(error_string),"Height need to be a multiple of 16 (current height is %d)\n", height);
if (width%16)
snprintf(error_string, sizeof(error_string),"Width need to be a multiple of 16 (current Width is %d)\n", width);
#endif
stream += 8;
for (i=0; i<nr_components; i++) {
// 该颜色分量的ID
cid = *stream++;
// 该分量的水平、垂直采样因子
sampling_factor = *stream++;
// 该分量使用的量化表序号
Q_table = *stream++;
// 指向该分量的结构体指针
c = &priv->component_infos[i];
#if SANITY_CHECK
c->cid = cid;
if (Q_table >= COMPONENTS)
snprintf(error_string, sizeof(error_string),"Bad Quantization table index (got %d, max allowed %d)\n", Q_table, COMPONENTS-1);
#endif
c->Vfactor = sampling_factor&0xf; // 垂直采样因子(低四位)
c->Hfactor = sampling_factor>>4; // 水平采样因子(高四位)
c->Q_table = priv->Q_tables[Q_table];
#if TRACE
fprintf(p_trace,"Component:%d factor:%dx%d Quantization table:%d\n",
cid, c->Hfactor, c->Hfactor, Q_table );
fflush(p_trace);
#endif
}
priv->width = width;
priv->height = height;
#if TRACE
fprintf(p_trace,"< SOF marker\n");
fflush(p_trace);
#endif
return 0;
}
parse_DHT函数:解析DHT,得到 Huffman 表的类型(AC、DC)、序号, 依据数据重建 Huffman 表
任务四:以txt文件输出所有的HUFFMAN码表
static int parse_DHT(struct jdec_private *priv, const unsigned char *stream)
{
unsigned int count, i;
// 定义存储码长从1至16的码字个数的数组
unsigned char huff_bits[17];
int length, index;
// 表长(减去标记代码所占字节数后的表长,可能包含多张表)
length = be16_to_cpu(stream) - 2;
stream += 2; /* Skip length */
#if TRACE
fprintf(p_trace,"> DHT marker (length=%d)\n", length);
fflush(p_trace);
#endif
// 是否还有表
while (length>0) {
index = *stream++;
/* We need to calculate the number of bytes 'vals' will takes */
huff_bits[0] = 0;
count = 0;
for (i=1; i<17; i++)
{
// 各码长的码字个数分别赋值
huff_bits[i] = *stream++;
// 总码字数
// count个字节对应的就是每个字符对应的权值
// 这些权值的含义即为DC系数经DPCM编码后幅度值的位长
count += huff_bits[i];
}
#if SANITY_CHECK
if (count >= HUFFMAN_BITS_SIZE)
snprintf(error_string, sizeof(error_string),"No more than %d bytes is allowed to describe a huffman table", HUFFMAN_BITS_SIZE);
if ( (index &0xf) >= HUFFMAN_TABLES)
snprintf(error_string, sizeof(error_string),"No more than %d Huffman tables is supported (got %d)\n", HUFFMAN_TABLES, index&0xf);
#if TRACE
fprintf(p_trace,"Huffman table %s[%d] length=%d\n", (index&0xf0)?"AC":"DC", index&0xf, count);
fflush(p_trace);
#endif
#endif
/* 添加:以txt文件输出所有的HUFFMAN码表 */
FILE *fp = fopen("huffman.txt", "a");
if (fp == NULL)
{
printf("Fail to open file!\n");
exit(0); //退出程序(结束程序)
}
/* index:Huffman表ID号和类型:1字节
高四位为表的类型,0:DC 直流;1:AC 交流
低四位为Huffman表的ID */
// 高四位为1是AC表
if (index & 0xf0)
// (index&0xf)获得Huffman表序号
{
fprintf(fp, "huffman_table AC%d号表\n", (index & 0xf));
build_huffman_table(huff_bits, stream, &priv->HTAC[index & 0xf]);
}
// 高四位为0是DC表
else
{
fprintf(fp, "huffman_table DC%d号表\n", (index & 0xf));
build_huffman_table(huff_bits, stream, &priv->HTDC[index & 0xf]);
}
length -= 1;
length -= 16;
length -= count;
stream += count;
}
#if TRACE
fprintf(p_trace,"< DHT marker\n");
fflush(p_trace);
#endif
return 0;
}
static void build_huffman_table(const unsigned char *bits, const unsigned char *vals, struct huffman_table *table)
{
unsigned int i, j, code, code_size, val, nbits;
unsigned char huffsize[HUFFMAN_BITS_SIZE+1], *hz;
unsigned int huffcode[HUFFMAN_BITS_SIZE+1], *hc;
int next_free_entry;
/*
* Build a temp array
* huffsize[X] => numbers of bits to write vals[X]
*/
hz = huffsize;
// 码长为1-16
for (i=1; i<=16; i++)
{
// 码长为1-16的码字个数
for (j=1; j<=bits[i]; j++)
// 每个码字的长度
*hz++ = i;
}
*hz = 0;
memset(table->lookup, 0xff, sizeof(table->lookup));
for (i=0; i<(16-HUFFMAN_HASH_NBITS); i++)
table->slowtable[i][0] = 0;
/* Build a temp array
* huffcode[X] => code used to write vals[X]
*/
// 第一个码字必定为0
code = 0;
// 指向码字
hc = huffcode;
// 重新指向码长
hz = huffsize;
nbits = *hz;
// 码长大于0时
while (*hz)
{
while (*hz == nbits)
{
// 码长未改变时,码字加1并且指向下一个码字
*hc++ = code++;
hz++;
}
// 如果码长比前面的码字的码长大,则加1补0
code <<= 1;
nbits++;
}
/*
* Build the lookup table, and the slowtable if needed.
*/
/* 添加:以txt文件输出所有的HUFFMAN码表 */
FILE *fp = fopen("huffman.txt", "a");
if (fp == NULL)
{
printf("Fail to open file!\n");
exit(0); //退出程序(结束程序)
}
next_free_entry = -1;
for (i=0; huffsize[i]; i++)
{
// 每个码字的权值,即解码时需要再读入的bit位数
val = vals[i];
// 码字
code = huffcode[i];
// 码长
code_size = huffsize[i];
#if TRACE
fprintf(p_trace,"val=%2.2x code=%8.8x codesize=%2.2d\n", val, code, code_size);
fflush(p_trace);
fprintf(fp, "val=%2.2x code=%8.8x codesize=%2.2d\n", val, code, code_size);
fflush(fp);
#endif
table->code_size[val] = code_size;
if (code_size <= HUFFMAN_HASH_NBITS)
{
/*
* Good: val can be put in the lookup table, so fill all value of this
* column with value val
*/
int repeat = 1UL<<(HUFFMAN_HASH_NBITS - code_size);
code <<= HUFFMAN_HASH_NBITS - code_size;
while ( repeat-- )
table->lookup[code++] = val;
}
else
{
/* Perhaps sorting the array will be an optimization */
uint16_t *slowtable = table->slowtable[code_size-HUFFMAN_HASH_NBITS-1];
while(slowtable[0])
slowtable+=2;
slowtable[0] = code;
slowtable[1] = val;
slowtable[2] = 0;
/* TODO: NEED TO CHECK FOR AN OVERFLOW OF THE TABLE */
}
}
}
parse_SOS函数:解析SOS,得到解析每个颜色分量的 DC、AC 值所使用的 Huffman 表序号(与 DHT 中序号对应)
static int parse_SOS(struct jdec_private *priv, const unsigned char *stream)
{
unsigned int i, cid, table;
// 得到颜色分量数
unsigned int nr_components = stream[2];
#if TRACE
fprintf(p_trace,"> SOS marker\n");
fflush(p_trace);
#endif
#if SANITY_CHECK
if (nr_components != 3)
snprintf(error_string, sizeof(error_string),"We only support YCbCr image\n");
#endif
stream += 3;
// 对所有分量
for (i=0;i<nr_components;i++) {
// 得到ID
cid = *stream++;
table = *stream++;
#if SANITY_CHECK
if ((table&0xf)>=4)
snprintf(error_string, sizeof(error_string),"We do not support more than 2 AC Huffman table\n");
if ((table>>4)>=4)
snprintf(error_string, sizeof(error_string),"We do not support more than 2 DC Huffman table\n");
if (cid != priv->component_infos[i].cid)
snprintf(error_string, sizeof(error_string),"SOS cid order (%d:%d) isn't compatible with the SOF marker (%d:%d)\n",
i, cid, i, priv->component_infos[i].cid);
#if TRACE
fprintf(p_trace,"ComponentId:%d tableAC:%d tableDC:%d\n", cid, table&0xf, table>>4);
fflush(p_trace);
#endif
#endif
// Huffman表序号,高四位:DC 低四位:AC
priv->component_infos[i].AC_table = &priv->HTAC[table&0xf];
priv->component_infos[i].DC_table = &priv->HTDC[table>>4];
}
// 指向熵编码数据流的开始
priv->stream = stream+3;
#if TRACE
fprintf(p_trace,"< SOS marker\n");
fflush(p_trace);
#endif
return 0;
}
Tinyjpeg_decode函数:依据每个分量的水平、垂直采样因子计算 MCU 的大小,并得到每个 MCU 中 8*8 宏块的个数
任务五:输出DC图像和某一个AC值图像并分别统计其概率分布(使用第三个实验中的Huffman编码器)
// 定义两个文件指针分别打开两个yuv文件:DC图像和AC图像
// add
FILE *DC_image = fopen("DCimage.yuv", "wb");
FILE *AC_image = fopen("ACimage.yuv", "wb");
// 为创建的指针分配空间
// 每个8x8的块只有一个直流分量
priv->dc = (int *)malloc(sizeof(int) * (priv->width * priv->height) / 64);
// 因为只输出某一AC值图像,所以AC和DC所占空间相同
priv->ac = (int *)malloc(sizeof(int) * (priv->width * priv->height) / 64);
//add
// 初始化为 4:4:4 时的情况,即MCU的宽和高均为8x8
xstride_by_mcu = ystride_by_mcu = 8;
// 如果Y分量的垂直和水平采样因子相等,则每个MCU就包括1个Y分量
if ((priv->component_infos[cY].Hfactor | priv->component_infos[cY].Vfactor) == 1) {
// 4:4:4
decode_MCU = decode_mcu_table[0];
convert_to_pixfmt = colorspace_array_conv[0];
#if TRACE
fprintf(p_trace,"Use decode 1x1 sampling\n");
fflush(p_trace);
#endif
} else if (priv->component_infos[cY].Hfactor == 1) {
//如果水平采样因子为1,垂直为2,则每个MCU包含2个Y分量
decode_MCU = decode_mcu_table[1];
convert_to_pixfmt = colorspace_array_conv[1];
// 一个 MCU 的高为 16
ystride_by_mcu = 16;
#if TRACE
fprintf(p_trace,"Use decode 1x2 sampling (not supported)\n");
fflush(p_trace);
#endif
} else if (priv->component_infos[cY].Vfactor == 2) {
// 如果水平采样因子为2,垂直为2,则每个MCU包含4个Y分量
decode_MCU = decode_mcu_table[3];
convert_to_pixfmt = colorspace_array_conv[3];
// 一个 MCU 的宽为 16
xstride_by_mcu = 16;
// 一个 MCU 的高为 16
ystride_by_mcu = 16;
#if TRACE
fprintf(p_trace,"Use decode 2x2 sampling\n");
fflush(p_trace);
#endif
} else {
// 如果水平采样因子为2,垂直为1,则每个MCU包含2个Y分量
decode_MCU = decode_mcu_table[2];
convert_to_pixfmt = colorspace_array_conv[2];
// 一个 MCU 的宽为 16
xstride_by_mcu = 16;
#if TRACE
fprintf(p_trace,"Use decode 2x1 sampling\n");
fflush(p_trace);
#endif
}
resync(priv);
/* Don't forget to that block can be either 8 or 16 lines */
bytes_per_blocklines[0] *= ystride_by_mcu;
bytes_per_blocklines[1] *= ystride_by_mcu;
bytes_per_blocklines[2] *= ystride_by_mcu;
bytes_per_mcu[0] *= xstride_by_mcu/8;
bytes_per_mcu[1] *= xstride_by_mcu/8;
bytes_per_mcu[2] *= xstride_by_mcu/8;
/* Just the decode the image by macroblock (size is 8x8, 8x16, or 16x16) */
// 行循环
for (y=0; y < priv->height/ystride_by_mcu; y++)
{
//trace("Decoding row %d\n", y);
priv->plane[0] = priv->components[0] + (y * bytes_per_blocklines[0]);
priv->plane[1] = priv->components[1] + (y * bytes_per_blocklines[1]);
priv->plane[2] = priv->components[2] + (y * bytes_per_blocklines[2]);
// 列循环
for (x=0; x < priv->width; x+=xstride_by_mcu)
{
// 解码(Huffman 解码 + IDCT)
// 每解码一个8x8的宏块,调用getdcac函数将DC(DCT[0])和AC(这里是DCT[1])写入内存
getdcac(priv);
decode_MCU(priv);
convert_to_pixfmt(priv);
priv->plane[0] += bytes_per_mcu[0];
priv->plane[1] += bytes_per_mcu[1];
priv->plane[2] += bytes_per_mcu[2];
if (priv->restarts_to_go>0)
{
priv->restarts_to_go--;
if (priv->restarts_to_go == 0)
{
priv->stream -= (priv->nbits_in_reservoir/8);
// 清空 preDC(所有颜色分量)
resync(priv);
// 查找 RST 标记
if (find_next_rst_marker(priv) < 0)
return -1;
}
}
}
}
#if TRACE
fprintf(p_trace,"Input file size: %d\n", priv->stream_length+2);
fprintf(p_trace,"Input bytes actually read: %d\n", priv->stream - priv->stream_begin + 2);
fflush(p_trace);
#endif
/* 根据DCT的能量守恒特性,反DCT之后的DC取值最大可达到8*256,
且反差分编码后数据可能为负数
因此需要对得到的DC系数进行归一化处理,使其值分布在[0,255]之间,
AC系数亦是如此*/
// 调用outdcac函数进行归一化处理并输出到文件中
outdcac(priv, DC_image, AC_image);
return 0;
}
static void getdcac(struct jdec_private *priv)
{
static int i = 0;
if (i < priv->width * priv->height / 64)
{
priv->dc[i] = priv->component_infos[0].DCT[0];
priv->ac[i] = priv->component_infos[0].DCT[1];
}
i++;
}
static void outdcac(struct jdec_private *priv,FILE *DC_image, FILE *AC_image)
{
priv->outdc = (unsigned char*)malloc(sizeof(unsigned char) * (priv->width * priv->height) / 64);
priv->outac = (unsigned char*)malloc(sizeof(unsigned char) * (priv->width * priv->height) / 64);
// 添加
int dcmax = priv->dc[0];
int acmax = priv->ac[0];
int dcmin = priv->dc[0];
int acmin = priv->ac[0];
for (int j = 0; j < priv->width * priv->height / 64; j++)
{
if (priv->dc[j] > dcmax)
dcmax = priv->dc[j];
if (priv->dc[j] < dcmin)
dcmin = priv->dc[j];
if (priv->ac[j] > acmax)
acmax = priv->ac[j];
if (priv->ac[j] < acmin)
acmin = priv->ac[j];
}
}
// 归一化处理
for (int k = 0; k < priv->width * priv->height / 64; k++)
{
priv->outdc[k] = (unsigned char)(255 * (priv->dc[k] - dcmin) / (dcmax - dcmin));
priv->outac[k] = (unsigned char)(255 * (priv->ac[k] - acmin) / (acmax - acmin));
}
fwrite(priv->outdc, 1, priv->width * priv->height / 64, DC_image);
fwrite(priv->outac, 1, priv->width * priv->height / 64, AC_image);
// 添加结束
}
对每个 MCU 解码(依照各分量水平、垂直采样因子对 MCU 中每个分量宏块解码)
1. 对每个宏块进行 Huffman 解码,得到 DCT 系数
2. 对每个宏块的 DCT 系数进行 IDCT,得到 Y、Cb、Cr
3. 遇到 Segment Marker RST 时,清空之前的 DC DCT 系数
/*
* Decode all the 3 components for 1x1
*/
// 4:4:4
static void decode_MCU_1x1_3planes(struct jdec_private *priv)
{
// Y
// 以 8x8 宏块为单位进行 Huffman 解码
process_Huffman_data_unit(priv, cY);
// 对得到的 DCT 系数进行 IDCT
IDCT(&priv->component_infos[cY], priv->Y, 8);
// Cb
process_Huffman_data_unit(priv, cCb);
IDCT(&priv->component_infos[cCb], priv->Cb, 8);
// Cr
process_Huffman_data_unit(priv, cCr);
IDCT(&priv->component_infos[cCr], priv->Cr, 8);
}
/*
* Decode a 1x1 directly in 1 color
*/
static void decode_MCU_1x1_1plane(struct jdec_private *priv)
{
// Y
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y, 8);
// Cb
process_Huffman_data_unit(priv, cCb);
IDCT(&priv->component_infos[cCb], priv->Cb, 8);
// Cr
process_Huffman_data_unit(priv, cCr);
IDCT(&priv->component_infos[cCr], priv->Cr, 8);
}
/*
* Decode a 2x1
* .-------.
* | 1 | 2 |
* `-------'
*/
// 4:2:2
static void decode_MCU_2x1_3planes(struct jdec_private *priv)
{
// Y
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y, 16);
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y+8, 16);
// Cb
process_Huffman_data_unit(priv, cCb);
IDCT(&priv->component_infos[cCb], priv->Cb, 8);
// Cr
process_Huffman_data_unit(priv, cCr);
IDCT(&priv->component_infos[cCr], priv->Cr, 8);
}
/*
* Decode a 2x1
* .-------.
* | 1 | 2 |
* `-------'
*/
static void decode_MCU_2x1_1plane(struct jdec_private *priv)
{
// Y
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y, 16);
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y+8, 16);
// Cb
process_Huffman_data_unit(priv, cCb);
// Cr
process_Huffman_data_unit(priv, cCr);
}
/*
* Decode a 2x2
* .-------.
* | 1 | 2 |
* |---+---|
* | 3 | 4 |
* `-------'
*/
// 4:2:0
static void decode_MCU_2x2_3planes(struct jdec_private *priv)
{
// Y
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y, 16);
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y+8, 16);
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y+64*2, 16);
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y+64*2+8, 16);
// Cb
process_Huffman_data_unit(priv, cCb);
IDCT(&priv->component_infos[cCb], priv->Cb, 8);
// Cr
process_Huffman_data_unit(priv, cCr);
IDCT(&priv->component_infos[cCr], priv->Cr, 8);
}
/*
* Decode a 2x2 directly in GREY format (8bits)
* .-------.
* | 1 | 2 |
* |---+---|
* | 3 | 4 |
* `-------'
*/
static void decode_MCU_2x2_1plane(struct jdec_private *priv)
{
// Y
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y, 16);
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y+8, 16);
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y+64*2, 16);
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y+64*2+8, 16);
// Cb
process_Huffman_data_unit(priv, cCb);
// Cr
process_Huffman_data_unit(priv, cCr);
}
/*
* Decode a 1x2 mcu
* .---.
* | 1 |
* |---|
* | 2 |
* `---'
*/
static void decode_MCU_1x2_3planes(struct jdec_private *priv)
{
// Y
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y, 8);
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y+64, 8);
// Cb
process_Huffman_data_unit(priv, cCb);
IDCT(&priv->component_infos[cCb], priv->Cb, 8);
// Cr
process_Huffman_data_unit(priv, cCr);
IDCT(&priv->component_infos[cCr], priv->Cr, 8);
}
/*
* Decode a 1x2 mcu
* .---.
* | 1 |
* |---|
* | 2 |
* `---'
*/
static void decode_MCU_1x2_1plane(struct jdec_private *priv)
{
// Y
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y, 8);
process_Huffman_data_unit(priv, cY);
IDCT(&priv->component_infos[cY], priv->Y+64, 8);
// Cb
process_Huffman_data_unit(priv, cCb);
// Cr
process_Huffman_data_unit(priv, cCr);
}
process_Huffman_data_unit函数:以 8x8 宏块为单位进行 Huffman 解码
static void process_Huffman_data_unit(struct jdec_private *priv, int component)
{
unsigned char j;
unsigned int huff_code;
unsigned char size_val, count_0;
struct component *c = &priv->component_infos[component];
short int DCT[64];
/* Initialize the DCT coef table */
memset(DCT, 0, sizeof(DCT));
/* DC coefficient decoding */
huff_code = get_next_huffman_code(priv, c->DC_table);
//trace("+ %x\n", huff_code);
if (huff_code) {
// 查表的 DC DCT 系数(残值)
get_nbits(priv->reservoir, priv->nbits_in_reservoir, priv->stream, huff_code, DCT[0]);
// DC系数采用差分编码,恢复原值
DCT[0] += c->previous_DC;
c->previous_DC = DCT[0];
} else {
DCT[0] = c->previous_DC;
}
/* AC coefficient decoding */
j = 1;
while (j<64)
{
huff_code = get_next_huffman_code(priv, c->AC_table);
//trace("- %x\n", huff_code);
// Amplitude 幅度
size_val = huff_code & 0xF;
// 零游程长度
count_0 = huff_code >> 4;
// 0 不是一个有效的 Amplitude 值,这里做零游程标志
if (size_val == 0)
{ /* RLE */ /* 零游程 */
if (count_0 == 0)
break; /* EOB found, go out */
else if (count_0 == 0xF)
j += 16; /* skip 16 zeros */
}
else
{
j += count_0; /* skip count_0 zeroes *//* 忽略零游程 */
if (__unlikely(j >= 64)) // 出错了
{
snprintf(error_string, sizeof(error_string), "Bad huffman data (buffer overflow)");
break;
}
// 查表得到 AC DCT 系数
get_nbits(priv->reservoir, priv->nbits_in_reservoir, priv->stream, size_val, DCT[j]);
j++;
}
}
// 以 zig-zag 序保存
for (j = 0; j < 64; j++)
c->DCT[j] = DCT[zigzag[j]];
}
解完所有 MCU,解码结束
四、实验结果及分析
-
逐步调试JPEG解码器程序。将输入的JPG文件进行解码,将输出文件保存为可供YUVViewer观看的YUV文件。
-
(详细阐述见“三、关键代码分析及实验任务”)
(1)理解程序设计的整体框架
(2)理解三个结构体的设计目的
(3)理解在视音频编解码调试中TRACE的目的和含义
①会打开和关闭TRACE
②会根据自己的要求修改TRACE:若要修改TRACE,只需修改#if
TRACE…#endif内的内容,如下图所示:
- 以txt文件输出所有的量化矩阵和所有的HUFFMAN码表
量化矩阵table.txt
Huffman码表huffman.txt
- 输出DC图像和出某一个AC值图像并统计其概率分布
五 、总结与反思
分析代码能力有待加强,对于不明白的问题要及时解决,否则当下没有表现出来慢慢会在学习过程中体现。
- 以txt文件输出所有的量化矩阵和所有的HUFFMAN码表