ffmpeg的tutorial中文版学习笔记(七)

最新推荐文章于 2024-03-16 09:37:56 发布

坤哥玩csdn

最新推荐文章于 2024-03-16 09:37:56 发布

阅读量1.6k

点赞数

分类专栏： Ffmpeg 文章标签： ffmpeg

Ffmpeg 专栏收录该内容

9 篇文章 1 订阅

订阅专栏

指导7：快进快退

处理快进快退命令

源代码:tutorial07.c

现在我们来为我们的播放器加入一些快进和快退的功能，因为如果你不能全局搜索一部电影是很让人讨厌的。同时，这将告诉你av_seek_frame函数是多么容易使用。

我们将在电影播放中使用左方向键和右方向键来表示向后和向前一小段，使用向上和向下键来表示向前和向后一大段。这里一小段是10秒，一大段是60 秒。所以我们需要设置我们的主循环来捕捉键盘事件。然而当我们捕捉到键盘事件后我们不能直接调用av_seek_frame函数。我们要主要的解码线程 decode_thread的循环中做这些。所以，我们要添加一些变量到大结构体VideoState中，用来包含新的跳转位置和一些跳转标志：

int seek_req;
int seek_flags;
int64_t seek_pos;

现在让我们在主循环中捕捉按键：

	for(;;) 
	{
		double incr, pos;
		SDL_WaitEvent(&event);
		switch(event.type) 
		{
			case SDL_KEYDOWN:
				switch(event.key.keysym.sym) 
				{
					case SDLK_LEFT:
						incr = -10.0;
						goto do_seek;
					case SDLK_RIGHT:
						incr = 10.0;
						goto do_seek;
					case SDLK_UP:
						incr = 60.0;
						goto do_seek;
					case SDLK_DOWN:
						incr = -60.0;
						goto do_seek;
do_seek:
		if(global_video_state) 
		{
			pos = get_master_clock(global_video_state);
			pos += incr;
			stream_seek(global_video_state, (int64_t)(pos * AV_TIME_BASE), incr);
		}
						break;
					default:
						break;
				}
				break;

为了检测按键，我们先查了一下是否有SDL_KEYDOWN事件。然后我们使用event.key.keysym.sym来判断哪个按键被按下。一旦我们知道了如何来跳转，我们就来计算新的时间，方法为把增加的时间值加到从函数get_master_clock中得到的时间值上。然后我们调用 stream_seek函数来设置seek_pos等变量。我们把新的时间转换成为avcodec中的内部时间戳单位。在流中调用那个时间戳将使用帧而不是用秒来计算，公式为seconds = frames * time_base(fps)（感觉错了，应该为frames/time_base）。默认的avcodec值为1,000,000fps（所以2秒的内部时间戳为2,000,000）。

时间基转换公式

timestamp(ffmpeg内部时间戳) = AV_TIME_BASE * time(秒)
time(秒) = AV_TIME_BASE_Q * timestamp(ffmpeg内部时间戳)

在后面我们来看一下为什么要把这个值进行一下转换。

这就是我们的stream_seek函数。请注意我们设置了一个标志为后退服务：

void stream_seek(VideoState *is, int64_t pos, int rel) 
{
	if(!is->seek_req) 
	{
		is->seek_pos = pos;
		is->seek_flags = rel < 0 ? AVSEEK_FLAG_BACKWARD : 0;
		is->seek_req = 1;
	}
}

现在让我们看一下如果在decode_thread中实现跳转。你会注意到我们已经在源文件中标记了一个叫做“seek stuff goes here”的部分。现在我们将把代码写在这里。跳转是围绕着av_seek_frame函数的。这个函数用到了一个格式上下文，一个流，一个时间戳和一组标记来作为它的参数。这个函数将会跳转到你所给的时间戳的位置。时间戳的单位是你传递给函数的流的时基time_base。然而，你并不是必需要传给它一个流（流可以用-1来代替）。如果你这样做了，时基time_base将会是avcodec中的内部时间戳单位，或者是1000000fps。这就是为什么我们在设置seek_pos的时候会把位置乘以AV_TIME_BASER的原因。

但是，如果给av_seek_frame函数的stream参数传递传-1,你有时会在播放某些文件的时候遇到问题（比较少见），所以我们会取文件中的第一个流并且把它传递到av_seek_frame函数。不要忘记我们也要把时间戳timestamp的单位进行转化。

		// seek stuff goes here
		if(is->seek_req) 
		{
			int stream_index= -1;
			int64_t seek_target = is->seek_pos;
			if (is->videoStream >= 0) 
				stream_index = is->videoStream;
			else if(is->audioStream >= 0) 
				stream_index = is->audioStream;
			if(stream_index>=0)
			{
				seek_target= av_rescale_q(seek_target, AV_TIME_BASE_Q, pFormatCtx->streams[stream_index]->time_base);
			}
			if(av_seek_frame(is->pFormatCtx, stream_index, seek_target, is->seek_flags) < 0) 
			{
				fprintf(stderr, "%s: error while seeking\n", is->pFormatCtx->filename);
			} 
			else 
			{

这里av_rescale_q(a,b,c)是用来把时间戳从一个时基调整到另外一个时基时候用的函数。它基本的动作是计算a*b/c，但是这个函数还是必需的，因为直接计算会有溢出的情况发生。AV_TIME_BASE_Q是AV_TIME_BASE作为分母后的版本。它们是很不相同的：AV_TIME_BASE * time_in_seconds = avcodec_timestamp而AV_TIME_BASE_Q * avcodec_timestamp = time_in_seconds（注意AV_TIME_BASE_Q实际上是一个AVRational对象，所以你必需使用avcodec中特定的q函数来处理它）。

清空我们的缓冲

我们已经正确设定了跳转位置，但是我们还没有结束。记住我们有一个堆放了很多包的队列。既然我们跳到了不同的位置，我们必需把队列中的内容清空否则电影是不会跳转的。不仅如此，avcodec也有它自己的内部缓冲，也需要每次被清空。

要实现这个，我们需要首先写一个函数来清空我们的包队列。然后我们需要一种命令声音和视频线程来清空avcodec内部缓冲的办法。我们可以在清空队列后把特定的包放入到队列中，然后当它们检测到特定的包的时候，它们就会把自己的内部缓冲清空。

让我们开始写清空函数。其实很简单的，所以我直接把代码写在下面：

static void packet_queue_flush(PacketQueue *q) 
{
	AVPacketList *pkt, *pkt1;
	SDL_LockMutex(q->mutex);
	for(pkt = q->first_pkt; pkt != NULL; pkt = pkt1) 
	{
		pkt1 = pkt->next;
		av_free_packet(&pkt->pkt);
		av_freep(&pkt);
	}
	q->last_pkt = NULL;
	q->first_pkt = NULL;
	q->nb_packets = 0;
	q->size = 0;
	SDL_UnlockMutex(q->mutex);
}

既然队列已经清空了，我们放入“清空包”。但是开始我们要定义和创建这个包：

AVPacket flush_pkt;
<pre name="code" class="cpp">...

int main( int argc, char *argv[])
{
... av_init_packet(&flush_pkt); flush_pkt. data = ( unsigned char *) "FLUSH"; ...}

现在我们把这个包放到队列中：

			} 
			else 
			{
				if(is->audioStream >= 0) 
				{
					packet_queue_flush(&is->audioq);
					packet_queue_put(&is->audioq, &flush_pkt);
				}
				if(is->videoStream >= 0) 
				{
					packet_queue_flush(&is->videoq);
					packet_queue_put(&is->videoq, &flush_pkt);
				}
			}
			is->seek_req = 0;
		}

（这些代码片段是接着前面decode_thread中的代码片段的）我们也需要修改packet_queue_put函数才不至于直接简单复制了这个包：

int packet_queue_put(PacketQueue *q, AVPacket *pkt) 
{
	AVPacketList *pkt1;
	if(pkt != &flush_pkt && av_dup_packet(pkt) < 0) 
	{
		return -1;
	}

然后在声音线程和视频线程中，我们在packet_queue_get后立即调用函数avcodec_flush_buffers：

/* next packet */
if(packet_queue_get(&is->audioq, pkt, 1) < 0) {
return -1;
}
if(pkt->data == flush_pkt.data) {
avcodec_flush_buffers(is->audio_st->codec);//清空或重置avcodec自己的内部缓冲
continue;
}

上面的代码片段与视频线程中的一样，只要把“audio”换成“video”。

就这样，让我们编译我们的播放器：

gcc ./tutorial07.c -o ./tutorial07 -lavutil -lavformat -lavcodec -lswscale -lz -lm `sdl-config --cflags --libs` 
-I /home/Jiakun/ffmpeg_build/include/ -L /home/Jiakun/ffmpeg_build/lib/ -I /usr/include/SDL/

试一下！我们几乎已经都做完了；下次我们只要做一点小的改动就好了，那就是检测ffmpeg提供的小的软件缩放采样。

源代码：见这里的github;

Windows下的源代码(在VS2010下的包含重采样的代码)：

// Ffmpeg_Tutorial7_Console.cpp : 定义控制台应用程序的入口点。

#pragma once

#include "stdafx.h"

// tutorial07.c

// A pedagogical video player that really works!

// This tutorial was written by Stephen Dranger (dranger@gmail.com).

// and a tutorial by Martin Bohme (boehme@inb.uni-luebeckREMOVETHIS.de)

// Tested on Gentoo, CVS version 5/01/07 compiled with GCC 4.1.1

// Use the Makefile to build all the samples.

// Run using

// tutorial07 myvideofile.mpg

// to play the video.

extern "C"

{

#include "libavcodec/avcodec.h"

#include "libavformat/avformat.h"

#include "libavformat/avio.h"

#include "libswscale/swscale.h"

#include "libavutil/avstring.h"

#include "libavutil/time.h"

//重采样用到了下面的头文件

#include "libswresample/swresample.h"

#include "libavutil/opt.h"

#include <SDL/SDL.h>

#include <SDL/SDL_thread.h>

}

//#ifdef __MINGW32__

#undef main /* Prevents SDL from overriding main() */

//#endif

#include <stdio.h>

#include <math.h>

#define SDL_AUDIO_BUFFER_SIZE 1024

#define MAX_AUDIO_FRAME_SIZE 192000

#define MAX_AUDIOQ_SIZE (5 * 16 * 1024)

#define MAX_VIDEOQ_SIZE (5 * 256 * 1024)

#define AV_SYNC_THRESHOLD 0.01

#define AV_NOSYNC_THRESHOLD 10.0

#define SAMPLE_CORRECTION_PERCENT_MAX 10

#define AUDIO_DIFF_AVG_NB 20

#define FF_ALLOC_EVENT (SDL_USEREVENT)

#define FF_REFRESH_EVENT (SDL_USEREVENT + 1)

#define FF_QUIT_EVENT (SDL_USEREVENT + 2)

#define VIDEO_PICTURE_QUEUE_SIZE 1

#define DEFAULT_AV_SYNC_TYPE AV_SYNC_VIDEO_MASTER

//#define DEFAULT_AV_SYNC_TYPE AV_SYNC_EXTERNAL_MASTER

typedef struct PacketQueue {

AVPacketList *first_pkt , *last_pkt;

int nb_packets ;

int size ;

SDL_mutex *mutex ;

SDL_cond *cond ;

} PacketQueue;

typedef struct VideoPicture {

SDL_Overlay *bmp ;

int width , height; /* source height & width */

int allocated ;

double pts ;

} VideoPicture;

typedef struct VideoState

{

AVFormatContext *pFormatCtx ;

int videoStream , audioStream;

int av_sync_type ;

double external_clock ; /* external clock base */

int64_t external_clock_time ;

double audio_clock ;

AVStream *audio_st ;

PacketQueue audioq ;

AVFrame audio_frame ;

uint8_t audio_buf [(MAX_AUDIO_FRAME_SIZE * 3) / 2];

unsigned int audio_buf_size;

unsigned int audio_buf_index;

AVPacket audio_pkt ;

uint8_t *audio_pkt_data ;

int audio_pkt_size ;

int audio_hw_buf_size ;

double audio_diff_cum ; /* used for AV difference average computation */

double audio_diff_avg_coef ;

double audio_diff_threshold ;

int audio_diff_avg_count ;

double frame_timer ;

double frame_last_pts ;

double frame_last_delay ;

double video_clock ; ///<pts of last decoded frame / predicted pts of next decoded frame

double video_current_pts ; ///<current displayed pts (different from video_clock if frame fifos are used)

int64_t video_current_pts_time ; ///<time (av_gettime) at which we updated video_current_pts - used to have running video pts

AVStream *video_st ;

PacketQueue videoq ;

VideoPicture pictq [VIDEO_PICTURE_QUEUE_SIZE];

int pictq_size , pictq_rindex, pictq_windex;

SDL_mutex *pictq_mutex ;

SDL_cond *pictq_cond ;

SDL_Thread *parse_tid ;

SDL_Thread *video_tid ;

char filename [1024];

int quit ;

AVIOContext *io_context ;

struct SwsContext *sws_ctx;

//下面三个为在第七个快进快退教程中添加的变量。包含新的跳转位置和一些跳转标志

int seek_req ;

int seek_flags ;

int64_t seek_pos ;

} VideoState;

enum {

AV_SYNC_AUDIO_MASTER,

AV_SYNC_VIDEO_MASTER,

AV_SYNC_EXTERNAL_MASTER,

};

SDL_Surface *screen;

AVPacket flush_pkt ;

/* Since we only have one decoding thread, the Big Struct

can be global in case we need it. */

VideoState *global_video_state ;

重采样函数

int AudioResampling (AVCodecContext * audio_dec_ctx,AVFrame * pAudioDecodeFrame,

int out_sample_fmt ,int out_channels ,int out_sample_rate , uint8_t * out_buf );

static inline double rint(double x)

{

return x >= 0 ? floor( x + 0.5) : ceil(x - 0.5);

}

void packet_queue_init (PacketQueue * q)

{

memset(q , 0, sizeof( PacketQueue));

q->mutex = SDL_CreateMutex();

q->cond = SDL_CreateCond();

}

int packet_queue_put (PacketQueue * q, AVPacket *pkt)

{

AVPacketList *pktl ;

if(pkt !=&flush_pkt && av_dup_packet(pkt ) < 0)

{

return -1;

}

pktl = (AVPacketList *)av_malloc(sizeof( AVPacketList));

if (!pktl )

return -1;

pktl->pkt = *pkt;

pktl->next = NULL;

SDL_LockMutex(q ->mutex);

if (!q ->last_pkt)

q->first_pkt = pktl;

else

q->last_pkt ->next = pktl;

q->last_pkt = pktl;

q->nb_packets ++;

q->size += pktl-> pkt.size ;

SDL_CondSignal(q ->cond);

SDL_UnlockMutex(q ->mutex);

return 0;

}

/**

*从队列中获取包

static int packet_queue_get( PacketQueue *q , AVPacket * pkt, int block)

{

AVPacketList *pkt1 ;

int ret ;

SDL_LockMutex(q ->mutex);

for(;;) {

if(global_video_state ->quit) {

ret = -1;

break;

}

pkt1 = q ->first_pkt;

if (pkt1 ) {

q->first_pkt = pkt1-> next;

if (!q ->first_pkt)

q->last_pkt = NULL;

q->nb_packets --;

q->size -= pkt1-> pkt.size ;

* pkt = pkt1 ->pkt;

av_free(pkt1 );

ret = 1;

break;

} else if (!block) {

ret = 0;

break;

} else {

SDL_CondWait(q ->cond, q->mutex );

}

SDL_UnlockMutex(q ->mutex);

return ret ;

}

double get_audio_clock (VideoState * is)

{

double pts ;

int hw_buf_size , bytes_per_sec, n;

pts = is ->audio_clock; /* maintained in the audio thread */

hw_buf_size = is ->audio_buf_size - is->audio_buf_index ;

bytes_per_sec = 0;

n = is ->audio_st-> codec->channels * 2;

if(is ->audio_st)

{

bytes_per_sec = is ->audio_st-> codec->sample_rate * n;

}

if(bytes_per_sec )

{

pts -= (double )hw_buf_size / bytes_per_sec;

}

return pts ;

}

double get_video_clock (VideoState * is)

{

double delta ;

delta = (av_gettime () - is-> video_current_pts_time) / 1000000.0;

return is ->video_current_pts + delta;

}

double get_external_clock (VideoState * is)

{

return av_gettime () / 1000000.0;

}

double get_master_clock (VideoState * is)

{

if(is ->av_sync_type == AV_SYNC_VIDEO_MASTER)

{

return get_video_clock (is);

}

else if (is-> av_sync_type == AV_SYNC_AUDIO_MASTER )

{

return get_audio_clock (is);

} else

{

return get_external_clock (is);

}

/* Add or subtract samples to get a better sync, return new audio buffer size */

int synchronize_audio (VideoState * is, short *samples, int samples_size , double pts)

{

int n ;

double ref_clock ;

n = 2 * is ->audio_st-> codec->channels ;

if(is ->av_sync_type != AV_SYNC_AUDIO_MASTER)

{

double diff , avg_diff;

int wanted_size , min_size, max_size /*, nb_samples */ ;

ref_clock = get_master_clock (is);

diff = get_audio_clock (is) - ref_clock;

if(diff < AV_NOSYNC_THRESHOLD) //声音时钟和视频时钟的差异在我们的阀值范围内，

{

// accumulate the diffs

is->audio_diff_cum = diff + is->audio_diff_avg_coef * is-> audio_diff_cum;//用公式diff_sum=new_diff+diff_sum*c来计算差异,

if(is ->audio_diff_avg_count < AUDIO_DIFF_AVG_NB)

{

is->audio_diff_avg_count ++;

}

else

{

avg_diff = is ->audio_diff_cum * (1.0 - is->audio_diff_avg_coef ); //当准备好去找平均差异的时候，我们用简单的计算方式：avg_diff=diff_sum*(1-c)来平均差异；

if(fabs (avg_diff) >= is->audio_diff_threshold ) //音频现在时钟大于视频现在时钟

{

wanted_size = samples_size + ((int)( diff * is->audio_st ->codec-> sample_rate) * n );//记住audio_length*(sample_rate）*channels*2就是audio_length秒时间的样本数。

min_size = samples_size * ((100 - SAMPLE_CORRECTION_PERCENT_MAX ) / 100);

max_size = samples_size * ((100 + SAMPLE_CORRECTION_PERCENT_MAX ) / 100);

if(wanted_size < min_size)

{

wanted_size = min_size ;

}

else if (wanted_size > max_size)

{

wanted_size = max_size ;

}

if(wanted_size < samples_size)

{

/* remove samples */

samples_size = wanted_size ;

}

else if (wanted_size > samples_size)

{

uint8_t *samples_end , *q;

int nb ;

/* add samples by copying final sample*/

nb = (wanted_size -samples_size); //源代码中写反了，被我改过来了。

samples_end = (uint8_t *)samples + samples_size - n ;

q = samples_end + n;

while(nb > 0)

{

memcpy(q , samples_end, n);

q += n ;

nb -= n ;

}

samples_size = wanted_size ;

}

else //声音时钟和视频时钟的差异大于我们的阀值。失去同步

{

/* difference is TOO big; reset diff stuff */

is->audio_diff_avg_count = 0;

is->audio_diff_cum = 0;

}

return samples_size ;

}

/**

*把音频包解码成帧

int audio_decode_frame (VideoState * is, double *pts_ptr)

{

int len1 , data_size = 0, n;

AVPacket *pkt = &is-> audio_pkt;

double pts ;

AVCodecContext *pCodecCtx =is-> audio_st->codec ;

///

for(;;)

{

while(is ->audio_pkt_size > 0)

{

int got_frame = 0;

len1 = avcodec_decode_audio4 (is-> audio_st->codec , &is-> audio_frame, &got_frame, pkt );

if(len1 < 0)

{

/* if error, skip frame */

is->audio_pkt_size = 0;

break;

}

if (got_frame )

{

data_size=AudioResampling (pCodecCtx,&( is->audio_frame ),AV_SAMPLE_FMT_S16,2,44100, is->audio_buf );

//原来ffmpeg tutorial6代码

//data_size =

// av_samples_get_buffer_size

// (

// NULL,

// is->audio_st->codec->channels,

// is->audio_frame.nb_samples,

// is->audio_st->codec->sample_fmt,

// 1

// );

//memcpy(is->audio_buf, is->audio_frame.data[0], data_size);

}

is->audio_pkt_data += len1;

is->audio_pkt_size -= len1;

if(data_size <= 0)

{

/* No data yet, get more frames */

continue;

}

pts = is ->audio_clock;

* pts_ptr = pts ;

n = 2 * is ->audio_st-> codec->channels ;

is->audio_clock += (double) data_size /(double )(n * is->audio_st ->codec-> sample_rate);

/* We have data, return it and come back for more later */

return data_size ;

}

if(pkt ->data)

av_free_packet(pkt );

if(is ->quit)

{

return -1;

}

/* next packet */

if(packet_queue_get (&is-> audioq, pkt , 1) < 0)

{

return -1;

}

if (pkt ->data== flush_pkt.data )

{

avcodec_flush_buffers(is ->audio_st-> codec);

continue;

}

is->audio_pkt_data = pkt-> data;

is->audio_pkt_size = pkt-> size;

/* if update, update the audio clock w/pts */

if(pkt ->pts != AV_NOPTS_VALUE)

{

is->audio_clock = av_q2d( is->audio_st ->time_base)* pkt->pts ;

}

void audio_callback (void * userdata, Uint8 *stream, int len )

{

VideoState *is = (VideoState *) userdata;

int len1 , audio_size;

double pts ;

while(len > 0)

{

if(is ->audio_buf_index >= is->audio_buf_size )

{

/* We have already sent all our data; get more */

audio_size = audio_decode_frame (is, & pts);

if(audio_size < 0)

{

/* If error, output silence */

is->audio_buf_size = 1024;

memset(is ->audio_buf, 0, is->audio_buf_size );

}

else

{

audio_size = synchronize_audio (is, ( int16_t *)is->audio_buf ,audio_size, pts);

is->audio_buf_size = audio_size;

}

is->audio_buf_index = 0;

}

len1 = is ->audio_buf_size - is->audio_buf_index ;

if(len1 > len)

len1 = len ;

memcpy(stream , (uint8_t *) is->audio_buf + is-> audio_buf_index, len1 );

len -= len1 ;

stream += len1 ;

is->audio_buf_index += len1;

}

static Uint32 sdl_refresh_timer_cb( Uint32 interval , void * opaque)

{

SDL_Event event ;

event.type = FF_REFRESH_EVENT;

event.user .data1 = opaque;

SDL_PushEvent(&event );

return 0; /* 0 means stop timer */

}

/* schedule a video refresh in 'delay' ms */

static void schedule_refresh( VideoState *is , int delay)

{

SDL_AddTimer(delay , sdl_refresh_timer_cb, is);

}

void video_display (VideoState * is)

{

SDL_Rect rect ;

VideoPicture *vp ;

//AVPicture pict;

float aspect_ratio ;

int w , h, x, y ;

//int i;

vp = &is ->pictq[ is->pictq_rindex ];

if(vp ->bmp)

{

if(is ->video_st-> codec->sample_aspect_ratio .num == 0)

{

aspect_ratio = 0;

} else

{

aspect_ratio = av_q2d (is-> video_st->codec ->sample_aspect_ratio) *is->video_st ->codec-> width / is ->video_st-> codec->height ;

}

if(aspect_ratio <= 0.0)

{

aspect_ratio = (float )is-> video_st->codec ->width /(float)is ->video_st-> codec->height ;

}

h = screen ->h;

w = ((int )rint( h * aspect_ratio )) & -3;

if(w > screen-> w)

{

w = screen ->w;

h = ((int )rint( w / aspect_ratio )) & -3;

}

x = (screen ->w - w) / 2;

y = (screen ->h - h) / 2;

rect.x = x;

rect.y = y;

rect.w = w;

rect.h = h;

SDL_DisplayYUVOverlay(vp ->bmp, & rect);

}

void video_refresh_timer (void * userdata) {

VideoState *is = (VideoState *) userdata;

VideoPicture *vp ;

double actual_delay , delay, sync_threshold, ref_clock , diff;

if(is ->video_st)

{

if(is ->pictq_size == 0)

{

schedule_refresh(is , 1);

}

else

{

vp = &is ->pictq[ is->pictq_rindex ];

is->video_current_pts = vp-> pts;

is->video_current_pts_time = av_gettime();

delay = vp ->pts - is->frame_last_pts ; /* the pts from last time */

if(delay <= 0 || delay >= 1.0)

{

/* if incorrect delay, use previous one */

delay = is ->frame_last_delay;

}

/* save for next time */

is->frame_last_delay = delay;

is->frame_last_pts = vp-> pts;

/* update delay to sync to audio if not master source */

if(is ->av_sync_type != AV_SYNC_VIDEO_MASTER)

{

ref_clock = get_master_clock (is);

diff = vp ->pts - ref_clock;

/* Skip or repeat the frame. Take delay into account

FFPlay still doesn't "know if this is the best guess." */

sync_threshold = (delay > AV_SYNC_THRESHOLD) ? delay : AV_SYNC_THRESHOLD;

if(fabs (diff) < AV_NOSYNC_THRESHOLD)

{

if(diff <= -sync_threshold) //vp->pts小于ref_clock,即当前播放视频帧慢。

{

delay = 0;

}

else if (diff >= sync_threshold)

{

delay = 2 * delay ;

}

is->frame_timer += delay;

/* computer the REAL delay */

actual_delay = is ->frame_timer - ( av_gettime() / 1000000.0);

if(actual_delay < 0.010)

{

/* Really it should skip the picture instead */

actual_delay = 0.010;

}

schedule_refresh(is , (int)( actual_delay * 1000 + 0.5));

/* show the picture! */

video_display(is );

/* update queue for next picture! */

if(++is ->pictq_rindex == VIDEO_PICTURE_QUEUE_SIZE)

{

is->pictq_rindex = 0;

}

SDL_LockMutex(is ->pictq_mutex);

is->pictq_size --;

SDL_CondSignal(is ->pictq_cond);

SDL_UnlockMutex(is ->pictq_mutex);

}

else

{

schedule_refresh(is , 100);

}

void alloc_picture (void * userdata)

{

VideoState *is = (VideoState *) userdata;

VideoPicture *vp ;

vp = &is ->pictq[ is->pictq_windex ];

if(vp ->bmp) {

// we already have one make another, bigger/smaller

SDL_FreeYUVOverlay(vp ->bmp);

}

// Allocate a place to put our YUV image on that screen

vp->bmp = SDL_CreateYUVOverlay( is->video_st ->codec-> width,

is->video_st ->codec-> height,

SDL_YV12_OVERLAY,

screen);

vp->width = is-> video_st->codec ->width;

vp->height = is-> video_st->codec ->height;

SDL_LockMutex(is ->pictq_mutex);

vp->allocated = 1;

SDL_CondSignal(is ->pictq_cond);

SDL_UnlockMutex(is ->pictq_mutex);

}

int queue_picture (VideoState * is, AVFrame *pFrame, double pts )

{

VideoPicture *vp ;

AVPicture pict ;

/* wait until we have space for a new pic */

SDL_LockMutex(is ->pictq_mutex);

while(is ->pictq_size >= VIDEO_PICTURE_QUEUE_SIZE &&!is ->quit)

{

SDL_CondWait(is ->pictq_cond, is->pictq_mutex );

}

SDL_UnlockMutex(is ->pictq_mutex);

if(is ->quit)

return -1;

// windex is set to 0 initially

vp = &is ->pictq[ is->pictq_windex ];

/* allocate or resize the buffer! */

if(!vp ->bmp || vp->width != is-> video_st->codec ->width || vp->height != is->video_st ->codec-> height)

{

SDL_Event event ;

vp->allocated = 0;

/* we have to do it in the main thread */

event.type = FF_ALLOC_EVENT;

event.user .data1 = is;

SDL_PushEvent(&event );

/* wait until we have a picture allocated */

SDL_LockMutex(is ->pictq_mutex);

while(!vp ->allocated && ! is->quit )

{

SDL_CondWait(is ->pictq_cond, is->pictq_mutex );

}

SDL_UnlockMutex(is ->pictq_mutex);

if(is ->quit)

{

return -1;

}

/* We have a place to put our picture on the queue */

/* If we are skipping a frame, do we set this to null

but still return vp->allocated = 1? */

if(vp ->bmp)

{

SDL_LockYUVOverlay(vp ->bmp);

/* point pict at the queue */

pict.data [0] = vp-> bmp->pixels[0];

pict.data [1] = vp-> bmp->pixels[2];

pict.data [2] = vp-> bmp->pixels[1];

pict.linesize [0] = vp-> bmp->pitches[0];

pict.linesize [1] = vp-> bmp->pitches[2];

pict.linesize [2] = vp-> bmp->pitches[1];

// Convert the image into YUV format that SDL uses

sws_scale

(

is->sws_ctx ,

( uint8_t const * const *) pFrame->data ,

pFrame->linesize ,

is->video_st ->codec-> height,

pict.data ,

pict.linesize

);

SDL_UnlockYUVOverlay(vp ->bmp);

vp->pts = pts;

/* now we inform our display thread that we have a pic ready */

if(++is ->pictq_windex == VIDEO_PICTURE_QUEUE_SIZE)

{

is->pictq_windex = 0;

}

SDL_LockMutex(is ->pictq_mutex);

is->pictq_size ++;

SDL_UnlockMutex(is ->pictq_mutex);

}

return 0;

}

double synchronize_video (VideoState * is, AVFrame *src_frame, double pts )

{

double frame_delay ;

if(pts != 0)

{

/* if we have pts, set video clock to it */

is->video_clock = pts;

}

else

{

/* if we aren't given a pts, set it to the clock */

pts = is ->video_clock;

}

/* update the video clock */

frame_delay = av_q2d (is-> video_st->codec ->time_base);

/* if we are repeating a frame, adjust clock accordingly */

frame_delay += src_frame ->repeat_pict * ( frame_delay * 0.5);

is->video_clock += frame_delay;

return pts ;

}

uint64_t global_video_pkt_pts = AV_NOPTS_VALUE;

/* These are called whenever we allocate a frame

* buffer. We use this to store the global_pts in

* a frame at the time it is allocated.

int our_get_buffer (struct AVCodecContext *c , AVFrame * pic) {

int ret = avcodec_default_get_buffer( c, pic );

uint64_t *pts = (uint64_t *)av_malloc( sizeof(uint64_t ));

* pts = global_video_pkt_pts ;

pic->opaque = pts;

return ret ;

}

void our_release_buffer (struct AVCodecContext *c , AVFrame * pic) {

if(pic ) av_freep(& pic->opaque );

avcodec_default_release_buffer(c , pic);

}

//清空函数，清空包队列

static void packet_queue_flush( PacketQueue *q )

{

AVPacketList *pkt ,*pktl;

SDL_LockMutex(q ->mutex);

for (pkt =q-> first_pkt;pkt !=NULL; pkt=pktl )

{

pktl=pkt ->next;

av_free_packet(&pkt ->pkt);

av_freep(&pkt );

}

q->last_pkt =NULL;

q->first_pkt =NULL;

q->nb_packets =0;

q->size =0;

SDL_UnlockMutex(q ->mutex);

}

int video_thread (void * arg)

{

VideoState *is = (VideoState *) arg;

AVPacket pkt1 , *packet = & pkt1;

int frameFinished ;

AVFrame *pFrame ;

double pts ;

pFrame = av_frame_alloc ();

for(;;)

{

if(packet_queue_get (&is-> videoq, packet , 1) < 0)

{

// means we quit getting packets

break;

}

if (packet ->data== flush_pkt.data )

{

avcodec_flush_buffers(is ->video_st-> codec);

continue;

}

pts = 0;

// Save global pts to be stored in pFrame in first call

global_video_pkt_pts = packet ->pts;

// Decode video frame

avcodec_decode_video2(is ->video_st-> codec, pFrame , &frameFinished, packet);

if(packet ->dts == AV_NOPTS_VALUE&& pFrame ->opaque && *(uint64_t*)pFrame ->opaque != AV_NOPTS_VALUE)

{

pts = *(uint64_t *)pFrame-> opaque;

}

else if (packet-> dts != AV_NOPTS_VALUE )

{

pts = packet ->dts;

}

else

{

pts = 0;

}

pts *= av_q2d (is-> video_st->time_base);

// Did we get a video frame?

if(frameFinished )

{

pts = synchronize_video (is, pFrame, pts );

if(queue_picture (is, pFrame, pts ) < 0)

{

break;

}

av_free_packet(packet );

}

av_free(pFrame );

return 0;

}

int stream_component_open (VideoState * is, int stream_index)

{

AVFormatContext *pFormatCtx = is-> pFormatCtx;

AVCodecContext *codecCtx = NULL;

AVCodec *codec = NULL;

AVDictionary *optionsDict = NULL;

SDL_AudioSpec wanted_spec , spec;

if(stream_index < 0 || stream_index >= pFormatCtx->nb_streams )

{

return -1;

}

// Get a pointer to the codec context for the video stream

codecCtx = pFormatCtx ->streams[ stream_index]->codec ;

if(codecCtx ->codec_type == AVMEDIA_TYPE_AUDIO)

{

// Set audio settings from codec info

wanted_spec.freq = codecCtx-> sample_rate;

wanted_spec.format = AUDIO_S16SYS;

wanted_spec.channels = codecCtx-> channels;

wanted_spec.silence = 0;

wanted_spec.samples = SDL_AUDIO_BUFFER_SIZE;

wanted_spec.callback = audio_callback;

wanted_spec.userdata = is;

if(SDL_OpenAudio (&wanted_spec, & spec) < 0)

{

fprintf(stderr , "SDL_OpenAudio: %s\n", SDL_GetError());

return -1;

}

is->audio_hw_buf_size = spec. size;

}

codec = avcodec_find_decoder (codecCtx-> codec_id);

if(!codec || (avcodec_open2( codecCtx, codec , &optionsDict) < 0))

{

fprintf(stderr , "Unsupported codec!\n");

return -1;

}

switch(codecCtx ->codec_type)

{

case AVMEDIA_TYPE_AUDIO :

is->audioStream = stream_index;

is->audio_st = pFormatCtx-> streams[stream_index ];

is->audio_buf_size = 0;

is->audio_buf_index = 0;

/* averaging filter for audio sync */

is->audio_diff_avg_coef = exp( log(0.01 / AUDIO_DIFF_AVG_NB ));

is->audio_diff_avg_count = 0;

/* Correct audio only if larger error than this */

is->audio_diff_threshold = 2.0 * SDL_AUDIO_BUFFER_SIZE / codecCtx->sample_rate ;

memset(&is ->audio_pkt, 0, sizeof(is ->audio_pkt));

packet_queue_init(&is ->audioq);

SDL_PauseAudio(0);

break;

case AVMEDIA_TYPE_VIDEO :

is->videoStream = stream_index;

is->video_st = pFormatCtx-> streams[stream_index ];

is->frame_timer = (double) av_gettime() / 1000000.0;//1微秒等于百万分之一秒

is->frame_last_delay = 40e-3;

is->video_current_pts_time = av_gettime();

packet_queue_init(&is ->videoq);

is->video_tid = SDL_CreateThread( video_thread, is );

is->sws_ctx =

sws_getContext

(

is->video_st ->codec-> width,

is->video_st ->codec-> height,

is->video_st ->codec-> pix_fmt,

is->video_st ->codec-> width,

is->video_st ->codec-> height,

PIX_FMT_YUV420P,

SWS_BILINEAR,

NULL,

NULL

);

codecCtx->get_buffer = our_get_buffer; //此处代码中原为codecCtx->get_buffer2 = our_get_buffer;

codecCtx->release_buffer = our_release_buffer;

break;

default:

break;

}

return 0;

}

int decode_interrupt_cb (void * opaque)

{

return (global_video_state && global_video_state-> quit);

}

int decode_thread (void * arg)

{

VideoState *is = (VideoState *) arg;

AVFormatContext *pFormatCtx = NULL;

AVPacket pkt1 , *packet = & pkt1;

AVDictionary *io_dict = NULL;

AVIOInterruptCB callback ;

int video_index = -1;

int audio_index = -1;

int i ;

is->videoStream =-1;

is->audioStream =-1;

global_video_state = is ;

// will interrupt blocking functions if we quit!

callback.callback = decode_interrupt_cb; //自定义IO层的中断函数.用来防止阻塞。每次从文件中读到一个包后都会调用一次。

callback.opaque = is;

if (avio_open2 (&is-> io_context, is ->filename, 0, & callback, &io_dict ))

{

fprintf(stderr , "Unable to open I/O for %s\n", is ->filename);

return -1;

}

// Open video file

if(avformat_open_input (&pFormatCtx, is->filename , NULL, NULL)!=0)

return -1; // Couldn't open file

is->pFormatCtx = pFormatCtx;

// Retrieve stream information

if(avformat_find_stream_info (pFormatCtx, NULL)<0)

return -1; // Couldn't find stream information

// Dump information about file onto standard error

av_dump_format(pFormatCtx , 0, is-> filename, 0);

// Find the first video stream

for(i =0; i< pFormatCtx->nb_streams ; i++)

{

if(pFormatCtx ->streams[ i]->codec ->codec_type== AVMEDIA_TYPE_VIDEO &&video_index < 0)

{

video_index=i ;

}

if(pFormatCtx ->streams[ i]->codec ->codec_type== AVMEDIA_TYPE_AUDIO &&audio_index < 0)

{

audio_index=i ;

}

if(audio_index >= 0)

{

stream_component_open(is , audio_index);

}

if(video_index >= 0)

{

stream_component_open(is , video_index);

}

if(is ->videoStream < 0 || is->audioStream < 0)

{

fprintf(stderr , "%s: could not open codecs\n", is ->filename);

goto fail ;

}

// main decode loop

for(;;)

{

if(is ->quit)

{

break;

}

//快进快退

if (is ->seek_req)

{

int stream_index = -1;

int64_t seek_target =is-> seek_pos;

if(is ->videoStream>=0)

stream_index=is ->videoStream;

else if (is-> audioStream>=0)

stream_index=is ->audioStream;

if (stream_index >=0)

{

AVRational avr ={1, AV_TIME_BASE};

seek_target=av_rescale_q (seek_target, avr,pFormatCtx ->streams[ stream_index]->time_base);//a*b/c:把时间戳从一个时基调整到另外一个时基的时候用的函数。

}

if (av_seek_frame (is-> pFormatCtx,stream_index ,seek_target, is->seek_flags )<0)

{

fprintf(stderr ,"%s:error while seeking\n",is ->pFormatCtx-> filename);

}

else

{

if(is ->audioStream>=0)

{

packet_queue_flush(&is ->audioq);

packet_queue_put(&is ->audioq,& flush_pkt);

}

if (is ->videoStream>=0)

{

packet_queue_flush(&is ->videoq);

packet_queue_put(&is ->videoq,& flush_pkt);

}

is->seek_req =0;

}

if(is ->audioq. size > MAX_AUDIOQ_SIZE ||is-> videoq.size > MAX_VIDEOQ_SIZE)

{

SDL_Delay(10);

continue;

}

if(av_read_frame (is-> pFormatCtx, packet ) < 0)

{

if(is ->pFormatCtx-> pb->error == 0)

{

SDL_Delay(100); /* no error; wait for user input */

continue;

}

else

{

break;

}

// Is this a packet from the video stream?

if(packet ->stream_index == is->videoStream )

{

packet_queue_put(&is ->videoq, packet);

}

else if (packet-> stream_index == is ->audioStream)

{

packet_queue_put(&is ->audioq, packet);

}

else

{

av_free_packet(packet );

}

/* all done - wait for it */

while(!is ->quit)

{

SDL_Delay(100);

}

fail:

{

SDL_Event event ;

event.type = FF_QUIT_EVENT;

event.user .data1 = is;

SDL_PushEvent(&event );

}

return 0;

}

//快退快退的定位，rel区别出事快退还是快进

void stream_seek (VideoState * is,int64_t pos, int rel )

{

if (!is ->seek_req)

{

is->seek_pos =pos;

is->seek_flags =rel<0? AVSEEK_FLAG_BACKWARD:0;//AVSEEK_FLAG_BACKWARD表示快退

is->seek_req =1;

}

//程序开始执行的地方

int main (int argc, char *argv[])

{

SDL_Event event ;

VideoState *is ;

is = (VideoState *)av_mallocz(sizeof( VideoState));

av_init_packet(&flush_pkt );

flush_pkt.data = (unsigned char *)"FLUSH" ;

if(argc < 2)

{

fprintf(stderr , "Usage: test <file>\n");

exit(1);

}

// Register all formats and codecs

av_register_all();

avformat_network_init();

if(SDL_Init (SDL_INIT_VIDEO | SDL_INIT_AUDIO | SDL_INIT_TIMER ))

{

fprintf(stderr , "Could not initialize SDL - %s\n", SDL_GetError ());

exit(1);

}

// Make a screen to put our video

#ifndef __DARWIN__

screen = SDL_SetVideoMode (640, 480, 0, 0);

#else

screen = SDL_SetVideoMode (640, 480, 24, 0);

#endif

if(!screen )

{

fprintf(stderr , "SDL: could not set video mode - exiting\n");

exit(1);

}

av_strlcpy(is ->filename, argv[1], 1024);

is->pictq_mutex = SDL_CreateMutex();

is->pictq_cond = SDL_CreateCond();

schedule_refresh(is , 40);

is->av_sync_type = DEFAULT_AV_SYNC_TYPE;

is->parse_tid = SDL_CreateThread( decode_thread, is );

if(!is ->parse_tid)

{

av_free(is );

return -1;

}

for(;;)

{

double incr ,pos;

SDL_WaitEvent(&event );//和SDL_PollEvent(SDL_Event *event)相比，wait是停下来等，poll是继续做其他事情的同时等。

switch(event .type)

{

case SDL_KEYDOWN :

{

switch (event .key. keysym.sym )

{

case SDLK_UP :

incr=60.0;

goto do_seek ;

case SDLK_DOWN :

incr=-60.0;

goto do_seek ;

case SDLK_LEFT :

incr=-10.0;

goto do_seek ;

case SDLK_RIGHT :

incr=10.0;

goto do_seek ;

do_seek:

if (global_video_state )

{

pos=get_master_clock (global_video_state);

pos+=incr ;

//设置seek_pos等变量。把新的时间转换成为avcodec中的内部时间戳单位。

//在流中调用那个时间戳将使用帧而不是用秒来计算。

stream_seek(global_video_state ,(int64_t)( pos*AV_TIME_BASE ),incr);

}

break;

default:

break;

}

break;

case FF_QUIT_EVENT :

case SDL_QUIT :

is->quit = 1;

* If the video has finished playing, then both the picture and

* audio queues are waiting for more data. Make them stop

* waiting and terminate normally.

SDL_CondSignal(is ->audioq. cond);

SDL_CondSignal(is ->videoq. cond);

SDL_Quit();

exit(0);

break;

case FF_ALLOC_EVENT :

alloc_picture(event .user. data1);

break;

case FF_REFRESH_EVENT :

video_refresh_timer(event .user. data1);

break;

default:

break;

}

return 0;

}

int AudioResampling (AVCodecContext * audio_dec_ctx,AVFrame * pAudioDecodeFrame,

int out_sample_fmt ,int out_channels ,int out_sample_rate , uint8_t * out_buf )

{

SwrContext * swr_ctx = NULL;

int data_size = 0;

int ret = 0;

int64_t src_ch_layout = AV_CH_LAYOUT_STEREO; //初始化这样根据不同文件做调整

int64_t dst_ch_layout = AV_CH_LAYOUT_STEREO; //这里设定ok

int dst_nb_channels = 0;

int dst_linesize = 0;

int src_nb_samples = 0;

int dst_nb_samples = 0;

int max_dst_nb_samples = 0;

uint8_t **dst_data = NULL;

int resampled_data_size = 0;

//重新采样

if (swr_ctx )

{

swr_free(&swr_ctx );

}

//创建重采样

swr_ctx = swr_alloc ();

if (!swr_ctx )

{

printf("swr_alloc error \n" );

return -1;

}

src_ch_layout = (audio_dec_ctx ->channel_layout &&

audio_dec_ctx->channels ==

av_get_channel_layout_nb_channels(audio_dec_ctx ->channel_layout)) ?

audio_dec_ctx->channel_layout :

av_get_default_channel_layout(audio_dec_ctx ->channels);

if (out_channels == 1)

{

dst_ch_layout = AV_CH_LAYOUT_MONO ;

}

else if (out_channels == 2)

{

dst_ch_layout = AV_CH_LAYOUT_STEREO ;

}

else

{

//可扩展

}

if (src_ch_layout <= 0)

{

printf("src_ch_layout error \n" );

return -1;

}

src_nb_samples = pAudioDecodeFrame ->nb_samples;

if (src_nb_samples <= 0)

{

printf("src_nb_samples error \n" );

return -1;

}

/* set options */

av_opt_set_int(swr_ctx , "in_channel_layout", src_ch_layout, 0);

av_opt_set_int(swr_ctx , "in_sample_rate", audio_dec_ctx->sample_rate , 0);

av_opt_set_sample_fmt(swr_ctx , "in_sample_fmt", audio_dec_ctx->sample_fmt , 0);

av_opt_set_int(swr_ctx , "out_channel_layout", dst_ch_layout, 0);

av_opt_set_int(swr_ctx , "out_sample_rate", out_sample_rate, 0);

av_opt_set_sample_fmt(swr_ctx , "out_sample_fmt", ( AVSampleFormat)out_sample_fmt , 0);

swr_init(swr_ctx );

//int64_t av_rescale_rnd(int64_t a,int64_t b,int64_t c,enum AVRounding rnd);作用是计算"a*b/c"的值并分五种方式来取整。

//在FFMPEG中是将以"时钟基c"表示的数值a转换成以"时钟基b"来表示。具体见印象笔记。

max_dst_nb_samples = dst_nb_samples = av_rescale_rnd( src_nb_samples, out_sample_rate , audio_dec_ctx->sample_rate , AV_ROUND_UP);

if (max_dst_nb_samples <= 0)

{

printf("av_rescale_rnd error \n" );

return -1;

}

dst_nb_channels = av_get_channel_layout_nb_channels (dst_ch_layout);

ret = av_samples_alloc_array_and_samples (&dst_data, & dst_linesize, dst_nb_channels ,

dst_nb_samples, (AVSampleFormat )out_sample_fmt, 0);

if (ret < 0)

{

printf("av_samples_alloc_array_and_samples error \n" );

return -1;

}

dst_nb_samples = av_rescale_rnd (swr_get_delay( swr_ctx, audio_dec_ctx ->sample_rate) +

src_nb_samples, out_sample_rate , audio_dec_ctx-> sample_rate,AV_ROUND_UP );

if (dst_nb_samples <= 0)

{

printf("av_rescale_rnd error \n" );

return -1;

}

if (dst_nb_samples > max_dst_nb_samples)

{

av_free(dst_data [0]);

ret = av_samples_alloc (dst_data, & dst_linesize, dst_nb_channels ,

dst_nb_samples, (AVSampleFormat )out_sample_fmt, 1);

max_dst_nb_samples = dst_nb_samples ;

}

data_size = av_samples_get_buffer_size (NULL, audio_dec_ctx->channels ,

pAudioDecodeFrame->nb_samples ,

audio_dec_ctx->sample_fmt , 1);

if (data_size <= 0)

{

printf("av_samples_get_buffer_size error \n" );

return -1;

}

resampled_data_size = data_size ;

if (swr_ctx )

{

ret = swr_convert (swr_ctx, dst_data, dst_nb_samples ,

( const uint8_t **)pAudioDecodeFrame-> data, pAudioDecodeFrame->nb_samples );

if (ret <= 0)

{

printf("swr_convert error \n" );

return -1;

}

resampled_data_size = av_samples_get_buffer_size (&dst_linesize, dst_nb_channels,

ret, (AVSampleFormat )out_sample_fmt, 1);

if (resampled_data_size <= 0)

{

printf("av_samples_get_buffer_size error \n" );

return -1;

}

else

{

printf("swr_ctx null error \n" );

return -1;

}

//将值返回去

memcpy(out_buf ,dst_data[0], resampled_data_size);

if (dst_data )

{

av_freep(&dst_data [0]);

}

av_freep(&dst_data );

dst_data = NULL ;

if (swr_ctx )

{

swr_free(&swr_ctx );

}

return resampled_data_size ;

}