ffmpeg录制系统声音，麦克风声音，桌面视频，并且桌面视频和声音分两个线程写

最新推荐文章于 2024-04-27 02:15:00 发布
最新推荐文章于 2024-04-27 02:15:00 发布
阅读量1.9k 收藏
点赞数
文章标签： 音视频
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/tusong86/article/details/122388408
版权

本人之前写过ffmpeg 本地麦克风声音和系统声音混音后，再混合本地桌面成最终的mp4文件-修正

这篇博客里面，如果设置抓取帧率为30帧，而系统没有能力在一秒钟抓取30帧时，就会出现问题，为此我又写了ffmpeg gdi录制桌面视频时注意事项
录制视频时，确保尽力交付，如果一秒钟能录30帧，则最终的视频帧率是30，如果一秒钟只能录20帧，则最终的视频帧率是20。

如今将尽力交付放在音视频同时录制时，却出了问题，录制的视频帧数很少，连10都不到，而实际上，本人的电脑，一秒钟大概可以抓20张，这就出现了问题，排查时发现，按照下面方式写文件时，该写视频文件的时候没有写，具体原因不多说，我是打印了相关日志，一点点看，发现问题的，读者可以自己尝试。
在这里插入图片描述

为此我将音频和视频分两个线程分别进行写入，之前的音视频混合线程就被注释掉了，如下所示：

m_hAudioInnerCapture = CreateThread(NULL, 0, AudioInnerCaptureProc, this, 0, NULL);
m_hAudioInnerResample = CreateThread(NULL, 0, AudioInnerResampleProc, this, 0, NULL);
m_hAudioMicCapture = CreateThread(NULL, 0, AudioMicCaptureProc, this, 0, NULL);
m_hAudioMix = CreateThread(NULL, 0, AudioMixProc, this, 0, NULL);
m_hScreenCapture = CreateThread(NULL, 0, ScreenCaptureProc, this, 0, NULL);
//m_hScreenAudioMix = CreateThread(NULL, 0, ScreenAudioMixProc, this, 0, NULL);
m_hScreenWrite = CreateThread(NULL, 0, ScreenWriteProc, this, 0, NULL);
m_hAudioWrite = CreateThread(NULL, 0, AudioWriteProc, this, 0, NULL);

这样一来，av_write_frame会在m_hScreenWrite和m_hAudioWrite两个线程中同时进行，需要加锁，否则时间长了，写入出问题，最终在关闭文件时，出现崩溃。
加锁的话这样写。

EnterCriticalSection(&m_csPacketWriteSection);
av_write_frame(m_pFormatCtx_Out, &packet);
LeaveCriticalSection(&m_csPacketWriteSection);

最终的录制文件是ok的，变量m_iFrameRate的值是可以调的，比如我设置为30，而最终的视频文件里面，帧率是19，这表示我电脑一秒钟只能抓19帧。而m_iFrameRate设置为10时，我中间会Sleep，确保一秒只抓10帧。

工程结构如下：
在这里插入图片描述
下面分别给出内容：
FfmpegVideoAudioAndMicOneFileSeparateWriteTest.cpp的内容如下：

#include "ULinkRecord.h"
#include <stdio.h>
#include <conio.h>








int main()
{
	ULinkRecord cULinkRecord;

	cULinkRecord.SetMicName(L"麦克风 (2- Synaptics HD Audio)");
	cULinkRecord.SetRecordPath("E:\\learn\\ffmpeg\\FfmpegTest\\x64\\Release");

	RECT rect;
	rect.left = 0;
	rect.top = 0;
	rect.right = 1920;
	rect.bottom = 1080;

	cULinkRecord.SetRecordRect(rect);

	cULinkRecord.StartRecord();

	Sleep(120000);

	printf("begin StopRecord\n");
	cULinkRecord.StopRecord();
	printf("end StopRecord\n");
	return 0;
}

CaptureScreen.h的内容如下：

#ifndef _CCAPTURE_SCREEN_HH
#define _CCAPTURE_SCREEN_HH

#include<time.h>
#include <d3d9.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <windows.h>

#include <tchar.h>
#include <winbase.h>
#include <winreg.h>
#include <Strsafe.h>


//
// ---抓屏类----
//
class CCaptureScreen
{
public:
	CCaptureScreen(void);
	~CCaptureScreen(void);

public:
	/*-----------定义外部调用函数-----------*/
	int Init(int iPosX, int iPosY, int iWidth, int iHeight);//初始化
	BYTE* CaptureImage(); //抓取屏幕

private:
	/*-----------定义内部调用函数-----------*/
	void* CaptureScreenFrame(int, int, int, int);//抓屏
	HCURSOR FetchCursorHandle(); //获取鼠标光标

private:
	/*-----------定义私有变量-----------*/
	int m_iXPos;
	int m_iYPos;
	int m_width;
	int m_height;
	UINT   wLineLen;
	DWORD  dwSize;
	DWORD  wColSize;

	//设备句柄
	HDC hScreenDC;
	HDC hMemDC;
	//图像RGB内存缓存
	PRGBTRIPLE m_hdib;
	//位图头信息结构体
	BITMAPINFO pbi;

	HBITMAP hbm = NULL;
	//鼠标光标
	HCURSOR m_hSavedCursor;


};

#endif //--_CCAPTURE_SCREEN_HH

CaptureScreen.cpp的内容如下：

//#include "stdafx.h"
#include "CaptureScreen.h"

CCaptureScreen::CCaptureScreen(void)
{
	m_hdib = NULL;
	m_hSavedCursor = NULL;
	hScreenDC = NULL;
	hMemDC = NULL;
	hbm = NULL;
	m_width = 1920;
	m_height = 1080;
	FetchCursorHandle();
}
//
// 释放资源
//
CCaptureScreen::~CCaptureScreen(void)
{
	if (hbm)
	{
		DeleteObject(hbm);
		hbm = NULL;
	}

	if (m_hdib){

		free(m_hdib);
		m_hdib = NULL;
	}
	if (hScreenDC){

		::ReleaseDC(NULL, hScreenDC);
	}
	if (hMemDC) {

		DeleteDC(hMemDC);
	}
}

//
// 初始化
//
int CCaptureScreen::Init(int iPosX, int iPosY, int iWidth, int iHeight)
{
	HWND hDeskTop = GetDesktopWindow();
	RECT rc;
	GetWindowRect(hDeskTop, &rc);
	hScreenDC = ::GetDC(NULL);
	if (hScreenDC == NULL) return 0;

	hMemDC = ::CreateCompatibleDC(hScreenDC);
	if (hMemDC == NULL) return 0;

	m_iXPos = iPosX;
	m_iYPos = iPosY;
	m_width = iWidth;
	m_height = iHeight;

	if (!m_hdib){
		m_hdib = (PRGBTRIPLE)malloc(m_width * m_height * 3);//24位图像大小
	}
	//位图头信息结构体
	pbi.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
	pbi.bmiHeader.biWidth = m_width;
	pbi.bmiHeader.biHeight = m_height;
	pbi.bmiHeader.biPlanes = 1;
	pbi.bmiHeader.biBitCount = 24;
	pbi.bmiHeader.biCompression = BI_RGB;

	hbm = CreateCompatibleBitmap(hScreenDC, m_width, m_height);
	SelectObject(hMemDC, hbm);

	wLineLen = ((m_width * 24 + 31) & 0xffffffe0) / 8;
	wColSize = sizeof(RGBQUAD)* ((24 <= 8) ? 1 << 24 : 0);
	dwSize = (DWORD)(UINT)wLineLen * (DWORD)(UINT)m_height;

	return 1;
}

//抓取屏幕数据
BYTE* CCaptureScreen::CaptureImage()
{

	VOID*  alpbi = CaptureScreenFrame(m_iXPos, m_iYPos, m_width, m_height);
	return (BYTE*)(alpbi);
}


void* CCaptureScreen::CaptureScreenFrame(int left, int top, int width, int height)
{
	if (hbm == NULL || hMemDC == NULL || hScreenDC == NULL) return NULL;
	
	//BitBlt(hMemDC, 0, 0, width, height, hScreenDC, left, top, SRCCOPY);
	BOOL bRet = StretchBlt(hMemDC, 0, 0, width, height, hScreenDC, left, top, width, height, SRCCOPY);
	/*-------------------------捕获鼠标-------------------------------*/
	{
		POINT xPoint;
		GetCursorPos(&xPoint);
		HCURSOR hcur = FetchCursorHandle();
		xPoint.x -= left;
		xPoint.y -= top;

		ICONINFO iconinfo;
		BOOL ret;
		ret = GetIconInfo(hcur, &iconinfo);
		if (ret){
			xPoint.x -= iconinfo.xHotspot;
			xPoint.y -= iconinfo.yHotspot;

			if (iconinfo.hbmMask) DeleteObject(iconinfo.hbmMask);
			if (iconinfo.hbmColor) DeleteObject(iconinfo.hbmColor);
		}
		/*画鼠标*/
		::DrawIcon(hMemDC, xPoint.x, xPoint.y, hcur);
	}
	//动态分配的内存
	PRGBTRIPLE hdib = m_hdib;
	if (!hdib)
		return hdib;

	GetDIBits(hMemDC, hbm, 0, m_height, hdib, (LPBITMAPINFO)&pbi, DIB_RGB_COLORS);
	return hdib;
}

//
// 获取窗体鼠标光标
//
HCURSOR CCaptureScreen::FetchCursorHandle()
{
	if (m_hSavedCursor == NULL)
	{
		m_hSavedCursor = GetCursor();
	}
	return m_hSavedCursor;
}

ULinkRecord.h的内容如下：

#pragma once

#include <string>
#include <Windows.h>

#ifdef	__cplusplus
extern "C"
{
#endif
#include "libavcodec/avcodec.h"
#include "libavformat/avformat.h"
#include "libswscale/swscale.h"
#include "libswresample/swresample.h"
#include "libavdevice/avdevice.h"
#include "libavutil/audio_fifo.h"
#include "libavutil/avutil.h"
#include "libavutil/fifo.h"
#include "libavutil/frame.h"
#include "libavutil/imgutils.h"

#include "libavfilter/avfilter.h"
#include "libavfilter/buffersink.h"
#include "libavfilter/buffersrc.h"


#pragma comment(lib, "avcodec.lib")
#pragma comment(lib, "avformat.lib")
#pragma comment(lib, "avutil.lib")
#pragma comment(lib, "avdevice.lib")
#pragma comment(lib, "avfilter.lib")
#pragma comment(lib, "postproc.lib")
#pragma comment(lib, "swresample.lib")
#pragma comment(lib, "swscale.lib")


#ifdef __cplusplus
};
#endif

class ULinkRecord
{
public:
	ULinkRecord();
	~ULinkRecord();
public:
	void SetMicName(const wchar_t* pMicName);
	void SetRecordPath(const char* pRecordPath);
	void SetRecordRect(RECT rectRecord);
	int StartRecord();
	void StopRecord();
	void SetMute(bool bMute);
private:
	int OpenAudioInnerCapture();
	int OpenAudioMicCapture();
	int OpenOutPut();
	int InitFilter(const char* filter_desc);
	void Clear();
private:
	static DWORD WINAPI AudioInnerCaptureProc(LPVOID lpParam);
	void AudioInnerCapture();

	static DWORD WINAPI AudioInnerResampleProc(LPVOID lpParam);
	void AudioInnerResample();

	static DWORD WINAPI AudioMicCaptureProc(LPVOID lpParam);
	void AudioMicCapture();

	static DWORD WINAPI AudioMixProc(LPVOID lpParam);
	void AudioMix();

	static DWORD WINAPI ScreenCaptureProc(LPVOID lpParam);
	void ScreenCapture();

	static DWORD WINAPI ScreenAudioMixProc(LPVOID lpParam);
	void ScreenAudioMix();

	static DWORD WINAPI ScreenWriteProc(LPVOID lpParam);
	void ScreenWrite();

	static DWORD WINAPI AudioWriteProc(LPVOID lpParam);
	void AudioWrite();
private:
	std::wstring m_wstrMicName;
	std::string m_strRecordPath;
	std::string m_strFilePrefix;
private:
	CRITICAL_SECTION m_csVideoSection;
	CRITICAL_SECTION m_csAudioInnerSection;
	CRITICAL_SECTION m_csAudioInnerResampleSection;
	CRITICAL_SECTION m_csAudioMicSection;
	CRITICAL_SECTION m_csAudioMixSection;

	CRITICAL_SECTION m_csPacketWriteSection;

	AVFifoBuffer *m_pVideoFifo = NULL;
	AVAudioFifo *m_pAudioInnerFifo = NULL;
	AVAudioFifo *m_pAudioInnerResampleFifo = NULL;
	AVAudioFifo *m_pAudioMicFifo = NULL;
	AVAudioFifo *m_pAudioMixFifo = NULL;

	AVFormatContext *m_pFormatCtx_Out = NULL;
	AVFormatContext	*m_pFormatCtx_AudioInner = NULL;
	AVFormatContext	*m_pFormatCtx_AudioMic = NULL;

	AVCodecContext *m_pReadCodecCtx_AudioInner = NULL;
	AVCodecContext *m_pReadCodecCtx_AudioMic = NULL;
	AVCodec *m_pReadCodec_Video = NULL;

	AVCodecContext	*m_pCodecEncodeCtx_Video = NULL;
	AVCodecContext	*m_pCodecEncodeCtx_Audio = NULL;
	AVCodec			*m_pCodecEncode_Audio = NULL;

	SwrContext *m_pAudioInnerResampleCtx = NULL;
	SwrContext *m_pAudioConvertCtx = NULL;


	AVFilterGraph* m_pFilterGraph = NULL;
	AVFilterContext* m_pFilterCtxSrcInner = NULL;
	AVFilterContext* m_pFilterCtxSrcMic = NULL;
	AVFilterContext* m_pFilterCtxSink = NULL;

	int m_iVideoStreamIndex = 0;
	int m_iAudioStreamIndex = 0;
	bool m_bRecord = false;
	bool m_bMute = false;

	HANDLE m_hAudioInnerCapture = NULL;
	HANDLE m_hAudioInnerResample = NULL;
	HANDLE m_hAudioMicCapture = NULL;
	HANDLE m_hAudioMix = NULL;
	HANDLE m_hScreenCapture = NULL;
	HANDLE m_hScreenAudioMix = NULL;
	HANDLE m_hScreenWrite = NULL;
	HANDLE m_hAudioWrite = NULL;

	int m_iYuv420FrameSize = 0;

	int m_iRecordPosX = 0;
	int m_iRecordPosY = 0;
	int m_iRecordWidth = 0;
	int m_iRecordHeight = 0;

	int m_iFrameNumber = 0;
	int m_iFrameRate = 30;
};

ULinkRecord.cpp的内容如下：

#include "ULinkRecord.h"
#include "log/log.h"
#include "appfun/appfun.h"

#include "CaptureScreen.h"



typedef struct BufferSourceContext {
	const AVClass    *bscclass;
	AVFifoBuffer     *fifo;
	AVRational        time_base;     ///< time_base to set in the output link
	AVRational        frame_rate;    ///< frame_rate to set in the output link
	unsigned          nb_failed_requests;
	unsigned          warning_limit;

	/* video only */
	int               w, h;
	enum AVPixelFormat  pix_fmt;
	AVRational        pixel_aspect;
	char              *sws_param;

	AVBufferRef *hw_frames_ctx;

	/* audio only */
	int sample_rate;
	enum AVSampleFormat sample_fmt;
	int channels;
	uint64_t channel_layout;
	char    *channel_layout_str;

	int got_format_from_params;
	int eof;
} BufferSourceContext;


static char *dup_wchar_to_utf8(const wchar_t *w)
{
	char *s = NULL;
	int l = WideCharToMultiByte(CP_UTF8, 0, w, -1, 0, 0, 0, 0);
	s = (char *)av_malloc(l);
	if (s)
		WideCharToMultiByte(CP_UTF8, 0, w, -1, s, l, 0, 0);
	return s;
}


/* just pick the highest supported samplerate */
static int select_sample_rate(const AVCodec *codec)
{
	const int *p;
	int best_samplerate = 0;

	if (!codec->supported_samplerates)
		return 44100;

	p = codec->supported_samplerates;
	while (*p) {
		if (!best_samplerate || abs(44100 - *p) < abs(44100 - best_samplerate))
			best_samplerate = *p;
		p++;
	}
	return best_samplerate;
}




/* select layout with the highest channel count */
static int select_channel_layout(const AVCodec *codec)
{
	const uint64_t *p;
	uint64_t best_ch_layout = 0;
	int best_nb_channels = 0;

	if (!codec->channel_layouts)
		return AV_CH_LAYOUT_STEREO;

	p = codec->channel_layouts;
	while (*p) {
		int nb_channels = av_get_channel_layout_nb_channels(*p);

		if (nb_channels > best_nb_channels) {
			best_ch_layout = *p;
			best_nb_channels = nb_channels;
		}
		p++;
	}
	return best_ch_layout;
}


unsigned char clip_value(unsigned char x, unsigned char min_val, unsigned char  max_val) {
	if (x > max_val) {
		return max_val;
	}
	else if (x < min_val) {
		return min_val;
	}
	else {
		return x;
	}
}

//RGB to YUV420
bool RGB24_TO_YUV420(unsigned char *RgbBuf, int w, int h, unsigned char *yuvBuf)
{
	unsigned char*ptrY, *ptrU, *ptrV, *ptrRGB;
	memset(yuvBuf, 0, w*h * 3 / 2);
	ptrY = yuvBuf;
	ptrU = yuvBuf + w * h;
	ptrV = ptrU + (w*h * 1 / 4);
	unsigned char y, u, v, r, g, b;
	for (int j = h - 1; j >= 0; j--) {
		ptrRGB = RgbBuf + w * j * 3;
		for (int i = 0; i < w; i++) {

			b = *(ptrRGB++);
			g = *(ptrRGB++);
			r = *(ptrRGB++);


			y = (unsigned char)((66 * r + 129 * g + 25 * b + 128) >> 8) + 16;
			u = (unsigned char)((-38 * r - 74 * g + 112 * b + 128) >> 8) + 128;
			v = (unsigned char)((112 * r - 94 * g - 18 * b + 128) >> 8) + 128;
			*(ptrY++) = clip_value(y, 0, 255);
			if (j % 2 == 0 && i % 2 == 0) {
				*(ptrU++) = clip_value(u, 0, 255);
			}
			else {
				if (i % 2 == 0) {
					*(ptrV++) = clip_value(v, 0, 255);
				}
			}
		}
	}
	return true;
}


ULinkRecord::ULinkRecord()
{
	InitializeCriticalSection(&m_csVideoSection);
	InitializeCriticalSection(&m_csAudioInnerSection);
	InitializeCriticalSection(&m_csAudioInnerResampleSection);
	InitializeCriticalSection(&m_csAudioMicSection);
	InitializeCriticalSection(&m_csAudioMixSection);

	InitializeCriticalSection(&m_csPacketWriteSection);

	

	avdevice_register_all();
}

ULinkRecord::~ULinkRecord()
{
	DeleteCriticalSection(&m_csVideoSection);
	DeleteCriticalSection(&m_csAudioInnerSection);
	DeleteCriticalSection(&m_csAudioInnerResampleSection);
	DeleteCriticalSection(&m_csAudioMicSection);
	DeleteCriticalSection(&m_csAudioMixSection);

	DeleteCriticalSection(&m_csPacketWriteSection);
}

void ULinkRecord::SetMicName(const wchar_t* pMicName)
{
	m_wstrMicName = pMicName;
}

void ULinkRecord::SetRecordPath(const char* pRecordPath)
{
	m_strRecordPath = pRecordPath;
	if (!m_strRecordPath.empty())
	{
		if (m_strRecordPath[m_strRecordPath.length() - 1] != '\\')
		{
			m_strRecordPath = m_strRecordPath + "\\";
		}
	}
}

void ULinkRecord::SetRecordRect(RECT rectRecord)
{
	m_iRecordPosX = rectRecord.left;
	m_iRecordPosY = rectRecord.top;
	m_iRecordWidth = rectRecord.right - rectRecord.left;
	m_iRecordHeight = rectRecord.bottom - rectRecord.top;
}

int ULinkRecord::StartRecord()
{
	int iRet = -1;
	do 
	{
		Clear();

		m_pAudioConvertCtx = swr_alloc();
		av_opt_set_channel_layout(m_pAudioConvertCtx, "in_channel_layout", AV_CH_LAYOUT_STEREO, 0);
		av_opt_set_channel_layout(m_pAudioConvertCtx, "out_channel_layout", AV_CH_LAYOUT_STEREO, 0);
		av_opt_set_int(m_pAudioConvertCtx, "in_sample_rate", 44100, 0);
		av_opt_set_int(m_pAudioConvertCtx, "out_sample_rate", 44100, 0);
		av_opt_set_sample_fmt(m_pAudioConvertCtx, "in_sample_fmt", AV_SAMPLE_FMT_S16, 0);
		//av_opt_set_sample_fmt(audio_convert_ctx, "in_sample_fmt", AV_SAMPLE_FMT_FLTP, 0);
		av_opt_set_sample_fmt(m_pAudioConvertCtx, "out_sample_fmt", AV_SAMPLE_FMT_FLTP, 0);

		iRet = swr_init(m_pAudioConvertCtx);


		m_pAudioInnerResampleCtx = swr_alloc();
		av_opt_set_channel_layout(m_pAudioInnerResampleCtx, "in_channel_layout", AV_CH_LAYOUT_STEREO, 0);
		av_opt_set_channel_layout(m_pAudioInnerResampleCtx, "out_channel_layout", AV_CH_LAYOUT_STEREO, 0);
		av_opt_set_int(m_pAudioInnerResampleCtx, "in_sample_rate", 48000, 0);
		av_opt_set_int(m_pAudioInnerResampleCtx, "out_sample_rate", 44100, 0);
		av_opt_set_sample_fmt(m_pAudioInnerResampleCtx, "in_sample_fmt", AV_SAMPLE_FMT_S16, 0);
		av_opt_set_sample_fmt(m_pAudioInnerResampleCtx, "out_sample_fmt", AV_SAMPLE_FMT_S16, 0);

		iRet = swr_init(m_pAudioInnerResampleCtx);


		if (OpenAudioInnerCapture() < 0)
		{
			break;
		}

		if (OpenAudioMicCapture() < 0)
		{
			break;
		}

		if (OpenOutPut() < 0)
		{
			break;
		}

		const char* filter_desc = "[in0][in1]amix=inputs=2[out]";
		LOG_INFO("begin InitFilter");
		iRet = InitFilter(filter_desc);
		LOG_INFO("end InitFilter");
		if (iRet < 0)
		{
			break;
		}

		if (NULL == m_pAudioInnerResampleFifo)
		{
			m_pAudioInnerResampleFifo = av_audio_fifo_alloc((AVSampleFormat)m_pFormatCtx_AudioInner->streams[0]->codecpar->format,
				m_pFormatCtx_AudioInner->streams[0]->codecpar->channels, 3000 * 1024);
		}

		if (NULL == m_pAudioInnerFifo)
		{
			m_pAudioInnerFifo = av_audio_fifo_alloc((AVSampleFormat)m_pFormatCtx_AudioInner->streams[0]->codecpar->format,
				m_pFormatCtx_AudioInner->streams[0]->codecpar->channels, 3000 * 1024);
		}

		if (NULL == m_pAudioMicFifo)
		{
			m_pAudioMicFifo = av_audio_fifo_alloc((AVSampleFormat)m_pFormatCtx_AudioMic->streams[0]->codecpar->format,
				m_pFormatCtx_AudioMic->streams[0]->codecpar->channels, 3000 * 1024);
		}

		if (NULL == m_pAudioMixFifo)
		{
			m_pAudioMixFifo = av_audio_fifo_alloc((AVSampleFormat)m_pFormatCtx_AudioInner->streams[0]->codecpar->format,
				m_pFormatCtx_AudioInner->streams[0]->codecpar->channels, 3000 * 1024);
		}

		m_iYuv420FrameSize = av_image_get_buffer_size(AV_PIX_FMT_YUV420P, m_iRecordWidth, m_iRecordHeight, 1);

		//申请30帧缓存
		m_pVideoFifo = av_fifo_alloc(30 * m_iYuv420FrameSize);

		m_bRecord = true;

		m_hAudioInnerCapture = CreateThread(NULL, 0, AudioInnerCaptureProc, this, 0, NULL);
		m_hAudioInnerResample = CreateThread(NULL, 0, AudioInnerResampleProc, this, 0, NULL);
		m_hAudioMicCapture = CreateThread(NULL, 0, AudioMicCaptureProc, this, 0, NULL);
		m_hAudioMix = CreateThread(NULL, 0, AudioMixProc, this, 0, NULL);
		m_hScreenCapture = CreateThread(NULL, 0, ScreenCaptureProc, this, 0, NULL);
		//m_hScreenAudioMix = CreateThread(NULL, 0, ScreenAudioMixProc, this, 0, NULL);
		m_hScreenWrite = CreateThread(NULL, 0, ScreenWriteProc, this, 0, NULL);
		m_hAudioWrite = CreateThread(NULL, 0, AudioWriteProc, this, 0, NULL);

		iRet = 0;
	} while (0);
	
	if (0 == iRet)
	{
		LOG_INFO("StartRecord succeed");
	}
	else
	{
		Clear();
	}


	return iRet;
}


void ULinkRecord::StopRecord()
{
	m_bRecord = false;
	if (m_hAudioInnerCapture == NULL && m_hAudioInnerResample == NULL && m_hAudioMicCapture == NULL && m_hAudioMix == NULL && m_hScreenCapture == NULL && m_hScreenWrite == NULL && m_hAudioWrite == NULL)
	{
		return;
	}
	Sleep(1000);
	HANDLE hThreads[7];
	hThreads[0] = m_hAudioInnerCapture;
	hThreads[1] = m_hAudioInnerResample;
	hThreads[2] = m_hAudioMicCapture;
	hThreads[3] = m_hAudioMix;
	hThreads[4] = m_hScreenCapture;
	hThreads[5] = m_hScreenWrite;
	hThreads[6] = m_hAudioWrite;

	WaitForMultipleObjects(6, hThreads, TRUE, INFINITE);

	CloseHandle(m_hAudioInnerCapture);
	m_hAudioInnerCapture = NULL;

	CloseHandle(m_hAudioInnerResample);
	m_hAudioInnerResample = NULL;

	CloseHandle(m_hAudioMicCapture);
	m_hAudioMicCapture = NULL;

	CloseHandle(m_hAudioMix);
	m_hAudioMix = NULL;

	
	if (m_hScreenCapture != NULL)
	{
		CloseHandle(m_hScreenCapture);
		m_hScreenCapture = NULL;
	}

	if (m_hScreenAudioMix != NULL)
	{
		CloseHandle(m_hScreenAudioMix);
		m_hScreenAudioMix = NULL;
	}

	if (m_hScreenWrite != NULL)
	{
		CloseHandle(m_hScreenWrite);
		m_hScreenWrite = NULL;
	}

	if (m_hAudioWrite != NULL)
	{
		CloseHandle(m_hAudioWrite);
		m_hAudioWrite = NULL;
	}

	av_write_trailer(m_pFormatCtx_Out);
	avio_close(m_pFormatCtx_Out->pb);


	Clear();
}

void ULinkRecord::SetMute(bool bMute)
{
	LOG_INFO("SetMute, bMute is %d", bMute);
	m_bMute = bMute;
}

int ULinkRecord::OpenAudioInnerCapture()
{
	int iRet = -1;

	do 
	{
		//查找输入方式
		const AVInputFormat *pAudioInputFmt = av_find_input_format("dshow");

		//以Direct Show的方式打开设备，并将 输入方式 关联到格式上下文
		//const char * psDevName = dup_wchar_to_utf8(L"audio=麦克风 (2- Synaptics HD Audio)");
		char * psDevName = dup_wchar_to_utf8(L"audio=virtual-audio-capturer");

		if (avformat_open_input(&m_pFormatCtx_AudioInner, psDevName, pAudioInputFmt, NULL) < 0)
		{
			LOG_ERROR("avformat_open_input failed, m_pFormatCtx_AudioInner");
			break;
		}

		if (avformat_find_stream_info(m_pFormatCtx_AudioInner, NULL) < 0)
		{
			break;
		}

		if (m_pFormatCtx_AudioInner->streams[0]->codecpar->codec_type != AVMEDIA_TYPE_AUDIO)
		{
			LOG_ERROR("Couldn't find audio stream information");
			break;
		}


		const AVCodec *tmpCodec = avcodec_find_decoder(m_pFormatCtx_AudioInner->streams[0]->codecpar->codec_id);

		m_pReadCodecCtx_AudioInner = avcodec_alloc_context3(tmpCodec);

		m_pReadCodecCtx_AudioInner->sample_rate = m_pFormatCtx_AudioInner->streams[0]->codecpar->sample_rate;
		m_pReadCodecCtx_AudioInner->channel_layout = select_channel_layout(tmpCodec);
		m_pReadCodecCtx_AudioInner->channels = av_get_channel_layout_nb_channels(m_pReadCodecCtx_AudioInner->channel_layout);

		m_pReadCodecCtx_AudioInner->sample_fmt = (AVSampleFormat)m_pFormatCtx_AudioInner->streams[0]->codecpar->format;
		//pReadCodecCtx_Audio->sample_fmt = AV_SAMPLE_FMT_FLTP;

		if (0 > avcodec_open2(m_pReadCodecCtx_AudioInner, tmpCodec, NULL))
		{
			LOG_ERROR("avcodec_open2 failed, m_pReadCodecCtx_AudioInner");
			break;
		}

		avcodec_parameters_from_context(m_pFormatCtx_AudioInner->streams[0]->codecpar, m_pReadCodecCtx_AudioInner);

		iRet = 0;
	} while (0);

	if (iRet != 0)
	{
		if (m_pReadCodecCtx_AudioInner != NULL)
		{
			avcodec_free_context(&m_pReadCodecCtx_AudioInner);
			m_pReadCodecCtx_AudioInner = NULL;
		}

		if (m_pFormatCtx_AudioInner != NULL)
		{
			avformat_close_input(&m_pFormatCtx_AudioInner);
			m_pFormatCtx_AudioInner = NULL;
		}
	}

	return iRet;
}


int ULinkRecord::OpenAudioMicCapture()
{
	int iRet = -1;
	do 
	{
		//查找输入方式
		const AVInputFormat *pAudioInputFmt = av_find_input_format("dshow");

		//以Direct Show的方式打开设备，并将 输入方式 关联到格式上下文
		std::wstring strDeviceName = L"audio=";
		strDeviceName += m_wstrMicName;
		const char * psDevName = dup_wchar_to_utf8(strDeviceName.c_str());

		if (avformat_open_input(&m_pFormatCtx_AudioMic, psDevName, pAudioInputFmt, NULL) < 0)
		{
			LOG_ERROR("avformat_open_input failed, m_pFormatCtx_AudioMic");
			break;
		}

		if (m_pFormatCtx_AudioMic->streams[0]->codecpar->codec_type != AVMEDIA_TYPE_AUDIO)
		{
			LOG_ERROR("Couldn't find audio stream information");
			break;
		}


		const AVCodec *tmpCodec = avcodec_find_decoder(m_pFormatCtx_AudioMic->streams[0]->codecpar->codec_id);

		m_pReadCodecCtx_AudioMic = avcodec_alloc_context3(tmpCodec);

		m_pReadCodecCtx_AudioMic->sample_rate = select_sample_rate(tmpCodec);
		m_pReadCodecCtx_AudioMic->channel_layout = select_channel_layout(tmpCodec);
		m_pReadCodecCtx_AudioMic->channels = av_get_channel_layout_nb_channels(m_pReadCodecCtx_AudioMic->channel_layout);

		m_pReadCodecCtx_AudioMic->sample_fmt = (AVSampleFormat)m_pFormatCtx_AudioMic->streams[0]->codecpar->format;
		//pReadCodecCtx_Audio->sample_fmt = AV_SAMPLE_FMT_FLTP;

		if (0 > avcodec_open2(m_pReadCodecCtx_AudioMic, tmpCodec, NULL))
		{
			LOG_ERROR("avcodec_open2 failed, m_pReadCodecCtx_AudioMic");
			break;
		}

		avcodec_parameters_from_context(m_pFormatCtx_AudioMic->streams[0]->codecpar, m_pReadCodecCtx_AudioMic);

		iRet = 0;
	} while (0);

	if (iRet != 0)
	{
		if (m_pReadCodecCtx_AudioMic != NULL)
		{
			avcodec_free_context(&m_pReadCodecCtx_AudioMic);
			m_pReadCodecCtx_AudioMic = NULL;
		}

		if (m_pFormatCtx_AudioMic != NULL)
		{
			avformat_close_input(&m_pFormatCtx_AudioMic);
			m_pFormatCtx_AudioMic = NULL;
		}
	}
	return iRet;
}


int ULinkRecord::OpenOutPut()
{
	std::string strFileName = m_strRecordPath;
	m_strFilePrefix = strFileName + time_tSimpleString(time(NULL));

	int iRet = -1;

	AVStream *pAudioStream = NULL;
	AVStream *pVideoStream = NULL;

	do
	{
		std::string strFileName = m_strRecordPath;
		strFileName += time_tSimpleString(time(NULL));
		strFileName += ".mp4";

		const char *outFileName = strFileName.c_str();
		avformat_alloc_output_context2(&m_pFormatCtx_Out, NULL, NULL, outFileName);

		{
			m_iVideoStreamIndex = 0;

			AVCodec* pCodecEncode_Video = (AVCodec *)avcodec_find_encoder(m_pFormatCtx_Out->oformat->video_codec);

			m_pCodecEncodeCtx_Video = avcodec_alloc_context3(pCodecEncode_Video);
			if (!m_pCodecEncodeCtx_Video)
			{
				LOG_ERROR("avcodec_alloc_context3 failed, m_pCodecEncodeCtx_Audio");
				break;
			}

			pVideoStream = avformat_new_stream(m_pFormatCtx_Out, pCodecEncode_Video);
			if (!pVideoStream)
			{
				break;
			}

			
			m_pCodecEncodeCtx_Video->flags |= AV_CODEC_FLAG_QSCALE;
			m_pCodecEncodeCtx_Video->bit_rate = 4000000;
			m_pCodecEncodeCtx_Video->rc_min_rate = 4000000;
			m_pCodecEncodeCtx_Video->rc_max_rate = 4000000;
			m_pCodecEncodeCtx_Video->bit_rate_tolerance = 4000000;
			m_pCodecEncodeCtx_Video->time_base.den = m_iFrameRate;
			m_pCodecEncodeCtx_Video->time_base.num = 1;

			m_pCodecEncodeCtx_Video->width = m_iRecordWidth;
			m_pCodecEncodeCtx_Video->height = m_iRecordHeight;
			//pH264Encoder->pCodecCtx->frame_number = 1;
			m_pCodecEncodeCtx_Video->gop_size = 12;
			m_pCodecEncodeCtx_Video->max_b_frames = 0;
			m_pCodecEncodeCtx_Video->thread_count = 4;
			m_pCodecEncodeCtx_Video->pix_fmt = AV_PIX_FMT_YUV420P;
			m_pCodecEncodeCtx_Video->codec_id = AV_CODEC_ID_H264;
			m_pCodecEncodeCtx_Video->codec_type = AVMEDIA_TYPE_VIDEO;

			av_opt_set(m_pCodecEncodeCtx_Video->priv_data, "b-pyramid", "none", 0);
			av_opt_set(m_pCodecEncodeCtx_Video->priv_data, "preset", "superfast", 0);
			av_opt_set(m_pCodecEncodeCtx_Video->priv_data, "tune", "zerolatency", 0);

			if (m_pFormatCtx_Out->oformat->flags & AVFMT_GLOBALHEADER)
				m_pCodecEncodeCtx_Video->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;

			if (avcodec_open2(m_pCodecEncodeCtx_Video, pCodecEncode_Video, 0) < 0)
			{
				//编码器打开失败，退出程序
				LOG_ERROR("avcodec_open2 failed, m_pCodecEncodeCtx_Audio");
				break;
			}
		}


		if (m_pFormatCtx_AudioInner->streams[0]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && m_pFormatCtx_AudioMic->streams[0]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO)
		{
			pAudioStream = avformat_new_stream(m_pFormatCtx_Out, NULL);

			m_iAudioStreamIndex = 1;

			m_pCodecEncode_Audio = (AVCodec *)avcodec_find_encoder(m_pFormatCtx_Out->oformat->audio_codec);

			m_pCodecEncodeCtx_Audio = avcodec_alloc_context3(m_pCodecEncode_Audio);
			if (!m_pCodecEncodeCtx_Audio)
			{
				LOG_ERROR("avcodec_alloc_context3 failed, m_pCodecEncodeCtx_Audio");
				break;
			}


			//pCodecEncodeCtx_Audio->codec_id = pFormatCtx_Out->oformat->audio_codec;
			m_pCodecEncodeCtx_Audio->sample_fmt = m_pCodecEncode_Audio->sample_fmts ? m_pCodecEncode_Audio->sample_fmts[0] : AV_SAMPLE_FMT_FLTP;
			m_pCodecEncodeCtx_Audio->bit_rate = 64000;
			m_pCodecEncodeCtx_Audio->sample_rate = 44100;
			m_pCodecEncodeCtx_Audio->channel_layout = AV_CH_LAYOUT_STEREO;
			m_pCodecEncodeCtx_Audio->channels = av_get_channel_layout_nb_channels(m_pCodecEncodeCtx_Audio->channel_layout);


			AVRational timeBase;
			timeBase.den = m_pCodecEncodeCtx_Audio->sample_rate;
			timeBase.num = 1;
			pAudioStream->time_base = timeBase;

			if (avcodec_open2(m_pCodecEncodeCtx_Audio, m_pCodecEncode_Audio, 0) < 0)
			{
				//编码器打开失败，退出程序
				LOG_ERROR("avcodec_open2 failed, m_pCodecEncodeCtx_Audio");
				break;
			}
		}


		if (!(m_pFormatCtx_Out->oformat->flags & AVFMT_NOFILE))
		{
			if (avio_open(&m_pFormatCtx_Out->pb, outFileName, AVIO_FLAG_WRITE) < 0)
			{
				LOG_ERROR("avio_open failed, m_pFormatCtx_Out->pb");
				break;
			}
		}

		avcodec_parameters_from_context(pVideoStream->codecpar, m_pCodecEncodeCtx_Video);
		avcodec_parameters_from_context(pAudioStream->codecpar, m_pCodecEncodeCtx_Audio);

		if (avformat_write_header(m_pFormatCtx_Out, NULL) < 0)
		{
			LOG_ERROR("avio_open avformat_write_header,m_pFormatCtx_Out");
			break;
		}

		iRet = 0;
	} while (0);


	if (iRet != 0)
	{
		if (m_pCodecEncodeCtx_Video != NULL)
		{
			avcodec_free_context(&m_pCodecEncodeCtx_Video);
			m_pCodecEncodeCtx_Video = NULL;
		}

		if (m_pCodecEncodeCtx_Audio != NULL)
		{
			avcodec_free_context(&m_pCodecEncodeCtx_Audio);
			m_pCodecEncodeCtx_Audio = NULL;
		}

		if (m_pFormatCtx_Out != NULL)
		{
			avformat_free_context(m_pFormatCtx_Out);
			m_pFormatCtx_Out = NULL;
		}
	}

	return iRet;
}


int ULinkRecord::InitFilter(const char* filter_desc)
{
	int ret = 0;
	char args_inner[512];
	const char* pad_name_inner = "in0";
	char args_mic[512];
	const char* pad_name_mic = "in1";

	AVFilter* filter_src_spk = (AVFilter *)avfilter_get_by_name("abuffer");
	AVFilter* filter_src_mic = (AVFilter *)avfilter_get_by_name("abuffer");
	AVFilter* filter_sink = (AVFilter *)avfilter_get_by_name("abuffersink");
	AVFilterInOut* filter_output_inner = avfilter_inout_alloc();
	AVFilterInOut* filter_output_mic = avfilter_inout_alloc();
	AVFilterInOut* filter_input = avfilter_inout_alloc();
	m_pFilterGraph = avfilter_graph_alloc();

	/*sprintf_s(args_inner, sizeof(args_inner), "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=0x%I64x",
		m_pReadCodecCtx_AudioInner->time_base.num,
		m_pReadCodecCtx_AudioInner->time_base.den,
		m_pReadCodecCtx_AudioInner->sample_rate,
		av_get_sample_fmt_name((AVSampleFormat)m_pReadCodecCtx_AudioInner->sample_fmt),
		m_pReadCodecCtx_AudioInner->channel_layout);*/

	sprintf_s(args_inner, sizeof(args_inner), "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=0x%I64x",
		m_pReadCodecCtx_AudioMic->time_base.num,
		m_pReadCodecCtx_AudioMic->time_base.den,
		m_pReadCodecCtx_AudioMic->sample_rate,
		av_get_sample_fmt_name((AVSampleFormat)m_pReadCodecCtx_AudioMic->sample_fmt),
		m_pReadCodecCtx_AudioMic->channel_layout);

	sprintf_s(args_mic, sizeof(args_mic), "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=0x%I64x",
		m_pReadCodecCtx_AudioMic->time_base.num,
		m_pReadCodecCtx_AudioMic->time_base.den,
		m_pReadCodecCtx_AudioMic->sample_rate,
		av_get_sample_fmt_name((AVSampleFormat)m_pReadCodecCtx_AudioMic->sample_fmt),
		m_pReadCodecCtx_AudioMic->channel_layout);

	AVFilterInOut* filter_outputs[2];
	do 
	{
		ret = avfilter_graph_create_filter(&m_pFilterCtxSrcInner, filter_src_spk, pad_name_inner, args_inner, NULL, m_pFilterGraph);
		if (ret < 0)
		{
			LOG_ERROR("Filter: failed to call avfilter_graph_create_filter -- src inner");
			break;
		}
		ret = avfilter_graph_create_filter(&m_pFilterCtxSrcMic, filter_src_mic, pad_name_mic, args_mic, NULL, m_pFilterGraph);
		if (ret < 0)
		{
			LOG_ERROR("Filter: failed to call avfilter_graph_create_filter -- src mic");
			break;
		}

		ret = avfilter_graph_create_filter(&m_pFilterCtxSink, filter_sink, "out", NULL, NULL, m_pFilterGraph);
		if (ret < 0)
		{
			LOG_ERROR("Filter: failed to call avfilter_graph_create_filter -- sink");
			break;
		}
		AVCodecContext* encodec_ctx = m_pCodecEncodeCtx_Audio;
		//ret = av_opt_set_bin(_filter_ctx_sink, "sample_fmts", (uint8_t*)&encodec_ctx->sample_fmt, sizeof(encodec_ctx->sample_fmt), AV_OPT_SEARCH_CHILDREN);
		ret = av_opt_set_bin(m_pFilterCtxSink, "sample_fmts", (uint8_t*)&m_pReadCodecCtx_AudioInner->sample_fmt, sizeof(m_pReadCodecCtx_AudioInner->sample_fmt), AV_OPT_SEARCH_CHILDREN);
		if (ret < 0)
		{
			LOG_ERROR("Filter: failed to call av_opt_set_bin -- sample_fmts");
			break;
		}
		ret = av_opt_set_bin(m_pFilterCtxSink, "channel_layouts", (uint8_t*)&encodec_ctx->channel_layout, sizeof(encodec_ctx->channel_layout), AV_OPT_SEARCH_CHILDREN);
		if (ret < 0)
		{
			LOG_ERROR("Filter: failed to call av_opt_set_bin -- channel_layouts");
			break;
		}
		ret = av_opt_set_bin(m_pFilterCtxSink, "sample_rates", (uint8_t*)&encodec_ctx->sample_rate, sizeof(encodec_ctx->sample_rate), AV_OPT_SEARCH_CHILDREN);
		if (ret < 0)
		{
			LOG_ERROR("Filter: failed to call av_opt_set_bin -- sample_rates");
			break;
		}

		filter_output_inner->name = av_strdup(pad_name_inner);
		filter_output_inner->filter_ctx = m_pFilterCtxSrcInner;
		filter_output_inner->pad_idx = 0;
		filter_output_inner->next = filter_output_mic;

		filter_output_mic->name = av_strdup(pad_name_mic);
		filter_output_mic->filter_ctx = m_pFilterCtxSrcMic;
		filter_output_mic->pad_idx = 0;
		filter_output_mic->next = NULL;

		filter_input->name = av_strdup("out");
		filter_input->filter_ctx = m_pFilterCtxSink;
		filter_input->pad_idx = 0;
		filter_input->next = NULL;

		filter_outputs[0] = filter_output_inner;
		filter_outputs[1] = filter_output_mic;

		ret = avfilter_graph_parse_ptr(m_pFilterGraph, filter_desc, &filter_input, filter_outputs, NULL);
		if (ret < 0)
		{
			LOG_ERROR("Filter: failed to call avfilter_graph_parse_ptr");
			break;
		}

		ret = avfilter_graph_config(m_pFilterGraph, NULL);
		if (ret < 0)
		{
			LOG_ERROR("Filter: failed to call avfilter_graph_config");
			break;
		}

		ret = 0;

	} while (0);
	

	avfilter_inout_free(&filter_input);
	av_free(filter_src_spk);
	av_free(filter_src_mic);
	avfilter_inout_free(filter_outputs);

	char* temp = avfilter_graph_dump(m_pFilterGraph, NULL);

	return ret;
}

void ULinkRecord::Clear()
{
	if (m_pReadCodecCtx_AudioInner)
	{
		avcodec_free_context(&m_pReadCodecCtx_AudioInner);
		m_pReadCodecCtx_AudioInner = NULL;
	}

	if (m_pReadCodecCtx_AudioMic != NULL)
	{
		avcodec_free_context(&m_pReadCodecCtx_AudioMic);
		m_pReadCodecCtx_AudioMic = NULL;
	}

	if (m_pCodecEncodeCtx_Video != NULL)
	{
		avcodec_free_context(&m_pCodecEncodeCtx_Video);
		m_pCodecEncodeCtx_Video = NULL;
	}

	if (m_pCodecEncodeCtx_Audio != NULL)
	{
		avcodec_free_context(&m_pCodecEncodeCtx_Audio);
		m_pCodecEncodeCtx_Audio = NULL;
	}

	if (m_pAudioConvertCtx != NULL)
	{
		swr_free(&m_pAudioConvertCtx);
		m_pAudioConvertCtx = NULL;
	}

	if (m_pAudioInnerResampleCtx != NULL)
	{
		swr_free(&m_pAudioInnerResampleCtx);
		m_pAudioInnerResampleCtx = NULL;
	}

	if (m_pVideoFifo != NULL)
	{
		av_fifo_free(m_pVideoFifo);
		m_pVideoFifo = NULL;
	}

	if (m_pAudioInnerFifo != NULL)
	{
		av_audio_fifo_free(m_pAudioInnerFifo);
		m_pAudioInnerFifo = NULL;
	}

	if (m_pAudioInnerResampleFifo != NULL)
	{
		av_audio_fifo_free(m_pAudioInnerResampleFifo);
		m_pAudioInnerResampleFifo = NULL;
	}

	if (m_pAudioMicFifo != NULL)
	{
		av_audio_fifo_free(m_pAudioMicFifo);
		m_pAudioMicFifo = NULL;
	}

	if (m_pAudioMixFifo != NULL)
	{
		av_audio_fifo_free(m_pAudioMixFifo);
		m_pAudioMixFifo = NULL;
	}

	if (m_pFormatCtx_AudioInner != NULL)
	{
		avformat_close_input(&m_pFormatCtx_AudioInner);
		m_pFormatCtx_AudioInner = NULL;
	}

	if (m_pFormatCtx_AudioMic != NULL)
	{
		avformat_close_input(&m_pFormatCtx_AudioMic);
		m_pFormatCtx_AudioMic = NULL;
	}

	if (m_pFormatCtx_Out != NULL)
	{
		avformat_free_context(m_pFormatCtx_Out);
		m_pFormatCtx_Out = NULL;
	}

	if (m_hAudioInnerCapture != NULL)
	{
		CloseHandle(m_hAudioInnerCapture);
		m_hAudioInnerCapture = NULL;
	}

	if (m_hAudioInnerResample != NULL)
	{
		CloseHandle(m_hAudioInnerResample);
		m_hAudioInnerResample = NULL;
	}

	if (m_hAudioMicCapture != NULL)
	{
		CloseHandle(m_hAudioMicCapture);
		m_hAudioMicCapture = NULL;
	}

	if (m_hAudioMix != NULL)
	{
		CloseHandle(m_hAudioMix);
		m_hAudioMix = NULL;
	}

	if (m_hScreenCapture != NULL)
	{
		CloseHandle(m_hScreenCapture);
		m_hScreenCapture = NULL;
	}

	if (m_hScreenAudioMix != NULL)
	{
		CloseHandle(m_hScreenAudioMix);
		m_hScreenAudioMix = NULL;
	}
}

DWORD WINAPI ULinkRecord::AudioInnerCaptureProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->AudioInnerCapture();
	}
	return 0;
}

void ULinkRecord::AudioInnerCapture()
{
	AVFrame *pFrame;
	pFrame = av_frame_alloc();

	AVPacket packet = { 0 };
	int ret = 0;
	while (m_bRecord)
	{
		av_packet_unref(&packet);
		ret = av_read_frame(m_pFormatCtx_AudioInner, &packet);
		if (ret < 0)
		{
			//LOG_INFO("AudioInnerCapture, av_read_frame, ret is %d", ret);
			continue;
		}

		ret = avcodec_send_packet(m_pReadCodecCtx_AudioInner, &packet);
		if (ret >= 0)
		{
			ret = avcodec_receive_frame(m_pReadCodecCtx_AudioInner, pFrame);
			if (ret == AVERROR(EAGAIN))
			{
				//LOG_INFO("AudioInnerCapture, avcodec_receive_frame, ret is %d", ret);
				continue;
			}
			else if (ret == AVERROR_EOF)
			{
				break;
			}
			else if (ret < 0) {
				LOG_ERROR("avcodec_receive_frame in AudioInnerCapture error");
				break;
			}

			int buf_space = av_audio_fifo_space(m_pAudioInnerFifo);
			if (buf_space >= pFrame->nb_samples)
			{
				//AudioSection
				EnterCriticalSection(&m_csAudioInnerSection);
				ret = av_audio_fifo_write(m_pAudioInnerFifo, (void **)pFrame->data, pFrame->nb_samples);
				LeaveCriticalSection(&m_csAudioInnerSection);
			}



			av_packet_unref(&packet);
		}

	}

	LOG_INFO("AudioInnerCapture end");
	av_frame_free(&pFrame);
}

DWORD WINAPI ULinkRecord::AudioInnerResampleProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->AudioInnerResample();
	}
	return 0;
}

void ULinkRecord::AudioInnerResample()
{
	int ret = 0;

	while (1)
	{
		if (av_audio_fifo_size(m_pAudioInnerFifo) >=
			(m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024))
		{
			AVFrame *frame_audio_inner = NULL;
			frame_audio_inner = av_frame_alloc();

			frame_audio_inner->nb_samples = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024;
			frame_audio_inner->channel_layout = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->channel_layout;
			frame_audio_inner->format = m_pFormatCtx_AudioInner->streams[0]->codecpar->format;
			frame_audio_inner->sample_rate = m_pFormatCtx_AudioInner->streams[0]->codecpar->sample_rate;
			av_frame_get_buffer(frame_audio_inner, 0);

			EnterCriticalSection(&m_csAudioInnerSection);
			int readcount = av_audio_fifo_read(m_pAudioInnerFifo, (void **)frame_audio_inner->data,
				(m_pFormatCtx_AudioInner->streams[0]->codecpar->frame_size > 0 ? m_pFormatCtx_AudioInner->streams[0]->codecpar->frame_size : 1024));
			LeaveCriticalSection(&m_csAudioInnerSection);

			AVFrame *frame_audio_inner_resample = NULL;
			frame_audio_inner_resample = av_frame_alloc();

			//int iDelaySamples = swr_get_delay(audio_convert_ctx, frame_mic->sample_rate);
			int iDelaySamples = 0;
			//int dst_nb_samples = av_rescale_rnd(iDelaySamples + frame_mic->nb_samples, frame_mic->sample_rate, pCodecEncodeCtx_Audio->sample_rate, AVRounding(1));
			int dst_nb_samples = av_rescale_rnd(iDelaySamples + frame_audio_inner->nb_samples, m_pCodecEncodeCtx_Audio->sample_rate, frame_audio_inner->sample_rate, AV_ROUND_UP);


			frame_audio_inner_resample->nb_samples = m_pCodecEncodeCtx_Audio->frame_size;
			frame_audio_inner_resample->channel_layout = m_pCodecEncodeCtx_Audio->channel_layout;
			frame_audio_inner_resample->format = m_pFormatCtx_AudioInner->streams[0]->codecpar->format;
			frame_audio_inner_resample->sample_rate = m_pCodecEncodeCtx_Audio->sample_rate;
			av_frame_get_buffer(frame_audio_inner_resample, 0);

			uint8_t* out_buffer = (uint8_t*)frame_audio_inner_resample->data[0];

			int nb = swr_convert(m_pAudioInnerResampleCtx, &out_buffer, dst_nb_samples, (const uint8_t**)frame_audio_inner->data, frame_audio_inner->nb_samples);



			//if (av_audio_fifo_space(fifo_audio_resample) >= pFrame->nb_samples)
			{
				EnterCriticalSection(&m_csAudioInnerResampleSection);
				ret = av_audio_fifo_write(m_pAudioInnerResampleFifo, (void **)frame_audio_inner_resample->data, dst_nb_samples);
				LeaveCriticalSection(&m_csAudioInnerResampleSection);
			}

			av_frame_free(&frame_audio_inner);
			av_frame_free(&frame_audio_inner_resample);

			if (!m_bRecord)
			{
				if (av_audio_fifo_size(m_pAudioInnerFifo) < 1024)
				{
					break;
				}
			}
		}
		else
		{
			Sleep(1);
			if (!m_bRecord)
			{
				break;
			}
		}
	}
}

DWORD WINAPI ULinkRecord::AudioMicCaptureProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->AudioMicCapture();
	}
	return 0;
}

void ULinkRecord::AudioMicCapture()
{
	AVFrame *pFrame;
	pFrame = av_frame_alloc();

	AVPacket packet = { 0 };
	int ret = 0;
	while (m_bRecord)
	{
		av_packet_unref(&packet);
		if (av_read_frame(m_pFormatCtx_AudioMic, &packet) < 0)
		{
			continue;
		}

		ret = avcodec_send_packet(m_pReadCodecCtx_AudioMic, &packet);
		if (ret >= 0)
		{
			ret = avcodec_receive_frame(m_pReadCodecCtx_AudioMic, pFrame);
			if (ret == AVERROR(EAGAIN))
			{
				continue;
			}
			else if (ret == AVERROR_EOF)
			{
				break;
			}
			else if (ret < 0) {
				LOG_ERROR("avcodec_receive_frame error, ret is %d", ret);
				break;
			}

			if (m_bMute)
			{
				av_samples_set_silence(pFrame->data, 0, pFrame->nb_samples, pFrame->channels, (AVSampleFormat)pFrame->format);
			}
			int buf_space = av_audio_fifo_space(m_pAudioMicFifo);
			if (buf_space >= pFrame->nb_samples)
			{
				EnterCriticalSection(&m_csAudioMicSection);
				ret = av_audio_fifo_write(m_pAudioMicFifo, (void **)pFrame->data, pFrame->nb_samples);
				LeaveCriticalSection(&m_csAudioMicSection);
			}



			av_packet_unref(&packet);
		}

	}

	av_frame_free(&pFrame);

	LOG_INFO("AudioMicCapture end");
}


DWORD WINAPI ULinkRecord::AudioMixProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->AudioMix();
	}
	return 0;
}

void ULinkRecord::AudioMix()
{
	int ret = 0;

	int iAudioFrameMixedIndex = 0;

	AVFrame *frame_audio_inner_resample = NULL;
	frame_audio_inner_resample = av_frame_alloc();

	AVFrame *frame_audio_mic = NULL;
	frame_audio_mic = av_frame_alloc();


	while (m_bRecord)
	{
		if (NULL == m_pAudioInnerResampleFifo)
		{
			continue;
		}
		if (NULL == m_pAudioMicFifo)
		{
			continue;
		}

		int fifo_inner_resample_size = av_audio_fifo_size(m_pAudioInnerResampleFifo);
		int fifo_mic_size = av_audio_fifo_size(m_pAudioMicFifo);
		//int frame_inner_min_size = pReadCodecCtx_Audio->frame_size;
		//int frame_mic_min_size = pReadCodecCtx_AudioMic->frame_size;

		int frame_inner_resample_min_size = 1024;
		int frame_mic_min_size = 1024;

		if (fifo_inner_resample_size >= frame_inner_resample_min_size && fifo_mic_size >= frame_mic_min_size)
		{
			frame_audio_inner_resample->nb_samples = frame_inner_resample_min_size;
			frame_audio_inner_resample->channel_layout = 3;
			frame_audio_inner_resample->format = m_pFormatCtx_AudioInner->streams[0]->codecpar->format;
			frame_audio_inner_resample->sample_rate = m_pFormatCtx_AudioMic->streams[0]->codecpar->sample_rate;
			av_frame_get_buffer(frame_audio_inner_resample, 0);

			frame_audio_mic->nb_samples = frame_mic_min_size;
			frame_audio_mic->channel_layout = 3;
			frame_audio_mic->format = m_pFormatCtx_AudioMic->streams[0]->codecpar->format;
			frame_audio_mic->sample_rate = m_pFormatCtx_AudioMic->streams[0]->codecpar->sample_rate;
			av_frame_get_buffer(frame_audio_mic, 0);

			EnterCriticalSection(&m_csAudioInnerResampleSection);
			int readcount = av_audio_fifo_read(m_pAudioInnerResampleFifo, (void **)frame_audio_inner_resample->data, frame_inner_resample_min_size);
			LeaveCriticalSection(&m_csAudioInnerResampleSection);


			EnterCriticalSection(&m_csAudioMicSection);
			readcount = av_audio_fifo_read(m_pAudioMicFifo, (void**)frame_audio_mic->data, frame_mic_min_size);
			LeaveCriticalSection(&m_csAudioMicSection);


			frame_audio_inner_resample->pts = iAudioFrameMixedIndex * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;
			frame_audio_mic->pts = iAudioFrameMixedIndex * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;

			BufferSourceContext* s = (BufferSourceContext*)m_pFilterCtxSrcInner->priv;
			bool b1 = (s->sample_fmt != frame_audio_inner_resample->format);
			bool b2 = (s->sample_rate != frame_audio_inner_resample->sample_rate);
			bool b3 = (s->channel_layout != frame_audio_inner_resample->channel_layout);
			bool b4 = (s->channels != frame_audio_inner_resample->channels);

			ret = av_buffersrc_add_frame(m_pFilterCtxSrcInner, frame_audio_inner_resample);
			if (ret < 0)
			{
				LOG_ERROR("Mixer: failed to call av_buffersrc_add_frame (inner)");
				break;
			}

			ret = av_buffersrc_add_frame(m_pFilterCtxSrcMic, frame_audio_mic);
			if (ret < 0)
			{
				LOG_ERROR("Mixer: failed to call av_buffersrc_add_frame (microphone)");
				break;
			}


			while (1)
			{
				AVFrame* pFrame_out = av_frame_alloc();

				ret = av_buffersink_get_frame_flags(m_pFilterCtxSink, pFrame_out, 0);
				if (ret < 0)
				{
					av_frame_free(&pFrame_out);
					//printf("Mixer: failed to call av_buffersink_get_frame_flags\n");
					break;
				}
				iAudioFrameMixedIndex++;
				EnterCriticalSection(&m_csAudioMixSection);
				ret = av_audio_fifo_write(m_pAudioMixFifo, (void **)pFrame_out->data, pFrame_out->nb_samples);
				LeaveCriticalSection(&m_csAudioMixSection);

				av_frame_free(&pFrame_out);
			}
		}
		else
		{
			Sleep(1);
		}
	}


	av_frame_free(&frame_audio_inner_resample);
	av_frame_free(&frame_audio_mic);
}


DWORD WINAPI ULinkRecord::ScreenCaptureProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->ScreenCapture();
	}
	return 0;
}


void ULinkRecord::ScreenCapture()
{
	CCaptureScreen* ccs = new CCaptureScreen();

	ccs->Init(m_iRecordPosX, m_iRecordPosY, m_iRecordWidth, m_iRecordHeight);

	AVFrame *pFrameYUV = av_frame_alloc();
	pFrameYUV->format = AV_PIX_FMT_YUV420P;
	pFrameYUV->width = m_iRecordWidth;
	pFrameYUV->height = m_iRecordHeight;

	int y_size = m_iRecordWidth * m_iRecordHeight;

	int frame_size = av_image_get_buffer_size(AV_PIX_FMT_YUV420P, m_iRecordWidth, m_iRecordHeight, 1);
	BYTE* out_buffer_yuv420 = new BYTE[frame_size];
	av_image_fill_arrays(pFrameYUV->data, pFrameYUV->linesize, out_buffer_yuv420, AV_PIX_FMT_YUV420P, m_iRecordWidth, m_iRecordHeight, 1);

	AVPacket* packet = av_packet_alloc();
	m_iFrameNumber = 0;

	int ret = 0;

	DWORD dwBeginTime = ::GetTickCount();
	while(m_bRecord)
	{
		BYTE* frameimage = ccs->CaptureImage();

		RGB24_TO_YUV420(frameimage, m_iRecordWidth, m_iRecordHeight, out_buffer_yuv420);
		//Sframe->pkt_dts = frame->pts = frameNumber * avCodecCtx_Out->time_base.num * avStream->time_base.den / (avCodecCtx_Out->time_base.den *  avStream->time_base.num);
		DWORD dwCurrTime = ::GetTickCount();
		int dts = dwCurrTime - dwBeginTime;

		AVRational millBase;
		millBase.den = 1000;
		millBase.num = 1;

		pFrameYUV->pkt_dts = pFrameYUV->pts = av_rescale_q_rnd(dts, millBase, m_pFormatCtx_Out->streams[m_iVideoStreamIndex]->time_base, (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
		//pFrameYUV->pkt_dts = pFrameYUV->pts = av_rescale_q_rnd(m_iFrameNumber, m_pCodecEncodeCtx_Video->time_base, m_pFormatCtx_Out->streams[m_iVideoStreamIndex]->time_base, (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
		pFrameYUV->pkt_duration = 0;
		pFrameYUV->pkt_pos = -1;

		if (av_fifo_space(m_pVideoFifo) >= frame_size)
		{
			EnterCriticalSection(&m_csVideoSection);
			av_fifo_generic_write(m_pVideoFifo, pFrameYUV->data[0], y_size, NULL);
			av_fifo_generic_write(m_pVideoFifo, pFrameYUV->data[1], y_size / 4, NULL);
			av_fifo_generic_write(m_pVideoFifo, pFrameYUV->data[2], y_size / 4, NULL);
			LeaveCriticalSection(&m_csVideoSection);

			m_iFrameNumber++;

			DWORD dwCurrentTime = ::GetTickCount();
			int dwPassedMillSeconds = dwCurrentTime - dwBeginTime;
			int dwDiff = m_iFrameNumber * 1000 / m_iFrameRate - dwPassedMillSeconds;
			if (dwDiff > 0)
			{
				Sleep(dwDiff);
			}
		}
	}
	av_packet_free(&packet);
	av_frame_free(&pFrameYUV);
	delete[] out_buffer_yuv420;
	delete ccs;
}


DWORD WINAPI ULinkRecord::ScreenAudioMixProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->ScreenAudioMix();
	}
	return 0;
}

void ULinkRecord::ScreenAudioMix()
{
	int ret = 0;
	int cur_pts_v = 0;
	int cur_pts_a = 0;

	AVFrame *pFrameYUVInMain = av_frame_alloc();
	uint8_t *out_buffer_yuv420 = (uint8_t *)av_malloc(m_iYuv420FrameSize);
	av_image_fill_arrays(pFrameYUVInMain->data, pFrameYUVInMain->linesize, out_buffer_yuv420, AV_PIX_FMT_YUV420P, m_iRecordWidth, m_iRecordHeight, 1);

	int AudioFrameIndex_mic = 1;
	AVPacket packet = { 0 };

	int iPicCount = 0;
	while (m_bRecord)
	{
		if (NULL == m_pVideoFifo)
		{
			continue;
		}

		if (NULL == m_pAudioMixFifo)
		{
			continue;
		}

		/*if (!m_bRecord)
		{
			///已经停止时，需要保证队列里面的已经录制的都写进去
			int iVideoSize = av_fifo_size(m_pVideoFifo);
			int iAudioSize = av_audio_fifo_size(m_pAudioMixFifo);

			if (iVideoSize < m_iYuv420FrameSize && iAudioSize < m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size)
			{
				break;
			}
		}*/

		if (av_compare_ts(cur_pts_v, m_pFormatCtx_Out->streams[m_iVideoStreamIndex]->time_base,
			cur_pts_a, m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->time_base) <= 0)
		{
			if (av_fifo_size(m_pVideoFifo) >= m_iYuv420FrameSize)
			{
				EnterCriticalSection(&m_csVideoSection);
				av_fifo_generic_read(m_pVideoFifo, out_buffer_yuv420, m_iYuv420FrameSize, NULL);
				LeaveCriticalSection(&m_csVideoSection);

				pFrameYUVInMain->pkt_dts = pFrameYUVInMain->pts = av_rescale_q_rnd(iPicCount, m_pCodecEncodeCtx_Video->time_base, m_pFormatCtx_Out->streams[m_iVideoStreamIndex]->time_base, (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
				pFrameYUVInMain->pkt_duration = 0;
				pFrameYUVInMain->pkt_pos = -1;

				pFrameYUVInMain->width = m_iRecordWidth;
				pFrameYUVInMain->height = m_iRecordHeight;
				pFrameYUVInMain->format = AV_PIX_FMT_YUV420P;

				cur_pts_v = packet.pts;

				ret = avcodec_send_frame(m_pCodecEncodeCtx_Video, pFrameYUVInMain);

				ret = avcodec_receive_packet(m_pCodecEncodeCtx_Video, &packet);

				if (packet.size > 0)
				{
					//av_packet_rescale_ts(packet, avCodecCtx_Out->time_base, avStream->time_base);
					av_write_frame(m_pFormatCtx_Out, &packet);
					iPicCount++;
				}
			}
			else
			{
				Sleep(1);
			}
		}
		else
		{
			if (av_audio_fifo_size(m_pAudioMixFifo) >=
				(m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024))
			{
				AVFrame *frame_mix = NULL;
				frame_mix = av_frame_alloc();

				frame_mix->nb_samples = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024;
				frame_mix->channel_layout = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->channel_layout;
				//frame_mix->format = pFormatCtx_Out->streams[iAudioStreamIndex]->codecpar->format;
				frame_mix->format = 1;
				frame_mix->sample_rate = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->sample_rate;
				av_frame_get_buffer(frame_mix, 0);

				EnterCriticalSection(&m_csAudioMixSection);
				int readcount = av_audio_fifo_read(m_pAudioMixFifo, (void **)frame_mix->data,
					(m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024));
				LeaveCriticalSection(&m_csAudioMixSection);

				AVPacket pkt_out_mic = { 0 };

				pkt_out_mic.data = NULL;
				pkt_out_mic.size = 0;

				frame_mix->pts = AudioFrameIndex_mic * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;


				AVFrame *frame_mic_encode = NULL;
				frame_mic_encode = av_frame_alloc();

				frame_mic_encode->nb_samples = m_pCodecEncodeCtx_Audio->frame_size;
				frame_mic_encode->channel_layout = m_pCodecEncodeCtx_Audio->channel_layout;
				frame_mic_encode->format = m_pCodecEncodeCtx_Audio->sample_fmt;
				frame_mic_encode->sample_rate = m_pCodecEncodeCtx_Audio->sample_rate;
				av_frame_get_buffer(frame_mic_encode, 0);


				int iDelaySamples = 0;
				int dst_nb_samples = av_rescale_rnd(iDelaySamples + frame_mix->nb_samples, frame_mix->sample_rate, frame_mix->sample_rate, AVRounding(1));

				//uint8_t *audio_buf = NULL;
				uint8_t *audio_buf[2] = { 0 };
				audio_buf[0] = (uint8_t *)frame_mic_encode->data[0];
				audio_buf[1] = (uint8_t *)frame_mic_encode->data[1];

				int nb = swr_convert(m_pAudioConvertCtx, audio_buf, dst_nb_samples, (const uint8_t**)frame_mix->data, frame_mix->nb_samples);

				ret = avcodec_send_frame(m_pCodecEncodeCtx_Audio, frame_mic_encode);

				ret = avcodec_receive_packet(m_pCodecEncodeCtx_Audio, &pkt_out_mic);
				if (ret == AVERROR(EAGAIN))
				{
					continue;
				}
				av_frame_free(&frame_mix);
				av_frame_free(&frame_mic_encode);
				{
					pkt_out_mic.stream_index = m_iAudioStreamIndex;
					pkt_out_mic.pts = AudioFrameIndex_mic * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;
					pkt_out_mic.dts = AudioFrameIndex_mic * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;
					pkt_out_mic.duration = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;

					cur_pts_a = pkt_out_mic.pts;
					av_write_frame(m_pFormatCtx_Out, &pkt_out_mic);
					//int ret2 = av_interleaved_write_frame(m_pFormatCtx_Out, &pkt_out_mic);
					av_packet_unref(&pkt_out_mic);
				}
				AudioFrameIndex_mic++;
			}
			else
			{
				Sleep(1);
			}
		}
	}


	Sleep(100);
	av_write_trailer(m_pFormatCtx_Out);

	avio_close(m_pFormatCtx_Out->pb);

	av_frame_free(&pFrameYUVInMain);
}


DWORD WINAPI ULinkRecord::ScreenWriteProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->ScreenWrite();
	}
	return 0;
}

void ULinkRecord::ScreenWrite()
{
	int ret = 0;

	AVFrame *pFrameYUVInMain = av_frame_alloc();
	uint8_t *out_buffer_yuv420 = (uint8_t *)av_malloc(m_iYuv420FrameSize);
	av_image_fill_arrays(pFrameYUVInMain->data, pFrameYUVInMain->linesize, out_buffer_yuv420, AV_PIX_FMT_YUV420P, m_iRecordWidth, m_iRecordHeight, 1);

	AVPacket packet = { 0 };

	int iPicCount = 0;
	DWORD dwBeginTime = ::GetTickCount();
	while (m_bRecord)
	{
		if (NULL == m_pVideoFifo)
		{
			continue;
		}

		if (av_fifo_size(m_pVideoFifo) >= m_iYuv420FrameSize)
		{
			EnterCriticalSection(&m_csVideoSection);
			av_fifo_generic_read(m_pVideoFifo, out_buffer_yuv420, m_iYuv420FrameSize, NULL);
			LeaveCriticalSection(&m_csVideoSection);

			DWORD dwCurrTime = ::GetTickCount();
			int dts = dwCurrTime - dwBeginTime;

			AVRational millBase;
			millBase.den = 1000;
			millBase.num = 1;

			pFrameYUVInMain->pkt_dts = pFrameYUVInMain->pts = av_rescale_q_rnd(dts, millBase, m_pFormatCtx_Out->streams[m_iVideoStreamIndex]->time_base, (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
			//pFrameYUVInMain->pkt_dts = pFrameYUVInMain->pts = av_rescale_q_rnd(iPicCount, m_pCodecEncodeCtx_Video->time_base, m_pFormatCtx_Out->streams[m_iVideoStreamIndex]->time_base, (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
			pFrameYUVInMain->pkt_duration = 0;
			pFrameYUVInMain->pkt_pos = -1;

			pFrameYUVInMain->width = m_iRecordWidth;
			pFrameYUVInMain->height = m_iRecordHeight;
			pFrameYUVInMain->format = AV_PIX_FMT_YUV420P;

			ret = avcodec_send_frame(m_pCodecEncodeCtx_Video, pFrameYUVInMain);

			ret = avcodec_receive_packet(m_pCodecEncodeCtx_Video, &packet);

			if (packet.size > 0)
			{
				//av_packet_rescale_ts(packet, avCodecCtx_Out->time_base, avStream->time_base);
				EnterCriticalSection(&m_csPacketWriteSection);
				av_write_frame(m_pFormatCtx_Out, &packet);
				LeaveCriticalSection(&m_csPacketWriteSection);
				iPicCount++;
			}
		}
		else
		{
			Sleep(1);
		}
		
	}


	Sleep(100);

	av_frame_free(&pFrameYUVInMain);
}

DWORD WINAPI ULinkRecord::AudioWriteProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->AudioWrite();
	}
	return 0;
}

void ULinkRecord::AudioWrite()
{
	int ret = 0;


	int AudioFrameIndex_mic = 1;
	AVPacket packet = { 0 };
	while (m_bRecord)
	{
		if (NULL == m_pAudioMixFifo)
		{
			continue;
		}

		if (av_audio_fifo_size(m_pAudioMixFifo) >=
			(m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024))
		{
			AVFrame *frame_mix = NULL;
			frame_mix = av_frame_alloc();

			frame_mix->nb_samples = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024;
			frame_mix->channel_layout = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->channel_layout;
			//frame_mix->format = pFormatCtx_Out->streams[iAudioStreamIndex]->codecpar->format;
			frame_mix->format = 1;
			frame_mix->sample_rate = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->sample_rate;
			av_frame_get_buffer(frame_mix, 0);

			EnterCriticalSection(&m_csAudioMixSection);
			int readcount = av_audio_fifo_read(m_pAudioMixFifo, (void **)frame_mix->data,
				(m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024));
			LeaveCriticalSection(&m_csAudioMixSection);

			AVPacket pkt_out_mic = { 0 };

			pkt_out_mic.data = NULL;
			pkt_out_mic.size = 0;

			frame_mix->pts = AudioFrameIndex_mic * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;


			AVFrame *frame_mic_encode = NULL;
			frame_mic_encode = av_frame_alloc();

			frame_mic_encode->nb_samples = m_pCodecEncodeCtx_Audio->frame_size;
			frame_mic_encode->channel_layout = m_pCodecEncodeCtx_Audio->channel_layout;
			frame_mic_encode->format = m_pCodecEncodeCtx_Audio->sample_fmt;
			frame_mic_encode->sample_rate = m_pCodecEncodeCtx_Audio->sample_rate;
			av_frame_get_buffer(frame_mic_encode, 0);


			int iDelaySamples = 0;
			int dst_nb_samples = av_rescale_rnd(iDelaySamples + frame_mix->nb_samples, frame_mix->sample_rate, frame_mix->sample_rate, AVRounding(1));

			//uint8_t *audio_buf = NULL;
			uint8_t *audio_buf[2] = { 0 };
			audio_buf[0] = (uint8_t *)frame_mic_encode->data[0];
			audio_buf[1] = (uint8_t *)frame_mic_encode->data[1];

			int nb = swr_convert(m_pAudioConvertCtx, audio_buf, dst_nb_samples, (const uint8_t**)frame_mix->data, frame_mix->nb_samples);

			ret = avcodec_send_frame(m_pCodecEncodeCtx_Audio, frame_mic_encode);

			ret = avcodec_receive_packet(m_pCodecEncodeCtx_Audio, &pkt_out_mic);
			if (ret == AVERROR(EAGAIN))
			{
				continue;
			}
			av_frame_free(&frame_mix);
			av_frame_free(&frame_mic_encode);
			{
				pkt_out_mic.stream_index = m_iAudioStreamIndex;
				pkt_out_mic.pts = AudioFrameIndex_mic * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;
				pkt_out_mic.dts = AudioFrameIndex_mic * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;
				pkt_out_mic.duration = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;

				EnterCriticalSection(&m_csPacketWriteSection);
				av_write_frame(m_pFormatCtx_Out, &pkt_out_mic);
				LeaveCriticalSection(&m_csPacketWriteSection);
				//int ret2 = av_interleaved_write_frame(m_pFormatCtx_Out, &pkt_out_mic);
				av_packet_unref(&pkt_out_mic);
			}
			AudioFrameIndex_mic++;
		}
		else
		{
			Sleep(1);
		}
	}
}
tusong86
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 5
    评论
ffmpeg录屏并保存为MP4文件
Cesaroy的博客
06-10 1255
利用FFMPEG对桌面进行截屏，并保存成MP4的格式。
c++ Dxgi+FFmpeg 完成对截图屏幕，以及将屏幕数据保存为MP4格式，以完成录屏功能
weixin_43174052的博客
04-18 3650
c++ Dxgi+FFmpeg 完成对截图屏幕，以及将屏幕数据保存为MP4格式，以完成录屏功能
【FFmpeg】音视频录制 ③ ( 使用 ffmpeg 录制 Screen Capturer Recorder 软件生成的虚拟音视频设备 | 录制桌面 | 录制音频 | 同时录制 音频和视频数据 )
最新发布
让 学习 成为一种 习惯 ( 韩曙亮 の 技术博客 )
04-27 5616
一、Screen Capturer Recorder 生成的虚拟音视频设备 1、Screen Capturer Recorder 生成的虚拟音视频设备 2、查看上述虚拟音视频设备 二、使用 ffmpeg 录制 Screen Capturer Recorder 软件生成的虚拟音视频设备 1、录制桌面 2、录制音频 3、录制多路音频 - 配置 filter_complex 混音参数 4、同时录制 系统音频 + 桌面视频 5、同时录制 系统音频 + 麦克风音频 + 桌面视频
FFmpeg 命令：从入门到精通 | ffmpeg 命令视频录制
ProgramNovice的博客
10-02 1659
FFmpeg 命令：从入门到精通 | ffmpeg 命令视频录制
C++音视频开发从放弃到入门 （基于FFmpeg+OpenCV）
西部秋虫的博客
11-12 1万+
音视频开发一定要学C++吗？答案是肯定的。虽然其它语言也能搞音视频开发，甚至使用起来更简单，但“语言越高级，离真相就越远”，当你的功能需求日益增多，程序的性能需求越来越迫切，你想进一步了解程序实现的细节时，使用其它语言往往会面临“无法解决”的困境，最后不得不使用C++来解决问题，我们何不从一开始就使用C++呢？FFmpeg及OpenCV是开源、跨平台的音视频开发SDK，搞音视频开发基本都需要用到它。
ffmpeg录制音频 命令行与C++实现
sheqianweilong的博客
03-12 2129
关于音视频，看了很多的博客，发现都很难正确的打开设备与录制。然后就自己瞎搞，突然发现能正确的录制与打开了，故此记录一下。 一、官方wiki https://trac.ffmpeg.org/wiki/Capture/ALSA 提取一句关于音频录制的命令： ffmpeg -f alsa -channels 1 -sample_rate 44100 -i hw:0 -t 30 out.wav linux选择alsa的设备,mac上是avfoundation,sample_rate指的是采样率，channel
QT+FFMPEG屏幕录制及声音抓取完整代码（包含音视频同步）
05-20
1.文件说明：screenrecord主控界面，控制录制的开始和结束，包含开始/结束按钮；audiocaptureprocess声音录制类；screencaptureprocess屏幕录制类；muxerprocess音视频合并类；myglobals全局参数； 2.使用方法：将...
FFmpeg - 录制windows系统声音（代码方式）
02-08
本资源包含完整的Vs+Qt+FFmpeg代码。 资源介绍：https://blog.csdn.net/automoblie0/article/details/126107825 资源视频介绍：https://www.bilibili.com/video/BV1uG4y1v7ye/?spm_id_from=333.999.0.0 【视频中展示...
FFmpeg - 录制声音生成aac文件
02-08
本资源包含完整的Vs+Qt+FFmpeg+sdl代码。 资源介绍：https://blog.csdn.net/automoblie0/article/details/109079515 资源视频介绍：https://www.bilibili.com/video/BV1nf4y1A7h9/?spm_id_from=333.999.0.0 【视频中...
FFmpeg - 录制Windows屏幕与麦克风(音视频同步)，生成Mp4文件（附源码）
04-16
■ 本源码运行环境为Vs+FFmpeg，主要功能录制Windows屏幕与麦克风(音视频同步)，生成Mp4文件 ■ 本源码对应的文章介绍：https://blog.csdn.net/automoblie0/article/details/126056946 ■ 如需协助，csdn私信或...
FFmpeg - 编码/录制麦克风声音并保存为mp3文件
04-16
■ 本源码运行环境为Vs+FFmpeg，主要功能编码/录制麦克风声音并保存为mp3文件 ■ 本源码对应的文章介绍：https://blog.csdn.net/automoblie0/article/details/126108806 ■ 本源码对应的运行效果介绍：...
音频-麦克风-ffmpeg-aac语音编码VS2010
01-22
从麦克风采集音频，用ffmpeg编码为aac，并保存到本地，要用vs2010打开，音频设备名要改为自己的音频设备名，程序决定可以使用。
ffmpeg命令录制windows音视频
09-26
ffmpeg命令录制windows音视频
ffmpeg开发:打开摄像头和麦克风
MR_DUAN1105的博客
09-03 3079
使用ffmpeg打开电脑的摄像头。用ffmpeg打开摄像头网上雷神有写，不过他的博客把摄像头的地址写死了。当遇到摄像头是英文的描述符时还好说，但是当遇到麦克风或者摄像头是中文描述符就非常坑爹了。当初打开麦克风时，麦克风描述符有中文，用win32的MultiByteToWideChar函数open_input提示打开错误，确实很崩溃。其实当设备有中文 可以使用它的Moniker名，Moniker名
菜鸟学ffmpeg音视频技术之一实现命令录制
xpg4321的博客
11-16 717
ffmpeg就不解释了，太多资料了，著名的开源做音视频的库 学些录制的动力是目前所在公司使用OBS录屏存在一些问题，录制窗口有些显卡会黑屏，录制屏幕会把电脑的其他软件也会录制进去，理想的录制方式是组合录制方式，把想录制的录制进去，不像录制的去掉。比如一个软件中有多个窗口，只想录制其中一些。希望后续能成功立一个flag。 ffmpeg使用的方式有两种，一种是命令模式，一种是api模式，要实现我想要录制应该是api模式能实现，这里我先使用api实现录制，了解一下录制原理。为了方便依旧使用qt来实现。 .
ffmpeg录制桌面，麦克风和系统声音独立成2路音轨
tusong86的博客
12-24 1506
之前本人写过下面一篇博客： ffmpeg 本地麦克风声音和系统声音混音后，再混合本地桌面成最终的mp4文件-修正 里面的麦克风声音和系统声音利用滤镜进行了混音处理，昨天夜晚再想如果麦克风声音和系统声音不混合，各自成一路音轨，最终生成的mp4文件能否播放，是否能达到预期的效果。 结果就是不及预期，生成视频文件2021-12-24T12-09-09.mp4的两路音轨，只能播放序号小的音轨，没法同时播放，我尝试了很多播放器，都是这样，vlc和potplayer做的好一些，可以检测到音轨的数量，并让用户选择播放哪一
ffmpeg- 部分错误码-av_interleaved_write_frame/av_write_frame
shizheng163的博客
05-28 1万+
部分返回值错误码 Linux error_base.h av_interleaved_write_frame/av_write_frame 返回值 如有错误，请指正，谢谢。 部分返回值错误码 ffmpeg源码部分返回值形式为： **AVERROR（EINVAL）** **AVERROR（EPIPE）** **AVERROR（ENOMEM）** ...... 再看一下ffmp...
FFmpeg 录制桌面、麦克风、摄像头
热门推荐
勤能补拙
12-31 1万+
FFmpeg 录制桌面、麦克风、摄像头前言老师要我们试试能不能用手机拍摄视频然后发送到树莓派上。可能以后要然树莓派处理视频之类。老师描述的场景好像实时的。虽然需求不明确，我们就先试试吧。我的计划是在树莓派上搭建好流媒体服务器，然后手机拍摄视频并推送到服务器。但是找了好久也找不到靠谱的可以实时拍摄视频并推送到服务器的 Android 应用。我们也不会 Android 开发。好不容器把树莓派上的 rtm
ffmpeg 录制桌面声卡输出和麦克风输出
06-08
使用 FFmpeg 可以录制桌面声卡输出和麦克风输出。以下是一个简单的使用 FFmpeg 进行录制的命令行示例： ``` ffmpeg -f dshow -i audio="麦克风" -f gdigrab -framerate 30 -i desktop -filter_complex amix=inputs=2 -vcodec libx264 -preset ultrafast -acodec aac output.mp4 ``` 这个命令行会录制桌面和麦克风的声音，并保存为 MP4 格式的文件。其中，`-f dshow -i audio="麦克风"` 表示使用 DirectShow 来录制麦克风的声音；`-f gdigrab -framerate 30 -i desktop` 表示使用 GDI 来录制桌面的视频；`-filter_complex amix=inputs=2` 表示将两个输入的音频混合为一个输出的音频；`-vcodec libx264 -preset ultrafast -acodec aac` 表示使用 H.264 和 AAC 格式来编码视频和音频；`output.mp4` 表示输出的文件名。 需要注意的是，使用 FFmpeg 进行录制可能会涉及到版权问题，需要获得相关内容的授权或者使用自己拍摄的素材。另外，录制时需要注意性能问题，如果硬件配置较低或者录制视频分辨率较高，可能会导致录制过程中出现卡顿或者花屏等问题。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 5
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值