ffmpeg 本地麦克风声音和系统声音混音后，再混合本地桌面成最终的mp4文件-修正

最新推荐文章于 2022-08-29 16:42:15 发布
最新推荐文章于 2022-08-29 16:42:15 发布
阅读量2.1k 收藏 3
点赞数 2
文章标签： 音视频
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/tusong86/article/details/121863390
版权

之前本人写过一篇博客：
ffmpeg 本地麦克风声音和系统声音混音后，再混合本地桌面成最终的mp4文件

但是存在着下面两个问题：
1.系统声音和麦克风对应的设备的采样率不一样，没有进行重采样，比如系统声音设备的采样率是 48000，若不进行重采样，则最终系统声音播放出来，会发现播放变慢。
2.av_read_frame采集的视频图像，在用下面两个函数编码时，avcodec_receive_packet经常返回 AVERROR(EAGAIN)，若单纯写一个桌面录制的功能，在主线程中进行抓图，编码，则大概率发现是正常的，但是如果创建一个子线程，然后在这个线程里面进行抓图，编码，就会发现出错概率很高。
我曾经将ffmpeg命令行在子线程中调用，发现录制1分钟时，生成的视频文件只有2M。

ret = avcodec_send_frame(pCodecEncodeCtx_Video, pFrameYUV);
if (ret == AVERROR(EAGAIN))
{
	continue;
}

ret = avcodec_receive_packet(pCodecEncodeCtx_Video, &packet);
if (ret == AVERROR(EAGAIN))
{
	continue;
}

针对第一个问题：系统声音设备的采样率是48000，麦克风的是44100，最终编码的声音要求是44100，则先将系统声音重采样成44100，再和麦克风的混音即可。

针对第二个问题：估计是自己目前学艺不精的问题，av_read_frame出现了大问题，故本人自己采取gdi抓图，不用ffmpeg的库函数av_read_frame，关于这个的具体细节，读者可以参考我写的一篇博客：ffmpeg录制桌面(自己用gdi抓图)

下面讲讲本人录制桌面视频，系统音频，麦克风音频的大致方法。

m_hAudioInnerCapture = CreateThread(NULL, 0, AudioInnerCaptureProc, this, 0, NULL);
m_hAudioInnerResample = CreateThread(NULL, 0, AudioInnerResampleProc, this, 0, NULL);
m_hAudioMicCapture = CreateThread(NULL, 0, AudioMicCaptureProc, this, 0, NULL);
m_hAudioMix = CreateThread(NULL, 0, AudioMixProc, this, 0, NULL);
m_hScreenCapture = CreateThread(NULL, 0, ScreenCaptureProc, this, 0, NULL);
m_hScreenAudioMix = CreateThread(NULL, 0, ScreenAudioMixProc, this, 0, NULL);

上面代码中，一共创建了6个线程，其中：
m_hAudioInnerCapture代码的是系统声音抓取。
m_hAudioInnerResample代表的是系统声音重采样。
m_hAudioMicCapture代表的是麦克风声音抓取。
m_hAudioMix代表的是混音，是将重采样后的系统声音和麦克风声音混合。
m_hScreenCapture代表的是桌面图像抓取
m_hScreenAudioMix代表的是桌面图像和混合后的音频进行混合，生成最终的mp4文件。

如下所示，本人定义了五个队列，其中m_pVideoFifo代表的是桌面图像，m_hScreenCapture线程往这个队列里面写数据，m_hScreenAudioMix从这个队列里面读取数据。
m_pAudioInnerFifo代表的系统声音，原始的，m_hAudioInnerCapture线程往这个队列里面写数据，m_hAudioInnerResample线程从这个队列里面读取数据，进行重采样，然后将采样后的结果放入队列
m_pAudioInnerResampleFifo。

AVFifoBuffer *m_pVideoFifo = NULL;
	AVAudioFifo *m_pAudioInnerFifo = NULL;
	AVAudioFifo *m_pAudioInnerResampleFifo = NULL;
	AVAudioFifo *m_pAudioMicFifo = NULL;
	AVAudioFifo *m_pAudioMixFifo = NULL;

下面我给出自己的代码结构：
在这里插入图片描述
其中appfun和log两个文件夹，没有具体业务含义，大家不用管。
CaptureScreen.cpp用于gdi抓取桌面图像。
ULinkRecord.cpp里面完成了音视频的抓取，混音，以及声音和视频混合。

FfmpegVideoAudioAndMicOneFileTest.cpp是调用方，其代码如下：

#include "ULinkRecord.h"
#include <stdio.h>
#include <conio.h>








int main()
{
	ULinkRecord cULinkRecord;

	cULinkRecord.SetMicName(L"麦克风 (2- Synaptics HD Audio)");
	cULinkRecord.SetRecordPath("E:\\learn\\ffmpeg\\FfmpegTest\\x64\\Release");

	RECT rect;
	rect.left = 0;
	rect.top = 0;
	rect.right = 1920;
	rect.bottom = 1080;

	cULinkRecord.SetRecordRect(rect);

	cULinkRecord.StartRecord();

	Sleep(60000);

	printf("begin StopRecord\n");
	cULinkRecord.StopRecord();
	printf("end StopRecord\n");
	return 0;
}

可以看出，本次抓取的时长为1分钟。

下面再给出其他4个主要文件的内容，尽管很长，我还是觉得贴出来会比较好。
CaptureScreen.h内容如下：

#ifndef _CCAPTURE_SCREEN_HH
#define _CCAPTURE_SCREEN_HH

#include<time.h>
#include <d3d9.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <windows.h>

#include <tchar.h>
#include <winbase.h>
#include <winreg.h>
#include <Strsafe.h>


//
// ---抓屏类----
//
class CCaptureScreen
{
public:
	CCaptureScreen(void);
	~CCaptureScreen(void);

public:
	/*-----------定义外部调用函数-----------*/
	int Init(int&, int&);//初始化
	BYTE* CaptureImage(); //抓取屏幕

private:
	/*-----------定义内部调用函数-----------*/
	void* CaptureScreenFrame(int, int, int, int);//抓屏
	HCURSOR FetchCursorHandle(); //获取鼠标光标

private:
	/*-----------定义私有变量-----------*/
	int m_width;
	int m_height;
	UINT   wLineLen;
	DWORD  dwSize;
	DWORD  wColSize;

	//设备句柄
	HDC hScreenDC;
	HDC hMemDC;
	//图像RGB内存缓存
	PRGBTRIPLE m_hdib;
	//位图头信息结构体
	BITMAPINFO pbi;

	HBITMAP hbm;
	//鼠标光标
	HCURSOR m_hSavedCursor;


};

#endif //--_CCAPTURE_SCREEN_HH

CaptureScreen.cpp内容如下：

//#include "stdafx.h"
#include "CaptureScreen.h"

CCaptureScreen::CCaptureScreen(void)
{
	m_hdib = NULL;
	m_hSavedCursor = NULL;
	hScreenDC = NULL;
	hMemDC = NULL;
	hbm = NULL;
	m_width = 1920;
	m_height = 1080;
	FetchCursorHandle();
}
//
// 释放资源
//
CCaptureScreen::~CCaptureScreen(void)
{
	DeleteObject(hbm);
	if (m_hdib){

		free(m_hdib);
		m_hdib = NULL;
	}
	if (hScreenDC){

		::ReleaseDC(NULL, hScreenDC);
	}
	if (hMemDC) {

		DeleteDC(hMemDC);
	}
	if (hbm)
	{
		DeleteObject(hbm);
	}
}

//
// 初始化
//
int CCaptureScreen::Init(int& src_VideoWidth, int& src_VideoHeight)
{
	hScreenDC = ::GetDC(GetDesktopWindow());
	if (hScreenDC == NULL) return 0;

	int m_nMaxxScreen = GetDeviceCaps(hScreenDC, HORZRES);
	int m_nMaxyScreen = GetDeviceCaps(hScreenDC, VERTRES);

	hMemDC = ::CreateCompatibleDC(hScreenDC);
	if (hMemDC == NULL) return 0;

	m_width = m_nMaxxScreen;
	m_height = m_nMaxyScreen;

	if (!m_hdib){
		m_hdib = (PRGBTRIPLE)malloc(m_width * m_height * 3);//24位图像大小
	}
	//位图头信息结构体
	pbi.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
	pbi.bmiHeader.biWidth = m_width;
	pbi.bmiHeader.biHeight = m_height;
	pbi.bmiHeader.biPlanes = 1;
	pbi.bmiHeader.biBitCount = 24;
	pbi.bmiHeader.biCompression = BI_RGB;

	src_VideoWidth = m_width;
	src_VideoHeight = m_height;

	hbm = CreateCompatibleBitmap(hScreenDC, m_width, m_height);
	SelectObject(hMemDC, hbm);

	wLineLen = ((m_width * 24 + 31) & 0xffffffe0) / 8;
	wColSize = sizeof(RGBQUAD)* ((24 <= 8) ? 1 << 24 : 0);
	dwSize = (DWORD)(UINT)wLineLen * (DWORD)(UINT)m_height;

	return 1;
}

//抓取屏幕数据
BYTE* CCaptureScreen::CaptureImage()
{

	VOID*  alpbi = CaptureScreenFrame(0, 0, m_width, m_height);
	return (BYTE*)(alpbi);
}

void* CCaptureScreen::CaptureScreenFrame(int left, int top, int width, int height)
{

	if (hbm == NULL || hMemDC == NULL || hScreenDC == NULL) return NULL;

	BitBlt(hMemDC, 0, 0, width, height, hScreenDC, left, top, SRCCOPY);
	/*-------------------------捕获鼠标-------------------------------*/
	{
		POINT xPoint;
		GetCursorPos(&xPoint);
		HCURSOR hcur = FetchCursorHandle();
		xPoint.x -= left;
		xPoint.y -= top;

		ICONINFO iconinfo;
		BOOL ret;
		ret = GetIconInfo(hcur, &iconinfo);
		if (ret){
			xPoint.x -= iconinfo.xHotspot;
			xPoint.y -= iconinfo.yHotspot;

			if (iconinfo.hbmMask) DeleteObject(iconinfo.hbmMask);
			if (iconinfo.hbmColor) DeleteObject(iconinfo.hbmColor);
		}
		/*画鼠标*/
		::DrawIcon(hMemDC, xPoint.x, xPoint.y, hcur);
	}
	//动态分配的内存
	PRGBTRIPLE hdib = m_hdib;
	if (!hdib)
		return hdib;

	GetDIBits(hMemDC, hbm, 0, m_height, hdib, (LPBITMAPINFO)&pbi, DIB_RGB_COLORS);
	return hdib;
}

//
// 获取窗体鼠标光标
//
HCURSOR CCaptureScreen::FetchCursorHandle()
{
	if (m_hSavedCursor == NULL)
	{
		m_hSavedCursor = GetCursor();
	}
	return m_hSavedCursor;
}

ULinkRecord.h的内容如下：

#pragma once

#include <string>
#include <Windows.h>

#ifdef	__cplusplus
extern "C"
{
#endif
#include "libavcodec/avcodec.h"
#include "libavformat/avformat.h"
#include "libswscale/swscale.h"
#include "libswresample/swresample.h"
#include "libavdevice/avdevice.h"
#include "libavutil/audio_fifo.h"
#include "libavutil/avutil.h"
#include "libavutil/fifo.h"
#include "libavutil/frame.h"
#include "libavutil/imgutils.h"

#include "libavfilter/avfilter.h"
#include "libavfilter/buffersink.h"
#include "libavfilter/buffersrc.h"


#pragma comment(lib, "avcodec.lib")
#pragma comment(lib, "avformat.lib")
#pragma comment(lib, "avutil.lib")
#pragma comment(lib, "avdevice.lib")
#pragma comment(lib, "avfilter.lib")
#pragma comment(lib, "postproc.lib")
#pragma comment(lib, "swresample.lib")
#pragma comment(lib, "swscale.lib")


#ifdef __cplusplus
};
#endif

class ULinkRecord
{
public:
	ULinkRecord();
	~ULinkRecord();
public:
	void SetMicName(const wchar_t* pMicName);
	void SetRecordPath(const char* pRecordPath);
	void SetRecordRect(RECT rectRecord);
	int StartRecord();
	void StopRecord();
private:
	int OpenAudioInnerCapture();
	int OpenAudioMicCapture();
	int OpenOutPut();
	int InitFilter(const char* filter_desc);
	void Clear();
private:
	static DWORD WINAPI AudioInnerCaptureProc(LPVOID lpParam);
	void AudioInnerCapture();

	static DWORD WINAPI AudioInnerResampleProc(LPVOID lpParam);
	void AudioInnerResample();

	static DWORD WINAPI AudioMicCaptureProc(LPVOID lpParam);
	void AudioMicCapture();

	static DWORD WINAPI AudioMixProc(LPVOID lpParam);
	void AudioMix();

	static DWORD WINAPI ScreenCaptureProc(LPVOID lpParam);
	void ScreenCapture();

	static DWORD WINAPI ScreenAudioMixProc(LPVOID lpParam);
	void ScreenAudioMix();
private:
	std::wstring m_wstrMicName;
	std::string m_strRecordPath;
	std::string m_strFilePrefix;
private:
	CRITICAL_SECTION m_csVideoSection;
	CRITICAL_SECTION m_csAudioInnerSection;
	CRITICAL_SECTION m_csAudioInnerResampleSection;
	CRITICAL_SECTION m_csAudioMicSection;
	CRITICAL_SECTION m_csAudioMixSection;

	AVFifoBuffer *m_pVideoFifo = NULL;
	AVAudioFifo *m_pAudioInnerFifo = NULL;
	AVAudioFifo *m_pAudioInnerResampleFifo = NULL;
	AVAudioFifo *m_pAudioMicFifo = NULL;
	AVAudioFifo *m_pAudioMixFifo = NULL;

	AVFormatContext *m_pFormatCtx_Out = NULL;
	AVFormatContext	*m_pFormatCtx_AudioInner = NULL;
	AVFormatContext	*m_pFormatCtx_AudioMic = NULL;

	AVCodecContext *m_pReadCodecCtx_AudioInner = NULL;
	AVCodecContext *m_pReadCodecCtx_AudioMic = NULL;
	AVCodec *m_pReadCodec_Video = NULL;

	AVCodecContext	*m_pCodecEncodeCtx_Video = NULL;
	AVCodecContext	*m_pCodecEncodeCtx_Audio = NULL;
	AVCodec			*m_pCodecEncode_Audio = NULL;

	SwsContext *m_pImgConvertCtx = NULL;
	SwrContext *m_pAudioInnerResampleCtx = NULL;
	SwrContext *m_pAudioConvertCtx = NULL;


	AVFilterGraph* m_pFilterGraph = NULL;
	AVFilterContext* m_pFilterCtxSrcInner = NULL;
	AVFilterContext* m_pFilterCtxSrcMic = NULL;
	AVFilterContext* m_pFilterCtxSink = NULL;

	int m_iVideoStreamIndex = 0;
	int m_iAudioStreamIndex = 0;
	bool m_bRecord;

	HANDLE m_hAudioInnerCapture = NULL;
	HANDLE m_hAudioInnerResample = NULL;
	HANDLE m_hAudioMicCapture = NULL;
	HANDLE m_hAudioMix = NULL;
	HANDLE m_hScreenCapture = NULL;
	HANDLE m_hScreenAudioMix = NULL;

	int m_iYuv420FrameSize = 0;

	int m_iRecordPosX = 0;
	int m_iRecordPosY = 0;
	int m_iRecordWidth = 0;
	int m_iRecordHeight = 0;

	int m_iFrameNumber = 0;
};

ULinkRecord.cpp的内容如下：

#include "ULinkRecord.h"
#include "log/log.h"
#include "appfun/appfun.h"

#include "LocalRecord.h"
#include "CaptureScreen.h"



typedef struct BufferSourceContext {
	const AVClass    *bscclass;
	AVFifoBuffer     *fifo;
	AVRational        time_base;     ///< time_base to set in the output link
	AVRational        frame_rate;    ///< frame_rate to set in the output link
	unsigned          nb_failed_requests;
	unsigned          warning_limit;

	/* video only */
	int               w, h;
	enum AVPixelFormat  pix_fmt;
	AVRational        pixel_aspect;
	char              *sws_param;

	AVBufferRef *hw_frames_ctx;

	/* audio only */
	int sample_rate;
	enum AVSampleFormat sample_fmt;
	int channels;
	uint64_t channel_layout;
	char    *channel_layout_str;

	int got_format_from_params;
	int eof;
} BufferSourceContext;


static char *dup_wchar_to_utf8(const wchar_t *w)
{
	char *s = NULL;
	int l = WideCharToMultiByte(CP_UTF8, 0, w, -1, 0, 0, 0, 0);
	s = (char *)av_malloc(l);
	if (s)
		WideCharToMultiByte(CP_UTF8, 0, w, -1, s, l, 0, 0);
	return s;
}


/* just pick the highest supported samplerate */
static int select_sample_rate(const AVCodec *codec)
{
	const int *p;
	int best_samplerate = 0;

	if (!codec->supported_samplerates)
		return 44100;

	p = codec->supported_samplerates;
	while (*p) {
		if (!best_samplerate || abs(44100 - *p) < abs(44100 - best_samplerate))
			best_samplerate = *p;
		p++;
	}
	return best_samplerate;
}




/* select layout with the highest channel count */
static int select_channel_layout(const AVCodec *codec)
{
	const uint64_t *p;
	uint64_t best_ch_layout = 0;
	int best_nb_channels = 0;

	if (!codec->channel_layouts)
		return AV_CH_LAYOUT_STEREO;

	p = codec->channel_layouts;
	while (*p) {
		int nb_channels = av_get_channel_layout_nb_channels(*p);

		if (nb_channels > best_nb_channels) {
			best_ch_layout = *p;
			best_nb_channels = nb_channels;
		}
		p++;
	}
	return best_ch_layout;
}


unsigned char clip_value(unsigned char x, unsigned char min_val, unsigned char  max_val) {
	if (x > max_val) {
		return max_val;
	}
	else if (x < min_val) {
		return min_val;
	}
	else {
		return x;
	}
}

//RGB to YUV420
bool RGB24_TO_YUV420(unsigned char *RgbBuf, int w, int h, unsigned char *yuvBuf)
{
	unsigned char*ptrY, *ptrU, *ptrV, *ptrRGB;
	memset(yuvBuf, 0, w*h * 3 / 2);
	ptrY = yuvBuf;
	ptrU = yuvBuf + w * h;
	ptrV = ptrU + (w*h * 1 / 4);
	unsigned char y, u, v, r, g, b;
	for (int j = h - 1; j >= 0; j--) {
		ptrRGB = RgbBuf + w * j * 3;
		for (int i = 0; i < w; i++) {

			b = *(ptrRGB++);
			g = *(ptrRGB++);
			r = *(ptrRGB++);


			y = (unsigned char)((66 * r + 129 * g + 25 * b + 128) >> 8) + 16;
			u = (unsigned char)((-38 * r - 74 * g + 112 * b + 128) >> 8) + 128;
			v = (unsigned char)((112 * r - 94 * g - 18 * b + 128) >> 8) + 128;
			*(ptrY++) = clip_value(y, 0, 255);
			if (j % 2 == 0 && i % 2 == 0) {
				*(ptrU++) = clip_value(u, 0, 255);
			}
			else {
				if (i % 2 == 0) {
					*(ptrV++) = clip_value(v, 0, 255);
				}
			}
		}
	}
	return true;
}


ULinkRecord::ULinkRecord()
{
	InitializeCriticalSection(&m_csVideoSection);
	InitializeCriticalSection(&m_csAudioInnerSection);
	InitializeCriticalSection(&m_csAudioInnerResampleSection);
	InitializeCriticalSection(&m_csAudioMicSection);
	InitializeCriticalSection(&m_csAudioMixSection);

	avdevice_register_all();
}

ULinkRecord::~ULinkRecord()
{
	DeleteCriticalSection(&m_csVideoSection);
	DeleteCriticalSection(&m_csAudioInnerSection);
	DeleteCriticalSection(&m_csAudioInnerResampleSection);
	DeleteCriticalSection(&m_csAudioMicSection);
	DeleteCriticalSection(&m_csAudioMixSection);
}

void ULinkRecord::SetMicName(const wchar_t* pMicName)
{
	m_wstrMicName = pMicName;
}

void ULinkRecord::SetRecordPath(const char* pRecordPath)
{
	m_strRecordPath = pRecordPath;
	if (!m_strRecordPath.empty())
	{
		if (m_strRecordPath[m_strRecordPath.length() - 1] != '\\')
		{
			m_strRecordPath = m_strRecordPath + "\\";
		}
	}
}

void ULinkRecord::SetRecordRect(RECT rectRecord)
{
	m_iRecordPosX = rectRecord.left;
	m_iRecordPosY = rectRecord.top;
	m_iRecordWidth = rectRecord.right - rectRecord.left;
	m_iRecordHeight = rectRecord.bottom - rectRecord.top;
}

int ULinkRecord::StartRecord()
{
	int iRet = -1;
	do 
	{
		m_pAudioConvertCtx = swr_alloc();
		av_opt_set_channel_layout(m_pAudioConvertCtx, "in_channel_layout", AV_CH_LAYOUT_STEREO, 0);
		av_opt_set_channel_layout(m_pAudioConvertCtx, "out_channel_layout", AV_CH_LAYOUT_STEREO, 0);
		av_opt_set_int(m_pAudioConvertCtx, "in_sample_rate", 44100, 0);
		av_opt_set_int(m_pAudioConvertCtx, "out_sample_rate", 44100, 0);
		av_opt_set_sample_fmt(m_pAudioConvertCtx, "in_sample_fmt", AV_SAMPLE_FMT_S16, 0);
		//av_opt_set_sample_fmt(audio_convert_ctx, "in_sample_fmt", AV_SAMPLE_FMT_FLTP, 0);
		av_opt_set_sample_fmt(m_pAudioConvertCtx, "out_sample_fmt", AV_SAMPLE_FMT_FLTP, 0);

		iRet = swr_init(m_pAudioConvertCtx);


		m_pAudioInnerResampleCtx = swr_alloc();
		av_opt_set_channel_layout(m_pAudioInnerResampleCtx, "in_channel_layout", AV_CH_LAYOUT_STEREO, 0);
		av_opt_set_channel_layout(m_pAudioInnerResampleCtx, "out_channel_layout", AV_CH_LAYOUT_STEREO, 0);
		av_opt_set_int(m_pAudioInnerResampleCtx, "in_sample_rate", 48000, 0);
		av_opt_set_int(m_pAudioInnerResampleCtx, "out_sample_rate", 44100, 0);
		av_opt_set_sample_fmt(m_pAudioInnerResampleCtx, "in_sample_fmt", AV_SAMPLE_FMT_S16, 0);
		av_opt_set_sample_fmt(m_pAudioInnerResampleCtx, "out_sample_fmt", AV_SAMPLE_FMT_S16, 0);

		iRet = swr_init(m_pAudioInnerResampleCtx);


		if (OpenAudioInnerCapture() < 0)
		{
			break;
		}

		if (OpenAudioMicCapture() < 0)
		{
			break;
		}

		if (OpenOutPut() < 0)
		{
			break;
		}

		const char* filter_desc = "[in0][in1]amix=inputs=2[out]";
		iRet = InitFilter(filter_desc);
		if (iRet < 0)
		{
			break;
		}

		if (NULL == m_pAudioInnerResampleFifo)
		{
			m_pAudioInnerResampleFifo = av_audio_fifo_alloc((AVSampleFormat)m_pFormatCtx_AudioInner->streams[0]->codecpar->format,
				m_pFormatCtx_AudioInner->streams[0]->codecpar->channels, 3000 * 1024);
		}

		if (NULL == m_pAudioInnerFifo)
		{
			m_pAudioInnerFifo = av_audio_fifo_alloc((AVSampleFormat)m_pFormatCtx_AudioInner->streams[0]->codecpar->format,
				m_pFormatCtx_AudioInner->streams[0]->codecpar->channels, 3000 * 1024);
		}

		m_iYuv420FrameSize = av_image_get_buffer_size(AV_PIX_FMT_YUV420P, m_iRecordWidth, m_iRecordHeight, 1);

		//申请30帧缓存
		m_pVideoFifo = av_fifo_alloc(30 * m_iYuv420FrameSize);

		m_hAudioInnerCapture = CreateThread(NULL, 0, AudioInnerCaptureProc, this, 0, NULL);
		m_hAudioInnerResample = CreateThread(NULL, 0, AudioInnerResampleProc, this, 0, NULL);
		m_hAudioMicCapture = CreateThread(NULL, 0, AudioMicCaptureProc, this, 0, NULL);
		m_hAudioMix = CreateThread(NULL, 0, AudioMixProc, this, 0, NULL);
		m_hScreenCapture = CreateThread(NULL, 0, ScreenCaptureProc, this, 0, NULL);
		m_hScreenAudioMix = CreateThread(NULL, 0, ScreenAudioMixProc, this, 0, NULL);

		iRet = 0;
	} while (0);
	
	if (0 == iRet)
	{
		m_bRecord = true;
	}
	else
	{
		Clear();
	}


	return iRet;
}


void ULinkRecord::StopRecord()
{
	m_bRecord = false;
	if (m_hAudioInnerCapture == NULL && m_hAudioInnerResample == NULL && m_hAudioMicCapture == NULL && m_hAudioMix == NULL && m_hScreenCapture == NULL && m_hScreenAudioMix == NULL)
	{
		return;
	}
	Sleep(1000);
	HANDLE hThreads[6];
	hThreads[0] = m_hAudioInnerCapture;
	hThreads[1] = m_hAudioInnerResample;
	hThreads[2] = m_hAudioMicCapture;
	hThreads[3] = m_hAudioMix;
	hThreads[4] = m_hScreenCapture;

	WaitForMultipleObjects(6, hThreads, TRUE, INFINITE);

	CloseHandle(m_hAudioInnerCapture);
	m_hAudioInnerCapture = NULL;

	CloseHandle(m_hAudioInnerResample);
	m_hAudioInnerResample = NULL;

	CloseHandle(m_hAudioMicCapture);
	m_hAudioMicCapture = NULL;

	CloseHandle(m_hAudioMix);
	m_hAudioMix = NULL;

	
	if (m_hScreenCapture != NULL)
	{
		CloseHandle(m_hScreenCapture);
		m_hScreenCapture = NULL;
	}

	if (m_hScreenAudioMix != NULL)
	{
		CloseHandle(m_hScreenAudioMix);
		m_hScreenAudioMix = NULL;
	}
	

	Clear();
}

int ULinkRecord::OpenAudioInnerCapture()
{
	int iRet = -1;

	do 
	{
		//查找输入方式
		const AVInputFormat *pAudioInputFmt = av_find_input_format("dshow");

		//以Direct Show的方式打开设备，并将 输入方式 关联到格式上下文
		//const char * psDevName = dup_wchar_to_utf8(L"audio=麦克风 (2- Synaptics HD Audio)");
		char * psDevName = dup_wchar_to_utf8(L"audio=virtual-audio-capturer");

		if (avformat_open_input(&m_pFormatCtx_AudioInner, psDevName, pAudioInputFmt, NULL) < 0)
		{
			LOG_ERROR("avformat_open_input failed, m_pFormatCtx_AudioInner");
			break;
		}

		if (avformat_find_stream_info(m_pFormatCtx_AudioInner, NULL) < 0)
		{
			break;
		}

		if (m_pFormatCtx_AudioInner->streams[0]->codecpar->codec_type != AVMEDIA_TYPE_AUDIO)
		{
			LOG_ERROR("Couldn't find audio stream information");
			break;
		}


		const AVCodec *tmpCodec = avcodec_find_decoder(m_pFormatCtx_AudioInner->streams[0]->codecpar->codec_id);

		m_pReadCodecCtx_AudioInner = avcodec_alloc_context3(tmpCodec);

		m_pReadCodecCtx_AudioInner->sample_rate = m_pFormatCtx_AudioInner->streams[0]->codecpar->sample_rate;
		m_pReadCodecCtx_AudioInner->channel_layout = select_channel_layout(tmpCodec);
		m_pReadCodecCtx_AudioInner->channels = av_get_channel_layout_nb_channels(m_pReadCodecCtx_AudioInner->channel_layout);

		m_pReadCodecCtx_AudioInner->sample_fmt = (AVSampleFormat)m_pFormatCtx_AudioInner->streams[0]->codecpar->format;
		//pReadCodecCtx_Audio->sample_fmt = AV_SAMPLE_FMT_FLTP;

		if (0 > avcodec_open2(m_pReadCodecCtx_AudioInner, tmpCodec, NULL))
		{
			LOG_ERROR("avcodec_open2 failed, m_pReadCodecCtx_AudioInner");
			break;
		}

		avcodec_parameters_from_context(m_pFormatCtx_AudioInner->streams[0]->codecpar, m_pReadCodecCtx_AudioInner);

		iRet = 0;
	} while (0);

	if (iRet != 0)
	{
		if (m_pReadCodecCtx_AudioInner != NULL)
		{
			avcodec_free_context(&m_pReadCodecCtx_AudioInner);
			m_pReadCodecCtx_AudioInner = NULL;
		}

		if (m_pFormatCtx_AudioInner != NULL)
		{
			avformat_close_input(&m_pFormatCtx_AudioInner);
			m_pFormatCtx_AudioInner = NULL;
		}
	}

	return iRet;
}


int ULinkRecord::OpenAudioMicCapture()
{
	int iRet = -1;
	do 
	{
		//查找输入方式
		const AVInputFormat *pAudioInputFmt = av_find_input_format("dshow");

		//以Direct Show的方式打开设备，并将 输入方式 关联到格式上下文
		std::wstring strDeviceName = L"audio=";
		strDeviceName += m_wstrMicName;
		const char * psDevName = dup_wchar_to_utf8(strDeviceName.c_str());

		if (avformat_open_input(&m_pFormatCtx_AudioMic, psDevName, pAudioInputFmt, NULL) < 0)
		{
			LOG_ERROR("avformat_open_input failed, m_pFormatCtx_AudioMic");
			break;
		}

		if (m_pFormatCtx_AudioMic->streams[0]->codecpar->codec_type != AVMEDIA_TYPE_AUDIO)
		{
			LOG_ERROR("Couldn't find audio stream information");
			break;
		}


		const AVCodec *tmpCodec = avcodec_find_decoder(m_pFormatCtx_AudioMic->streams[0]->codecpar->codec_id);

		m_pReadCodecCtx_AudioMic = avcodec_alloc_context3(tmpCodec);

		m_pReadCodecCtx_AudioMic->sample_rate = select_sample_rate(tmpCodec);
		m_pReadCodecCtx_AudioMic->channel_layout = select_channel_layout(tmpCodec);
		m_pReadCodecCtx_AudioMic->channels = av_get_channel_layout_nb_channels(m_pReadCodecCtx_AudioMic->channel_layout);

		m_pReadCodecCtx_AudioMic->sample_fmt = (AVSampleFormat)m_pFormatCtx_AudioMic->streams[0]->codecpar->format;
		//pReadCodecCtx_Audio->sample_fmt = AV_SAMPLE_FMT_FLTP;

		if (0 > avcodec_open2(m_pReadCodecCtx_AudioMic, tmpCodec, NULL))
		{
			LOG_ERROR("avcodec_open2 failed, m_pReadCodecCtx_AudioMic");
			break;
		}

		avcodec_parameters_from_context(m_pFormatCtx_AudioMic->streams[0]->codecpar, m_pReadCodecCtx_AudioMic);

		iRet = 0;
	} while (0);

	if (iRet != 0)
	{
		if (m_pReadCodecCtx_AudioMic != NULL)
		{
			avcodec_free_context(&m_pReadCodecCtx_AudioMic);
			m_pReadCodecCtx_AudioMic = NULL;
		}

		if (m_pFormatCtx_AudioMic != NULL)
		{
			avformat_close_input(&m_pFormatCtx_AudioMic);
			m_pFormatCtx_AudioMic = NULL;
		}
	}
	return iRet;
}


int ULinkRecord::OpenOutPut()
{
	std::string strFileName = m_strRecordPath;
	m_strFilePrefix = strFileName + time_tSimpleString(time(NULL));

	int iRet = -1;

	AVStream *pAudioStream = NULL;
	AVStream *pVideoStream = NULL;

	do
	{
		std::string strFileName = m_strRecordPath;
		strFileName += time_tSimpleString(time(NULL));
		strFileName += ".mp4";

		const char *outFileName = strFileName.c_str();
		avformat_alloc_output_context2(&m_pFormatCtx_Out, NULL, NULL, outFileName);

		{
			m_iVideoStreamIndex = 0;

			AVCodec* pCodecEncode_Video = (AVCodec *)avcodec_find_encoder(m_pFormatCtx_Out->oformat->video_codec);

			m_pCodecEncodeCtx_Video = avcodec_alloc_context3(pCodecEncode_Video);
			if (!m_pCodecEncodeCtx_Video)
			{
				LOG_ERROR("avcodec_alloc_context3 failed, m_pCodecEncodeCtx_Audio");
				break;
			}

			pVideoStream = avformat_new_stream(m_pFormatCtx_Out, pCodecEncode_Video);
			if (!pVideoStream)
			{
				break;
			}

			int frameRate = 10;
			m_pCodecEncodeCtx_Video->flags |= AV_CODEC_FLAG_QSCALE;
			m_pCodecEncodeCtx_Video->bit_rate = 4000000;
			m_pCodecEncodeCtx_Video->rc_min_rate = 4000000;
			m_pCodecEncodeCtx_Video->rc_max_rate = 4000000;
			m_pCodecEncodeCtx_Video->bit_rate_tolerance = 4000000;
			m_pCodecEncodeCtx_Video->time_base.den = frameRate;
			m_pCodecEncodeCtx_Video->time_base.num = 1;

			m_pCodecEncodeCtx_Video->width = m_iRecordWidth;
			m_pCodecEncodeCtx_Video->height = m_iRecordHeight;
			//pH264Encoder->pCodecCtx->frame_number = 1;
			m_pCodecEncodeCtx_Video->gop_size = 12;
			m_pCodecEncodeCtx_Video->max_b_frames = 0;
			m_pCodecEncodeCtx_Video->thread_count = 4;
			m_pCodecEncodeCtx_Video->pix_fmt = AV_PIX_FMT_YUV420P;
			m_pCodecEncodeCtx_Video->codec_id = AV_CODEC_ID_H264;
			m_pCodecEncodeCtx_Video->codec_type = AVMEDIA_TYPE_VIDEO;

			av_opt_set(m_pCodecEncodeCtx_Video->priv_data, "b-pyramid", "none", 0);
			av_opt_set(m_pCodecEncodeCtx_Video->priv_data, "preset", "superfast", 0);
			av_opt_set(m_pCodecEncodeCtx_Video->priv_data, "tune", "zerolatency", 0);

			if (m_pFormatCtx_Out->oformat->flags & AVFMT_GLOBALHEADER)
				m_pCodecEncodeCtx_Video->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;

			if (avcodec_open2(m_pCodecEncodeCtx_Video, pCodecEncode_Video, 0) < 0)
			{
				//编码器打开失败，退出程序
				LOG_ERROR("avcodec_open2 failed, m_pCodecEncodeCtx_Audio");
				break;
			}
		}


		if (m_pFormatCtx_AudioInner->streams[0]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && m_pFormatCtx_AudioMic->streams[0]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO)
		{
			pAudioStream = avformat_new_stream(m_pFormatCtx_Out, NULL);

			m_iAudioStreamIndex = 1;

			m_pCodecEncode_Audio = (AVCodec *)avcodec_find_encoder(m_pFormatCtx_Out->oformat->audio_codec);

			m_pCodecEncodeCtx_Audio = avcodec_alloc_context3(m_pCodecEncode_Audio);
			if (!m_pCodecEncodeCtx_Audio)
			{
				LOG_ERROR("avcodec_alloc_context3 failed, m_pCodecEncodeCtx_Audio");
				break;
			}


			//pCodecEncodeCtx_Audio->codec_id = pFormatCtx_Out->oformat->audio_codec;
			m_pCodecEncodeCtx_Audio->sample_fmt = m_pCodecEncode_Audio->sample_fmts ? m_pCodecEncode_Audio->sample_fmts[0] : AV_SAMPLE_FMT_FLTP;
			m_pCodecEncodeCtx_Audio->bit_rate = 64000;
			m_pCodecEncodeCtx_Audio->sample_rate = 44100;
			m_pCodecEncodeCtx_Audio->channel_layout = AV_CH_LAYOUT_STEREO;
			m_pCodecEncodeCtx_Audio->channels = av_get_channel_layout_nb_channels(m_pCodecEncodeCtx_Audio->channel_layout);


			AVRational timeBase;
			timeBase.den = m_pCodecEncodeCtx_Audio->sample_rate;
			timeBase.num = 1;
			pAudioStream->time_base = timeBase;

			if (avcodec_open2(m_pCodecEncodeCtx_Audio, m_pCodecEncode_Audio, 0) < 0)
			{
				//编码器打开失败，退出程序
				LOG_ERROR("avcodec_open2 failed, m_pCodecEncodeCtx_Audio");
				break;
			}
		}


		if (!(m_pFormatCtx_Out->oformat->flags & AVFMT_NOFILE))
		{
			if (avio_open(&m_pFormatCtx_Out->pb, outFileName, AVIO_FLAG_WRITE) < 0)
			{
				LOG_ERROR("avio_open failed, m_pFormatCtx_Out->pb");
				break;
			}
		}

		avcodec_parameters_from_context(pVideoStream->codecpar, m_pCodecEncodeCtx_Video);
		avcodec_parameters_from_context(pAudioStream->codecpar, m_pCodecEncodeCtx_Audio);

		if (avformat_write_header(m_pFormatCtx_Out, NULL) < 0)
		{
			LOG_ERROR("avio_open avformat_write_header,m_pFormatCtx_Out");
			break;
		}

		iRet = 0;
	} while (0);


	if (iRet != 0)
	{
		if (m_pCodecEncodeCtx_Video != NULL)
		{
			avcodec_free_context(&m_pCodecEncodeCtx_Video);
			m_pCodecEncodeCtx_Video = NULL;
		}

		if (m_pCodecEncodeCtx_Audio != NULL)
		{
			avcodec_free_context(&m_pCodecEncodeCtx_Audio);
			m_pCodecEncodeCtx_Audio = NULL;
		}

		if (m_pFormatCtx_Out != NULL)
		{
			avformat_free_context(m_pFormatCtx_Out);
			m_pFormatCtx_Out = NULL;
		}
	}

	return iRet;
}


int ULinkRecord::InitFilter(const char* filter_desc)
{
	char args_inner[512];
	const char* pad_name_inner = "in0";
	char args_mic[512];
	const char* pad_name_mic = "in1";

	AVFilter* filter_src_spk = (AVFilter *)avfilter_get_by_name("abuffer");
	AVFilter* filter_src_mic = (AVFilter *)avfilter_get_by_name("abuffer");
	AVFilter* filter_sink = (AVFilter *)avfilter_get_by_name("abuffersink");
	AVFilterInOut* filter_output_inner = avfilter_inout_alloc();
	AVFilterInOut* filter_output_mic = avfilter_inout_alloc();
	AVFilterInOut* filter_input = avfilter_inout_alloc();
	m_pFilterGraph = avfilter_graph_alloc();

	/*sprintf_s(args_inner, sizeof(args_inner), "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=0x%I64x",
		m_pReadCodecCtx_AudioInner->time_base.num,
		m_pReadCodecCtx_AudioInner->time_base.den,
		m_pReadCodecCtx_AudioInner->sample_rate,
		av_get_sample_fmt_name((AVSampleFormat)m_pReadCodecCtx_AudioInner->sample_fmt),
		m_pReadCodecCtx_AudioInner->channel_layout);*/

	sprintf_s(args_inner, sizeof(args_inner), "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=0x%I64x",
		m_pReadCodecCtx_AudioMic->time_base.num,
		m_pReadCodecCtx_AudioMic->time_base.den,
		m_pReadCodecCtx_AudioMic->sample_rate,
		av_get_sample_fmt_name((AVSampleFormat)m_pReadCodecCtx_AudioMic->sample_fmt),
		m_pReadCodecCtx_AudioMic->channel_layout);

	sprintf_s(args_mic, sizeof(args_mic), "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=0x%I64x",
		m_pReadCodecCtx_AudioMic->time_base.num,
		m_pReadCodecCtx_AudioMic->time_base.den,
		m_pReadCodecCtx_AudioMic->sample_rate,
		av_get_sample_fmt_name((AVSampleFormat)m_pReadCodecCtx_AudioMic->sample_fmt),
		m_pReadCodecCtx_AudioMic->channel_layout);

	//sprintf_s(args_spk, sizeof(args_spk), "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=0x%I64x", _fmt_ctx_out->streams[_index_a_out]->codec->time_base.num, _fmt_ctx_out->streams[_index_a_out]->codec->time_base.den, _fmt_ctx_out->streams[_index_a_out]->codec->sample_rate, av_get_sample_fmt_name(_fmt_ctx_out->streams[_index_a_out]->codec->sample_fmt), _fmt_ctx_out->streams[_index_a_out]->codec->channel_layout);
	//sprintf_s(args_mic, sizeof(args_mic), "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=0x%I64x", _fmt_ctx_out->streams[_index_a_out]->codec->time_base.num, _fmt_ctx_out->streams[_index_a_out]->codec->time_base.den, _fmt_ctx_out->streams[_index_a_out]->codec->sample_rate, av_get_sample_fmt_name(_fmt_ctx_out->streams[_index_a_out]->codec->sample_fmt), _fmt_ctx_out->streams[_index_a_out]->codec->channel_layout);


	int ret = 0;
	ret = avfilter_graph_create_filter(&m_pFilterCtxSrcInner, filter_src_spk, pad_name_inner, args_inner, NULL, m_pFilterGraph);
	if (ret < 0)
	{
		printf("Filter: failed to call avfilter_graph_create_filter -- src inner\n");
		return -1;
	}
	ret = avfilter_graph_create_filter(&m_pFilterCtxSrcMic, filter_src_mic, pad_name_mic, args_mic, NULL, m_pFilterGraph);
	if (ret < 0)
	{
		printf("Filter: failed to call avfilter_graph_create_filter -- src mic\n");
		return -1;
	}

	ret = avfilter_graph_create_filter(&m_pFilterCtxSink, filter_sink, "out", NULL, NULL, m_pFilterGraph);
	if (ret < 0)
	{
		printf("Filter: failed to call avfilter_graph_create_filter -- sink\n");
		return -1;
	}
	AVCodecContext* encodec_ctx = m_pCodecEncodeCtx_Audio;
	//ret = av_opt_set_bin(_filter_ctx_sink, "sample_fmts", (uint8_t*)&encodec_ctx->sample_fmt, sizeof(encodec_ctx->sample_fmt), AV_OPT_SEARCH_CHILDREN);
	ret = av_opt_set_bin(m_pFilterCtxSink, "sample_fmts", (uint8_t*)&m_pReadCodecCtx_AudioInner->sample_fmt, sizeof(m_pReadCodecCtx_AudioInner->sample_fmt), AV_OPT_SEARCH_CHILDREN);
	if (ret < 0)
	{
		printf("Filter: failed to call av_opt_set_bin -- sample_fmts\n");
		return -1;
	}
	ret = av_opt_set_bin(m_pFilterCtxSink, "channel_layouts", (uint8_t*)&encodec_ctx->channel_layout, sizeof(encodec_ctx->channel_layout), AV_OPT_SEARCH_CHILDREN);
	if (ret < 0)
	{
		printf("Filter: failed to call av_opt_set_bin -- channel_layouts\n");
		return -1;
	}
	ret = av_opt_set_bin(m_pFilterCtxSink, "sample_rates", (uint8_t*)&encodec_ctx->sample_rate, sizeof(encodec_ctx->sample_rate), AV_OPT_SEARCH_CHILDREN);
	if (ret < 0)
	{
		printf("Filter: failed to call av_opt_set_bin -- sample_rates\n");
		return -1;
	}

	filter_output_inner->name = av_strdup(pad_name_inner);
	filter_output_inner->filter_ctx = m_pFilterCtxSrcInner;
	filter_output_inner->pad_idx = 0;
	filter_output_inner->next = filter_output_mic;

	filter_output_mic->name = av_strdup(pad_name_mic);
	filter_output_mic->filter_ctx = m_pFilterCtxSrcMic;
	filter_output_mic->pad_idx = 0;
	filter_output_mic->next = NULL;

	filter_input->name = av_strdup("out");
	filter_input->filter_ctx = m_pFilterCtxSink;
	filter_input->pad_idx = 0;
	filter_input->next = NULL;

	AVFilterInOut* filter_outputs[2];
	filter_outputs[0] = filter_output_inner;
	filter_outputs[1] = filter_output_mic;

	ret = avfilter_graph_parse_ptr(m_pFilterGraph, filter_desc, &filter_input, filter_outputs, NULL);
	if (ret < 0)
	{
		printf("Filter: failed to call avfilter_graph_parse_ptr\n");
		return -1;
	}

	ret = avfilter_graph_config(m_pFilterGraph, NULL);
	if (ret < 0)
	{
		printf("Filter: failed to call avfilter_graph_config\n");
		return -1;
	}

	avfilter_inout_free(&filter_input);
	av_free(filter_src_spk);
	av_free(filter_src_mic);
	avfilter_inout_free(filter_outputs);
	//av_free(filter_outputs);

	char* temp = avfilter_graph_dump(m_pFilterGraph, NULL);
	printf("%s\n", temp);

	return 0;
}

void ULinkRecord::Clear()
{
	if (m_pReadCodecCtx_AudioInner)
	{
		avcodec_free_context(&m_pReadCodecCtx_AudioInner);
		m_pReadCodecCtx_AudioInner = NULL;
	}

	if (m_pReadCodecCtx_AudioMic != NULL)
	{
		avcodec_free_context(&m_pReadCodecCtx_AudioMic);
		m_pReadCodecCtx_AudioMic = NULL;
	}

	if (m_pCodecEncodeCtx_Audio != NULL)
	{
		avcodec_free_context(&m_pCodecEncodeCtx_Audio);
		m_pCodecEncodeCtx_Audio = NULL;
	}

	if (m_pAudioConvertCtx != NULL)
	{
		swr_free(&m_pAudioConvertCtx);
		m_pAudioConvertCtx = NULL;
	}

	if (m_pAudioInnerFifo != NULL)
	{
		av_audio_fifo_free(m_pAudioInnerFifo);
		m_pAudioInnerFifo = NULL;
	}

	if (m_pAudioInnerResampleFifo != NULL)
	{
		av_audio_fifo_free(m_pAudioInnerResampleFifo);
		m_pAudioInnerResampleFifo = NULL;
	}

	if (m_pAudioMicFifo != NULL)
	{
		av_audio_fifo_free(m_pAudioMicFifo);
		m_pAudioMicFifo = NULL;
	}

	if (m_pAudioMixFifo != NULL)
	{
		av_audio_fifo_free(m_pAudioMixFifo);
		m_pAudioMixFifo = NULL;
	}

	if (m_pFormatCtx_AudioInner != NULL)
	{
		avformat_close_input(&m_pFormatCtx_AudioInner);
		m_pFormatCtx_AudioInner = NULL;
	}

	if (m_pFormatCtx_AudioMic != NULL)
	{
		avformat_close_input(&m_pFormatCtx_AudioMic);
		m_pFormatCtx_AudioMic = NULL;
	}

	if (m_pFormatCtx_Out != NULL)
	{
		avformat_free_context(m_pFormatCtx_Out);
		m_pFormatCtx_Out = NULL;
	}

	if (m_hScreenCapture != NULL)
	{
		CloseHandle(m_hScreenCapture);
		m_hScreenCapture = NULL;
	}

	if (m_hScreenAudioMix != NULL)
	{
		CloseHandle(m_hScreenAudioMix);
		m_hScreenAudioMix = NULL;
	}
}

DWORD WINAPI ULinkRecord::AudioInnerCaptureProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->AudioInnerCapture();
	}
	return 0;
}

void ULinkRecord::AudioInnerCapture()
{
	AVFrame *pFrame;
	pFrame = av_frame_alloc();

	AVPacket packet = { 0 };
	int ret = 0;
	while (m_bRecord)
	{
		av_packet_unref(&packet);
		if (av_read_frame(m_pFormatCtx_AudioInner, &packet) < 0)
		{
			continue;
		}

		ret = avcodec_send_packet(m_pReadCodecCtx_AudioInner, &packet);
		if (ret >= 0)
		{
			ret = avcodec_receive_frame(m_pReadCodecCtx_AudioInner, pFrame);
			if (ret == AVERROR(EAGAIN))
			{
				continue;
			}
			else if (ret == AVERROR_EOF)
			{
				break;
			}
			else if (ret < 0) {
				fprintf(stderr, "Error during decoding\n");
				exit(1);
			}

			int buf_space = av_audio_fifo_space(m_pAudioInnerFifo);
			if (buf_space >= pFrame->nb_samples)
			{
				//AudioSection
				EnterCriticalSection(&m_csAudioInnerSection);
				ret = av_audio_fifo_write(m_pAudioInnerFifo, (void **)pFrame->data, pFrame->nb_samples);
				LeaveCriticalSection(&m_csAudioInnerSection);
			}



			av_packet_unref(&packet);
		}

	}

	av_frame_free(&pFrame);
}

DWORD WINAPI ULinkRecord::AudioInnerResampleProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->AudioInnerResample();
	}
	return 0;
}

void ULinkRecord::AudioInnerResample()
{
	int ret = 0;

	while (1)
	{
		if (av_audio_fifo_size(m_pAudioInnerFifo) >=
			(m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024))
		{
			AVFrame *frame_audio_inner = NULL;
			frame_audio_inner = av_frame_alloc();

			frame_audio_inner->nb_samples = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024;
			frame_audio_inner->channel_layout = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->channel_layout;
			frame_audio_inner->format = m_pFormatCtx_AudioInner->streams[0]->codecpar->format;
			frame_audio_inner->sample_rate = m_pFormatCtx_AudioInner->streams[0]->codecpar->sample_rate;
			av_frame_get_buffer(frame_audio_inner, 0);

			EnterCriticalSection(&m_csAudioInnerSection);
			int readcount = av_audio_fifo_read(m_pAudioInnerFifo, (void **)frame_audio_inner->data,
				(m_pFormatCtx_AudioInner->streams[0]->codecpar->frame_size > 0 ? m_pFormatCtx_AudioInner->streams[0]->codecpar->frame_size : 1024));
			LeaveCriticalSection(&m_csAudioInnerSection);

			AVFrame *frame_audio_inner_resample = NULL;
			frame_audio_inner_resample = av_frame_alloc();

			//int iDelaySamples = swr_get_delay(audio_convert_ctx, frame_mic->sample_rate);
			int iDelaySamples = 0;
			//int dst_nb_samples = av_rescale_rnd(iDelaySamples + frame_mic->nb_samples, frame_mic->sample_rate, pCodecEncodeCtx_Audio->sample_rate, AVRounding(1));
			int dst_nb_samples = av_rescale_rnd(iDelaySamples + frame_audio_inner->nb_samples, m_pCodecEncodeCtx_Audio->sample_rate, frame_audio_inner->sample_rate, AV_ROUND_UP);


			frame_audio_inner_resample->nb_samples = m_pCodecEncodeCtx_Audio->frame_size;
			frame_audio_inner_resample->channel_layout = m_pCodecEncodeCtx_Audio->channel_layout;
			frame_audio_inner_resample->format = m_pFormatCtx_AudioInner->streams[0]->codecpar->format;
			frame_audio_inner_resample->sample_rate = m_pCodecEncodeCtx_Audio->sample_rate;
			av_frame_get_buffer(frame_audio_inner_resample, 0);

			uint8_t* out_buffer = (uint8_t*)frame_audio_inner_resample->data[0];

			int nb = swr_convert(m_pAudioInnerResampleCtx, &out_buffer, dst_nb_samples, (const uint8_t**)frame_audio_inner->data, frame_audio_inner->nb_samples);



			//if (av_audio_fifo_space(fifo_audio_resample) >= pFrame->nb_samples)
			{
				EnterCriticalSection(&m_csAudioInnerResampleSection);
				ret = av_audio_fifo_write(m_pAudioInnerResampleFifo, (void **)frame_audio_inner_resample->data, dst_nb_samples);
				LeaveCriticalSection(&m_csAudioInnerResampleSection);
			}

			av_frame_free(&frame_audio_inner);
			av_frame_free(&frame_audio_inner_resample);

			if (!m_bRecord)
			{
				if (av_audio_fifo_size(m_pAudioInnerFifo) < 1024)
				{
					break;
				}
			}
		}
		else
		{
			if (!m_bRecord)
			{
				break;
			}
		}
	}
}

DWORD WINAPI ULinkRecord::AudioMicCaptureProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->AudioMicCapture();
	}
	return 0;
}

void ULinkRecord::AudioMicCapture()
{
	AVFrame *pFrame;
	pFrame = av_frame_alloc();

	AVPacket packet = { 0 };
	int ret = 0;
	while (m_bRecord)
	{
		av_packet_unref(&packet);
		if (av_read_frame(m_pFormatCtx_AudioMic, &packet) < 0)
		{
			continue;
		}

		ret = avcodec_send_packet(m_pReadCodecCtx_AudioMic, &packet);
		if (ret >= 0)
		{
			ret = avcodec_receive_frame(m_pReadCodecCtx_AudioMic, pFrame);
			if (ret == AVERROR(EAGAIN))
			{
				continue;
			}
			else if (ret == AVERROR_EOF)
			{
				break;
			}
			else if (ret < 0) {
				break;
			}

			if (NULL == m_pAudioMicFifo)
			{
				m_pAudioMicFifo = av_audio_fifo_alloc((AVSampleFormat)m_pFormatCtx_AudioMic->streams[0]->codecpar->format,
					m_pFormatCtx_AudioMic->streams[0]->codecpar->channels, 3000 * pFrame->nb_samples);
			}

			int buf_space = av_audio_fifo_space(m_pAudioMicFifo);
			if (buf_space >= pFrame->nb_samples)
			{
				EnterCriticalSection(&m_csAudioMicSection);
				ret = av_audio_fifo_write(m_pAudioMicFifo, (void **)pFrame->data, pFrame->nb_samples);
				LeaveCriticalSection(&m_csAudioMicSection);
			}



			av_packet_unref(&packet);
		}

	}

	av_frame_free(&pFrame);
}


DWORD WINAPI ULinkRecord::AudioMixProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->AudioMix();
	}
	return 0;
}

void ULinkRecord::AudioMix()
{
	int ret = 0;

	int iAudioFrameMixedIndex = 0;

	AVFrame *frame_audio_inner_resample = NULL;
	frame_audio_inner_resample = av_frame_alloc();

	AVFrame *frame_audio_mic = NULL;
	frame_audio_mic = av_frame_alloc();


	if (NULL == m_pAudioMixFifo)
	{
		m_pAudioMixFifo = av_audio_fifo_alloc((AVSampleFormat)m_pFormatCtx_AudioInner->streams[0]->codecpar->format,
			m_pFormatCtx_AudioInner->streams[0]->codecpar->channels, 3000 * 1024);
	}


	while (m_bRecord)
	{
		if (NULL == m_pAudioInnerResampleFifo)
		{
			continue;
		}
		if (NULL == m_pAudioMicFifo)
		{
			continue;
		}

		int fifo_inner_resample_size = av_audio_fifo_size(m_pAudioInnerResampleFifo);
		int fifo_mic_size = av_audio_fifo_size(m_pAudioMicFifo);
		//int frame_inner_min_size = pReadCodecCtx_Audio->frame_size;
		//int frame_mic_min_size = pReadCodecCtx_AudioMic->frame_size;

		int frame_inner_resample_min_size = 1024;
		int frame_mic_min_size = 1024;

		if (fifo_inner_resample_size >= frame_inner_resample_min_size && fifo_mic_size >= frame_mic_min_size)
		{
			frame_audio_inner_resample->nb_samples = frame_inner_resample_min_size;
			frame_audio_inner_resample->channel_layout = 3;
			frame_audio_inner_resample->format = m_pFormatCtx_AudioInner->streams[0]->codecpar->format;
			frame_audio_inner_resample->sample_rate = m_pFormatCtx_AudioMic->streams[0]->codecpar->sample_rate;
			av_frame_get_buffer(frame_audio_inner_resample, 0);

			frame_audio_mic->nb_samples = frame_mic_min_size;
			frame_audio_mic->channel_layout = 3;
			frame_audio_mic->format = m_pFormatCtx_AudioMic->streams[0]->codecpar->format;
			frame_audio_mic->sample_rate = m_pFormatCtx_AudioMic->streams[0]->codecpar->sample_rate;
			av_frame_get_buffer(frame_audio_mic, 0);

			EnterCriticalSection(&m_csAudioInnerResampleSection);
			int readcount = av_audio_fifo_read(m_pAudioInnerResampleFifo, (void **)frame_audio_inner_resample->data, frame_inner_resample_min_size);
			LeaveCriticalSection(&m_csAudioInnerResampleSection);


			EnterCriticalSection(&m_csAudioMicSection);
			readcount = av_audio_fifo_read(m_pAudioMicFifo, (void**)frame_audio_mic->data, frame_mic_min_size);
			LeaveCriticalSection(&m_csAudioMicSection);


			frame_audio_inner_resample->pts = iAudioFrameMixedIndex * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;
			frame_audio_mic->pts = iAudioFrameMixedIndex * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;

			BufferSourceContext* s = (BufferSourceContext*)m_pFilterCtxSrcInner->priv;
			bool b1 = (s->sample_fmt != frame_audio_inner_resample->format);
			bool b2 = (s->sample_rate != frame_audio_inner_resample->sample_rate);
			bool b3 = (s->channel_layout != frame_audio_inner_resample->channel_layout);
			bool b4 = (s->channels != frame_audio_inner_resample->channels);

			ret = av_buffersrc_add_frame(m_pFilterCtxSrcInner, frame_audio_inner_resample);
			if (ret < 0)
			{
				printf("Mixer: failed to call av_buffersrc_add_frame (speaker)\n");
				break;
			}

			ret = av_buffersrc_add_frame(m_pFilterCtxSrcMic, frame_audio_mic);
			if (ret < 0)
			{
				printf("Mixer: failed to call av_buffersrc_add_frame (microphone)\n");
				break;
			}


			while (1)
			{
				AVFrame* pFrame_out = av_frame_alloc();

				ret = av_buffersink_get_frame_flags(m_pFilterCtxSink, pFrame_out, 0);
				if (ret < 0)
				{
					av_frame_free(&pFrame_out);
					//printf("Mixer: failed to call av_buffersink_get_frame_flags\n");
					break;
				}
				iAudioFrameMixedIndex++;
				EnterCriticalSection(&m_csAudioMixSection);
				ret = av_audio_fifo_write(m_pAudioMixFifo, (void **)pFrame_out->data, pFrame_out->nb_samples);
				LeaveCriticalSection(&m_csAudioMixSection);

				av_frame_free(&pFrame_out);
			}
		}
	}


	av_frame_free(&frame_audio_inner_resample);
	av_frame_free(&frame_audio_mic);
}


DWORD WINAPI ULinkRecord::ScreenCaptureProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->ScreenCapture();
	}
	return 0;
}


void ULinkRecord::ScreenCapture()
{
	CCaptureScreen* ccs = new CCaptureScreen();
	int width = 0;
	int height = 0;

	ccs->Init(width, height);

	AVFrame *pFrameYUV = av_frame_alloc();
	pFrameYUV->format = AV_PIX_FMT_YUV420P;
	pFrameYUV->width = width;
	pFrameYUV->height = height;

	int y_size = m_iRecordWidth * m_iRecordHeight;

	int frame_size = av_image_get_buffer_size(AV_PIX_FMT_YUV420P, m_iRecordWidth, m_iRecordHeight, 1);
	BYTE* out_buffer_yuv420 = new BYTE[frame_size];
	av_image_fill_arrays(pFrameYUV->data, pFrameYUV->linesize, out_buffer_yuv420, AV_PIX_FMT_YUV420P, m_iRecordWidth, m_iRecordHeight, 1);

	AVPacket* packet = av_packet_alloc();
	m_iFrameNumber = 0;

	int ret = 0;

	DWORD dwBeginTime = ::GetTickCount();
	while(m_bRecord)
	{
		BYTE* frameimage = ccs->CaptureImage();

		RGB24_TO_YUV420(frameimage, width, height, out_buffer_yuv420);
		//Sframe->pkt_dts = frame->pts = frameNumber * avCodecCtx_Out->time_base.num * avStream->time_base.den / (avCodecCtx_Out->time_base.den *  avStream->time_base.num);
		pFrameYUV->pkt_dts = pFrameYUV->pts = av_rescale_q_rnd(m_iFrameNumber, m_pCodecEncodeCtx_Video->time_base, m_pFormatCtx_Out->streams[m_iVideoStreamIndex]->time_base, (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
		pFrameYUV->pkt_duration = 0;
		pFrameYUV->pkt_pos = -1;

		if (av_fifo_space(m_pVideoFifo) >= frame_size)
		{
			EnterCriticalSection(&m_csVideoSection);
			av_fifo_generic_write(m_pVideoFifo, pFrameYUV->data[0], y_size, NULL);
			av_fifo_generic_write(m_pVideoFifo, pFrameYUV->data[1], y_size / 4, NULL);
			av_fifo_generic_write(m_pVideoFifo, pFrameYUV->data[2], y_size / 4, NULL);
			LeaveCriticalSection(&m_csVideoSection);

			m_iFrameNumber++;

			DWORD dwCurrentTime = ::GetTickCount();
			int dwPassedMillSeconds = dwCurrentTime - dwBeginTime;
			int dwDiff = m_iFrameNumber * 100 - dwPassedMillSeconds;
			if (dwDiff > 0)
			{
				Sleep(dwDiff);
			}
		}
	}
	av_packet_free(&packet);
	av_frame_free(&pFrameYUV);
	delete[] out_buffer_yuv420;
}


DWORD WINAPI ULinkRecord::ScreenAudioMixProc(LPVOID lpParam)
{
	ULinkRecord *pULinkRecord = (ULinkRecord *)lpParam;
	if (pULinkRecord != NULL)
	{
		pULinkRecord->ScreenAudioMix();
	}
	return 0;
}

void ULinkRecord::ScreenAudioMix()
{
	int ret = 0;
	int cur_pts_v = 0;
	int cur_pts_a = 0;

	AVFrame *pFrameYUVInMain = av_frame_alloc();
	uint8_t *out_buffer_yuv420 = (uint8_t *)av_malloc(m_iYuv420FrameSize);
	av_image_fill_arrays(pFrameYUVInMain->data, pFrameYUVInMain->linesize, out_buffer_yuv420, AV_PIX_FMT_YUV420P, m_iRecordWidth, m_iRecordHeight, 1);

	int AudioFrameIndex_mic = 1;
	AVPacket packet = { 0 };

	int iPicCount = 0;
	while (m_bRecord)
	{
		if (NULL == m_pVideoFifo)
		{
			continue;
		}

		if (NULL == m_pAudioMixFifo)
		{
			continue;
		}

		if (av_compare_ts(cur_pts_v, m_pFormatCtx_Out->streams[m_iVideoStreamIndex]->time_base,
			cur_pts_a, m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->time_base) <= 0)
		{
			if (av_fifo_size(m_pVideoFifo) >= m_iYuv420FrameSize)
			{
				EnterCriticalSection(&m_csVideoSection);
				av_fifo_generic_read(m_pVideoFifo, out_buffer_yuv420, m_iYuv420FrameSize, NULL);
				LeaveCriticalSection(&m_csVideoSection);

				pFrameYUVInMain->pkt_dts = pFrameYUVInMain->pts = av_rescale_q_rnd(iPicCount, m_pCodecEncodeCtx_Video->time_base, m_pFormatCtx_Out->streams[m_iVideoStreamIndex]->time_base, (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
				pFrameYUVInMain->pkt_duration = 0;
				pFrameYUVInMain->pkt_pos = -1;

				pFrameYUVInMain->width = m_iRecordWidth;
				pFrameYUVInMain->height = m_iRecordHeight;
				pFrameYUVInMain->format = AV_PIX_FMT_YUV420P;

				cur_pts_v = packet.pts;

				ret = avcodec_send_frame(m_pCodecEncodeCtx_Video, pFrameYUVInMain);

				ret = avcodec_receive_packet(m_pCodecEncodeCtx_Video, &packet);

				if (packet.size > 0)
				{
					//av_packet_rescale_ts(packet, avCodecCtx_Out->time_base, avStream->time_base);
					av_write_frame(m_pFormatCtx_Out, &packet);
					iPicCount++;
				}
			}
		}
		else
		{
			if (av_audio_fifo_size(m_pAudioMixFifo) >=
				(m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024))
			{
				AVFrame *frame_mix = NULL;
				frame_mix = av_frame_alloc();

				frame_mix->nb_samples = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024;
				frame_mix->channel_layout = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->channel_layout;
				//frame_mix->format = pFormatCtx_Out->streams[iAudioStreamIndex]->codecpar->format;
				frame_mix->format = 1;
				frame_mix->sample_rate = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->sample_rate;
				av_frame_get_buffer(frame_mix, 0);

				EnterCriticalSection(&m_csAudioMixSection);
				int readcount = av_audio_fifo_read(m_pAudioMixFifo, (void **)frame_mix->data,
					(m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size > 0 ? m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size : 1024));
				LeaveCriticalSection(&m_csAudioMixSection);

				AVPacket pkt_out_mic = { 0 };

				pkt_out_mic.data = NULL;
				pkt_out_mic.size = 0;

				frame_mix->pts = AudioFrameIndex_mic * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;


				AVFrame *frame_mic_encode = NULL;
				frame_mic_encode = av_frame_alloc();

				frame_mic_encode->nb_samples = m_pCodecEncodeCtx_Audio->frame_size;
				frame_mic_encode->channel_layout = m_pCodecEncodeCtx_Audio->channel_layout;
				frame_mic_encode->format = m_pCodecEncodeCtx_Audio->sample_fmt;
				frame_mic_encode->sample_rate = m_pCodecEncodeCtx_Audio->sample_rate;
				av_frame_get_buffer(frame_mic_encode, 0);



				int dst_nb_samples = av_rescale_rnd(swr_get_delay(m_pAudioConvertCtx, frame_mix->sample_rate) + frame_mix->nb_samples, frame_mix->sample_rate, frame_mix->sample_rate, AVRounding(1));

				//uint8_t *audio_buf = NULL;
				uint8_t *audio_buf[2] = { 0 };
				audio_buf[0] = (uint8_t *)frame_mic_encode->data[0];
				audio_buf[1] = (uint8_t *)frame_mic_encode->data[1];

				int nb = swr_convert(m_pAudioConvertCtx, audio_buf, dst_nb_samples, (const uint8_t**)frame_mix->data, frame_mix->nb_samples);

				ret = avcodec_send_frame(m_pCodecEncodeCtx_Audio, frame_mic_encode);

				ret = avcodec_receive_packet(m_pCodecEncodeCtx_Audio, &pkt_out_mic);
				if (ret == AVERROR(EAGAIN))
				{
					continue;
				}
				av_frame_free(&frame_mix);
				av_frame_free(&frame_mic_encode);
				{
					pkt_out_mic.stream_index = m_iAudioStreamIndex;
					pkt_out_mic.pts = AudioFrameIndex_mic * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;
					pkt_out_mic.dts = AudioFrameIndex_mic * m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;
					pkt_out_mic.duration = m_pFormatCtx_Out->streams[m_iAudioStreamIndex]->codecpar->frame_size;

					cur_pts_a = pkt_out_mic.pts;
					av_write_frame(m_pFormatCtx_Out, &pkt_out_mic);
					//int ret2 = av_interleaved_write_frame(m_pFormatCtx_Out, &pkt_out_mic);
					av_packet_unref(&pkt_out_mic);
				}
				AudioFrameIndex_mic++;
			}
		}
	}


	Sleep(100);
	av_write_trailer(m_pFormatCtx_Out);

	avio_close(m_pFormatCtx_Out->pb);

	av_frame_free(&pFrameYUVInMain);
}
tusong86
关注
  • 2
    点赞
  • 踩
  • 3
    收藏
    觉得还不错? 一键收藏
  • 9
    评论
FFmpeg 录制桌面、麦克风、摄像头
勤能补拙
12-31 1万+
FFmpeg 录制桌面、麦克风、摄像头前言老师要我们试试能不能用手机拍摄视频然后发送到树莓派上。可能以后要然树莓派处理视频之类。老师描述的场景好像实时的。虽然需求不明确，我们就先试试吧。我的计划是在树莓派上搭建好流媒体服务器，然后手机拍摄视频并推送到服务器。但是找了好久也找不到靠谱的可以实时拍摄视频并推送到服务器的 Android 应用。我们也不会 Android 开发。好不容器把树莓派上的 rtm
均衡、变声、回声和混音的音效处理
05-15
本论文介绍了在matlab平台下的数字音效处理的实现。主要使用了matlab中的 GUI、FDAtools、audio函数、fft函数、filter函数等制作了图形用户界面、声音的采集和播放、信号在时域和频率的多种处理、滤波器的制作和应用。通过对声音的时域和频域分析，利用梳状滤波器、IIR2阶滤波器、信号加权线性叠加算法、频域差值算法等理论工具最后实现出对声音的均衡、变声、回声和混音的音效处理。
ffmpeg 本地麦克风声音和系统声音混音后，再混合本地桌面成最终的mp4文件
tusong86的博客
11-27 1989
之前写过ffmpeg录制麦克风声音和pc内部声音(如播放歌曲)—混音，现在再进一步，混音后，再和采集到的桌面视频合在一处。 现在说说大致思路，如下所示，创建了四个线程 HANDLE hThreadAudio = CreateThread(NULL, 0, AudioCapThreadProc, 0, 0, NULL); HANDLE hThreadAudioMic = CreateThread(NULL, 0, AudioMicCapThreadProc, 0, 0, NULL); HANDLE hT
ffmpeg录制麦克风声音和pc内部声音(如播放歌曲)---混音--修正
tusong86的博客
12-08 1749
之前写过一篇混音的文章，ffmpeg录制麦克风声音和pc内部声音(如播放歌曲)—混音 但是有瑕疵，主要是因为本地麦克风设备的采样频率是44100，而pc内部声音的采样率是48000，所以当最终输出的文件的采样率为44100时，文件里面的声音听起来会播放的慢一些。故在音频混合前，先将pc内部声音重采样成44100，重采样的转换过程如下： uint8_t* out_buffer = (uint8_t*)frame_audio_inner_resample->data[0]; int nb = sw
ffmpeg混音（混声）命令
张雨的博客
11-15 4519
ffmpeg命令目录 ffmpeg -i INPUT1 -i INPUT2 -i INPUT3 -filter_complex amix=inputs=3:duration=first:dropout_transition=3 OUTPUT inputs The number of inputs. If unspecified, it defaults to 2.//输入的数量，如果没有指明，默认为2. duration How to determine the end-of-stream.//决.
从零开始学习音视频编程技术（二十） 录屏软件开发之录屏生成MP4
qq214517703的专栏
04-27 4512
原文地址：http://blog.yundiantech.com/?log=blog&id=27 到这里我们已经知道如何生成视频文件了。 接下来就用屏幕图像和麦克风的声音合成视频。 首先是使用ffmpeg获取屏幕图像，捕获屏幕一样使用libavdevice。 在Windows系统使用libavdevice抓取屏幕数据有两种方法：gdigrab和d
FFmpeg - 编码/录制麦克风声音并保存为mp3文件
04-16
■ 本源码运行环境为Vs+FFmpeg，主要功能编码/录制麦克风声音并保存为mp3文件 ■ 本源码对应的文章介绍：https://blog.csdn.net/automoblie0/article/details/126108806 ■ 本源码对应的运行效果介绍：...
FFmpeg - 录制Windows屏幕与麦克风(音视频同步)，生成Mp4文件（附源码）
最新发布
04-16
■ 本源码运行环境为Vs+FFmpeg，主要功能录制Windows屏幕与麦克风(音视频同步)，生成Mp4文件 ■ 本源码对应的文章介绍：https://blog.csdn.net/automoblie0/article/details/126056946 ■ 如需协助，csdn私信或...
FFmpeg - 录制声音生成aac文件
02-08
本资源包含完整的Vs+Qt+FFmpeg+sdl代码。 资源介绍：https://blog.csdn.net/automoblie0/article/details/109079515 资源视频介绍：https://www.bilibili.com/video/BV1nf4y1A7h9/?spm_id_from=333.999.0.0 【视频中...
demo4.rar：《第4课 FFmpeg播放本地mp4文件》对应源文件
02-02
要显示视频画面，就要先用FFmpeg读取本地mp4文件，mp4文件中既有视频流也有音频流，FFmpeg对其中的视频流进行解码获取到视频帧，然后将视频帧用openCV转换为Mat数组，把Mat数组绘制到MFC窗口就可以达到上图显示的...
FFmpeg - 录制windows系统声音（代码方式）
02-08
本资源包含完整的Vs+Qt+FFmpeg代码。 资源介绍：https://blog.csdn.net/automoblie0/article/details/126107825 资源视频介绍：https://www.bilibili.com/video/BV1uG4y1v7ye/?spm_id_from=333.999.0.0 【视频中展示...
ffmpeg filter amix混音实现
m0_60259116的博客
08-29 1483
ffmpeg filter amix混音实现
Qt5+FFmpeg实现同时采集屏幕+麦克风+系统音频的录屏软件
热门推荐
心流剑的博客
08-11 1万+
标题
ffmpeg录制麦克风声音和pc内部声音(如播放歌曲)---混音
tusong86的博客
11-25 1801
vs版本：vs2017 ffmpeg版本： ffmpeg version N-102642-g864d1ef2fc Copyright © 2000-2021 the FFmpeg developers built with gcc 8.1.0 (x86_64-win32-seh-rev0, Built by MinGW-W64 project) configuration: --arch=x86_64 --disable-debug --enable-shared --disable-static --e
ffmpeg4教程6:采集桌面图像(desktop)和声音压缩为mp4或推流rtmp
Java_lilin的专栏
12-20 1537
基于vs2017 vc++  ffmpeg4.0.2下测试 ffmpeg 环境配置请百度（vs2017 ffmpeg ）图像和声音请安装screencapturer便于查找  部分方法在https://blog.csdn.net/Java_lilin/article/details/85118365中查找  需要pthread #include "pch.h" #include &lt;i...
FFmpeg实时采集麦克风延迟
yin_zhuang的博客
03-17 1347
FFmpeg实时采集遇到的问题(1) ffmpeg读取DirectShow，音频麦克风设备默认缓冲（通常为500ms的倍数），因此麦克风实时输出默认有500ms的延迟，这个值设置过地会降低性能。 directshow选项参数 audio_buffer_size: 以milliseconds为单位设置音频缓冲大小（它可以直接影响延迟，则取决于以来的设备），默认设备缓冲为500ms倍数。 pixel_
ffmpeg录制桌面，麦克风和系统声音独立成2路音轨
tusong86的博客
12-24 1538
之前本人写过下面一篇博客： ffmpeg 本地麦克风声音和系统声音混音后，再混合本地桌面成最终的mp4文件-修正 里面的麦克风声音和系统声音利用滤镜进行了混音处理，昨天夜晚再想如果麦克风声音和系统声音不混合，各自成一路音轨，最终生成的mp4文件能否播放，是否能达到预期的效果。 结果就是不及预期，生成视频文件2021-12-24T12-09-09.mp4的两路音轨，只能播放序号小的音轨，没法同时播放，我尝试了很多播放器，都是这样，vlc和potplayer做的好一些，可以检测到音轨的数量，并让用户选择播放哪一
ffmpeg录制桌面视频和麦克风音频(音视频同步)
tusong86的博客
11-20 1970
vs版本:2017 ffmpeg版本号： ffmpeg version N-102642-g864d1ef2fc Copyright © 2000-2021 the FFmpeg developers built with gcc 8.1.0 (x86_64-win32-seh-rev0, Built by MinGW-W64 project) configuration: --arch=x86_64 --prefix=/home/ffmpeg_static_x64 --disable-debug liba
FFMpeg-12、自带filter实现混音(系统音和麦克风音混合)
zw1996的博客
12-04 1888
获取系统音设备名称Setup Screen Capturer Recorder ，使用ffmpeg自带filter实现系统音和麦克风录制声音进行混音，以及一路静音的场景。
ffmpeg 读取本地MP4文件播放
07-22
要使用FFmpeg读取本地MP4文件并进行播放，您可以按照以下步骤进行操作： 1. 安装FFmpeg：首先，确保您的系统上已经安装了FFmpeg。您可以从FFmpeg官方网站（https://www.ffmpeg.org/）下载最新的二进制发行版，或者通过包管理器（如apt、yum等）在您的系统上安装。 2. 使用FFmpeg命令行工具：打开终端，并使用以下命令读取并播放MP4文件： ```shell ffmpeg -i input.mp4 -f sdl "My Video" ``` 其中，`input.mp4`是您要播放的MP4文件的路径。该命令将使用FFmpeg命令行工具读取输入的MP4文件，并将其显示在一个简单的SDL窗口中。 如果您系统中没有安装SDL库，可以尝试使用其他输出选项，例如： ```shell ffmpeg -i input.mp4 -f x11grab -video_size 640x480 -framerate 30 -i :0.0 -c:v rawvideo -pix_fmt yuv420p "My Video" ``` 这个命令将使用x11grab设备捕获屏幕上的图像，并将其显示在FFmpeg创建的窗口中。 注意：这些命令只是简单示例，您可以根据需要进行调整和扩展。 3. 自定义播放器：如果您想要更灵活和定制的播放器，可以使用FFmpeg的API和库来构建自己的播放器应用程序。FFmpeg提供了强大的多媒体处理功能和丰富的API，使您能够对媒体文件进行更细粒度的控制和处理。 您可以使用C或其他支持FFmpeg的编程语言（如Python）来编写自己的播放器应用程序。在您的代码中，您需要使用FFmpeg API打开、解码和渲染媒体文件。具体的实现细节会根据您的需求和编程语言而有所不同。 希望以上信息对您有所帮助！如果您还有其他问题，请随时提问。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 9
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值