FFmpeg+安卓

FFmpeg + Android

FFmpeg+安卓

July 25, 2023 by Samuel Mira | Comments

​2023年7月25日：Samuel Mira |评论

Qt 6.5 released a new Multimedia backend using the FFmpeg library.

Qt6.5使用FFmpeg库发布了一个新的多媒体后端。

Starting with Qt 6.5.2, we enabled this feature in Qt for Android! Now, you might ask what does it mean for Qt for Android to have the new FFmpeg multimedia backend? What does it provide?

从Qt 6.5.2开始，我们在Android版Qt中启用了此功能！现在，你可能会问，Qt for Android拥有新的FFmpeg多媒体后端意味着什么？它提供了什么？

This blog post will try to answer these questions. If you have other questions, please ask in the comments.

这篇博文将试图回答这些问题。如果还有其他问题，请在评论中提问。

Media playback using FFmpeg

使用FFmpeg进行媒体播放

A core functionality of the new multimedia backend is, of course, the playback of video files.

新的多媒体后端的核心功能当然是播放视频文件。

Using FFmpeg multimedia backend adds three main advantages for the playback of video files:

使用FFmpeg多媒体后端为视频文件的播放增加了三个主要优势：

• Compatibility with different file formats - supports several different codecs and file formats, some unavailable before.
• 兼容不同的文件格式-支持几种不同的编解码器和文件格式，有些以前不可用。
• Differences between devices - different devices support different video formats. Sometimes that depends on the vendor and the codecs provided. It is difficult for some video files to be played on all devices. Using FFmpeg helps by allowing software playback in case the hardware playback is impossible.
• 设备之间的差异-不同的设备支持不同的视频格式。有时这取决于供应商和所提供的编解码器。有些视频文件很难在所有设备上播放。使用FFmpeg有助于在硬件播放不可能的情况下允许软件播放。
• Performance gain - faster to load, but still using hardware acceleration.
• 性能提升-加载速度更快，但仍使用硬件加速。

On top of this, FFmpeg is a cross-platform library. Using it in Qt for Android will bring the functionality known on other platforms.

除此之外，FFmpeg是一个跨平台的库。在Qt for Android中使用它将带来其他平台上已知的功能。

Camera2 API

By developing a whole new media backend, we cleared up some technical debt on using the APIs supplied by Android. Previously, we used the Camera API, and we changed to support the Camera2 API.

通过开发一个全新的媒体后端，我们消除了使用安卓提供的API的一些技术债务。以前，我们使用了Camera API，现在改为支持Camera2 API。

The Camera API is very stable and works well for regular app development. But to support features available on the newest Android API levels, the Camera2 API is the way to go. It opens new possibilities by allowing much better control of the capture pipeline.

相机API非常稳定，适用于常规应用程序开发。但要支持最新Android API级别上可用的功能，Camera2 API是一个不错的选择。它允许更好地控制捕获管道，从而开启了新的可能性。

Development is ongoing to support both old and new camera options, so this is not a change that app developers can use today in Qt in its final form yet, rather than a stepping stone for new features in the future.

目前正在进行开发，以支持新旧相机选项，因此这还不是应用程序开发人员今天可以在Qt中使用的最终形式的更改，而不是未来新功能的垫脚石。

Media encoding using FFmpeg

使用FFmpeg的媒体编码

After setting up a new pipeline with Camera2 API, we needed to connect the Camera API with FFmpeg encoding.

在用Camera2 API建立了一个新的管道之后，我们需要用FFmpeg编码连接Camera API。

Media encoding on Android with FFmpeg was an "Achilles heel" since software encoding is not performant enough on mobile devices due to memory and processing constraints on most devices. 

使用FFmpeg在Android上进行媒体编码是一个“阿喀琉斯之踵”，因为由于大多数设备的内存和处理限制，软件编码在移动设备上的性能不够。

Earlier FFmpeg did not offer hardware encoding in Android. But that changed on FFmpeg 6.0 "Von Neumann" major release.

早些时候，FFmpeg没有在安卓系统中提供硬件编码。但在FFmpeg6.0“冯·诺依曼”的主要发行版上，情况发生了变化。

We looked closely into the development of the hardware acceleration feature in FFmpeg, and since its release, we contributed several patches to the FFmpeg media library.

我们仔细研究了FFmpeg中硬件加速功能的开发，自其发布以来，我们为FFmpeg媒体库提供了几个补丁。

The contributed patches enable AV1, VP9, and VP8 codec encoding using hardware acceleration. These patches are already merged into the master branch but have not been released yet. They will be available in future releases of FFmpeg and Qt.

所提供的补丁支持使用硬件加速进行AV1、VP9和VP8编解码器编码。这些修补程序已经合并到master分支中，但尚未发布。它们将在未来的FFmpeg和Qt版本中提供。