Android Dev Intro - MediaCodec Codec-specific Data

https://developer.android.com/reference/android/media/MediaCodec.html

Codec-specific Data

Some formats, notably AAC audio and MPEG4, H.264 and H.265 video formats require the actual data to be prefixed by a number of buffers containing setup data, or codec specific data. When processing such compressed formats, this data must be submitted to the codec after start() and before any frame data. Such data must be marked using the flag BUFFER_FLAG_CODEC_CONFIG in a call to queueInputBuffer.

Codec-specific data can also be included in the format passed to configure in ByteBuffer entries with keys "csd-0", "csd-1", etc. These keys are always included in the track MediaFormat obtained from the MediaExtractor. Codec-specific data in the format is automatically submitted to the codec uponstart(); you MUST NOT submit this data explicitly. If the format did not contain codec specific data, you can choose to submit it using the specified number of buffers in the correct order, according to the format requirements. Alternately, you can concatenate all codec-specific data and submit it as a single codec-config buffer.

Android uses the following codec-specific data buffers. These are also required to be set in the track format for proper MediaMuxer track configuration. Each parameter set and the codec-specific-data sections marked with (^*) must start with a start code of "\x00\x00\x00\x01".

Format	CSD buffer #0	CSD buffer #1	CSD buffer #2
AAC	Decoder-specific information from ESDS*	Not Used	Not Used
VORBIS	Identification header	Setup header	Not Used
OPUS	Identification header	Pre-skip in nanosecs (unsigned 64-bit native-order integer.) This overrides the pre-skip value in the identification header.	Seek Pre-roll in nanosecs (unsigned 64-bit native-orderinteger.)
MPEG-4	Decoder-specific information from ESDS*	Not Used	Not Used
H.264 AVC	SPS (Sequence Parameter Sets*)	PPS (Picture Parameter Sets*)	Not Used
H.265 HEVC	VPS (Video Parameter Sets*) + SPS (Sequence Parameter Sets*) + PPS (Picture Parameter Sets*)	Not Used	Not Used

Note: care must be taken if the codec is flushed immediately or shortly after start, before any output buffer or output format change has been returned, as the codec specific data may be lost during the flush. You must resubmit the data using buffers marked with BUFFER_FLAG_CODEC_CONFIGafter such flush to ensure proper codec operation.

Encoders (or codecs that generate compressed data) will create and return the codec specific data before any valid output buffer in output buffers marked with the codec-config flag. Buffers containing codec-specific-data have no meaningful timestamps.