ISP components and communication channels

ISP Hardware

The ISP hardware takes the video input and produces a video output according to the registers configuration. The hardware can take multiple different video inputs with its Multi-Channel Front End (MCFE) module. The hardware can also generate multiple video outputs with its Multi-Channel Back End (MCBE) module.

ISP Software

The ISP software controls both the sensor and the ISP hardware. It runs 2A algorithms with calibrated parameters, and also provides a command API interface for both debugging and tuning tasks.

Control Tool

The ACT software provides direct access to the ISP hardware and software for debugging and tuning purposes.

RAM

RAM is used as intermediate storage to keep the ISP register backup for a different slot. RAM is also used for the MCFE abstraction of stream context, and both raw and output buffers that retain the video input and output data.

External video Receiver

Any external hardware that processes the ISP video streaming output as an input signal.

Camera sensors

The camera sensors are a direct source of video input for the ISP hardware. The ISP can use multiple sensors in two different ways:

1. Local sensors: The first way is taking image stream on its front-end pipeline with its inputports.

2. Remote sensors: The second way is taking the image data directly from a buffer in thesystem memory.

The ISP supports up to four input ports and one context is available for each Multi-ContextFront-End slot, up to a maximum of 16. Therefore, the ISP can have a total of 16 sensors, andup to four local sensors on input ports. The ISP software solution only supports control for local sensors, and does not offer any control functionality for remote sensors.

1. Video I/O

The ISP can take two different types of video input and can produce two different types of video output. The following figure shows an overview of the blocks available in the Video I/O area of the ISP hardware:

略图

Streaming video input

Video input that is streamed from local sensors. The video input stream can be delivered directly to the MCFE in streaming mode, or indirectly through raw buffers to the MCFE in Time Division Multiplexed Frames(TDMF) mode. 

Streaming video output

The video output streams into the external video Rx. The Multi-Channel Back End(MCBE) supports up-to three streaming output planes.

例如：

1）直接流模式（Streaming Mode）Local sensors---Video input (streaming)---> Front-end pipeline--->MCFE--->Back-end pipeline--->MCBE--Video output(streaming)-->External video Receiver

2)时分复用帧模式（Time Division Multiplexed Frames，TDMF 模式）Local sensors---Video input (streaming)---> Front-end pipeline--->RAM(Raw buffers)--->MCFE--->Back-end pipeline--->MCBE--Video output(streaming)-->External video Receiver

Memory-to-Memory video input

Video input that has been dumped into the raw buffers by remote sensors. MCFE can take these inputs in its Memory-to-Memory(M2M)mode. M2M operation refers the combination of video input memory-to-memory and video output AXI, which takes memory both as an input and an output.

Video output AXI

Dumps the video output into RAM. The MCBE supports up-to three AXI output planes.

例如：

Remote sensors --Video input using Memory-to-Memory(M2M) --> RAM(Raw buffers) ---> MCFE ---> Back-end pipeline ---> MCBE --Video output(AXI)--> RAM(Out buffers) 

2. Sensor control channel

The sensor control channel is a data bus that is used to control sensor exposure time.

For implementing the sensor control channel, the I2C bus is commonly used. However, some snesors require SPI or other bus interfaces.

3. ISP I/O

The ISP software can access the ISP hardware on two different paths. One is accessing hardware registers directly for non-Configuration Direct Memory Access(CDMA) registers, and the other is accessing CDMA data on RAM for CDMA registers.

The ISP I/O is implemented in the platform library module, where it keeps the operating system dependant code. The ISP software provides Linux I/O implemented with memory-mapped I/O kernel functions, and the bare-metal implementation accesses the registers directly using the physical address.

例如：

Platform library in ISP software ---> CDMA data in RAM

Platform library in ISP software --ISP I/O--> Register / SRAM in ISP hardware

4. ISP interrupts

The ISP hardware raises two different types of interrupts: Hardware timing interrupts and fault interrupts.

There are four ISP hardware timing interrupt categories available:

1)Start of Frame (SOF)

2)End of Frame (EOF)

3)STATS

4)MCFE

The ISP software only handles hardware timing interrupts, and does not handle fault interrupts. The interrupt handler must be implemented according to the platform that the ISP software runs on. In the release package, the Linux implementation is only provided as a reference.

5. ISP Control Tool Channel

The Control Tool uses two different channels to directly access the ISP hardware and software.

The Control Tool hardware channel

Used to access the non-Configuration Direct Memory Access(CDMA) registers on hardware or the CDMA registers on RAM.

The Control Tool software channel

Used to control the ISP software and access the internal data. The Command API allows the ACT to control the software and gives access to internal software parameters. The Calibration API allows control to have access to tuning parameters.

6. ISP Configuration Direct Memory Access (CMDA)

The ISP hardware can support multiple sensor instances. Therefore, you can store the configurations for different sensors in memory.

CDMA in

When in operation, the CDMA copies data from RAM into the register space before the ISP starts to process the next frame.

CDMA out

Reads statistical data from the register space into RAM after the ISP finishes processing the frame for the slot.

例如：

CDMA in RAM <----CDMA----> Register / SRAM