java 解析mjpeg流,MJPEG流和解码

I want to receive JPEG images from an IP camera (over RTSP). For this, I tried cvCreateFileCapture_FFMPEG in OpenCV. But ffmpeg seems to have some problem with the MJPEG format of the streaming (since it automatically tries to detect the streaming info) and I end up with the following error

mjpeg: unsupported coding type

I, then, decided to use live555 for streaming. Till now, I can successfully establish streaming and capture (non-decoded) images through openRTSP.

The question is how can I do this in my application, e.g., in OpenCV. How can I use openRTSP in OpenCV to get images and save them in JPEG format?

I have heard that the data from openRTSP can be sent to a buffer (or a named pipe) and then read in OpenCV's IplImage. But I don't know how to do this.

I will really appreciate any help/suggestion in about this problem. I need answers of either of the following questions:

How can I disable ffmpeg's automatic stream information detection and specify my own format (mjpeg), or

How can I use openRTSP in OpenCV?

Regards,

解决方案

Is this an Axis IP camera? Either way, most IP cameras that provide MPEG4 RTSP stream that can be decoded using OpenCV using cvCreateFileCapture_FFMPEG. However, ffmpeg decoder's MJPEG codec has a widely known unresolved issues. I am sure you would have received an error similar to

[ingenient @ 0x97d20c0]Could not find codec parameters (Video: mjpeg)

Option1 : Using opencv, libcurl and libjpeg

To view mjpeg stream in opencv take a look at the following implementation

Option2: Using gstreamer (no opencv)

I would recommend looking at gstreamer if your goal is to just view or save jpeg images

To view MJPEG stream one may execute media pipeline string as follows

gst-launch -v souphttpsrc location="http://[ip]:[port]/[dir]/xxx.cgi" do-timestamp=true is_live=true ! multipartdemux ! jpegdec ! ffmpegcolorspace ! autovideosink

For RTSP

gst-launch -v rtspsrc location="rtsp://[user]:[pass]@[ip]:[port]/[dir]/xxx.amp" debug=1 ! rtpmp4vdepay ! mpeg4videoparse ! ffdec_mpeg4 ! ffmpegcolorspace! autovideosink

To work with C API see

For a simple example take a look at my other post on rtsp for constructing gstreamer C API media pipeline (This is same as gst-launch string but rather implemented as a C API)

To save MJPEG stream as multiple images the pipeline (Let us put a vertical flip BIN and connect the PADS to the previous and the next BINS to make it fancier)

gst-launch souphttpsrc location="http://[ip]:[port]/[dir]/xxx.cgi" do-timestamp=true is_live=true ! multipartdemux ! jpegdec ! videoflip method=vertical-flip ! jpegenc ! multifilesink location=image-out-%05d.jpg

Also maybe worthwhile have a look at gst-opencv

UPDATE:

Option3: Using gstreamer, Named Pipe and opencv

On Linux one may get mjpeg stream and convert it to mpeg4 and feed it to a named pipe. Then read the data from the named pipe in opencv

Step 1. Create Named Pipe

mkfifo stream_fifo

Step 2. Create opencvvideo_test.c

// compile with gcc -ggdb `pkg-config --cflags --libs opencv` opencvvideo_test.c -o opencvvideo_test

#include

#include "highgui.h"

#include "cv.h"

int main( int argc, char** argv){

IplImage *frame;

int key;

/* supply the AVI file to play */

assert( argc == 2 );

/* load the AVI file */

CvCapture *capture = cvCreateFileCapture(argv[1]) ;//cvCaptureFromAVI( argv[1] );

/* always check */

if( !capture ) return 1;

/* get fps, needed to set the delay */

int fps = ( int )cvGetCaptureProperty( capture, CV_CAP_PROP_FPS );

int frameH = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT);

int frameW = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH);

/* display video */

cvNamedWindow( "video", CV_WINDOW_AUTOSIZE );

while( key != 'q' ) {

double t1=(double)cvGetTickCount();

/* get a frame */

frame = cvQueryFrame( capture );

double t2=(double)cvGetTickCount();

printf("time: %gms fps: %.2g\n",(t2-t1)/(cvGetTickFrequency()*1000.), 1000./((t2-t1)/(cvGetTickFrequency()*1000.)));

/* always check */

if( !frame ) break;

/* display frame */

cvShowImage( "video", frame );

/* quit if user press 'q' */

key = cvWaitKey( 1000 / fps );

}

/* free memory */

cvReleaseCapture( &capture );

cvDestroyWindow( "video" );

return 0;

}

Step 3. Prepare To Convert From MJPEG to MPEG4 using gstreamer (rate of incoming frames critical)

gst-launch -v souphttpsrc location="http:///cgi_bin/.cgi" do-timestamp=true is_live=true ! multipartdemux ! jpegdec ! queue ! videoscale ! 'video/x-raw-yuv, width=640, height=480'! queue ! videorate ! 'video/x-raw-yuv,framerate=30/1' ! queue ! ffmpegcolorspace ! 'video/x-raw-yuv,format=(fourcc)I420' ! ffenc_mpeg4 ! queue ! filesink location=stream_fifo

Step 4. Display Stream in OpenCV

./opencvvideo_test stream_fifo