2021-10-15

NVIDIA Developer Forums
IP Camera RTSP + GSTREAMER + C++ (Stream Latency 4-5 seconds)
Intelligent Video Analytics
General
​
NVIDIA GTC 21: Access technical training and sessions built for developers

Register for Free - GTC 2021: #1 AI Conference

IP Camera RTSP + GSTREAMER + C++ (Stream Latency 4-5 seconds)
Accelerated Computing
Intelligent Video Analytics
General
Home

rohitnairkp
20 年 2 月
[b]Hi,

I have a the application reads rtsp stream address from the xml file where we mention the i.e rtsp or webcam and streams the rtsp camera with a latency of 4-5 seconds whereas usb webcam is in realtime.

I am using HPE DL385 servers Ubuntu 18.04 server edition OS with cuda 10.1 and two Tesla T4 GPU cards & opencv 3.4.1

I also tried the gstreamer pipeline which gives 0.3sec latency and thats almost a realtime stream

gst-launch-1.0 rtspsrc location=rtsp://root:Glueck321@10.0.1.36:554/axis-media/media.amp?streamprofile=H264 latency=200 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink
I wanted to merge the gstreamer pipeline to my c++ code where I capture the rtsp stream. I just added the whole pipeline into the c++ code as

// RTSP or Video File
void open(const std::string url_){
// this->url = url_;
// cap.open(this->url);
this->url=“rtspsrc location=”;
this->url.append(url_);
this->url.append(" latency=100 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! video/x-raw , format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink$
// this->url.append(" latency =100 ! nvvidconv ! video/x-raw(memory:NVMM),format=RGBA ! nvoverlaysink “);
cout<<url<<endl;
cap.open(this->url,CV_CAP_GSTREAMER);
if (url.find(“http”) != std::string::npos && url.find(”@") != std::string::npos){
this->isCamera_ = true;
this->restart = true;
}
else{
this->restart = false;
this->isCamera_ = false;
}

    }

I have also attached my old VideoManager part where i added the pipe along with this

class VideoManager{
public:
VideoManager(){this->isCamera_ = false;}
~VideoManager(){cap.release();}
// WebCam
void open(const int camera_id){
cap.open(camera_id);
this->restart = false;
this->isCamera_ = true;
}
// RTSP or Video File
void open(const std::string url_){
this->url = url_;
// cap.open(this->url);
cap(this->url);
if (url.find(“http”) != std::string::npos && url.find("@") != std::string::npos){
this->isCamera_ = true;
this->restart = true;
}
else{
this->restart = false;
this->isCamera_ = false;
}
}
// Retrieve all camera settings
void retrieve(cv::Mat &frame){
if (this->restart){
cap.open(this->url);
std::map<int, double>::iterator iter = this->camera_params.begin();
while (iter != this->camera_params.end()){
cap.set(iter->first, iter->second);
iter++;
}
}
cap >> frame;
}
// Set camera setting
void set(int propId, double value){
if (camera_params.find(propId) == camera_params.end())
camera_params.insert(std::pair<int, double>(propId, value));
else{
camera_params[propId] = value;
}
this->cap.set(propId, value);
}
// Get camera setting
double get(int propId){return this->cap.get(propId);}
bool isOpened(){return cap.isOpened();}
void release(){this->cap.release();}
bool isCamera(){return isCamera_;}
std::map<int, double> camera_params;
std::string url;
cv::VideoCapture cap;
bool isCamera_;
private:
bool restart;
};
I also tried changing the pipelines. But while compiling and running the code it gives a error’s as

• VIDEOIO ERROR: V4L: device rtspsrc location=rtsp://root:root@10.0.1.3:554/axis-media/media.amp?streamprofile=H264 latency=100 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! video/x$

• OpenCV(3.4.1) Error: Unspecified error (GStreamer: unable to start pipeline ) in icvStartPipeline, file /home/glueck/Downloads/opencv/modules/videoio/src/cap_gstreamer.cpp, line 450 terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(3.4.1) /home/glueck/Downloads/opencv/modules/videoio/src/cap_gstreamer.cpp:450: error: (-2) GStreamer: unable to start pipeline in function icvStartPipeline Aborted (core dumped)

I have installed Gstreamer 1.14.8 and also have compiled it along with opencv 3.4.1 by -DWITH_GSTREAMER=ON -DWITH_LIBV4L=ON -DWITH_V4L=ON

I am stuck at this place where I have to merge the pipeline into my application. Can anyone suggest a way to run the pipeline for the GPU through the c++ code mentioned above.

Can anyone help me on this?

[/b]

Hi, Firstly, you need to use appsink instead of autovideosink, since opencv need to get frame from GStreamer pipeline, but autovideosink can’t support it. After you change to appsink, if it still does not work, I think it may be related to your opencv installation. I tried below sample on Jetson w…

创建
20 年 2 月
最后回复
10 月 13 日
5
回复

mchi
Moderator
20 年 3 月
Can you run the exact same gstreamer command line with gst-launch-1.0 as you are using in the opencv gstreamer command line?

rohitnairkp
20 年 3 月
Can you run the exact same gstreamer command line with gst-launch-1.0 as you are using in the opencv gstreamer command line?

Yes I can but cannot run when integrated with the application.

mchi
Moderator

1
20 年 3 月
Hi,
Firstly, you need to use appsink instead of autovideosink, since opencv need to get frame from GStreamer pipeline, but autovideosink can’t support it.

After you change to appsink, if it still does not work, I think it may be related to your opencv installation. I tried below sample on Jetson which has OpenCV4 pre-installed, I can run the pipeline and capture the frame.

#include <opencv2/opencv.hpp>
#include <opencv2/videoio.hpp>
using namespace cv;

#include
using namespace std;

int main()
{
std::string pipe = “rtspsrc location=rtsp://freja.hiof.no:1935/rtplive/definst/hessdalen03.stream ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! appsink”;

VideoCapture cap(pipe, CAP_GSTREAMER);

if (!cap.isOpened()) {
    cerr <<"VideoCapture not opened"<<endl;
    exit(-1);
}

while (true) {
    Mat frame;

    cap.read(frame);

    imwrite("receiver.png", frame);

    getchar();
}

return 0;

}