目录
目录
2.2安装nginx-http-flv-module，与nginx链接
4.1jetson nano端两个gstreamer命令：（一个发双目rtmp视频流，一个把双目转单目rtmp视频流）
4.2服务器端gstreamer命令（用于保存双目摄像头的左右目视频）：
1. 简介
zed相机的数据采集需要一台带有gpu的计算机设备连接，经过调研，选用jetson nano边缘设备来连接zed相机，我们采用的方案是将zed相机采集数据传输到jetson nano，然后使用gstreamer结合zedsrc，将zed相机双目图像推流成rtmp数据流。一方面，在jetson nano段执行另一个gstreamer流，将zed双目图像数据流裁剪出单目rtmp流，以此来提供监控需求，另一方面，原先推出的双目图像rtmp流用于双目图像数据保存，对后续双目图像的处理提供条件。
具体架构如下图所示
2. nginx配置
2.1安装nginx
nginx需要安装nginx-http-flv-module以支持flv格式视频流，需要下载该模块，并从源码重新编译nginx，在编译nginx前还需要下载一些必要的依赖，以下参考了使用ffmpeg+nginx+flvjs实现web播放rtsp视频流
wget https://nginx.org/download/nginx-1.18.0.tar.gz
tar zxf nginx-1.18.0.tar.gz
2.2安装nginx-http-flv-module，与nginx链接
git clone https://github.com/winshining/nginx-http-flv-module.git2.2.1编译nginx-http-flv-module
首先进入刚刚解压的nginx文件夹目录下，用configure文件配置nginx-http-flv-module（如果出现找不到configure的错误，一般是/path/to/nginx-http-flv-module这个路径写错了，而且这个命令建议手打）
udo apt-get install libssl-dev
/path/to/nginx-http-flv-module为刚刚git clone的路径下的文件路径，如/home/nginx-http-flv-module
cd nginx-1.18.0
./configure --add-module=/path/to/nginx-http-flv-module
make
sudo make install
2.3nginx完整配置
sudo vi /usr/local/nginx/conf/nginx.conf 打开nginx配置文件
在这个配置里面，1935是监听rtmp流的，发送到该端口的rtmp流被接受，然后做成hls切片，做成m3u8后发到这个地址，然后前端就可以播放该地址的视频流
http://192.168.94.141:4399/hls/mystream.m3u8
#worker_processes 1; # Windows下应设置为1，因为不支持Unix域套接字
worker_processes auto; # 从1.3.8和1.2.5版本开始支持自动配置
#daemon off;
#worker_cpu_affinity 0001 0010 0100 1000; # 仅在FreeBSD和Linux上可用
#worker_cpu_affinity auto; # 从1.9.10版本开始支持自动配置
error_log logs/error.log error;
# 如果模块被编译为动态模块并且需要与RTMP相关的功能，必须指定以下命令，并且必须在events指令之前，否则模块将无法加载或在启动NGINX时加载失败
#load_module modules/ngx_http_flv_live_module.so;
events {
worker_connections 4096;
}
http {
include mime.types;
default_type application/octet-stream;
keepalive_timeout 65;
server {
listen 4399;
location / {
root /var/www;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
location /live {
flv_live on; # 开启flv实时流（订阅）
chunked_transfer_encoding on; # 开启'Transfer-Encoding: chunked'响应
add_header 'Access-Control-Allow-Origin' '*'; # 添加额外的HTTP标头
add_header 'Access-Control-Allow-Credentials' 'true'; # 添加额外的HTTP标头
}
location /hls {
types {
application/vnd.apple.mpegurl m3u8;
video/mp2t ts;
}
root /tmp;
expires -1;
add_header Cache-Control no-cache;
add_header 'Access-Control-Allow-Origin' '*';
add_header Access-Control-Allow-Credentials 'true';
add_header Access-Control-Allow-Headers "Origin, X-Requested-With, Content-Type, Accept";
add_header Access-Control-Allow-Methods "GET, POST, OPTIONS";
}
location /stat {
# 流媒体和录制统计的配置
rtmp_stat all;
rtmp_stat_stylesheet stat.xsl;
}
location /stat.xsl {
root /var/www/rtmp; # 指定stat.xsl所在的位置
}
# 如果需要JSON样式的统计，无需指定stat.xsl，而是使用新的指令rtmp_stat_format
#location /stat {
# rtmp_stat all;
# rtmp_stat_format json;
#}
location /control {
rtmp_control all; # rtmp控制模块的配置
}
}
}
rtmp_auto_push on;
rtmp_auto_push_reconnect 1s;
rtmp_socket_dir /tmp;
rtmp {
out_queue 4096;
out_cork 8;
max_streams 128;
timeout 15s;
drop_idle_publisher 15s;
log_interval 5s; # 日志模块用于记录在access.log中的时间间隔，对于调试非常有用
log_size 1m; # 日志模块用于记录access.log中的缓冲区大小
server {
listen 1935;
server_name www.test.*; # 用于后缀通配符匹配虚拟主机名
application myapp {
live on;
gop_cache on; # 开启GOP缓存，减少视频第一帧的等待时间
}
application hls {
live on;
hls on;
hls_path /tmp/hls;
hls_fragment 3s;
#hls_max_fragment 3s;
hls_cleanup on;
hls_playlist_length 9s;
}
}
}
3. gstreamer
3.1ubuntu18.04安装gstreamer
（也可以忽略此章，直接看下面zedsrc的安装，里面就有安装gstreamer）
安装
apt-get install libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio验证安装
dpkg -l | grep gstreamer（什么都没显示就是没安上了）
3.2gstreamer的zedsrc
我们的目标包括要获取zed的双目图像，其中可以尝试v4l2src，直接读取硬件端口的zed相机，但这种模式下，没办法使用zed相机自带的一些sdk的功能，而且也只能默认读取双目的，要想进一步拿单目的，也可以像下面做的一样，把rtmp裁剪
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=2560,height=720,framerate=30/1 ! videoconvert ! x264enc tune=zerolatency bitrate=5000 speed-preset=superfast ! h264parse ! mpegtsmux ! hlssink target-duration=5 playlist-root=http://192.168.94.141:5959/zedstream/安装zedsrc只需要安装官网的方法，首先正确安装zed sdk，然后安装gstreamer，然后再下载zedsrc的包，编译即可。具体操作流程如下链接。
https://github.com/stereolabs/zed-gstreamer
4. gstreamer推流
4.1jetson nano端两个gstreamer命令：（一个发双目rtmp视频流，一个把双目转单目rtmp视频流）
gst-launch-1.0 zedsrc camera-resolution=1 camera-fps=15 stream-type=2 ! videoconvert ! x264enc key-int-max=1 bitrate=20000 ! flvmux ! rtmpsink location="rtmp://172.17.75.4:1935/hls/mystream"
gst-launch-1.0 rtmpsrc location="rtmp://172.17.75.4:1935/hls/mystream" ! flvdemux name=demux demux.video ! queue ! h264parse ! avdec_h264 ! videoconvert ! videocrop top=0 bottom=1242 left=0 right=0 ! x264enc key-int-max=1 bitrate=20000 ! flvmux ! rtmpsink location="rtmp://172.17.75.4:1935/hls/mystream2"其中比较关键的参数在于key-int-max=1的设置，设置关键帧间隔为1，这样才能让nginx的hls在裁剪视频流时，找到关键帧，如果不设置关键帧，在nginx端接收到rtmp流时，nginx会报错，说在将rtmp流做成hls切片时，就算再nginx端设置了hls切片最大长度，也因为没有关键帧的原因导致hls切片没法确定切片，最后超过限制nginx强制切片为50s左右一个切片，这就会导致m3u8文件非常巨大，并且没有关键帧，非常卡顿。在设置key-int-max=1后效果提升显著。
其中videocrop top=0 bottom=1242 left=0 right=0 为将原本穿出的上下布局的左右目图片，保留一个作为监控视频流，原本的双目左右目视频流用于源双目视频数据获取。
其中camera-resolution=0为指定2k分辨率，streame-type=2为指定双目图像均采集，以上参数均可在zed gstreamer github观看对应参数。bitrate=20000为指定比特率，较高画质需指定高点的比特率，
4.2服务器端gstreamer命令（用于保存双目摄像头的左右目视频）：
然后这个gstream的命令是写入rtmp流的，所以先得配置nginx在一个服务器上，往那个服务器上推
gst-launch-1.0 rtmpsrc location="rtmp://172.17.75.4/hls/mystream" ! flvdemux name=demux demux.video ! queue ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw,format=I420 ! videorate ! video/x-raw,framerate=1/1 ! tee name=tee tee. ! queue ! videocrop top=0 bottom=720 ! videoconvert ! video/x-raw,format=I420 ! jpegenc ! multifilesink location="/opt/jupyter/zxg/pwcnet/images/testimage/left_frame_%04d.jpg" tee. ! queue ! videocrop top=720 bottom=0 ! videoconvert ! video/x-raw,format=I420 ! jpegenc ! multifilesink location="/opt/jupyter/zxg/pwcnet/images/testimage/right_frame_%04d.jpg"由于在我们的场景中，jetson nano和服务器在同一个局域网下，所以直接从jetson nano端发出的rtmp流用multifilesink保存图片，根据当前时间戳保存，用supervisor挂载在指定数据集文件夹下。
但由于zed相机fps设置限制，每秒帧数较多，导致保存的图片数量较大，目前尚未解决，加了一个定时删除文件夹下图片的脚本，但该脚本会和gstreamer命令中，按保存个数的形式命名文件相冲突left_frame_%04d.jpg。
5. 过程出现的问题
暂未解决问题：
1.如果摄像头选择2k、1080p的分辨率，真实帧率只能达到每秒5帧，最后的监控效果比较差，经常卡顿。所以目前使用的是720p的分辨率。目前像素较差。
2.zed相机和jetson nano，用supervisor的方式挂载zed相机发送gstreamer流，现在运行5-8个小时左右，会掉线报错the camera not detected。并且在硬件列表看不到zed的硬件，需要重启等待或拔插相机。
3.当前从gstreamer的rtmp流保存的图片数量较多，需5分钟清除一次保存图片，大概10小时就能存满200g的内存（可用脚本控制保存图片，暂未实现）
已解决问题的部分链接（只记得比较麻烦的问题）：
https://forums.developer.nvidia.com/t/zed-cam/75003
a website about zed python important
https://github.com/stereolabs/zed-sdk/issues/574
https://github.com/stereolabs/zed-sdk/issues/333
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
uff 0.6.9 requires protobuf>=3.3.0, but you have protobuf 3.0.0 which is incompatible.
problem:
running python.py
Illegal instruction (core dumped)
s:
export OPENBLAS_CORETYPE=ARMV8
https://github.com/stereolabs/zed-sdk/issues/333
一个安装numpy出现的问题，重新安装也没用，需要升级pip
Running setup.py bdist_wheel for numpy ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-qyl8lkuw/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp56_aqkjdpip-wheel- --python-tag cp36:
Running from numpy source directory.
Cythonizing sources
performance hint: _generator.pyx:707:41: Exception check after calling '_shuffle_int' will always require the GIL to be acquired.
Possible solutions:
1. Declare '_shuffle_int' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
2. Use an 'int' return type on '_shuffle_int' to allow an error code to be returned.
performance hint: _generator.pyx:736:45: Exception check after calling '_shuffle_int' will always require the GIL to be acquired.
Possible solutions:
1. Declare '_shuffle_int' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
2. Use an 'int' return type on '_shuffle_int' to allow an error code to be returned.
Error compiling Cython file:
------------------------------------------------------------
...
cdef sfc64_state rng_state
def __init__(self, seed=None):
BitGenerator.__init__(self, seed)
self._bitgen.state = <void *>&self.rng_state
self._bitgen.next_uint64 = &sfc64_uint64
^
------------------------------------------------------------
_sfc64.pyx:90:35: Cannot assign type 'uint64_t (*)(void *) except? -1 nogil' to 'uint64_t (*)(void *) noexcept nogil'. Exception values are incompatible. Suggest adding 'noexcept' to the type of the value being assigned.
Processing numpy/random/_bounded_integers.pxd.in
Processing numpy/random/_generator.pyx
Processing numpy/random/_sfc64.pyx
Traceback (most recent call last):
File "/tmp/pip-build-qyl8lkuw/numpy/tools/cythonize.py", line 235, in <module>
main()
File "/tmp/pip-build-qyl8lkuw/numpy/tools/cythonize.py", line 231, in main
find_process_files(root_dir)
File "/tmp/pip-build-qyl8lkuw/numpy/tools/cythonize.py", line 222, in find_process_files
process(root_dir, fromfile, tofile, function, hash_db)
File "/tmp/pip-build-qyl8lkuw/numpy/tools/cythonize.py", line 188, in process
processor_function(fromfile, tofile)
File "/tmp/pip-build-qyl8lkuw/numpy/tools/cythonize.py", line 78, in process_pyx
[sys.executable, '-m', 'cython'] + flags + ["-o", tofile, fromfile])
File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'cython', '-3', '--fast-fail', '-o', '_sfc64.c', '_sfc64.pyx']' returned non-zero exit status 1.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-qyl8lkuw/numpy/setup.py", line 508, in <module>
setup_package()
File "/tmp/pip-build-qyl8lkuw/numpy/setup.py", line 488, in setup_package
generate_cython()
File "/tmp/pip-build-qyl8lkuw/numpy/setup.py", line 285, in generate_cython
raise RuntimeError("Running cythonize failed!")
RuntimeError: Running cythonize failed!
----------------------------------------
Failed building wheel for numpy
Running setup.py clean for numpy
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-qyl8lkuw/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" clean --all:
Running from numpy source directory.
`setup.py clean` is not supported, use one of the following instead:
- `git clean -xdf` (cleans all files)
- `git clean -Xdf` (cleans all versioned files, doesn't touch
files that aren't checked into the git repo)
Add `--force` to your command to use it anyway if you must (unsupported).
----------------------------------------
Failed cleaning build dir for numpy
Failed to build numpy
Installing collected packages: numpy
Running setup.py install for numpy ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-qyl8lkuw/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-bo0k6la5-record/install-record.txt --single-version-externally-managed --compile --user --prefix=:
Running from numpy source directory.
Note: if you need reliable uninstall behavior, then install
with pip instead of using `setup.py install`:
- `pip install .` (from a git repo or downloaded source
release)
- `pip install numpy` (last NumPy release on PyPi)
Cythonizing sources
Error compiling Cython file:
------------------------------------------------------------
...
cdef sfc64_state rng_state
def __init__(self, seed=None):
BitGenerator.__init__(self, seed)
self._bitgen.state = <void *>&self.rng_state
self._bitgen.next_uint64 = &sfc64_uint64
^
------------------------------------------------------------
_sfc64.pyx:90:35: Cannot assign type 'uint64_t (*)(void *) except? -1 nogil' to 'uint64_t (*)(void *) noexcept nogil'. Exception values are incompatible. Suggest adding 'noexcept' to the type of the value being assigned.
numpy/random/_bounded_integers.pxd.in has not changed
numpy/random/_generator.pyx has not changed
Processing numpy/random/_sfc64.pyx
Traceback (most recent call last):
File "/tmp/pip-build-qyl8lkuw/numpy/tools/cythonize.py", line 235, in <module>
main()
File "/tmp/pip-build-qyl8lkuw/numpy/tools/cythonize.py", line 231, in main
find_process_files(root_dir)
File "/tmp/pip-build-qyl8lkuw/numpy/tools/cythonize.py", line 222, in find_process_files
process(root_dir, fromfile, tofile, function, hash_db)
File "/tmp/pip-build-qyl8lkuw/numpy/tools/cythonize.py", line 188, in process
processor_function(fromfile, tofile)
File "/tmp/pip-build-qyl8lkuw/numpy/tools/cythonize.py", line 78, in process_pyx
[sys.executable, '-m', 'cython'] + flags + ["-o", tofile, fromfile])
File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'cython', '-3', '--fast-fail', '-o', '_sfc64.c', '_sfc64.pyx']' returned non-zero exit status 1.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-qyl8lkuw/numpy/setup.py", line 508, in <module>
setup_package()
File "/tmp/pip-build-qyl8lkuw/numpy/setup.py", line 488, in setup_package
generate_cython()
File "/tmp/pip-build-qyl8lkuw/numpy/setup.py", line 285, in generate_cython
raise RuntimeError("Running cythonize failed!")
RuntimeError: Running cythonize failed!
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-qyl8lkuw/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-bo0k6la5-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-qyl8lkuw/numpy/
解决方法升级一下pip就可以了，
python3 -m pip install --upgrade pip
问题：zed的numpy好像是1.19.5,会报错，安装1.19.4就可以了
Illegal instruction (core dumped)
python3 --m pip install numpy==1.19.4
扫一扫
3921
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
