CornerNet-Lite是刚开源不久的实时目标检测方法，据说比YOLO3 牛逼，于是打算尝鲜一波。
首先说运行的环境要求：
1）Python 3.7
2）PyTorch 1.0.0
3）CUDA 10 （意思是你得有一张NVIDIA的显卡，比如我的是GTX1060）
4）GCC 4.9.2 or above
1. 去github下载源码
https://github.com/princeton-vl/CornerNet-Lite
git clone https://github.com/princeton-vl/CornerNet-Lite.git2. 跟着官方readme，设置环境，这会下载一大堆东西，比如opencv, pytorch
cd <CornerNet-Lite dir>
conda create --name CornerNet_Lite --file conda_packagelist.txt --channel pytorch这可能是一个漫长的过程（实验室这破网速，几kb的下载速度玩个XX），且在下载的pytorch的时候，可能会失败，失败，再失败，请不要气馁，奇迹总会出现，比如大清早，实验室的网速最佳，成功的几率更高哦。
3. active 你的环境
conda activate CornerNet_Lite注意，这里是 conda ，不是source。执行完后，你会发现终端命令行前面多了(CornerNet_Lite)
这个时候你还会发现，当你在该终端下运行python时，默认是3.7版本了
4. 编译Corner Pooling Layers
cd <CornerNet-Lite dir>/core/models/py_utils/_cpools/
python setup.py install --user5. 编译NMS
cd <CornerNet-Lite dir>/core/external
make6. 下载模型文件
原下载地址：
百度云走起：
链接: https://pan.baidu.com/s/1gmMpx6EUOVjfIVys_POGqQ 提取码: xtzx 复制这段内容后打开百度网盘手机App，操作更方便哦
把下载的文件放在正确的目录下：
Put the CornerNet-Saccade model under <CornerNet-Lite dir>/cache/nnet/CornerNet_Saccade/, CornerNet-Squeeze model under <CornerNet-Lite dir>/cache/nnet/CornerNet_Squeeze/ and CornerNet model under <CornerNet-Lite dir>/cache/nnet/CornerNet/
7. 测试一下demo，看看效果如何
python demo.py糟糕，有问题，由于本人安装了ROS，cv2.so冲突了吗？错误如下：
Traceback (most recent call last):
File "demo.py", line 7, in <module>
import cv2
ImportError: /opt/ros/kinetic/lib/python2.7/dist-packages/cv2.so: undefined symbol: PyCObject_Type
检查一下python路径
Python 3.7.1 (default, Oct 23 2018, 19:19:42)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/home/communicationgroup/rgbdslam2_catkin_ws/devel/lib/python2.7/dist-packages', '/opt/ros/kinetic/lib/python2.7/dist-packages', '/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python37.zip', '/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7', '/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7/lib-dynload', '/home/communicationgroup/.local/lib/python3.7/site-packages', '/home/communicationgroup/.local/lib/python3.7/site-packages/cpools-0.0.0-py3.7-linux-x86_64.egg', '/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7/site-packages', '/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7/site-packages/torchvision-0.2.1-py3.7.egg']
罪魁祸首正是'/opt/ros/kinetic/lib/python2.7/dist-packages'
那就删除它吧
对demo.py加入如下两行代码
#解决ros中cv2的冲突
import sys
sys.path.remove('/opt/ros/kinetic/lib/python2.7/dist-packages')2019/4/27更新：
这样的后果是不能在.py中 import rospy，若想嵌入到ROS中，则不能采用该方法。若不用嵌入ROS，使用该方法没得问题，若要嵌入ROS中，解决办法参考后面第8节的更新。
重新运行demo.py
python demo.py终端输出大概是这样的：
total parameters: 116969339
loading from /home/communicationgroup/CornerNet-Lite/core/../cache/nnet/CornerNet_Saccade/CornerNet_Saccade_500000.pkl
/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7/site-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))结果还会在CornerNet-Lite目录下保存一张名为 demo_out.jpg 的图片
可以看出，已经把小狗仔们框出来了。
其实也并不快嘛，我的电脑（GTX1060）跑一张图像需要0.5s左右，难道是没用上GPU？？？（2019/4/27注：平均测试速度单张图片244ms， Titan X (PASCAL) GPU）
8. 尝试一下能不能用在ROS工程中
思路：相机节点通过相机获取视频，并通过ROS消息发布，图像处理节点订阅图像消息，并调用CornerNet进行目标检测。
1)C++和python参数传递
将.cpp获取的图像作为参数传给.py，实现目标识别，再将.py得到的bboxes传回给.cpp
(以后慢慢做，等更新)
2) 写一个.py节点，来订阅图像消息，并发布bboxes
由于这个CornerNet既要python3.7，又要pytorch1.0.0，还要opencv3，而ros还没有完全支持python3，所以暂时放弃该尝试。
-------------------------------------------------------------我是分割线--------------------------------------------------------------------
2019/4/27更新：
经过一番波折，实现了 .py节点的方法。测试方式：Kinect2 获取图像，test_ros.py 订阅图像消息，并调用CornerNet实现目标检测。
该方法主要是解决Python3 环境下运行ROS的问题，其中主要涉及cv_bridge，cv2的问题。网上方法形形色色，但是到我这里基本是都不得行，还害得我重装了一遍ROS。为了让大家少走弯路，这里总结一下我对Python3环境下运行ROS，并调用CornerNet的探索。
8.1.1 首先，解决import cv2的问题
前面第7节提到的方案在这里就不适用了，sys.path.remove('/opt/ros/kinetic/lib/python2.7/dist-packages') 会导致 import rospy失败。
cd /opt/ros/kinetic/lib/python2.7/dist-packages/
sudo mv cv2.so cv2_ros.so意思就是重命名一下，真是简单粗暴还管用啊!
8.1.2 解决Python3 和 cv_bridge的问题
ROS默认使用python2.7，这时我们需要从源码编译cv_bridge，并在编译前设置好Python版本为Python3
1)新建ros工作空间，用于测试
mkdir py3_ws
cd py3_ws
mkdir src2)将前面测试通过的CornerNet-Lite目录全部拷贝到 src/目录下
现在，目录结构大概是这样的 py3_ws/src/CornerNet-Lite
3）使用conda激活CornerNet-Lite环境
cd src/CornerNet-Lite/
conda activate CornerNet_Lite跟前面一样，成功的话，终端命令行前面或多出 (CornerNet_Lite)
之所以激活该环境，是因为后续需要在该环境下调用ROS，所以得先切换到该环境下，安装rospkg等包。
4) 使用pip在CornerNet_Lite环境下安装rospkg、catkin_pkg等，这样才能在该环境下调用ROS
pip install catkin_pkg pyyaml empy rospkg numpy更改CornerNet_Lite环境下opencv版本为3.4.1（CornerNet_Lite环境下的opencv版本为3.4.2，该版本在调用cv2.imshow()函数时会报错，也不知道大家的存在这个问题不）
conda install opencv==3.4.1终端输出如下
Collecting package metadata: done
Solving environment: done
## Package Plan ##
environment location: /home/communicationgroup/anaconda3/envs/CornerNet_Lite
added / updated specs:
- opencv==3.4.1
The following packages will be downloaded:
package | build
---------------------------|-----------------
ffmpeg-3.4 | h7985aa0_0 8.0 MB
libopencv-3.4.1 | h8fa1ad8_3 41.5 MB
libprotobuf-3.5.2 | h6f1eeef_0 4.2 MB
opencv-3.4.1 | py37h6fd60c2_3 8 KB
py-opencv-3.4.1 | py37h8fa1ad8_3 1.2 MB
------------------------------------------------------------
Total: 54.9 MB
The following NEW packages will be INSTALLED:
libprotobuf pkgs/main/linux-64::libprotobuf-3.5.2-h6f1eeef_0
The following packages will be DOWNGRADED:
ffmpeg 4.0-hcdf2ecd_0 --> 3.4-h7985aa0_0
libopencv 3.4.2-hb342d67_1 --> 3.4.1-h8fa1ad8_3
opencv 3.4.2-py37h6fd60c2_1 --> 3.4.1-py37h6fd60c2_3
py-opencv 3.4.2-py37hb342d67_1 --> 3.4.1-py37h8fa1ad8_3
Proceed ([y]/n)? y
Downloading and Extracting Packages
py-opencv-3.4.1 | 1.2 MB | ##################################################################################################################################### | 100%
opencv-3.4.1 | 8 KB | ##################################################################################################################################### | 100%
libopencv-3.4.1 | 41.5 MB | ##################################################################################################################################### | 100%
libprotobuf-3.5.2 | 4.2 MB | ##################################################################################################################################### | 100%
ffmpeg-3.4 | 8.0 MB | ##################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done安装完成后，你可以使用conda list查看该环境下都有哪些包，主要关注以下opencv、 catkin_pkg、 pyyaml、 rospkg
conda list输出大概是这样的 （我这里多了个opencv-python 3.4.5.20，是我自己手贱安装的）
# packages in environment at /home/communicationgroup/anaconda3/envs/CornerNet_Lite:
#
# Name Version Build Channel
blas 1.0 mkl
bzip2 1.0.6 h14c3975_5
ca-certificates 2019.1.23 0
cairo 1.14.12 h8948797_3
catkin-pkg 0.4.12 pypi_0 pypi
certifi 2019.3.9 py37_0
cffi 1.11.5 py37he75722e_1
cuda100 1.0 0 pytorch
cycler 0.10.0 py37_0
cython 0.28.5 py37hf484d3e_0
dbus 1.13.2 h714fa37_1
defusedxml 0.6.0 pypi_0 pypi
docutils 0.14 pypi_0 pypi
empy 3.3.4 pypi_0 pypi
expat 2.2.6 he6710b0_0
ffmpeg 3.4 h7985aa0_0
fontconfig 2.13.0 h9420a91_0
freeglut 3.0.0 hf484d3e_5
freetype 2.9.1 h8a8886c_1
glib 2.56.2 hd408876_0
graphite2 1.3.12 h23475e2_2
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb453b48_1
harfbuzz 1.8.8 hffaf4a1_0
hdf5 1.10.2 hba1933b_1
icu 58.2 h9c2bf20_1
intel-openmp 2019.0 118
jasper 2.0.14 h07fcdf6_1
jpeg 9b h024ee3a_2
kiwisolver 1.0.1 py37hf484d3e_0
libedit 3.1.20170329 h6b74fdf_2
libffi 3.2.1 hd88cf55_4
libgcc-ng 8.2.0 hdf63c60_1
libgfortran-ng 7.3.0 hdf63c60_0
libglu 9.0.0 hf484d3e_1
libopencv 3.4.1 h8fa1ad8_3
libopus 1.2.1 hb9ed12e_0
libpng 1.6.35 hbc83047_0
libprotobuf 3.5.2 h6f1eeef_0
libstdcxx-ng 8.2.0 hdf63c60_1
libtiff 4.0.9 he85c1e1_2
libuuid 1.0.3 h1bed415_2
libvpx 1.7.0 h439df22_0
libxcb 1.13 h1bed415_1
libxml2 2.9.8 h26e45fe_1
matplotlib 3.0.2 py37h5429711_0
mkl 2018.0.3 1
mkl_fft 1.0.6 py37h7dd41cf_0
mkl_random 1.0.1 py37h4414c95_1
ncurses 6.1 hf484d3e_0
netifaces 0.10.9 pypi_0 pypi
ninja 1.8.2 py37h6bb024c_1
numpy 1.15.4 py37h1d66e8a_0
numpy-base 1.15.4 py37h81de0dd_0
olefile 0.46 py37_0
opencv 3.4.1 py37h6fd60c2_3
opencv-python 3.4.5.20 pypi_0 pypi
openssl 1.1.1b h7b6447c_1
pcre 8.42 h439df22_0
pillow 5.2.0 py37heded4f4_0
pip 19.1 pypi_0 pypi
pixman 0.34.0 hceecf20_3
py-opencv 3.4.1 py37h8fa1ad8_3
pycparser 2.18 py37_1
pyparsing 2.2.0 py37_1
pyqt 5.9.2 py37h05f1152_2
python 3.7.1 h0371630_3
python-dateutil 2.7.3 py37_0
pytorch 1.0.0 py3.7_cuda10.0.130_cudnn7.4.1_1 [cuda100] pytorch
pytz 2018.5 py37_0
pyyaml 5.1 pypi_0 pypi
qt 5.9.7 h5867ecd_1
readline 7.0 h7b6447c_5
rospkg 1.1.9 pypi_0 pypi
scikit-learn 0.19.1 py37hedc7406_0
scipy 1.1.0 py37hfa4b5c9_1
setuptools 40.2.0 py37_0
sip 4.19.8 py37hf484d3e_0
six 1.11.0 py37_1
sqlite 3.25.3 h7b6447c_0
tk 8.6.8 hbc83047_0
torchvision 0.2.1 py37_1 pytorch
tornado 5.1 py37h14c3975_0
tqdm 4.25.0 py37h28b3542_0
wheel 0.31.1 py37_0
xz 5.2.4 h14c3975_4
zlib 1.2.11 ha838bed_2 5）从源码编译cv_bridge
先装点依赖项吧，其实很多都有了
sudo apt-get update
sudo apt-get install python-catkin-tools python3-dev python3-catkin-pkg-modules python3-numpy python3-yaml ros-kinetic-cv-bridge初始化工作空间，并设置Python版本为Python3（注意，这里使用的是系统默认安装的Python3.5m，测试通过，管他呢，或许Python3都可以）
cd ../../ #回到py3_ws/目录
catkin init
catkin config -DPYTHON_EXECUTABLE=/usr/bin/python3 -DPYTHON_INCLUDE_DIR=/usr/include/python3.5m -DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.5m.so
catkin config --install下载cv_bridge源码并编译
git clone https://github.com/ros-perception/vision_opencv.git src/vision_opencv
注意，这里直接下载到了src/目录下，所以不用先cd src/
修改src/vision_opencv/cv_bridge/CMakeLists.txt 第11行内容（解决找不到boost_python3的问题）
gedit src/vision_opencv/cv_bridge/CMakeLists.txt将第11行的
find_package(Boost REQUIRED python3)修改为
find_package(Boost REQUIRED python-py35)修改src/vision_opencv/cv_bridge/setup.py第1行内容（话说这样才会在执行的时候使用Python3？？？）
gedit src/vision_opencv/cv_bridge/setup.py
将第1行的
#!/usr/bin/env python修改为
#!/usr/bin/env python3开始编译
catkin build cv_bridge
source install/setup.bash --extend此处使用catkin build编译cv_bridge包，并source了一下setup.bash，让系统知道cv_bridge要用咱们这里这个。
这时可以简单测试以下能不能用
python
Python 3.7.1 (default, Oct 23 2018, 19:19:42)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from cv_bridge.boost.cv_bridge_boost import getCvType
>>>
不报错或许就是莫得问题啦，那就开始写代码吧
8.1.3 来个test_ros.py脚本测试测试
cd src/CornerNet-Lite/
gedit test_ros.py
test_ros.py内容如下
#!/usr/bin/env python3
#!coding=utf-8
import sys
import rospy
from std_msgs.msg import String
from sensor_msgs.msg import Image
from cv_bridge import CvBridge, CvBridgeError
import cv2
print("cv2.__version__: ", cv2.__version__)
from core.detectors import CornerNet_Saccade
from core.vis_utils import draw_bboxes
def callback(data):
global count, bridge, detector
count += 1
if count ==1:
count = 0
cv_img = bridge.imgmsg_to_cv2(data, "bgr8")
bboxes = detector(cv_img)
cv_img = draw_bboxes(cv_img, bboxes)
cv2.imshow("frame", cv_img)
cv2.waitKey(3)
else:
pass
def image_pro():
# In ROS, nodes are uniquely named. If two nodes with the same
# name are launched, the previous one is kicked off. The
# anonymous=True flag means that rospy will choose a unique
# name for our 'listener' node so that multiple listeners can
# run simultaneously.
rospy.init_node('image_pro',anonymous=True)
global count,bridge,detector
detector = CornerNet_Saccade()
count = 0
bridge = CvBridge()
rospy.Subscriber("/kinect2/qhd/image_color_rect", Image, callback)
# spin() simply keeps python from exiting until this node is stopped
rospy.spin()
if __name__ == '__main__':
image_pro()
注意：
第一行 #!/usr/bin/env python3 很重要
请修改订阅图像信息的话题名，我这儿使用的Kinect2
rospy.Subscriber("/kinect2/sd/image_color_rect", Image, callback)
8.1.4 run起来，go！go！go！
新开一个终端，运行roscore
roscore再新开一个终端，运行相机节点（这里使用Kinect2）
rosrun kinect2_bridge kinect2_bridge在之前那个激活了CornerNet_Lite环境的终端下运行test_ros.py
./test_ros.py
或者
python test_ros.py 终端输出大概是这样的
cv2.__version__: 3.4.5
total parameters: 116969339
loading from /home/communicationgroup/py3_ws/src/CornerNet-Lite/core/../cache/nnet/CornerNet_Saccade/CornerNet_Saccade_500000.pkl
/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7/site-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
嘻嘻，稍等片刻，画面出现了，感谢刚哥友情出镜
效果不错啊，框出来这么多，美中不足就是有点卡。卡。卡。
8.1.5 出错总结：
1）如果出现如下类似错误，就是找不到'torch'，应该是你没有激活环境
Traceback (most recent call last):
File "./test_ros1.py", line 14, in <module>
from core.detectors import CornerNet_Saccade
File "/home/communicationgroup/py3_ws/src/CornerNet-Lite/core/detectors.py", line 1, in <module>
from .base import Base, load_cfg, load_nnet
File "/home/communicationgroup/py3_ws/src/CornerNet-Lite/core/base.py", line 3, in <module>
from .nnet.py_factory import NetworkFactory
File "/home/communicationgroup/py3_ws/src/CornerNet-Lite/core/nnet/py_factory.py", line 2, in <module>
import torch
ImportError: No module named 'torch'
解决办法：在终端执行
conda activate CornerNet_Lite2）如果出现如下错误，就是cv_bridge的错误，应该是你没有source 一下
[ERROR] [1556354686.771577]: bad callback: <function callback at 0x7f598b536268>
Traceback (most recent call last):
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/topics.py", line 750, in _invoke_callback
cb(msg)
File "./test_ros1.py", line 22, in callback
cv_img = bridge.imgmsg_to_cv2(data, "bgr8")
File "/opt/ros/kinetic/lib/python2.7/dist-packages/cv_bridge/core.py", line 163, in imgmsg_to_cv2
dtype, n_channels = self.encoding_to_dtype_with_channels(img_msg.encoding)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/cv_bridge/core.py", line 99, in encoding_to_dtype_with_channels
return self.cvtype2_to_dtype_with_channels(self.encoding_to_cvtype2(encoding))
File "/opt/ros/kinetic/lib/python2.7/dist-packages/cv_bridge/core.py", line 91, in encoding_to_cvtype2
from cv_bridge.boost.cv_bridge_boost import getCvType
ImportError: dynamic module does not define module export function (PyInit_cv_bridge_boost)解决办法：如果在py3_ws/src/CornerNet-Lite目录下的话，在终端执行
source ../../devel/setup.bash
-------------------------------------------------------------我是分割线--------------------------------------------------------------------
2019/4/28更新：
8.2 既然得到了目标的框框，那就发布出来吧，以待后续工程中能够应用
8.2.1 自定义BoundingBox消息
1）首先，在py3_ws/src目录下创建cornernet_ros_msgs包，里面主要是定义了我们自己的BoundingBox消息
cd py3_ws/src/
catkin_create_pkg cornernet_ros_msgs message_generation actionlib_msgs sensor_msgs std_msgs message_runtime
创建后会发现 py3_ws/src目录下生成了一个名为cornernet_ros_msgs的文件夹，打开该文件夹，我们可以看到里面生成了CMakeLists.txt 和 package.xml 。
2）进入cornernet_ros_msgs/目录，并创建msg文件夹，并在msg/目录下新建两个文件，分别为BoundingBox.msg 和BoundingBoxes.msg
cd cornernet_ros_msgs/
mkdir msg
cd msg/
gedit BoundingBox.msg BoundingBox.msg 中写入如下内容
string class
float64 prob
int64 x1
int64 y1
int64 x2
int64 y2意思就是自定义了名为 BoundingBox 的消息类型，其中class为类名，prob也就是得分嘛，可以根据这个得分滤掉低于设定阈值的框框，后面的x1,y1,x2,y2就是boundingbox的坐标了。
保存一下，再创建 BoundingBoxes.msg (很多时候框框不止一个)
gedit BoundingBoxes.msg BoundingBoxes.msg 内容如下
Header header
BoundingBox[] bounding_boxes意思就是里面有很多BoundingBox，存在数组（或者叫列表）中。
3）接下来就是修改修改CMakeLists.txt 和 package.xml 中的内容了。
CMakeLists.txt
cmake_minimum_required(VERSION 2.8.3)
project(cornernet_ros_msgs)
## Compile as C++11, supported in ROS Kinetic and newer
add_compile_options(-std=c++11)
find_package(catkin REQUIRED COMPONENTS
roscpp
rospy
message_generation
actionlib_msgs
message_runtime
sensor_msgs
std_msgs
)
# find_package(Boost REQUIRED COMPONENTS system)
add_message_files(
FILES
BoundingBox.msg
BoundingBoxes.msg
)
generate_messages(
DEPENDENCIES
actionlib_msgs
sensor_msgs
std_msgs
)
catkin_package(
CATKIN_DEPENDS
actionlib_msgs
message_runtime
sensor_msgs
std_msgs
)package.xml
<?xml version="1.0"?>
<package format="2">
<name>cornernet_ros_msgs</name>
<version>0.0.0</version>
<description>The cornernet_ros_msgs package</description>
<maintainer email="communicationgroup@todo.todo">communicationgroup</maintainer>
<license>TODO</license>
<buildtool_depend>catkin</buildtool_depend>
<build_depend>message_generation</build_depend>
<build_depend>actionlib_msgs</build_depend>
<build_depend>sensor_msgs</build_depend>
<build_depend>std_msgs</build_depend>
<build_depend>message_runtime</build_depend>
<build_export_depend>actionlib_msgs</build_export_depend>
<build_export_depend>sensor_msgs</build_export_depend>
<build_export_depend>std_msgs</build_export_depend>
<exec_depend>actionlib_msgs</exec_depend>
<exec_depend>message_runtime</exec_depend>
<exec_depend>sensor_msgs</exec_depend>
<exec_depend>std_msgs</exec_depend>
<export>
</export>
</package>4）那就开始编译吧（就像前面编译cv_bridge一样）
cd ../../../ #回到py3_ws/目录下
catkin init终端输出大概是这样的
Catkin workspace `/home/communicationgroup/py3_ws` is already initialized. No action taken.
--------------------------------------------------------------------------------
Profile: default
Extending: [cached] /home/communicationgroup/catkin_ws/devel:/home/communicationgroup/rgbdslam2_catkin_ws/devel:/opt/ros/kinetic
Workspace: /home/communicationgroup/py3_ws
--------------------------------------------------------------------------------
Build Space: [exists] /home/communicationgroup/py3_ws/build
Devel Space: [exists] /home/communicationgroup/py3_ws/devel
Install Space: [exists] /home/communicationgroup/py3_ws/install
Log Space: [exists] /home/communicationgroup/py3_ws/logs
Source Space: [exists] /home/communicationgroup/py3_ws/src
DESTDIR: [unused] None
--------------------------------------------------------------------------------
Devel Space Layout: linked
Install Space Layout: merged
--------------------------------------------------------------------------------
Additional CMake Args: -DPYTHON_EXECUTABLE=/usr/bin/python3 -DPYTHON_INCLUDE_DIR=/usr/include/python3.5m -DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.5m.so
Additional Make Args: None
Additional catkin Make Args: None
Internal Make Job Server: True
Cache Job Environments: False
--------------------------------------------------------------------------------
Whitelisted Packages: None
Blacklisted Packages: None
--------------------------------------------------------------------------------
Workspace configuration appears valid.
--------------------------------------------------------------------------------他说我已经initialized。。。（或许不必执行catkin init???）
既然这样，那就直接编译吧
catkin build cornernet_ros_msgs终端输出大概是这样的
--------------------------------------------------------------------------------
Profile: default
Extending: [cached] /home/communicationgroup/catkin_ws/devel:/home/communicationgroup/rgbdslam2_catkin_ws/devel:/opt/ros/kinetic
Workspace: /home/communicationgroup/py3_ws
--------------------------------------------------------------------------------
Build Space: [exists] /home/communicationgroup/py3_ws/build
Devel Space: [exists] /home/communicationgroup/py3_ws/devel
Install Space: [exists] /home/communicationgroup/py3_ws/install
Log Space: [exists] /home/communicationgroup/py3_ws/logs
Source Space: [exists] /home/communicationgroup/py3_ws/src
DESTDIR: [unused] None
--------------------------------------------------------------------------------
Devel Space Layout: linked
Install Space Layout: merged
--------------------------------------------------------------------------------
Additional CMake Args: -DPYTHON_EXECUTABLE=/usr/bin/python3 -DPYTHON_INCLUDE_DIR=/usr/include/python3.5m -DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.5m.so
Additional Make Args: None
Additional catkin Make Args: None
Internal Make Job Server: True
Cache Job Environments: False
--------------------------------------------------------------------------------
Whitelisted Packages: None
Blacklisted Packages: None
--------------------------------------------------------------------------------
Workspace configuration appears valid.
--------------------------------------------------------------------------------
[build] Found '5' packages in 0.0 seconds.
Starting >>> cornernet_ros_msgs
Finished <<< cornernet_ros_msgs [ 0.8 seconds ]
[build] Summary: All 1 packages succeeded!
[build] Ignored: 4 packages were skipped or are blacklisted.
[build] Warnings: None.
[build] Abandoned: None.
[build] Failed: None.
[build] Runtime: 0.9 seconds total. 我们最喜欢看到的就是succeed！
6) 敲代码，敲代码，把框框发布出去（只需要对前面的test_ros.py稍作修改）
暂且命名为test_ros1.py吧，内容如下，且放在py3_ws/src/CornerNet-Lite/目录下
#!/usr/bin/env python3
#!coding=utf-8
import sys
import numpy as np
import rospy
from std_msgs.msg import String
from sensor_msgs.msg import Image
from cv_bridge import CvBridge, CvBridgeError
#自己定义的BoundingBoxes消息
from cornernet_ros_msgs.msg import BoundingBoxes
from cornernet_ros_msgs.msg import BoundingBox
import cv2
print("cv2.__version__: ", cv2.__version__)
from core.detectors import CornerNet_Saccade
from core.vis_utils import draw_bboxes
def pub_bboxes(bboxes, thresh=0.5):
"""Public bounding boxes msgs.
Args:
bboxes: A dictionary representing bounding boxes of different object
categories, where the keys are the names of the categories and the
values are the bounding boxes. The bounding boxes of category should be
stored in a 2D NumPy array, where each row is a bounding box (x1, y1,
x2, y2, score).
thresh: (Optional) Only bounding boxes with scores above the threshold
will be drawn.
Returns:
None
"""
global pub
bboxes_msg = BoundingBoxes()
for cat_name in bboxes:
keep_inds = bboxes[cat_name][:, -1] > thresh
for bbox in bboxes[cat_name][keep_inds]:
bbox_msg = BoundingBox( str(cat_name), #class name
bbox[-1].astype(np.float), #prab
bbox[0].astype(np.int32), #x1
bbox[1].astype(np.int32), #y1
bbox[2].astype(np.int32), #x2
bbox[3].astype(np.int32)) #y2
bboxes_msg.bounding_boxes.append(bbox_msg)
# print(bboxes_msg)
pub.publish(bboxes_msg)
def callback(data):
global count, bridge, detector
count += 1
if count ==1:
count = 0
cv_img = bridge.imgmsg_to_cv2(data, "bgr8")
bboxes = detector(cv_img)
cv_img = draw_bboxes(cv_img, bboxes)
cv2.imshow("frame", cv_img)
pub_bboxes(bboxes) #发布消息
cv2.waitKey(3)
else:
pass
def image_pro():
# In ROS, nodes are uniquely named. If two nodes with the same
# name are launched, the previous one is kicked off. The
# anonymous=True flag means that rospy will choose a unique
# name for our 'listener' node so that multiple listeners can
# run simultaneously.
rospy.init_node('image_pro',anonymous=True)
global count,bridge,detector,pub
detector = CornerNet_Saccade()
count = 0
bridge = CvBridge()
rospy.Subscriber("/kinect2/qhd/image_color_rect", Image, callback)
pub = rospy.Publisher('boundingboxes',BoundingBoxes,queue_size=10)
# spin() simply keeps python from exiting until this node is stopped
rospy.spin()
if __name__ == '__main__':
image_pro()
7）试一试，看看消息能发布出来不（看看程序能跑不）
先激活CornerNet_Lite环境，并source一下工作空间
cd py3_ws/src/CornerNet-Lite
conda activate CornerNet_Lite
source ../../devel/setup.bash 运行test_ros1.py（假设你已经启动了roscore 和相机节点，参考8.1.4节）
./test_ros1.py 
东西框出来了，那消息发布出去了么？那就rostopic list看一看
新开一个终端
rostopic list 这是终端会输出ROS发布的所有消息，我的大概是这样的，很棒，第一个就是我们需要的
communicationgroup@Z370:~/py3_ws/src/CornerNet-Lite$ rostopic list
/boundingboxes
/kinect2/cap/camera_info
/kinect2/cap/cap_bgr
/kinect2/cap/cap_bgr/compressed
/kinect2/cap/cap_mono
/kinect2/cap/cap_mono/compressed
/kinect2/hd/camera_info
/kinect2/hd/image_color
/kinect2/hd/image_color/compressed
/kinect2/hd/image_color_rect
/kinect2/hd/image_color_rect/compressed
/kinect2/hd/image_depth_rect
/kinect2/hd/image_depth_rect/compressed
/kinect2/hd/image_mono
/kinect2/hd/image_mono/compressed
/kinect2/hd/image_mono_rect
/kinect2/hd/image_mono_rect/compressed
/kinect2/qhd/camera_info
/kinect2/qhd/image_color
/kinect2/qhd/image_color/compressed
/kinect2/qhd/image_color_rect
/kinect2/qhd/image_color_rect/compressed
/kinect2/qhd/image_depth_rect
/kinect2/qhd/image_depth_rect/compressed
/kinect2/qhd/image_mono
/kinect2/qhd/image_mono/compressed
/kinect2/qhd/image_mono_rect
/kinect2/qhd/image_mono_rect/compressed
/kinect2/sd/camera_info
/kinect2/sd/image_color_rect
/kinect2/sd/image_color_rect/compressed
/kinect2/sd/image_depth
/kinect2/sd/image_depth/compressed
/kinect2/sd/image_depth_rect
/kinect2/sd/image_depth_rect/compressed
/kinect2/sd/image_ir
/kinect2/sd/image_ir/compressed
/kinect2/sd/image_ir_rect
/kinect2/sd/image_ir_rect/compressed
/rosout
/rosout_agg
再用rostopic echo /boundingboxes 看看里面具体是些啥
communicationgroup@Z370:~/py3_ws/src/CornerNet-Lite$ rostopic echo /boundingboxes
ERROR: Cannot load message class for [cornernet_ros_msgs/BoundingBoxes]. Are your messages built?
哎呀，看到ERROR就尴尬了！其实这都是小问题啦，因为新开这个终端没有source工作空间，他怎么理解我们自己定义的消息呢，那就source一下
source ~/py3_ws/devel/setup.bash
然后再echo
rostopic echo /boundingboxes 终端输出大概是这样的
communicationgroup@Z370:~/py3_ws/src/CornerNet-Lite$ rostopic echo /boundingboxes
header:
seq: 1
stamp:
secs: 0
nsecs: 0
frame_id: ''
bounding_boxes:
-
class_: "bottle"
prob: 0.650504469872
x1: 1
y1: 300
x2: 43
y2: 444
-
class_: "bottle"
prob: 0.597384572029
x1: 123
y1: 310
x2: 154
y2: 381
-
class_: "cup"
prob: 0.739920139313
x1: 94
y1: 379
x2: 182
y2: 472
-
class_: "chair"
prob: 0.736968696117
x1: 719
y1: 368
x2: 917
y2: 536
-
class_: "tv"
prob: 0.689738988876
x1: 283
y1: 246
x2: 428
y2: 350
-
class_: "mouse"
prob: 0.543673157692
x1: 381
y1: 388
x2: 423
y2: 408
-
class_: "keyboard"
prob: 0.587634801865
x1: 228
y1: 386
x2: 369
y2: 427
-
class_: "book"
prob: 0.500504910946
x1: 501
y1: 291
x2: 590
y2: 306
---
没错，这就是我们想要的。以后具体应用的时候，还是该把Header加上，这样就可以根据时间戳等信息知道这些框框对应哪一帧图像的。
最后备注：
代码中detector=CornerNet_Saccade()，是否可以改一改呀？不是有三个models嘛，这个就留给大家去探索吧（悄悄告诉大家，CornerNet_Squeeze很快哦）
那就这样吧，本篇博客正式完结。
参考链接：
https://github.com/princeton-vl/CornerNet-Lite
https://community.bwbot.org/topic/499/%E5%9C%A8ros%E4%B8%AD%E4%BD%BF%E7%94%A8python3
1028
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
