Paper reading：A simple baseline for one-shot multi-object tracking（一）

最新推荐文章于 2024-06-20 09:00:00 发布

Felaim

最新推荐文章于 2024-06-20 09:00:00 发布

阅读量1.2k

点赞数 2

分类专栏：跟踪 paper Pytorch 文章标签：算法 python 人工智能深度学习 paper

本文链接：https://blog.csdn.net/Felaim/article/details/108364568

版权

paper 同时被 3 个专栏收录

32 篇文章 2 订阅

订阅专栏

Pytorch

20 篇文章 1 订阅

订阅专栏

跟踪

10 篇文章 7 订阅

订阅专栏

A simple baseline for one-shot multi-object tracking（一）

1. 背景介绍

作者来自华中科技大和微软亚亚研院，之前LZ是看过一点多目标跟踪的算法的，当时采用的是centernet+deepsort的的方法，速度呢在LZ的小笔记本上970m，在12fps左右，但是这篇论文中说速度已经是30fps，速度提升了一倍多，并且在2DMOT15，MOT16，MOT17，MOT20都是rank first，是个非常厉害的工作，并且速度和精度的提升，使得工业界也能够使用对应的思想，恩，为AI算法的落地又贡献了一步。

LZ也并不是想单纯的阅读对应的论文，做个翻译，对于数据处理，代码理解，包括一些实验也会包含进来，还是想好好理解这个算法，而不仅仅是跑个demo。

算法的概要是这样的，目前夺目标跟踪主要分为两个大类，一个是Two-Step MOT Methods，一个是One-Shot MOT Methods，这两个在之前的算法中是各有优缺点。

a. Two-Step MOT Methods

两步的算法主要是把多目标跟踪的任务分成两个独立的任务，一个是目标检测，第二个就是Re-ID，主要流程是先对图像进行目标检测，把检测框进行crop和resize后送到identity embedding network来提取Re-ID的特征，最后进行box linking操作，怎么进行box linking呢？

首先是通过Re-ID features和IoU计算一个对应的算是矩阵，然后通过Kalman Filter和Hungarian algorithm来完成linking的任务

优点：

因为检测和Re-ID feature提取任务是分开的，所以我们可以找到各自任务比较适合的网络结构
对于得到的框，送到identity embedding network是需要进行resize操作的，这样也可以解决对应的尺度变化的问题

缺点：

效果也不错，但就是慢

b. One-Shot MOT Methods

这个方法主要的目的是利用同一个网络，同时完成目标检测和indentity embedding（Re-ID features）来减少推断的时间，如果做项目的小伙伴就会深有感触，crop和resize操作在cpu比较弱的情况下还是会比较耗时的，能省一点时间是一点吧

优点：

只要做一次推断就可以得到对应的Re-ID features，速度肯定会快一些

缺点：

速度快，精度就下降了。。。鱼和熊掌不可兼得

这篇文章其实主要使用的是One-Shot MOT的方式，不仅速度快，精度也很高，所以，恩，很牛！！

2. 算法框架和代码初探

在这里插入图片描述

算法的主要流程其实真的非常简洁，就是先用一个特征提取器提取对应特征，然后对应不同的任务，分出两个分支，一个分支是目标检测分支，另外一个分支是获得对应维度的Re-ID特征，非常的清晰。

下面就是单纯是LZ的理解方式了，所以顺序可能会和paper顺序不一样了，建议还是要自己读一遍paper的

2.1 环境配置

LZ是不想再重新装笔记本了，所以开发环境基本上都放在服务器上，或者docker镜像里面了，而且LZ是2016年11月份买的笔记本，快四年了，折腾不动了，哭唧唧。。。

按照论文给的参考地址，在github下载对应代码，参考对应的README.md文档，先配置环境，为啥晚上装呢，人都下班了，网速快，之前也有一个教程是换源，这个真的亲测可以加速，感兴趣的小伙伴可以看一下！

conda create -n FairMOT
conda activate FairMOT
conda install pytorch==1.2.0 torchvision==0.4.0 cudatoolkit=10.0 -c pytorch
// LZ使用的这个版本的pytorch
conda install pytorch torchvision cudatoolkit=10.2 -c pytorch
cd ${FAIRMOT_ROOT}
pip install -r requirements.txt
cd src/lib/models/networks/DCNv2_new sh make.sh

DCNv2的编译细节请参考这里

2000 years later！！！

2. 2 数据准备

论文中说训练的时候用来6个公共数据集，ETH和 CityPerson只提供了bounding box的标注，所以我们只用这部分来训练检测的分支，剩余的四个CalTech，MOT17M，CUHK-SYSU和PRW不仅提供了bounding box，也提供了identity annotation，所以我们可以用这四个数据集训练对应的detection和identity embedding这两个分支

用了六个数据集，LZ下数据要下哭了。。。
在这里插入图片描述
训练数据肯定要知道数据集是啥，也就借这个机会了解一下各个数据集。数据集的一些简单介绍在这里

所有的上述数据集都要整理成对应的格式

Caltech
   |——————images
   |        └——————00001.jpg
   |        |—————— ...
   |        └——————0000N.jpg
   └——————labels_with_ids
            └——————00001.txt
            |—————— ...
            └——————0000N.txt

每个图片都有对应的txt文件，给定一个图片的路径，标注文件地址可以通过将路径中images替换成labels_with_ids，将.jpg替换成.txt的后缀

在标注文件中，每一行代表一个bounding box，并且按照如下的格式：

[class] [identity] [x_center] [y_center] [width] [height]

The field [class] should be 0. Only single-class multi-object tracking is supported in this version.
The field [identity] is an integer from 0 to num_identities - 1, or -1 if this box has no identity annotation.
Note that the values of [x_center] [y_center] [width] [height] are normalized by the width/height of the image, so they are floating point numbers ranging from 0 to 1.

下载的话，这个地址都给出了对应的下载链接，可以直接下载处理好的数据

2.3 初步训练

按照官网上的教程，我们需要先修改数据集的路径，在 src/lib/cfg/data.json和 ‘data_dir’ in src/lib/opts.py中进行修改

data.json的源文件如下，可以看到训练集就是我们之前介绍的6个数据集，测试集是mot15

{
    "root":"/data/yfzhang/MOT/JDE",
    "train":
    {
        "mot17":"./data/mot17.train",
        "caltech":"./data/caltech.train",
        "citypersons":"./data/citypersons.train",
        "cuhksysu":"./data/cuhksysu.train",
        "prw":"./data/prw.train",
        "eth":"./data/eth.train"
    },
    "test_emb":
    {
        "mot15":"./data/mot15.val"
    },
    "test":
    {
        "mot15":"./data/mot15.val"
    }
}

opts.py的具体内容后续再分析，目前关注的是data_dir：

        # mot
        self.parser.add_argument('--data_cfg', type=str,
                                 default='../src/lib/cfg/data.json',
                                 help='load data from cfg')
        self.parser.add_argument('--data_dir', type=str, default='')

如果实在不知道改哪，就先运行这个代码

sh experiments/all_dla34.sh

肯定会报路径的错误，然后按照实际的情况，进行修改就行了

OSError: /data/yfzhang/MOT/JDE/PRW/labels_with_ids/c1s1_000151.txt not found.

然后按照实际情况修改data.json即可

在terminal中出现显示如下，恭喜小伙伴，你已经成功运行代码啦，当然这只是第一步，代码都没好好看，这怎么好意思呢。。。

在这里插入图片描述

2.4 Tracking

Tacking部分其实就是inference阶段了，LZ就使用作者给的bashline尝试一下

直接运行对应的track.py文件，其实会报一个错误，具体错误如下：

  File "track.py", line 102, in main
    dataloader = datasets.LoadImages(osp.join(data_root, seq, 'img1'), opt.img_size)
  File "FairMOT\src\lib\datasets\dataset\jde.py", line 31, in __init__
    self.nF = len(self.files)  # number of image files
AttributeError: 'LoadImages' object has no attribute 'files'

后来在github上这里找到对应问题是数据集的位置没有写正确，写正确后，运行如下命令：

cd src

python track.py mot --load_model ../models/all_hrnet_v2_w18.pth --conf_thres 0.6 --arch hrnet_18 --reid_dim 128 --data_dir /your/path/to/data

在这里插入图片描述
这个结果是MOT15 TRAIN的测试结果，后期还会仔细分析这些结果的具体含义，因为LZ没有安装ffmpeg，所以把源码中对应ffmpeg代码全都改成OpenCV的接口来保存对应的视频了

2.5 Demo

python demo.py mot --load_model ../models/all_hrnet_v2_w18.pth --arch hrnet_18 --reid_dim 128 --conf_thres 0.4

最后放上一张跟踪的结果图吧

在这里插入图片描述

后面再开一篇专门对着论文和代码讲吧，不然一篇博客太长了。。。

参考地址：

代码地址：https://github.com/ifzhang/FairMOT

环境安装补充：

(FairMOT) root@2d6362a3620d:FairMOT-master# pip install -r requirements.txt 
Collecting yacs
  Downloading yacs-0.1.8-py3-none-any.whl (14 kB)
Collecting opencv-python
  Downloading opencv_python-4.4.0.42-cp38-cp38-manylinux2014_x86_64.whl (49.4 MB)
     |████████████████████████████████| 49.4 MB 2.1 MB/s 
Collecting cython-bbox
  Downloading cython_bbox-0.1.3.tar.gz (41 kB)
     |████████████████████████████████| 41 kB 372 kB/s 
    ERROR: Command errored out with exit status 1:
     command: /root/anaconda3/envs/FairMOT/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-x2aapxs_/cython-bbox/setup.py'"'"'; __file__='"'"'/tmp/pip-install-x2aapxs_/cython-bbox/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-3sptlang
         cwd: /tmp/pip-install-x2aapxs_/cython-bbox/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-x2aapxs_/cython-bbox/setup.py", line 10, in <module>
        from Cython.Build import cythonize
    ModuleNotFoundError: No module named 'Cython'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
(FairMOT) root@2d6362a3620d:FairMOT-master# conda install Cython
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /root/anaconda3/envs/FairMOT

  added / updated specs:
    - cython


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    cython-0.29.21             |   py38he6710b0_0         2.0 MB  defaults
    ------------------------------------------------------------
                                           Total:         2.0 MB

The following NEW packages will be INSTALLED:

  cython             pkgs/main/linux-64::cython-0.29.21-py38he6710b0_0


Proceed ([y]/n)? y


Downloading and Extracting Packages
cython-0.29.21       | 2.0 MB    | ############################################################################################################################################################ | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(FairMOT) root@2d6362a3620d:FairMOT-master# pip install -r requirements.txt 
Collecting yacs
  Using cached yacs-0.1.8-py3-none-any.whl (14 kB)
Collecting opencv-python
  Using cached opencv_python-4.4.0.42-cp38-cp38-manylinux2014_x86_64.whl (49.4 MB)
Collecting cython-bbox
  Using cached cython_bbox-0.1.3.tar.gz (41 kB)
Collecting scipy
  Downloading scipy-1.5.2-cp38-cp38-manylinux1_x86_64.whl (25.7 MB)
     |████████████████████████████████| 25.7 MB 2.5 MB/s 
Collecting numba
  Downloading numba-0.51.1-cp38-cp38-manylinux2014_x86_64.whl (3.1 MB)
     |████████████████████████████████| 3.1 MB 2.8 MB/s 
Collecting progress
  Downloading progress-1.5.tar.gz (5.8 kB)
Collecting motmetrics
  Downloading motmetrics-1.2.0-py3-none-any.whl (151 kB)
     |████████████████████████████████| 151 kB 2.7 MB/s 
Collecting matplotlib
  Downloading matplotlib-3.3.1-cp38-cp38-manylinux1_x86_64.whl (11.6 MB)
     |████████████████████████████████| 11.6 MB 2.4 MB/s 
Collecting lap
  Downloading lap-0.4.0.tar.gz (1.5 MB)
     |████████████████████████████████| 1.5 MB 2.4 MB/s 
Collecting openpyxl
  Downloading openpyxl-3.0.5-py2.py3-none-any.whl (242 kB)
     |████████████████████████████████| 242 kB 2.8 MB/s 
Requirement already satisfied: Pillow in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from -r requirements.txt (line 12)) (7.2.0)
Collecting tensorboardX
  Downloading tensorboardX-2.1-py2.py3-none-any.whl (308 kB)
     |████████████████████████████████| 308 kB 2.7 MB/s 
Collecting PyYAML
  Downloading PyYAML-5.3.1.tar.gz (269 kB)
     |████████████████████████████████| 269 kB 2.8 MB/s 
Requirement already satisfied: numpy>=1.17.3 in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from opencv-python->-r requirements.txt (line 2)) (1.19.1)
Collecting llvmlite<0.35,>=0.34.0.dev0
  Downloading llvmlite-0.34.0-cp38-cp38-manylinux2010_x86_64.whl (24.6 MB)
     |████████████████████████████████| 24.6 MB 2.2 MB/s 
Requirement already satisfied: setuptools in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from numba->-r requirements.txt (line 5)) (49.6.0.post20200814)
Collecting pytest-benchmark
  Downloading pytest_benchmark-3.2.3-py2.py3-none-any.whl (49 kB)
     |████████████████████████████████| 49 kB 3.6 MB/s 
Collecting xmltodict>=0.12.0
  Downloading xmltodict-0.12.0-py2.py3-none-any.whl (9.2 kB)
Collecting pandas>=0.23.1
  Downloading pandas-1.1.1-cp38-cp38-manylinux1_x86_64.whl (10.4 MB)
     |████████████████████████████████| 10.4 MB 2.5 MB/s 
Collecting flake8-import-order
  Downloading flake8_import_order-0.18.1-py2.py3-none-any.whl (15 kB)
Collecting flake8
  Downloading flake8-3.8.3-py2.py3-none-any.whl (72 kB)
     |████████████████████████████████| 72 kB 2.5 MB/s 
Collecting pytest
  Downloading pytest-6.0.1-py3-none-any.whl (270 kB)
     |████████████████████████████████| 270 kB 2.8 MB/s 
Collecting python-dateutil>=2.1
  Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
     |████████████████████████████████| 227 kB 2.8 MB/s 
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3
  Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
     |████████████████████████████████| 67 kB 3.1 MB/s 
Collecting cycler>=0.10
  Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.2.0-cp38-cp38-manylinux1_x86_64.whl (92 kB)
     |████████████████████████████████| 92 kB 147 kB/s 
Requirement already satisfied: certifi>=2020.06.20 in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 9)) (2020.6.20)
Collecting et-xmlfile
  Downloading et_xmlfile-1.0.1.tar.gz (8.4 kB)
Collecting jdcal
  Downloading jdcal-1.4.1-py2.py3-none-any.whl (9.5 kB)
Requirement already satisfied: six in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from tensorboardX->-r requirements.txt (line 13)) (1.15.0)
Collecting protobuf>=3.8.0
  Downloading protobuf-3.13.0-cp38-cp38-manylinux1_x86_64.whl (1.3 MB)
     |████████████████████████████████| 1.3 MB 2.8 MB/s 
Collecting py-cpuinfo
  Downloading py-cpuinfo-7.0.0.tar.gz (95 kB)
     |████████████████████████████████| 95 kB 2.3 MB/s 
Collecting pytz>=2017.2
  Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB)
     |████████████████████████████████| 510 kB 2.8 MB/s 
Collecting pycodestyle
  Downloading pycodestyle-2.6.0-py2.py3-none-any.whl (41 kB)
     |████████████████████████████████| 41 kB 1.3 MB/s 
Collecting mccabe<0.7.0,>=0.6.0
  Downloading mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)
Collecting pyflakes<2.3.0,>=2.2.0
  Downloading pyflakes-2.2.0-py2.py3-none-any.whl (66 kB)
     |████████████████████████████████| 66 kB 3.1 MB/s 
Collecting py>=1.8.2
  Downloading py-1.9.0-py2.py3-none-any.whl (99 kB)
     |████████████████████████████████| 99 kB 2.9 MB/s 
Collecting more-itertools>=4.0.0
  Downloading more_itertools-8.5.0-py3-none-any.whl (44 kB)
     |████████████████████████████████| 44 kB 4.5 MB/s 
Collecting packaging
  Downloading packaging-20.4-py2.py3-none-any.whl (37 kB)
Collecting iniconfig
  Downloading iniconfig-1.0.1-py3-none-any.whl (4.2 kB)
Collecting toml
  Downloading toml-0.10.1-py2.py3-none-any.whl (19 kB)
Collecting attrs>=17.4.0
  Downloading attrs-20.1.0-py2.py3-none-any.whl (49 kB)
     |████████████████████████████████| 49 kB 4.2 MB/s 
Collecting pluggy<1.0,>=0.12
  Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Building wheels for collected packages: cython-bbox, progress, lap, PyYAML, et-xmlfile, py-cpuinfo
  Building wheel for cython-bbox (setup.py) ... done
  Created wheel for cython-bbox: filename=cython_bbox-0.1.3-cp38-cp38-linux_x86_64.whl size=58568 sha256=3ebedb34e95b49afd367c1b62c1970e877f11136625c31e1e17c98b7be4bc15d
  Stored in directory: /root/.cache/pip/wheels/56/8d/2a/c7f38dc46bdfdee9aee8676cf3adf70d6e6e28b076cdb17963
  Building wheel for progress (setup.py) ... done
  Created wheel for progress: filename=progress-1.5-py3-none-any.whl size=8074 sha256=b4de011b051274a03887d309d45ca5e7d91c835e6d32cb7fda6e8c450e29a1fa
  Stored in directory: /root/.cache/pip/wheels/f1/5b/38/115b213dfbf5562108ea22df17c063f6378350205a819795aa
  Building wheel for lap (setup.py) ... done
  Created wheel for lap: filename=lap-0.4.0-cp38-cp38-linux_x86_64.whl size=1617987 sha256=2236f75be33da4b03820a1e420807f2394bf2e1ab4184302adbf3ab845049b12
  Stored in directory: /root/.cache/pip/wheels/86/ec/8a/2baa6fb3ea3a3bdc40a9c2c4b8957b5b544f207531f7a78a89
  Building wheel for PyYAML (setup.py) ... done
  Created wheel for PyYAML: filename=PyYAML-5.3.1-cp38-cp38-linux_x86_64.whl size=44619 sha256=369e890ee55fad482eec15f5ec8fe09d90aa23b399c829454ea96f2a5283f453
  Stored in directory: /root/.cache/pip/wheels/13/90/db/290ab3a34f2ef0b5a0f89235dc2d40fea83e77de84ed2dc05c
  Building wheel for et-xmlfile (setup.py) ... done
  Created wheel for et-xmlfile: filename=et_xmlfile-1.0.1-py3-none-any.whl size=8917 sha256=0265a2f3639bc7e9d3226db4ca8c4b9a4f7ce4fd1d90fadc3fcff49593e541b1
  Stored in directory: /root/.cache/pip/wheels/6e/df/38/abda47b884e3e25f9f9b6430e5ce44c47670758a50c0c51759
  Building wheel for py-cpuinfo (setup.py) ... done
  Created wheel for py-cpuinfo: filename=py_cpuinfo-7.0.0-py3-none-any.whl size=20070 sha256=2c40aeb8553a10a4257cbd1a02786903e4426b179c06dabec08350097795b8e6
  Stored in directory: /root/.cache/pip/wheels/b4/99/9f/8eb77fdf759c1380719071722f2c37dd0fa1f6aa477c51cb6c
Successfully built cython-bbox progress lap PyYAML et-xmlfile py-cpuinfo
Installing collected packages: PyYAML, yacs, opencv-python, cython-bbox, scipy, llvmlite, numba, progress, py, more-itertools, pyparsing, packaging, iniconfig, toml, attrs, pluggy, pytest, py-cpuinfo, pytest-benchmark, xmltodict, pytz, python-dateutil, pandas, pycodestyle, flake8-import-order, mccabe, pyflakes, flake8, motmetrics, cycler, kiwisolver, matplotlib, lap, et-xmlfile, jdcal, openpyxl, protobuf, tensorboardX
Successfully installed PyYAML-5.3.1 attrs-20.1.0 cycler-0.10.0 cython-bbox-0.1.3 et-xmlfile-1.0.1 flake8-3.8.3 flake8-import-order-0.18.1 iniconfig-1.0.1 jdcal-1.4.1 kiwisolver-1.2.0 lap-0.4.0 llvmlite-0.34.0 matplotlib-3.3.1 mccabe-0.6.1 more-itertools-8.5.0 motmetrics-1.2.0 numba-0.51.1 opencv-python-4.4.0.42 openpyxl-3.0.5 packaging-20.4 pandas-1.1.1 pluggy-0.13.1 progress-1.5 protobuf-3.13.0 py-1.9.0 py-cpuinfo-7.0.0 pycodestyle-2.6.0 pyflakes-2.2.0 pyparsing-2.4.7 pytest-6.0.1 pytest-benchmark-3.2.3 python-dateutil-2.8.1 pytz-2020.1 scipy-1.5.2 tensorboardX-2.1 toml-0.10.1 xmltodict-0.12.0 yacs-0.1.8
(FairMOT) root@2d6362a3620d