VISAGE代码安装过程记录

视频实例分割

挺有意思的模型，代码开源。GitHub - KimHanjung/VISAGE

环境搭建

conda create --name visage python=3.8 -y
conda activate visage
conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cudatoolkit=11.3 -c pytorch -c conda-forge
pip install -U opencv-python
git clone https://github.com/KimHanjung/VISAGE.git
cd VISAGE
pip install -r requirements.txt
git clone https://github.com/facebookresearch/detectron2.git
cd detectron2
pip install -e .
cd ..
cd mask2former/modeling/pixel_decoder/ops
sh make.sh

运行问题解决

加入cfg文件

继续训练，出现以下错误

好吧，绕不掉的问题。

3，2，1上脚本.以下是转换脚本。

import os
import json

# from vita.data.datasets.ytvis import (
#     COCO_TO_YTVIS_2019,
#     COCO_TO_YTVIS_2021,
# )
COCO_TO_YTVIS_2019 = {
    1:1, 2:21, 3:6, 4:21, 5:28, 7:17, 8:29, 9:34, 17:14, 18:8, 19:18, 21:15, 22:32, 23:20, 24:30, 25:22, 35:33, 36:33, 41:5, 42:27, 43:40
}
COCO_TO_YTVIS_2021 = {
    1:26, 2:23, 3:5, 4:23, 5:1, 7:36, 8:37, 9:4, 16:3, 17:6, 18:9, 19:19, 21:7, 22:12, 23:2, 24:40, 25:18, 34:14, 35:31, 36:31, 41:29, 42:33, 43:34
}

# from vita.data.datasets.ovis import (
#     COCO_TO_OVIS,
# )

COCO_TO_OVIS = {
    1:1, 2:21, 3:25, 4:22, 5:23, 6:25, 8:25, 9:24, 17:3, 18:4, 19:5, 20:6, 21:7, 22:8, 23:9, 24:10, 25:11,
}

_root = os.getenv("DETECTRON2_DATASETS", "datasets")

convert_list = [
    (COCO_TO_YTVIS_2019, 
        os.path.join(_root, "/home/rui/datasets/coco2017/annotations/instances_train2017.json"),
        os.path.join(_root, "/home/rui/VISAGE/annotations/coco2ytvis2019_train.json"), "COCO to YTVIS 2019:"),
    (COCO_TO_YTVIS_2019, 
        os.path.join(_root, "/home/rui/datasets/coco2017/annotations/instances_val2017.json"),
        os.path.join(_root, "/home/rui/VISAGE/annotations/coco2ytvis2019_val.json"), "COCO val to YTVIS 2019:"),
    (COCO_TO_YTVIS_2021, 
        os.path.join(_root, "/home/rui/datasets/coco2017/annotations/instances_train2017.json"),
        os.path.join(_root, "/home/rui/VISAGE/annotations/coco2ytvis2021_train.json"), "COCO to YTVIS 2021:"),
    (COCO_TO_YTVIS_2021, 
        os.path.join(_root, "/home/rui/datasets/coco2017/annotations/instances_train2017.json"),
        os.path.join(_root, "/home/rui/VISAGE/annotations/coco2ytvis2021_val.json"), "COCO val to YTVIS 2021:"),
    # (COCO_TO_OVIS,
    #     os.path.join(_root, "coco/annotations/instances_train2017.json"),
    #     os.path.join(_root, "coco/annotations/coco2ovis_train.json"), "COCO to OVIS:"),
]

for convert_dict, src_path, out_path, msg in convert_list:
    src_f = open(src_path, "r")
    out_f = open(out_path, "w")
    src_json = json.load(src_f)
    # print(src_json.keys())   dict_keys(['info', 'licenses', 'images', 'annotations', 'categories'])

    out_json = {}
    for k, v in src_json.items():
        if k != 'annotations':
            out_json[k] = v

    converted_item_num = 0
    out_json['annotations'] = []
    for anno in src_json['annotations']:
        if anno["category_id"] not in convert_dict:
            continue

        out_json['annotations'].append(anno)
        converted_item_num += 1

    json.dump(out_json, out_f)
    print(msg, converted_item_num, "items converted.")

至此，coco2ytvis转化完成。

换好路径，准备好权重文件

报错

找到报错地点将将int转换为int32.

再train一下。

将版本降低。

pip install setuptools==59.5.0

继续报错。

后面发现、能跑几个回合后报错，应是我的coco图像有问题.于是取消coco2ytvis这部分，如下：

然后报错修改路径VISAGE/mask2former_video/modeling/matcher.py 141行

。至此全部改完可以正常训练。

3090 bs设为2刚刚能跑。

结束语

VIS现在相对是组内靠谱的项目，但是及其吃设备（钱），祝诸位师弟好运。