CVPR2020交互式分割算法IOG的配置（Interactive Object Segmentation with Inside-Outside Guidance）

目标跟踪、目标检测、前景分割不分家，如SiamMask、SiamR-CNN

这篇文章针对目标框可起到很好地分割效果。

注：原代码的运行环境为Ubuntu，本文在Windows10系统下完成配置。

1、论文下载：

Interactive Object Segmentation with Inside-Outside Guidance. [paper][code]

2、代码下载：

https://github.com/shiyinzhang/Inside-Outside-Guidance

3、建立且激活虚拟环境IOG

 conda create -n IOG python=3.7.0

activate IOG

4、安装torch和torchvision

pip install torch===1.4.0 -f https://download.pytorch.org/whl/torch_stable.html

pip install torchvision===0.5.0 -f https://download.pytorch.org/whl/torch_stable.html

5、安装pycocotools和opencv-python

pip install pycocotools opencv-python

6、下载预训练模型放到工程路径

链接：https://pan.baidu.com/s/1-BViduSNAd-lwe81tMv-2A 
提取码：72sj 

7、下载PascalVOC2012数据集

Pascal VOC Dataset Mirror

只下载图中标注的数据集，然后解压即可。

8、打开mypath.py设置路径

9、打开test.py设置预训练模型路径

10、运行python test.py遇到错误

Traceback (most recent call last):
  File "test.py", line 2, in <module>
    import scipy.misc as sm
ModuleNotFoundError: No module named 'scipy'

错误原因是： 没有安装scipy，解决方法：pip install scipy

11、再次运行遇到错误

RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.BrokenPipeError: [Errno 32] Broken pipe
错误原因：线程冲突，解决办法：把test.py的71行num_workers=1去掉。

11、再次运行，出现错误

Traceback (most recent call last):
  File "test.py", line 91, in <module>
    sm.imsave(os.path.join(save_dir_res_list[0], metas['image'][0] + '-' + metas['object'][0] + '.png'), result)
AttributeError: module 'scipy.misc' has no attribute 'imsave'
错误原因：scipy版本不兼容问题，解决方法：将scipy降级到1.2.1版本之下即可，pip install scipy==1.2.1

12、再次运行成功。

结果存放在run_0/Results路径下

13、测试单张图片

1）新建 inference.py

from datetime import datetime
import scipy.misc as sm
from collections import OrderedDict
import glob
import numpy as np
import socket

# PyTorch includes
import torch
import torch.optim as optim
from torchvision import transforms
from torch.utils.data import DataLoader

# Custom includes
from dataloaders.combine_dbs import CombineDBs as combine_dbs
import dataloaders.pascal as pascal
import dataloaders.sbd as sbd
from dataloaders import custom_transforms as tr
from networks.loss import class_cross_entropy_loss 
from dataloaders.helpers import *
from networks.mainnetwork import *

import matplotlib.pyplot as plt

from PIL import Image
import cv2
import argparse


def process(image_name):

    # Set gpu_id to -1 to run in CPU mode, otherwise set the id of the corresponding gpu
    gpu_id = 0
    device = torch.device("cuda:"+str(gpu_id) if torch.cuda.is_available() else "cpu")
    if torch.cuda.is_available():
        print('Using GPU: {} '.format(gpu_id))

    # Setting parameters
    resume_epoch = 100  # test epoch
    nInputChannels = 5  # Number of input channels (RGB + heatmap of IOG points)

    # Network definition
    modelName = 'IOG_pascal'
    net = Network(nInputChannels=nInputChannels,
                  num_classes=1,
                  backbone='resnet101',
                  output_stride=16,
                  sync_bn=None,
                  freeze_bn=False)

    # load pretrain_dict
    pretrain_dict = torch.load('IOG_PASCAL_SBD.pth')

    net.load_state_dict(pretrain_dict)
    # net.to(device)

    # Generate result of the validation images
    net.eval()

    image = np.array(Image.open(image_name).convert('RGB'))
    im_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    roi = cv2.selectROI(im_rgb)
    image = image.astype(np.float32)

    bbox = np.zeros_like(image[..., 0])
    # bbox[0: 130, 220: 320] = 1 # for ponny
    # bbox[220: 390, 370: 570] = 1
    bbox[int(roi[1]):int(roi[1]+roi[3]), int(roi[0]):int(roi[0]+roi[2])] = 1
    void_pixels = 1 - bbox
    sample = {'image': image, 'gt': bbox, 'void_pixels': void_pixels}

    trns = transforms.Compose([
        tr.CropFromMask(crop_elems=('image', 'gt','void_pixels'), relax=30, zero_pad=True),
        tr.FixedResize(resolutions={'gt': None, 'crop_image': (512, 512), 'crop_gt': (512, 512), 'crop_void_pixels': (512, 512)},flagvals={'gt':cv2.INTER_LINEAR,'crop_image':cv2.INTER_LINEAR,'crop_gt':cv2.INTER_LINEAR,'crop_void_pixels': cv2.INTER_LINEAR}),
        tr.IOGPoints(sigma=10, elem='crop_gt',pad_pixel=10),
        tr.ToImage(norm_elem='IOG_points'), 
        tr.ConcatInputs(elems=('crop_image', 'IOG_points')),
        tr.ToTensor()])

    tr_sample = trns(sample)

    inputs = tr_sample['concat'][None]
    # inputs = inputs.to(device)
    outputs = net.forward(inputs)[-1]
    # outputs = fine_out.to(torch.device('cpu'))
    pred = np.transpose(outputs.data.numpy()[0, :, :, :], (1, 2, 0))
    pred = 1 / (1 + np.exp(-pred))
    pred = np.squeeze(pred)
    gt = tens2image(tr_sample['gt'])
    bbox = get_bbox(gt, pad=30, zero_pad=True)
    result = crop2fullmask(pred, bbox, gt, zero_pad=True, relax=0,mask_relax=False)

    light = np.zeros_like(image)
    light[:, :, 2] = 255.

    alpha = 0.8

    blending = (alpha * light + (1 - alpha) * image) * result[..., None] + (1 - result[..., None]) * image

    blending[blending > 255.] = 255

    cv2.imshow('resulting segmentation', cv2.cvtColor(blending.astype(np.uint8), cv2.COLOR_RGB2BGR))
    cv2.waitKey(0)
    cv2.destroyAllWindows()

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Run class agnostic segmentation')
    parser.add_argument('--image_name', type=str, default='samples/IMG-20201203-WA0023.jpg', 
        help='path to target image')

    args = parser.parse_args()

    process(args.image_name)
    

 2）运行python inference.py --image_name E:/Datasets/OTB2015/Lemming/img/0001.jpg

3）选取目标框后，回车

4）分割结果