CVPR 2020 运行SGMN遇到的问题及解决办法

文章简介

Graph-Structured Referring Expressions Reasoning in The Wild
代码链接
论文链接

Issues

在作者代码的README中，有环境配置的步骤，不过python2.7+pytorch0.4.0支持的cuda版本和本机显卡支持的cuda版本存在冲突，无法正常运行代码，会出现以下错误：

# 错误1
RuntimeError: cuDNN version mismatch: PyTorch was compiled against 7102 but linked against 7605
# 错误2
CUDNN_STATUS_EXECUTION_FAILED

错误1可以通过以下命令解决：

conda install cudnn=7.1.2

错误2是版本冲突，我没解决，所以没办法和作者用一样的环境。但是，在python3的环境下，作者的代码稍作调整也可以运行，具体配置步骤见下文。

环境配置

本机环境：

• ubuntu18.04
• rtx 2080ti
• NVIDIA-SMI 450.57

基于本机环境，我配置的conda虚拟环境中，各软件版本如下：

• python 3.6.2
• pytorch 1.4.0 - py3.6_cuda10.0.130_cudnn7.6.3_0
• torchvision 0.5.0 - py36_cu100
• scipy 1.5.2
• numpy 1.19.1
• h5py 2.10.0
• cudatoolkit 10.0.130

在上述环境中，对代码稍微修改一下（主要是python2$\to$ python3，如print xxx，改为print(xxx)），即可正常运行。

环境配置代码：

conda create -n lchpy36 python=3.6
source activate lchpy36
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
pip install h5py
pip install scipy

运行程序代码：

git clone https://github.com/sibeiyang/sgmn.git && cd sgmn
# 建立软链接一定要用绝对路径
ln -s /home/mcislab/dist4T/chuanhao/dataset/Ref-Reasoning/refvg/ /home/mcislab/dist4T/chuanhao/code/sgmn/data/
ln -s /home/mcislab/dist4T/chuanhao/dataset/Ref-Reasoning/gt_objects/ /home/mcislab/dist4T/chuanhao/code/sgmn/data/
ln -s /home/mcislab/dist4T/chuanhao/others/sgmn_glove/word_embedding/ /home/mcislab/dist4T/chuanhao/code/sgmn/data/
bash experiments/script/train.sh $GPUs

Ref-Reasoning数据集格式

train(val)_expression.json

{"825670": 
	{"num_nodes": 4, 
	 "referent_id": "4306103", 
	 "image_id": "2384229", 
	 "bbox": [215, 50, 43, 55], 
	 "expression": "There is a people to the right of man and to the left of a wine , and the man holding empty glass ."}, 
	
	...
}

train(val)_sgs.json

{"800000": 
	{"words_info": [[1, "det", 0, "a"], 
					[1, "amod", 0, "yellow"], 
					[1, "head", 0, "frisbee"], 
					[1, "none", 1, "to"], 
					[1, "none", 1, "the"], 
					[1, "none", 1, "left"], 
					[1, "none", 1, "of"], 
					[1, "head", 2, "man"], 
					[0, "none", 3, "that"], 
					[0, "none", 3, "is"], 
					[1, "none", 3, "wearing"], 
					[1, "amod", 4, "white"], 
					[1, "head", 4, "socks"], 
					[1, "none", 5, "and"], 
					[0, "none", 5, "is"], 
					[1, "none", 5, "wearing"], 
					[1, "head", 6, "shorts"], 
					[1, "none", 7, "."]], 
	 "co_index": {}},

	...
}

train(val)_sg_seqs.json

{"800000":
	 {"seq_sg": [{"seq_rel": [], "type": "S", "seq": [7]}, 
	 			 {"seq_rel": [], "type": "SPO", "seq": [2, 5, 6]}, 
	 			 {"seq_rel": [[2, 1, "SUBANDSUB"]], "type": "SPO", "seq": [2, 3, 4]},
	 			 {"seq_rel": [[3, 2, "OBJANDSUB"]], "type": "SPO", "seq": [0, 1, 2]}], 
	 "com_seq": [0, 3], 
	 "num_seq": 4, 
	 "split_to_seq": {"0": 3, "2": 3, "4": 2, "7": 0, "6": 1}},
	
	...
}

gt_objects_info.json

{"2379671":
	{"width": 500, 
	 "idx": 1541, 
	 "file": 2, 
	 "objectsNum": 15, 
	 "height": 438},
	
	 ...
}

*_sgs.json

basic info about language scene graph

{"expression_id":
	{"word_info": list of info (split id, dependent type, weight, word)
	 "co_index": dictionary from one split id to its coreference's split id
	},

	...
}

*_sg_seqs.json

structured info about language scene graph

{"expression_id":
	{"seq_sg": list of nodes and edges. Each node and edge includes its phrase listed by split ids, its relations to other nodes and edges, and its type info.
	 "com_seq": list of indexes of elements with zero out-degree in seq_sg
	 "num_seq": number of nodes and edges
	 "split_to_seq": dictionary from one split id to the index of seq_sg
	},

	...
}