(2021 ICCV) Disentangled High Quality Salient Object Detection (A类)

一、作者

​ Lv Tang ，Bo Li1，Yijie Zhong ，Shouhong Ding， Mofei Song2， Youtu Lab

二、地址

2.1 原文地址

ICCV 地址

2.2 代码地址

代码

三、摘要

​ Aiming at discovering and locating most distinctive objects from visual scenes, salient object detection (SOD) plays an essential role in various computer vision systems. Coming to the era of high resolution, SOD methods are facing new challenges. The major limitation of previous methods is that they try to identify the salient regions and estimate the accurate objects boundaries simultaneously with a single regression task at low-resolution. This practice ignores the inherent difference between the two difficult problems, resulting in poor detection quality. In this paper, we propose a novel deep learning framework for high-resolution SOD task, which disentangles the task into a low-resolution saliency classification network (LRSCN) and a high-resolution refinement network (HRRN). As apixel-wise classification task, LRSCN is designed to capture sufficient semantics at low-resolution to identify the definite salient(most pixels inside the salient object have the highest saliency value), background( most pixels in the background regions have the lowest salient value) and uncertain image regions(saliency values of the pixels at blurry object boundaries fluctuate between 0 and 1). HRRN is a regression task, which aims at accurately refining the saliency value of pixels in the uncertain region to preserve a clear object boundary at high-resolution with limited GPU memory. It is worth noting that by introducing uncertainty into the training process, our HRRN can well address the high-resolution refinement task without using any high-resolution training data. Extensive experiments on high-resolution saliency datasets as well as some widely used saliency benchmarks show that the method achieves superior performance compared to the state-of -the-art methods.

四、主要内容

4.1 主要工作

• We provide a new perspective that high-resolution salient object detection should be disentangled into two tasks, and demonstrate that the disentanglement of the two tasks is essential for improving the performance of DNN based SOD models.
• Motivated by the principle of disentanglement, we propose a novel framework for high-resolution salient object detection, which uses LRSCN to capture sufficient semantics at low-resolution and HRRN for accurate boundary refinement at high-resolution.
• We make the earliest efforts to introduce the uncertainty into SOD network training, which empowers HRRN to well address the high-resolution refinement task without any high-resolution training datasets.
• We perform extensive experiments to demonstrate the proposed method refreshes the SOTA performance on high-resolution saliency datasets as well as some widely used saliency benchmarks by a large margin.

4.2 网络结构 (VGG-16)

4.3 MECF and AGA

4.3.1 ME ( based on Global Convolutional Network (GCN))

4.3.2 CF (utilize cross-level feature fusion module)

4.3.3 SGA ( guarantees the alignment of trimap and saliecny map)

4.4 LOSS

4.4.1 LRSCN Loss

$ T^{gt}$ : trimap groundtruth

$$T^{gt}(x,y)=\{\begin{array}{l}2,T^{gt}(x,y)ϵdefinitesalient\\ 0,T^{gt}(x,y)ϵdefinitebackground\\ 1,T^{gt}(x,y)ϵdefiniteuncertainregion\end{array}$$

$$L_{trimap}=\frac{1}{N}\sum _i-log(\frac{e^{T_i}}{{\sum }_j{e}^{T_j}})$$

$$\begin{gathered} L_{LRSCN}=L_{saliency}+L_{trimap} \\ {L}_{saliency}:BCE+SSIM+F-measure \end{gathered}$$

4.4.2 HRRN Loss

​ uncertainty loss will make the weight of the loss in the uncertainty region be small and let the network ignore effects from noisy data as much as possible.

$$\begin{gathered} L_1=\frac{1}{E}\sum _{iϵE}|S_i^H-G_i^H| \\ E:numbeofpixels \end{gathered}$$

$$\begin{gathered} L_{uncertainty}=\frac{1}{U}\sum _{iϵU}\frac{{||S_i^H-G_i^H||}^2}{2{\sigma }_i^2}+\frac{1}{2}log{\sigma }_i^2 \\ U:totalnumberofpixelsinuncertainregion \end{gathered}$$

$$L_{HRRN}=L_{uncertainty}+L_1$$

五、评估材料

• MAE
• F-measure ($F_{\beta }$ and $F_{\beta }^{max}$)
• Structure Measure
• RP曲线
• BDE ( Boundary Displacement Error) 边界漂移误差
• $B_{\mu }$

六、结论

​ In this paper, we argue that there are two difficult and inherently different problems in high-resolution SOD. From this perspective, we propose a novel deep learning framework to disentangle the high-resolution SOD into two tasks: LRSCN and HRRN. LRSCN can identify the definite salient, background and uncertain regions at low-resolution with sufficient semantics. While HRRN can accurately refining the saliency value of pixels in the uncertain region to preserve a clear object boundary at high-resolution with limited GPU memory. We also make the earliest efforts to introduce the uncertainty into SOD network training, which empower HRRN to learn rich details without using any high-resolution training datasets. Extensive evaluations on high-resolution datasets and popular benchmark datasets not only verify the superiority of our method but also demonstrate the importance of disentanglement for SOD. We believe our novel disentanglement view in this work can contribute to other high-resolution computer vision tasks in the future.