目标检测之 IoU

IoU 作为目标检测算法性能 mAP 计算的一个非常重要的函数。

但纵观 IoU 计算的介绍知识，都是直接给出代码，给出计算方法，没有人彻底地分析过其中的逻辑，故本人书写该篇博客来介绍下其中的逻辑。

1. IoU的简介及原理解析

IoU 的全称为交并比（Intersection over Union），通过这个名称我们大概可以猜到 IoU 的计算方法。IoU 计算的是 “预测的边框” 和 “真实的边框” 的交集和并集的比值。

开始计算之前，我们首先进行分析下交集和并集到底应该怎么计算：我们首先需要计算交集，然后并集通过两个边框的面积的和减去交集部分即为并集，因此 IoU 的计算的难点在于交集的计算。

为了计算交集，你脑子里首先想到的方法应该是：考虑两个边框的相对位置，然后按照相对位置（左上，左下，右上，右下，包含，互不相交）分情况讨论，来计算交集。

上图就是你的直觉，这样想没有错。但计算一个交集，就要分多种情况讨论，要是程序真的按照这逻辑编写就太搞笑了。因此对这个问题进行进一步地研究显得十分有必要。

让我们重新思考一下两个框交集的计算。两个框交集的计算的实质是两个集合交集的计算，因此我们可以将两个框的交集的计算简化为：

通过简化，我们可以清晰地看到，交集计算的关键是交集上下界点（图中蓝点）的计算。

我们假设集合 A 为 [x1x_{1}x1​，x2x_{2}x2​]，集合 B 为 [y1y_{1}y1​，y2y_{2}y2​]。然后我们来求AB交集的上下界限。

交集计算的逻辑

• 交集下界 z1z_{1}z1​：max(x1,y1)\text{max}(x_{1}, y_{1})max(x1​,y1​)
• 交集上界 z2z_{2}z2​：min(x2,y2)\text{min}(x_{2}, y_{2})min(x2​,y2​)
• 如果 z2−z1z_{2}-z_{1}z2​−z1​ 小于0，则说明集合 A 和集合 B 没有交集。

下面使用Python来实现两个一维集合的 IoU 的计算：

<span style="color:#000000"><code class="language-python"><span style="color:#c678dd">def</span> <span style="color:#61aeee">iou</span><span style="color:#999999">(</span>set_a<span style="color:#999999">,</span> set_b<span style="color:#999999">)</span><span style="color:#999999">:</span>
    <span style="color:#669900">'''
    一维 iou 的计算
    '''</span>
    x1<span style="color:#999999">,</span> x2 <span style="color:#669900">=</span> set_a <span style="color:#5c6370"># (left, right)</span>
    y1<span style="color:#999999">,</span> y2 <span style="color:#669900">=</span> set_b <span style="color:#5c6370"># (left, right)</span>
    
    low <span style="color:#669900">=</span> <span style="color:#669900">max</span><span style="color:#999999">(</span>x1<span style="color:#999999">,</span> y1<span style="color:#999999">)</span>
    high <span style="color:#669900">=</span> <span style="color:#669900">min</span><span style="color:#999999">(</span>x2<span style="color:#999999">,</span> y2<span style="color:#999999">)</span>
    <span style="color:#5c6370"># intersection</span>
    <span style="color:#c678dd">if</span> high<span style="color:#669900">-</span>low<span style="color:#669900"><</span><span style="color:#98c379">0</span><span style="color:#999999">:</span>
        inter <span style="color:#669900">=</span> <span style="color:#98c379">0</span>
    <span style="color:#c678dd">else</span><span style="color:#999999">:</span>
        inter <span style="color:#669900">=</span> high<span style="color:#669900">-</span>low
    <span style="color:#5c6370"># union</span>
    union <span style="color:#669900">=</span> <span style="color:#999999">(</span>x2 <span style="color:#669900">-</span> x1<span style="color:#999999">)</span> <span style="color:#669900">+</span> <span style="color:#999999">(</span>y2 <span style="color:#669900">-</span> y1<span style="color:#999999">)</span> <span style="color:#669900">-</span> inter
    <span style="color:#5c6370"># iou</span>
    iou <span style="color:#669900">=</span> inter <span style="color:#669900">/</span> union
    <span style="color:#c678dd">return</span> iou
</code></span>

上面，我们计算了两个一维集合的 iou，将上面的程序进行扩展，即可得到两个框 IoU 计算的程序。

<span style="color:#000000"><code class="language-python"><span style="color:#c678dd">def</span> <span style="color:#61aeee">iou</span><span style="color:#999999">(</span>box1<span style="color:#999999">,</span> box2<span style="color:#999999">)</span><span style="color:#999999">:</span>
    <span style="color:#669900">'''
    两个框（二维）的 iou 计算
    
    注意：边框以左上为原点
    
    box:[top, left, bottom, right]
    '''</span>
    in_h <span style="color:#669900">=</span> <span style="color:#669900">min</span><span style="color:#999999">(</span>box1<span style="color:#999999">[</span><span style="color:#98c379">2</span><span style="color:#999999">]</span><span style="color:#999999">,</span> box2<span style="color:#999999">[</span><span style="color:#98c379">2</span><span style="color:#999999">]</span><span style="color:#999999">)</span> <span style="color:#669900">-</span> <span style="color:#669900">max</span><span style="color:#999999">(</span>box1<span style="color:#999999">[</span><span style="color:#98c379">0</span><span style="color:#999999">]</span><span style="color:#999999">,</span> box2<span style="color:#999999">[</span><span style="color:#98c379">0</span><span style="color:#999999">]</span><span style="color:#999999">)</span>
    in_w <span style="color:#669900">=</span> <span style="color:#669900">min</span><span style="color:#999999">(</span>box1<span style="color:#999999">[</span><span style="color:#98c379">3</span><span style="color:#999999">]</span><span style="color:#999999">,</span> box2<span style="color:#999999">[</span><span style="color:#98c379">3</span><span style="color:#999999">]</span><span style="color:#999999">)</span> <span style="color:#669900">-</span> <span style="color:#669900">max</span><span style="color:#999999">(</span>box1<span style="color:#999999">[</span><span style="color:#98c379">1</span><span style="color:#999999">]</span><span style="color:#999999">,</span> box2<span style="color:#999999">[</span><span style="color:#98c379">1</span><span style="color:#999999">]</span><span style="color:#999999">)</span>
    inter <span style="color:#669900">=</span> <span style="color:#98c379">0</span> <span style="color:#c678dd">if</span> in_h<span style="color:#669900"><</span><span style="color:#98c379">0</span> <span style="color:#669900">or</span> in_w<span style="color:#669900"><</span><span style="color:#98c379">0</span> <span style="color:#c678dd">else</span> in_h<span style="color:#669900">*</span>in_w
    union <span style="color:#669900">=</span> <span style="color:#999999">(</span>box1<span style="color:#999999">[</span><span style="color:#98c379">2</span><span style="color:#999999">]</span> <span style="color:#669900">-</span> box1<span style="color:#999999">[</span><span style="color:#98c379">0</span><span style="color:#999999">]</span><span style="color:#999999">)</span> <span style="color:#669900">*</span> <span style="color:#999999">(</span>box1<span style="color:#999999">[</span><span style="color:#98c379">3</span><span style="color:#999999">]</span> <span style="color:#669900">-</span> box1<span style="color:#999999">[</span><span style="color:#98c379">1</span><span style="color:#999999">]</span><span style="color:#999999">)</span> <span style="color:#669900">+</span> \
            <span style="color:#999999">(</span>box2<span style="color:#999999">[</span><span style="color:#98c379">2</span><span style="color:#999999">]</span> <span style="color:#669900">-</span> box2<span style="color:#999999">[</span><span style="color:#98c379">0</span><span style="color:#999999">]</span><span style="color:#999999">)</span> <span style="color:#669900">*</span> <span style="color:#999999">(</span>box2<span style="color:#999999">[</span><span style="color:#98c379">3</span><span style="color:#999999">]</span> <span style="color:#669900">-</span> box2<span style="color:#999999">[</span><span style="color:#98c379">1</span><span style="color:#999999">]</span><span style="color:#999999">)</span> <span style="color:#669900">-</span> inter
    iou <span style="color:#669900">=</span> inter <span style="color:#669900">/</span> union
    <span style="color:#c678dd">return</span> iou
</code></span>

2. 基于TensorFlow的IoU实现

上节介绍了IoU，及其的计算，下面我们给出其在 TensorFlow 上的实现：

<span style="color:#000000"><code class="language-python"><span style="color:#c678dd">import</span> tensorflow <span style="color:#c678dd">as</span> tf

<span style="color:#c678dd">def</span> <span style="color:#61aeee">IoU_calculator</span><span style="color:#999999">(</span>x<span style="color:#999999">,</span> y<span style="color:#999999">,</span> w<span style="color:#999999">,</span> h<span style="color:#999999">,</span> l_x<span style="color:#999999">,</span> l_y<span style="color:#999999">,</span> l_w<span style="color:#999999">,</span> l_h<span style="color:#999999">)</span><span style="color:#999999">:</span>
    <span style="color:#669900">"""calaulate IoU
    Args:
      x: net predicted x
      y: net predicted y
      w: net predicted width
      h: net predicted height
      l_x: label x
      l_y: label y
      l_w: label width
      l_h: label height
    
    Returns:
      IoU
    """</span>
    
    <span style="color:#5c6370"># convert to coner</span>
    x_max <span style="color:#669900">=</span> x <span style="color:#669900">+</span> w<span style="color:#669900">/</span><span style="color:#98c379">2</span>
    y_max <span style="color:#669900">=</span> y <span style="color:#669900">+</span> h<span style="color:#669900">/</span><span style="color:#98c379">2</span>
    x_min <span style="color:#669900">=</span> x <span style="color:#669900">-</span> w<span style="color:#669900">/</span><span style="color:#98c379">2</span>
    y_min <span style="color:#669900">=</span> y <span style="color:#669900">-</span> h<span style="color:#669900">/</span><span style="color:#98c379">2</span>
 
    l_x_max <span style="color:#669900">=</span> l_x <span style="color:#669900">+</span> l_w<span style="color:#669900">/</span><span style="color:#98c379">2</span>
    l_y_max <span style="color:#669900">=</span> l_y <span style="color:#669900">+</span> l_h<span style="color:#669900">/</span><span style="color:#98c379">2</span>
    l_x_min <span style="color:#669900">=</span> l_x <span style="color:#669900">-</span> l_w<span style="color:#669900">/</span><span style="color:#98c379">2</span>
    l_y_min <span style="color:#669900">=</span> l_y <span style="color:#669900">-</span> l_h<span style="color:#669900">/</span><span style="color:#98c379">2</span>
    <span style="color:#5c6370"># calculate the inter</span>
    inter_x_max <span style="color:#669900">=</span> tf<span style="color:#999999">.</span>minimum<span style="color:#999999">(</span>x_max<span style="color:#999999">,</span> l_x_max<span style="color:#999999">)</span>
    inter_x_min <span style="color:#669900">=</span> tf<span style="color:#999999">.</span>maximum<span style="color:#999999">(</span>x_min<span style="color:#999999">,</span> l_x_min<span style="color:#999999">)</span>
 
    inter_y_max <span style="color:#669900">=</span> tf<span style="color:#999999">.</span>minimum<span style="color:#999999">(</span>y_max<span style="color:#999999">,</span> l_y_max<span style="color:#999999">)</span>
    inter_y_min <span style="color:#669900">=</span> tf<span style="color:#999999">.</span>maximum<span style="color:#999999">(</span>y_min<span style="color:#999999">,</span> l_y_min<span style="color:#999999">)</span>
 
    inter_w <span style="color:#669900">=</span> inter_x_max <span style="color:#669900">-</span> inter_x_min
    inter_h <span style="color:#669900">=</span> inter_y_max <span style="color:#669900">-</span> inter_y_min
    
    inter <span style="color:#669900">=</span> tf<span style="color:#999999">.</span>cond<span style="color:#999999">(</span>tf<span style="color:#999999">.</span>logical_or<span style="color:#999999">(</span>tf<span style="color:#999999">.</span>less_equal<span style="color:#999999">(</span>inter_w<span style="color:#999999">,</span><span style="color:#98c379">0</span><span style="color:#999999">)</span><span style="color:#999999">,</span> tf<span style="color:#999999">.</span>less_equal<span style="color:#999999">(</span>inter_h<span style="color:#999999">,</span><span style="color:#98c379">0</span><span style="color:#999999">)</span><span style="color:#999999">)</span><span style="color:#999999">,</span> 
                    <span style="color:#c678dd">lambda</span><span style="color:#999999">:</span>tf<span style="color:#999999">.</span>cast<span style="color:#999999">(</span><span style="color:#98c379">0</span><span style="color:#999999">,</span>tf<span style="color:#999999">.</span>float32<span style="color:#999999">)</span><span style="color:#999999">,</span> 
                    <span style="color:#c678dd">lambda</span><span style="color:#999999">:</span>tf<span style="color:#999999">.</span>multiply<span style="color:#999999">(</span>inter_w<span style="color:#999999">,</span>inter_h<span style="color:#999999">)</span><span style="color:#999999">)</span>
    <span style="color:#5c6370"># calculate the union</span>
    union <span style="color:#669900">=</span> w<span style="color:#669900">*</span>h <span style="color:#669900">+</span> l_w<span style="color:#669900">*</span>l_h <span style="color:#669900">-</span> inter
    
    IoU <span style="color:#669900">=</span> inter <span style="color:#669900">/</span> union
    <span style="color:#c678dd">return</span> IoU
</code></span>