3 篇文章 0 订阅

Log-Sum-Exp Pooling

LSE Pooling

x p = 1 r ⋅ l o g [ 1 S ⋅ ∑ ( i , j ) ∈ S e x p ( r ⋅ x i j ) ] x_p=\frac{1}{r}\cdot log[\frac{1}{S}\cdot \sum_{(i,j)\in\mathbf{S}}exp(r\cdot x_{ij})]

The hyper-parameter r controls how smooth one wants the approximation to be: high r values implies having an effect similar to the max, very low values will have an effect similar to the score averaging. The advantage of this aggregation is that pixels having similar scores will have a similar weight in the training procedure, r controlling this notion of “similarity”.

By controlling the hyper-parameter, r, the pooled value ranges from the maximum in S (when r → ∞ r\to\infin ) to average ( r → 0 r\to0 ).

数学证明

x p = 1 r ⋅ l o g [ 1 n ⋅ ∑ i = 1 n e x p ( r ⋅ x i ) ] x_p=\frac{1}{r}\cdot log[\frac{1}{n}\cdot \sum_{i=1}^{n}exp(r\cdot x_i)]

证明 r → 0 r\to0 相当于 Average Pooling

a 1 + a 2 + . . . + a n n ≥ a 1 ⋅ a 2 . . . a n n \frac{a_1+a_2+...+a_n}{n}\ge\sqrt[n]{a_1\cdot a_2...a_n}

x p = 1 r ⋅ l o g [ 1 n ⋅ ∑ i = 1 n e x p ( r ⋅ x i ) ] = l o g ( 1 n ⋅ ∑ i = 1 n e r ⋅ x i ) 1 r \begin{aligned} x_p &amp;= \frac{1}{r}\cdot log[\frac{1}{n}\cdot \sum_{i=1}^{n}exp(r\cdot x_i)] \\ &amp;= log(\frac{1}{n}\cdot\sum_{i=1}^{n}e^{r\cdot x_i})^{\frac{1}{r}} \end{aligned}

( 1 n ⋅ ∑ i = 1 n e r ⋅ x i ) 1 r ≥ ( ∏ i = 1 n e r ⋅ x i ) 1 n ⋅ 1 r = ( ∏ i = 1 n e x i ) 1 n \begin{aligned} (\frac{1}{n}\cdot \sum_{i=1}^{n}e^{r\cdot x_i})^{\frac{1}{r}} &amp;\ge (\prod_{i=1}^{n} e^{r\cdot x_i})^{\frac{1}{n}\cdot\frac{1}{r}} \\ &amp;= (\prod_{i=1}^{n} e^{x_i})^{\frac{1}{n}} \end{aligned}
r = 0 r = 0 时，可取等号。代入整个式子：

x p = l o g ( 1 n ⋅ ∑ i = 1 n e r ⋅ x i ) 1 r ≥ l o g ( ∏ i = 1 n e x i ) 1 n = 1 n ∑ i = 1 n x i \begin{aligned} x_p &amp;= log(\frac{1}{n}\cdot\sum_{i=1}^{n}e^{r\cdot x_i})^{\frac{1}{r}} \\ &amp;\ge log(\prod_{i=1}^{n} e^{x_i})^{\frac{1}{n}} \\ &amp;= \frac{1}{n}\sum_{i=1}^{n}x_i \end{aligned}

证明 r → ∞ r\to \infin 相当于 Max Pooling

x p = 1 r ⋅ l o g [ 1 n ⋅ ∑ i = 1 n e x p ( r ⋅ x i ) ] = l o g ( ∑ i = 1 n e r ⋅ x i ) 1 r − 1 r ⋅ l o g ( n ) \begin{aligned} x_p &amp;= \frac{1}{r}\cdot log[\frac{1}{n}\cdot \sum_{i=1}^{n}exp(r\cdot x_i)] \\ &amp;= log(\sum_{i=1}^{n}e^{r\cdot x_i})^{\frac{1}{r}} - \frac{1}{r}\cdot log(n) \end{aligned}

m a x ( e r ⋅ x i ) 1 r ≤ ( ∑ i = 1 n e r ⋅ x i ) 1 r ≤ [ n ⋅ m a x ( e r ⋅ x i ) ] 1 r \begin{aligned} max(e^{r\cdot x_i})^{\frac{1}{r}} \le (\sum_{i=1}^{n}e^{r\cdot x_i})^{\frac{1}{r}} \le [n\cdot max(e^{r\cdot x_i})]^{\frac{1}{r}} \end{aligned}

m a x ( x i ) ≤ l o g ( ∑ i = 1 n e r ⋅ x i ) 1 r ≤ 1 r ⋅ l o g ( n ) + m a x ( x i ) max(x_i)\le log(\sum_{i=1}^{n}e^{r\cdot x_i})^{\frac{1}{r}} \le \frac{1}{r}\cdot log(n)+max(x_i)
r → ∞ r\to\infin 时有： 1 r ⋅ l o g ( n ) → 0 \frac{1}{r}\cdot log(n)\to0 ，故 r → ∞ r \to\infin 相当于 Max Pooling 得证

• 6
点赞
• 10
收藏
• 打赏
• 1
评论
08-14
12-09 552
05-06 250
01-07 1699
05-13 143
01-05 5万+
07-01 191
09-19 49
05-05 160
03-22 3295
11-28 648
03-25 5523
12-11 3715

“相关推荐”对你有帮助么？

• 非常没帮助
• 没帮助
• 一般
• 有帮助
• 非常有帮助

A_bigUncle

¥2 ¥4 ¥6 ¥10 ¥20

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、C币套餐、付费专栏及课程。