Wasserstein距离

前置知识：Computational optimal transport

Kantorovich’s optimal transport problem

$${\mathrm{L}}_{\mathbf{C}}(\mathbf{a},\mathbf{b})\stackrel{\text{ def. }}{=}\underset{\mathbf{P}\in \mathbf{U}(\mathbf{a},\mathbf{b})}{\min }⟨\mathbf{C},\mathbf{P}⟩$$

$$\mathbf{U}(\mathbf{a},\mathbf{b})\stackrel{\text{ def. }}{=}\left\{\mathbf{P}\in {\mathbb{R}}_{+}^{n\times m}:\mathbf{P}{1}_m=\mathbf{a}\text{ and }{\mathbf{P}}^{\mathrm{T}}{1}_n=\mathbf{b}\right\}$$

对偶

$${\mathrm{L}}_{\mathbf{C}}(\mathbf{a},\mathbf{b})=\underset{(\mathbf{f},\mathbf{g})\in \mathbf{R}(\mathbf{a},\mathbf{b})}{\max }⟨\mathbf{f},\mathbf{a}⟩+⟨\mathbf{g},\mathbf{b}⟩$$

R ( a , b ) =  def.  { ( f , g ) ∈ R n × R m : f ⊕ g ≤ C } \mathbf{R}(\mathbf{a}, \mathbf{b}) \stackrel{\text { def. }}{=}\left\{(\mathbf{f}, \mathbf{g}) \in \mathbb{R}^n \times \mathbb{R}^m: \mathbf{f} \oplus \mathbf{g} \leq \mathbf{C}\right\} R(a,b)= def. {(f,g)∈Rn×Rm:f⊕g≤C}

Wasserstein距离

考虑$n=m,p\ge 1$
令$\mathbf{C}={\mathbf{D}}^p\in {\mathbb{R}}^{n\times n}$
其中$\mathbf{D}$是距离，即满足
(1)$\mathbf{D}\in {\mathbb{R}}_{+}^{n\times n}$是对称的
(2)${\mathbf{D}}_{i,j}=0$当且仅当$i=j$
(3)$\forall i,j,k,{\mathbf{D}}_{i,k}\le {\mathbf{D}}_{i,j}+{\mathbf{D}}_{j,k}$

则
$${\mathrm{W}}_p(\mathbf{a},\mathbf{b})\stackrel{\text{ def. }}{=}{\mathrm{L}}_{{\mathbf{D}}^p}(\mathbf{a},\mathbf{b}{)}^{1/p}$$
称为p-Wasserstein distance，
可以证明p-Wasserstein distance也是距离

证明：
再说

对偶

$${\mathrm{W}}_p(\mathbf{a},\mathbf{b})=\underset{(\mathbf{f},\mathbf{g})\in \mathbf{R}(\mathbf{a},\mathbf{b})}{\max }⟨\mathbf{f},\mathbf{a}⟩+⟨\mathbf{g},\mathbf{b}⟩$$
R ( a , b ) =  def.  { ( f , g ) ∈ R n × R n : f ⊕ g ≤ D p } \mathbf{R}(\mathbf{a}, \mathbf{b}) \stackrel{\text { def. }}{=}\left\{(\mathbf{f}, \mathbf{g}) \in \mathbb{R}^n \times \mathbb{R}^n: \mathbf{f} \oplus \mathbf{g} \leq \mathbf{D}^p\right\} R(a,b)= def. {(f,g)∈Rn×Rn:f⊕g≤Dp}

而

$$\mathbf{f}\oplus \mathbf{g}\le {\mathbf{D}}^p\Rightarrow f_i+g_i\le 0$$
因此
$$⟨\mathbf{f},\mathbf{a}⟩+⟨\mathbf{g},\mathbf{b}⟩=\sum _{i=1}^n\left(f_i{a}_i+g_i{b}_i\right)\le \sum _{i=1}^n\left(f_i{a}_i-f_i{b}_i\right)$$

搬土距离

Earth Mover’s Distance
考虑有两个概率分布$P_r,P_{\theta }$
则
$$\mathrm{EMD}\left(P_r,P_{\theta }\right)=\underset{\gamma \in \Pi }{\inf }\sum _{x,y}\Vert x-y\Vert \gamma (x,y)=\underset{\gamma \in \Pi }{\inf }{\mathbb{E}}_{(x,y)\sim \gamma }\Vert x-y\Vert$$
考虑对偶
$$\mathrm{EMD}\left(P_r,P_{\theta }\right)=\underset{\Vert f{\Vert }_{L\le 1}}{\sup }{\mathbb{E}}_{x\sim P_r}f(x)-{\mathbb{E}}_{x\sim P_{\theta }}f(x).$$
$\Vert f{\Vert }_{L\le 1}$表示Lipschitz连续，其中Lipschitz常数为$L$
即$\left|f\left(\mathbf{x}\right)-f\left(\mathbf{y}\right)\right|\le L\Vert \mathbf{x}-\mathbf{y}\Vert$

参考：
从Wasserstein距离、对偶理论到WGAN
Wasserstein GAN and the Kantorovich-Rubinstein Duality