【基础储备】Differential Privacy 基础知识储备

onion routing, online tracking, privacy policies, genetic privacy, social networks

传统隐私保护方法概览

• k-anonymity
• k-map
• l-diversity
• δ-presence

Why differential privacy is awesome

1. We no longer need attack modeling.
   \quad We protect any kind of information about an individual. It doesn$’$t matter what the attacker wants to do.
   \quad It works no matter what the attacker knows about our data.
2. We can quantify the privacy loss.
   \quad When we use differential privacy, we can quantify the greatest possible information gain by the attacker.
3. We can compose multiple mechanisms.
   \quad Composition is a way to stay in control of the level of risk as new use cases appear and processes evolve.

Quantify the attacker${}^{\prime }$s knowledge   \, ( or, what it means to quantify information gain, &bound)

In the attacker$’$s model of the world, the actual database $\mathcal{D}$ can be either $D$ or $D^{\prime }$.
$\mathbb{P}[\mathcal{D}=D]$ is the initial suspicion of the attacker. Similarly, their another initial suspicion is $\mathbb{P}[\mathcal{D}=D^{\prime }]=1-\mathbb{P}[\mathcal{D}=D]$.
The updated suspicion $\mathbb{P}[\mathcal{D}=D|A(\mathcal{D})=\mathcal{O}]$ is the attacker$’$s model after seeing the mechanism returns output $\mathcal{O}$.

With differential privacy, the updated probability(suspicion) is never too far from the initial suspicion. The black line is what happens if the attacker didn$’$t get their suspicion updated at all. The blue lines are the lower and upper bounds on the updated suspicion: it can be anywhere between the two.
  \,
Proof :
  \,
Bayes$’$ rule : $\mathbb{P}[\mathcal{D}=D\mid A(\mathcal{D})=\mathcal{O}]=\frac{\mathbb{P}[\mathcal{D}=D]\cdot \mathbb{P}[A(\mathcal{D})=\mathcal{O}\mid \mathcal{D}=D]}{P[A(\mathcal{D})=\mathcal{O}]}$
  \,
$\frac{\mathbb{P}[\mathcal{D}=D\mid A(\mathcal{D})=\mathcal{O}]}{\mathbb{P}[\mathcal{D}=D^{\prime }\mid A(\mathcal{D})=\mathcal{O}]}=\frac{\mathbb{P}[\mathcal{D}=D]\cdot \mathbb{P}[A(\mathcal{D})=\mathcal{O}\mid \mathcal{D}=D]}{\mathbb{P}[\mathcal{D}=D^{\prime }]\cdot \mathbb{P}[A(\mathcal{D})=\mathcal{O}\mid \mathcal{D}=D^{\prime }]}=\frac{\mathbb{P}[\mathcal{D}=D]\cdot \mathbb{P}[A(D)=\mathcal{O}]}{\mathbb{P}[\mathcal{D}=D^{\prime }]\cdot \mathbb{P}[A(D^{\prime })=\mathcal{O}]}$
  \,
Differential privacy : $e^{-\epsilon }\le \frac{\mathbb{P}[A(D)=\mathcal{O}]}{\mathbb{P}[A(D^{\prime })=\mathcal{O}]}\le e^{\epsilon }$
  \,
$e^{-\epsilon }\cdot \frac{\mathbb{P}\left[\mathcal{D}=D\right]}{\mathbb{P}\left[\mathcal{D}=D^{\prime }\right]}\le \frac{\mathbb{P}\left[\mathcal{D}=D\mid A(\mathcal{D})=\mathcal{O}\right]}{\mathbb{P}\left[\mathcal{D}=D^{\prime }\mid A(\mathcal{D})=\mathcal{O}\right]}\le e^{\epsilon }\cdot \frac{\mathbb{P}\left[\mathcal{D}=D\right]}{\mathbb{P}\left[\mathcal{D}=D^{\prime }\right]}$
  \,
Replace $\mathbb{P}[\mathcal{D}=D^{\prime }]$ with $1-\mathbb{P}[\mathcal{D}=D]$, do the same for $\mathbb{P}\left[\mathcal{D}=D^{\prime }|A(\mathcal{D})=\mathcal{O}\right]$, and solve for $\mathbb{P}\left[\mathcal{D}=D|A(\mathcal{D})=\mathcal{O}\right]$
  \,
$\frac{\mathbb{P}\left[\mathcal{D}=D\right]}{e^{\epsilon }+\left(1-e^{\epsilon }\right)\cdot \mathbb{P}\left[\mathcal{D}=D\right]}\le \mathbb{P}\left[\mathcal{D}=D\mid A\left(\mathcal{D}\right)=\mathcal{O}\right]\le \frac{e^{\epsilon }\cdot \mathbb{P}\left[\mathcal{D}=D\right]}{1+\left(e^{\epsilon }-1\right)\cdot \mathbb{P}\left[\mathcal{D}=D\right]}$
  \,
We can draw a generalization of this graph for various values of $\epsilon$ :

The privacy loss random variable

Recall the proof above. We saw that $\epsilon$ bounds the evolution of betting odds : $\frac{\mathbb{P}\left[D=D_1\mid A(D)=O\right]}{\mathbb{P}\left[D=D_2\mid A(D)=O\right]}\le e^{\epsilon }\cdot \frac{\mathbb{P}\left[D=D_1\right]}{\mathbb{P}\left[D=D_2\right]}$
Let us define:
$${\mathcal{L}}_{D_1,D_2}(O)=\ln \frac{\frac{\mathbb{P}\left[D=D_1\mid A(D)=O\right]}{\mathbb{P}\left[D=D_2\mid A(D)=O\right]}}{\frac{\mathbb{P}\left[D=D_1\right]}{\mathbb{P}\left[D=D_2\right]}}=\ln \left(\frac{\mathbb{P}\left[A(D_1)=O\right]}{\mathbb{P}\left[A(D_2)=O\right]}\right)$$

Intuitively, the privacy loss random variable is the actual $\epsilon$ value for a specific output $O$.
在定义 $\mathbb{P}[A(D_1)=O]\le e^{\epsilon }\cdot \mathbb{P}[A(D_2)=O]$ 中取$\le$的体现：
     \quad\,\,\,\,

实际上只有当输出 $O$ 在 $O=A(D_1)$ 与 $O=A(D_2)$ 之间时才取 $<$ , 此时同样 ${\mathcal{L}}_{D_1,D_2}(O)\le \epsilon$ 只取 $<$

$\delta$ in $(\epsilon ,\delta )$-$DP$

$$\delta =\sum _O\mathbb{P}[A(D_1)=O]\cdot \max \left(0,1-e^{\epsilon -{\mathcal{L}}_{D_1,D_2}(O)}\right)={\mathbb{E}}_{O\sim A(D_1)}\left[\max \left(0,1-e^{\epsilon -{\mathcal{L}}_{D_1,D_2}(O)}\right)\right]$$

Intuitively, in $(\epsilon ,\delta )$-DP, the $\delta$ is the area highlighted below.

注 : 使用高斯噪声时, 若仅仅直观地将$\delta$理解为是 $(\epsilon ,0)$-$DP$即传统$\epsilon$-$DP$的定义可以被违反的概率, 则是选取了$\delta ={\delta }_1$
但实际上${\delta }_2$更紧。从上图可见 , ${\delta }_2<{\delta }_1$ , 因为${\delta }_1$在上图中也可以看作是长为${\delta }_1$宽为1的矩形面积, 显然大于阴影部分的面积${\delta }_2$
上述${\delta }_1$(   \, 即传统$\epsilon$-$DP$的定义可以被违反的概率   \, )   \, 的直观意义如下图 :

Proof :
  \,
The definition of $(\epsilon ,\delta )$-$DP$ : $\mathbb{P}[A(D_1)\in S]\le e^{\epsilon }\cdot \mathbb{P}[A(D_2)\in S]+\delta .$
Fix a mechanism $A$ and a $\epsilon \ge 0$. For each possible set of outputs $S$, we can compute:
  \,
$\delta ={\max }_S\left(\mathbb{P}[A(D_1)\in S]-e^{\epsilon }\cdot \mathbb{P}[A(D_2)\in S]\right)$
  \,
The set $S$ that maximizes the quantity above is:
  \,
S m a x = { O ∣ P [ A ( D 1 ) = O ] > e ε ⋅ P [ A ( D 2 ) = O ] } = { O ∣   L D 1 , D 2 ( O ) > ε    } S_{max} = \left\{O \mid \mathbb{P}[A(D_1)=O] > e^\varepsilon\cdot\mathbb{P}[A(D_2)=O]\right\} =\{ O \mid \, \mathcal{L}_{D_1,D_2}(O)>\varepsilon \,\, \} Smax​={O∣P[A(D1​)=O]>eε⋅P[A(D2​)=O]}={O∣LD1​,D2​​(O)>ε}
  \,
So we have:
  \,
$$\begin{gathered} \delta =\mathbb{P}[A(D_1)\in S_{max}]-e^{\epsilon }\cdot \mathbb{P}[A(D_2)\in S_{max}] \\ ={\sum }_{O\in S_{max}}\mathbb{P}[A(D_1)=O]-e^{\epsilon }\cdot \mathbb{P}[A(D_2)=O] \\ ={\sum }_{O\in S_{max}}\mathbb{P}[A(D_1)=O]\left(1-\frac{e^{\epsilon }}{e^{{\mathcal{L}}_{D_1,D_2}(O)}}\right) \end{gathered}$$
  \,
$\delta ={\sum }_O\mathbb{P}[A(D_1)=O]\cdot \max \left(0,1-e^{\epsilon -{\mathcal{L}}_{D_1,D_2}(O)}\right)={\mathbb{E}}_{O\sim A(D_1)}\left[\max \left(0,1-e^{\epsilon -{\mathcal{L}}_{D_1,D_2}(O)}\right)\right]$

Composition

$C$ is the algorithm which combines $A$ and $B$ : $C(\mathcal{D})=(A(\mathcal{D}),B(\mathcal{D}))$.
The output of this algorithm will be a pair of outputs: $\mathcal{O}=({\mathcal{O}}_A,{\mathcal{O}}_B)$.
$C$ is (${\epsilon }_A$+${\epsilon }_B$)-differentially private. / $C$ is (${\epsilon }_A$+${\epsilon }_B$ , ${\delta }_A$+${\delta }_B$)-differentially private.

ESA (Encode, Shuffle, Analyze) The best of both worlds (Local vs. Global differential privacy).

Until very recently, there was no middle ground between the two options. The choice was binary: either accept a much larger level of noise (Local differential privacy), or collect raw data (Global differential privacy). This is starting to change, thanks to recent work on a novel type of architecture called ESA.

• The encoder is a fancy name to say “user”. It collects the data, encrypts it twice in two layers, and passes it to the shuffler.
• The shuffler can only decrypt the first layer. It contains the user IDs, and something called “group ID”. This group ID describes what kind of data this is, but not what is the actual value of the data. First, The shuffler removes identifiers, and groups all group IDs together and counts how many users are in each group. Then, it passes them all to the analyzer if there are enough of them.
• The analyzer can then decrypt the second layer of the data, and compute the output.
  

eee

Ref

博文参考链接（Google）
博文参考链接（DP创始者之一）