Note-Towards Logical Specification of Statistical Machine Learning

Content

• Preliminaries

  前导知识，没怎么看懂，不过好像不影响后面

• Techniques for Conditional Indistinguishability

  • Counterfactual Epistemic Operators

    介绍了两个操作符，主要为形式化公平这个属性做准备

  • Conditional Indistinguishability via Counterfactual Knowledge

    如何用上文描述的两个操作符去表示’Conditional Indistinguishability’

• Formal Model for Statistical Classification

  • Statistical Classification Problems

    给出了一些定义:

    • $C:D\to L$, L be a finite set of class labels, D be the finite set of input data (called feature vectors) that we want to classify.
    • $f:D\times L\to R$: be a scoring function that gives a score f(v, ℓ) of predicting the class of an input datum (feature vector) v as a label ℓ.
    • $H(v)=l$: to represent that a label ℓ maximizes f(v, ℓ).

  • Modeling the Behaviours of Classifiers

    给出了两个公式

    $$\begin{array}{ll}s|=\psi (x,y)& \text{ iff }C\left({\sigma }_s(x)\right)={\sigma }_s(y)\\ s|=h(x,y)& \text{ iff }H\left({\sigma }_s(x)\right)={\sigma }_s(y)\end{array}$$

    ψ(x, y) to represent that C classifies a given input x as a class y.

    h(x, y) to represent that y is the actual class of an input x.

• Formalizing the Classification Performance

  • 形式化 correctness

  • 

  • true positive: $s|={\psi }_{\ell }(x)\wedge h_{\ell }(x)$.

  • the precision being within an interval I is given by:

    $\operatorname{Pr}\left[v \stackrel{$}{\leftarrow} \sigma_{w_{\mathrm{real}}}(x) : H(v)=\ell | C(v)=\ell\right] \in I​$ or $\operatorname{Pr}\left[s \stackrel{$}{\leftarrow} w_{\mathrm{real}} : s|=h_{\ell}(x) \ | \ s |= \psi_{\ell}(x)\right] \in I​$.

  • ${\text{Precisione}}_{\ell ,I}(x)\stackrel{\text{ def }}{=}{\psi }_{\ell }(x)\supset {\mathbb{P}}_I{h}_{\ell }(x)$ and $\text{ precision }=\frac{tp}{tp+fp}$.

  • ${\mathrm{Recall}}_{\ell ,I}(x)\stackrel{\text{ def }}{=}{h}_{\ell }(x)\supset {\mathbb{P}}_I{\psi }_{\ell }(x)$ and $\text{recall}=\frac{tp}{tp+fn}$.

  • $\begin{array}{l}{\text{Accuracy}}_{\ell ,I}(x)\stackrel{\text{ def }}{=}{\mathbb{P}}_I(\mathrm{tp}(x)\vee \mathrm{tn}(x))\end{array}$ and $\frac{TP+TN}{TP+TN+FP+FN}$.

• Formalizing the Robustness of Classifiers

  • Probabilistic Robustness against Targeted Attacks
    • 定义：When a robustness attack aims at misclassifying an input as a specific target label, then it is called a targeted attack.
    • ${\mathrm{K}}_{\epsilon }^D\phi$ represents that the classifier C is confident that ϕ is true as far as it classifies the test data that are perturbed by a level ε of noise.
    • D defined by $D\left({\sigma }_w(x)\Vert {\sigma }_{w^{\prime }}(x)\right)={\max }_{v,v^{\prime }}{\Vert v-v^{\prime }\Vert }_p$ where v and v′ range over the datasets supp(σw(x)) and supp(σw′ (x)) respectively.
    • 以下是给出的公式：
    • $h_{\text{panda }}(x)\supset K_{\epsilon }^D{\mathbb{P}}_0{\psi }_{\text{gibon }}(x)$, which represents that a panda’s photo x will not be recognized as a gibbon at all after the photo is perturbed by noise.
    • ${\text{ Target Robust}}_{panda,\delta }(x,\text{ gibbon })\stackrel{\text{ def }}{=}{K}_{\epsilon }^D\left(h_{\text{panda }}(x)\supset {\mathbb{P}}_{[0,\delta ]}{\psi }_{\text{gibbon }}(x)\right)$.
  • Probabilistic Robustness against Non-Targeted Attacks
    • ${\text{ TotalRobust}}_{\ell ,I}(x)\stackrel{\text{ def }}{=}{K}_{\epsilon }^D\left(h_{\ell }(x)\supset {\mathbb{P}}_I{\psi }_{\ell }(x)\right)=K_{\epsilon }^D{\mathrm{Recall}}_{\ell ,I}(x)$.
    • 结论：
    • ${\text{ TotalRobust}}_{panda,I}(x){\text{ implies TargetRobust }}_{\text{panda },\delta }(x,\text{ gibbon })$.
    • robustness can be regarded as recall in the presence of perturbed noise.

• Formalizing the Fairness of Classifiers

  • 符号定义
    • $s|={\eta }_G(x)\text{ iff }{\sigma }_s(x)\in G$.
    • $w|={\xi }_d\text{ iff }{\sigma }_w(x)=d$.
  • Group Fairness (Statistical Parity)
    • 定义：the property that the output distributions of the classifier are identical for different groups.
    • R ε =  def  { ( w , w ′ ) ∈ W × W ∣ D ( σ w ( y ) ∥ σ w ′ ( y ) ) ≤ ε } \mathcal{R}_{\varepsilon} \stackrel{\text { def }}{=}\left\{\left(w, w^{\prime}\right) \in \mathcal{W} \times \mathcal{W} | D\left(\sigma_{w}(y) \| \sigma_{w^{\prime}}(y)\right) \leq \varepsilon\right\} Rε​= def {(w,w′)∈W×W∣D(σw​(y)∥σw′​(y))≤ε}.
    • $\mathfrak{M},w=\overline{{\mathrm{P}}_{\epsilon }}\phi \text{ iff there exists a }{w}^{\prime }\text{ s.t. }\left(w,w^{\prime }\right)\notin {\mathcal{R}}_{\epsilon }\text{ and }\mathfrak{M},w^{\prime }|=\phi$.
    • $\text{ GrpFair }(x,y)\stackrel{\text{ def }}{=}\left({\eta }_{G_0}(x)\wedge \psi (x,y)\right)\supset \neg \overline{P_{\epsilon }^{\mathrm{t}\mathrm{v}}}{\mathbb{P}}_1\left({\xi }_d\wedge {\eta }_{G_1}(x)\wedge \psi (x,y)\right)$.
  • Individual Fairness (as Lipschitz Property)
    • the property that the classifier outputs similar labels given similar inputs.
    • ${\mathcal{R}}_{\epsilon }^{r,D}\stackrel{\text{ def }}{=}\left\{\left(w,w^{\prime }\right)\in \mathcal{W}\times \mathcal{W}|\begin{array}{c}v\in \mathrm{supp}\left({\sigma }_w(x)\right),v^{\prime }\in \mathrm{supp}\left({\sigma }_{w^{\prime }}(x)\right)\\ D\left({\sigma }_w(y)\Vert {\sigma }_{w^{\prime }}(y)\right)\le \epsilon \cdot r\left(v,v^{\prime }\right)\end{array}\right\}$.
    • $\mathrm{lndFair}(x,y)\stackrel{\text{ def }}{=}\psi (x,y)\supset \neg \overline{{\mathrm{P}}_{\epsilon }^{r,D}}{\mathbb{P}}_1\left({\xi }_d\wedge \psi (x,y)\right)$.
  • Equal Opportunity
    • the property that the recall (true positive rate) is the same for all the groups.
    • $\mathrm{E}\mathrm{q}\mathrm{O}\mathrm{p}\mathrm{p}(x)\stackrel{\text{ def }}{=}\left({\eta }_G(x)\wedge \psi (x,y)\right)\supset \neg \overline{{\mathrm{P}}_0^{\mathrm{t}\mathrm{v}}}{\mathbb{P}}_1\left({\xi }_d\wedge \neg {\eta }_G(x)\wedge \psi (x,y)\right)$.

Reference

统计学（statistical machine learning)

数学符号,|=,左边一个竖线,右边一个等号是什么符号

total variation 总变差