【论文笔记09】Differentially Private Hypothesis Transfer Learning 差分隐私迁移学习模型, ECML&PKDD 2018

系列传送

我的论文笔记频道

【Active Learning】
【论文笔记01】Learning Loss for Active Learning, CVPR 2019
【论文笔记02】Active Learning For Convolutional Neural Networks: A Core-Set Approch, ICLR 2018
【论文笔记03】Variational Adversarial Active Learning, ICCV 2019
【论文笔记04】Ranked Batch-Mode Active Learning，ICCV 2016

【Transfer Learning】
【论文笔记05】Active Transfer Learning, IEEE T CIRC SYST VID 2020
【论文笔记06】Domain-Adversarial Training of Neural Networks, JMLR 2016
【论文笔记10】A unified framework of active transfer learning for cross-system recommendation, AI 2017
【论文笔记14】Transfer Learning via Minimizing the Performance Gap Between Domains, NIPS 2019

【Differential Privacy】
【论文笔记07】A Survey on Differentially Private Machine Learning, IEEE CIM 2020
【论文笔记09】Differentially Private Hypothesis Transfer Learning, ECML&PKDD 2018
【论文笔记11】Deep Domain Adaptation With Differential Privay, IEEE TIFS 2020
【论文笔记12】Differential privacy based on importance weighting, Mach Learn 2013
【论文笔记13】Differentially Private Optimal Transport: Application to Domain Adaptation, IJCAI 2019

【Model inversion attack】
【论文笔记08】Model inversion attacks that exploit confidence information and basic countermeasures, SIGSAC 2015

Differentially Private Hypothesis Transfer Learning

原文传送

1 Abstract

2 Bg & Rw

更多关于差分隐私的背景可以看
差分隐私机器学习综述
模型反转攻击

Differential privacy essentially implies that the existence of any particular data point $s$ in a private data set $S$ cannot be determined by analyzing the output of a differentially private algorithm $\mathcal{M}$ applied on $S$.

Popular approaches to achieving differential privacy

• output perturbation, adding carefully calibrated noises to the parameters of the elarned hypothesis before releasing it.
• distributed privacy-preserving ML, where private data sets are collected by multiple parties.
• • One line of research involves exchanging differentially private information (e.g. gradients) among multiple parties during the iterative hypothesis training process.
• • An alternative line of work focuses on privacy-preserving model aggregation techniques.

The most related works to this paper are focusing on multi-task learning.

One of the key drawbacks of iterative differentially private methods is that privacy risks accumulate with each iteraion. So, according to the composition theorem of differential privacy, there is a limit on how many iterations can be performed on a specific private dataset under a certain level of total privacy budget and this severely affects the utility-privacy trade-off of iterative solutions. 换句话说就是utility会因为privacy的牵制下降。

3 Setting & The proposed methods

I think hypothesis means the same as model/mapping/function.

My understanding of the setting and algorithm:

1. This is a multi-source transfer learning.
2. In each source domain ${\mathcal{D}}^k$, there are i.i.d. labeled samples S l k = { ( x i k , y i k ) : 1 ⩽ i ⩽ n l k } S_l^k=\{(x_i^k, y_i^k): 1\leqslant i \leqslant n_l^k\} Slk​={ (xik​,yik​