【论文笔记13】Differentially Private Optimal Transport: Application to Domain Adaptation, IJCAI 2019

最新推荐文章于 2023-05-21 16:48:37 发布
最新推荐文章于 2023-05-21 16:48:37 发布
阅读量485 收藏 1
点赞数
分类专栏： 论文笔记 文章标签： 人工智能 机器学习 算法 迁移学习
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_45632492/article/details/115568841
版权
本文介绍了将差分隐私应用于最优传输（DPOT）来保护源数据隐私，进而解决域适应（DPDA）问题。研究在VisDA、Office-Home和Office-Caltech数据集上的实验，与非私有最优传输和半监督方法进行对比。
摘要由CSDN通过智能技术生成

目录导引

系列传送

我的论文笔记频道

【Active Learning】
【论文笔记01】Learning Loss for Active Learning, CVPR 2019
【论文笔记02】Active Learning For Convolutional Neural Networks: A Core-Set Approch, ICLR 2018
【论文笔记03】Variational Adversarial Active Learning, ICCV 2019
【论文笔记04】Ranked Batch-Mode Active Learning，ICCV 2016

【Transfer Learning】
【论文笔记05】Active Transfer Learning, IEEE T CIRC SYST VID 2020
【论文笔记06】Domain-Adversarial Training of Neural Networks, JMLR 2016
【论文笔记10】A unified framework of active transfer learning for cross-system recommendation, AI 2017
【论文笔记14】Transfer Learning via Minimizing the Performance Gap Between Domains, NIPS 2019

【Differential Privacy】
【论文笔记07】A Survey on Differentially Private Machine Learning, IEEE CIM 2020
【论文笔记09】Differentially Private Hypothesis Transfer Learning, ECML&PKDD 2018
【论文笔记11】 Deep Domain Adaptation With Differential Privay, IEEE TIFS 2020
【论文笔记12】Differential privacy based on importance weighting, Mach Learn 2013
【论文笔记13】Differentially Private Optimal Transport: Application to Domain Adaptation, IJCAI 2019

【Model inversion attack】
【论文笔记08】Model inversion attacks that exploit confidence information and basic countermeasures, SIGSAC 2015

最低0.47元/天 解锁文章
Brandon Bryant
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
Optimal Transport for Domain Adaptation
12-20
Domain adaptation is one of the most challenging tasks of modern data analytics. If the adaptation is done correctly, models built on a specific data representation become more robust when confronted to data depicting the same classes, but described by another observation system. Among the many strategies proposed, finding domain-invariant representations has shown excellent properties, in particular since it allows to train a unique classifier effective in all domains. In this paper, we propose a regularized unsupervised optimal transportation model to perform the alignment of the representations in the source and target domains. We learn a transportation plan matching both PDFs, which constrains labeled samples of the same class in the source domain to remain close during transport. This way, we exploit at the same time the labeled samples in the source and the distributions observed in both domains. Experiments on toy and challenging real visual adaptation examples show the interest of the method, that consistently outperforms state of the art approaches. In addition, numerical experiments show that our approach leads to better performances on domain invariant deep learning features and can be easily adapted to the semi-supervised case where few labeled samples are available in the target domain.
Differentially Private Deep Learning with Iterative Gradient Descent Optimization
daxuan1881的博客
04-27 493
motivation：梯度下降中不均衡的敏感度；不够紧的隐私界限。 methods：新的扰动梯度下降优化算法PIGDO，并证明该算法满足差分隐私；提出一种改进的MMA方法来进行隐私分析，与原始moment account方法相比，获得了更紧密的隐私损失约束。 1.本文提出了一种新的扰动迭代梯度下降优化(PIGDO)算法，该算法将梯度下降算法作为一个迭代分量，然后对迭代GDO计算的梯度进行梯度扰动，满足差分隐私。与之前针对每个梯度设计的噪声加法机制相比，我们的算法改进 从整体的角度验证了梯度的准确性，从而
论文解读《Differentially Private Label Protection in Split Learning》
wuxing610的博客
03-21 626
论文解读《Differentially Private Label Protection in Split Learning》 Threat Model 本文考虑的是Split Learning中的白盒攻击white box attack，黑盒攻击可以参考之前的解读。 白盒攻击：攻击者non-label party不仅可以得到label party传回来的梯度ggg，还可以得到label party模型的参数hhh。 基于上述假设，作者提出了一种最短距离攻击shortest distance attack
Differentially Private Federated Learning: A Client Level Perspective
sleepinghm的博客
10-17 3953
介绍 我们的目标并不是只保护一个数据点。相反，我们希望确保一个学习模型不会显示客户是否参与了分散的培训。这意味着客户机的整个数据集受到保护，不受来自其他客户机的差异攻击。 我们的主要贡献:首先，我们展示了在联合学习中保持较高的模型性能时，客户机的参与是可以隐藏的。我们证明了我们提出的算法可以在模型性能损失很小的情况下实现客户级差异隐私。与此同时发表的一项独立研究[6]提出了一个类似的客户级dp程序...
matlab矩阵随机投影,稀疏随机投影（aka Johnson Lindenstrauss变换）不保留2点之间的距离...
weixin_28746457的博客
03-21 813
我试图使用随机投影的方法(基本上，减少维度，同时保持2点之间的欧几里德距离)，最近我在网上发现了一些代码(MEX为MATLAB文件)：稀疏随机投影(aka Johnson Lindenstrauss变换)不保留2点之间的距离/** sjlt.c - Sparse Johnson-Lindenstrauss Transform** Creates a random sparse Johnson-Li...
Secure Spatial Query with Differentially Private Access Pattern
02-08
Although this mechanism will not reveal the original spatial data nor the query content to the service provider, the service provider can learn the relationship between the query tokens and the ...
Differentially private model publishing in cyber physical systems
02-07
该研究论文主要探讨了在赛博物理系统(Cyber Physical Systems, CPS)中如何利用差分隐私技术来保护数据发布过程中的隐私问题。随着CPS的快速发展及其在物联网、智能交通、健康监护等领域的广泛应用，隐私保护成为了...
Johnson-Lindenstrauss Theorem - EtoneWiki_files
10-07
Johnson-Lindenstrauss Theorem - EtoneWiki_files
Differentially-Private-SGD
05-02
学士学位论文代码：具有差异性私人更新的随机梯度下降作者：Rayel Hardwarsing数据集BankNoteData.mat-钞票认证数据集KDDCUPX.mat / KDDCUPY.mat-KDDCup数据集mnist.mat-MNIST数据集careerData.mat-占用检测数据集...
【最优传输论文笔记一】Optimal Transport for Domain Adaptation
m0_60231311的博客
03-24 2197
在本文中，描述了一种基于最优传输的新框架来解决无监督领域自适应问题。我们提出了两种正则化方案来在运输计划的估计过程中对源域中的类结构进行编码，从而强化了同一类的样本必须经历类似变换。并将这种OT正则化框架扩展到半监督域自适应情况，即目标域中几乎没有标签可用的情况。关于计算方面，建议使用条件梯度算法的修改版本，即广义条件梯度分裂，这使该方法能够扩展到真实世界的数据集。最后，我们将所提出的方法应用于合成数据集和现实世界数据集。结果表明，最优运输领域自适应方案往往优于竞争的最先进的方法。
Differentially Private Spatial Decompositions
天下月色共三分的博客
02-12 128
论文
Learning Rate Adaptation for Differentially Private Learning
daxuan1881的博客
09-20 332
中间定理7是MA的组合定理，是说隐私损失的总矩是小于等于各个机制之和的矩，因为算法G1和G2都是独立的，所以有了定理9，根据定理9，相同的参数值q，σ和C，算法2运行的迭代次数是DP-SGD的一半。并且通过MA说明了该算法的优点。最右边是ADADP算法，可以看到他和DPSGD不同的是，DPSGD是把数据划分为固定大小的批次，ADADP算法是在每一轮需要迭代两个小批次，用于分别计算G1和G2，因此需要迭代N/2B次，最后通过判断条件去改变模型的学习率。
【最优传输论文笔记二】（2017 NIPS）Joint distribution optimal transportation for domain adaptation
m0_60231311的博客
04-11 351
我们在本文中提出了用于域自适应的联合分布最优传输。JDOT假设存在将源域联合分布转换为目标域等效版本的转移映射。通过这种变换，可以操作特征空间和条件分布的对齐，从而能够设计出一种有效的算法，该算法同时优化和之间的耦合，以及解决传递问题的预测函数。还展示了我们方法的多功能性，可以适应几种类型的损失函数（mse、铰链）或一类假设（包括核机器或神经网络）。
【论文笔记11】 Deep Domain Adaptation With Differential Privay 差分隐私下的深度域泛化, IEEE TIFS 2020
weixin_45632492的博客
04-10 760
目录导引系列传送AbstractReference 系列传送 【Active Learning】 【论文笔记01】Learning Loss for Active Learning, CVPR 2019 【论文笔记02】Active Learning For Convolutional Neural Networks: A Core-Set Approch, ICLR 2018 【论文笔记03】Variational Adversarial Active Learning, ICCV 2019 【论文笔记0
最优传输论文（三）：Joint distribution optimal transportation for domain adaptation论文原理
lyy的博客
05-12 951
目录前言原理阐述文章介绍背景Joint distribution Optimal TransportOptimal transport in domain adaptationJoint distribution optimal transport loss其他 前言 本文属于我最优传输系列里的二篇，该专栏用于记录本人研究生阶段相关最优传输论文的原理阐述以及复现工作。 本专栏的文章主要内容为解释原理，论文具体的翻译及复现代码在文章的github中。 原理阐述 文章介绍 背景 当下域适应问题的解决方法之
【最优传输论文八】Optimal Transport for Multi-source Domain Adaptation under Target Shift
m0_60231311的博客
05-21 720
本文提出了一种基于最优传输(OT)的目标偏移校正算法。最初，OT解决了一个问题，即以一种最小化在它们之间移动单位质量的成本的方式对齐两个概率度量，同时保留原始的边际。最近出现的有效的OT公式允许其在数据处理中的应用，因为OT允许明确地学习将给定源pdf转换为目标样本的pdf。提出新的目标移位特定算法的动机源于这样一个事实，即许多流行的用于处理协变量移位的数据挖掘算法不能同样好地处理目标移位。
Complete the code for Differentially Private Stochastic Gradient Descent. a. Fill in the code for per-example clipping and adding Gaussian noise. b. Implement the privacy budget composition. Calculate the privacy budget of the training process, which means calculating $\epsilon$ based on the variance of Gaussian noise $\sigma^2$ and the given $\delta = 10^{-5}$ in different epochs. You can use basic composition to complete the code. If you correctly apply the Moments Accountant method, you will receive bonus points. import numpy as np from scipy import optimize from scipy.stats import norm import math """ Optionally you could use moments accountant to implement the epsilon calculation. """ def get_epsilon(epoch, delta, sigma, sensitivity, batch_size, training_nums): """ Compute epsilon with basic composition from given epoch, delta, sigma, sensitivity, batch_size and the number of training set. """ return epsilon
最新发布
06-12
Here is the completed code for Differentially Private Stochastic Gradient Descent, including per-example clipping and adding Gaussian noise as well as privacy budget composition: ```python import numpy as np from scipy import optimize from scipy.stats import norm import math def per_example_clipping(grad, clip_factor): """ Clip the gradient per example with a given clip factor. """ return np.clip(grad, -clip_factor, clip_factor) def add_gaussian_noise(grad, sigma): """ Add Gaussian noise to the gradient with a given standard deviation. """ return grad + np.random.normal(0, sigma, grad.shape) def get_epsilon(epoch, delta, sigma, sensitivity, batch_size, training_nums): """ Compute epsilon with basic composition from given epoch, delta, sigma, sensitivity, batch_size and the number of training set. """ steps = math.ceil(training_nums / batch_size) * epoch epsilon = sigma * math.sqrt(2 * math.log(1.25 / delta)) / sensitivity return epsilon * steps def dp_sgd(X, y, epochs, batch_size, clip_factor, sigma, delta): n, d = X.shape w = np.zeros(d) for epoch in range(epochs): for i in range(0, n, batch_size): X_batch = X[i:i+batch_size] y_batch = y[i:i+batch_size] grad = np.mean(X_batch * (sigmoid(X_batch.dot(w)) - y_batch).reshape(-1, 1), axis=0) clipped_grad = per_example_clipping(grad, clip_factor) noise_grad = add_gaussian_noise(clipped_grad, sigma) w -= noise_grad epsilon = get_epsilon(epoch+1, delta, sigma, clip_factor/batch_size, batch_size, n) print("Epoch {}: Epsilon = {}".format(epoch+1, epsilon)) return w ``` The `per_example_clipping` function clips the gradient per example with a given clip factor. The `add_gaussian_noise` function adds Gaussian noise to the gradient with a given standard deviation. The `get_epsilon` function computes epsilon with basic composition from given epoch, delta, sigma, sensitivity, batch_size and the number of training set. The `dp_sgd` function performs Differentially Private Stochastic Gradient Descent. For each epoch, it loops over the training set in batches and computes the gradient of the loss function using the sigmoid function. It then clips the gradient per example, adds Gaussian noise to the clipped gradient, and updates the weight vector. Finally, it computes the privacy budget using the `get_epsilon` function and prints it out. Note that the `get_epsilon` function uses basic composition to compute the privacy budget. It calculates the total number of steps based on the number of epochs and the batch size, and then uses the formula for epsilon with basic composition to compute the privacy budget for each epoch. It is worth noting that basic composition may not provide the tightest bound on privacy, and using the Moments Accountant method may provide a tighter bound.
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值