论文阅读 Local Differential Privacy-Based Federated Learning for Internet of Things

标题：Local Differential Privacy-Based Federated Learning for Internet of Things

期刊： IEEE Internet of Things Journal, Vol. 8, No. 11, June 1, 2021

---

1. 背景/已有方法存在问题

• 应用环境：Internet of Vehicles (IoV) 车辆网络
• 机器学习模型数据： real-time traffic information
• 隐私信息：users’ location information, traffic information, motor vehicle information, environmental information, etc.
• 车辆与云之间的频繁通信 （frequent communication）带来的通信成本问题

2. 论文解决方法

目的：保护隐私+降低通信成本

解决方案：联邦学习FL+本地差分隐私LDP

• 4个LDP机制来扰动梯度
• Three-Outputs mechanism ——》隐私预算$ϵ$小时，high accuracy——》编码成2位，降低通信成本
• optimal piecewise mechanism (PM-OPT)——》隐私预算$ϵ$大时，maximize the performance
• suboptimal mechanism (PM-SUB)——》PM-OPT
• hybrid mechanism by combining Three-Outputs and PM-SUB
• LDP-FedSGD algorithm——》train the model

3. 贡献/创新点

• LDP-FedSGD——》novel LDP机制——》Three-Outputs–3个输出（$ϵ$小时） 和PM-SUB–无限可能输出（$ϵ$大时）表现优异
  • PM-OPT——》次优PM-SUB
  • Three-Outputs + PM-SUB——》混合机制HM-TP
• 将 PM-SUB 和PM-OPT机制 连续的输出范围 离散化——》实用性+减小通信成本
• 实验——》real-world data sets and synthetic data sets（真实数据集+合成数据集）——》估计数据的平均频率和经验风险最小化——》较高准确性

4. 模型/方法

5. LDP-FedSGD算法

6. 与其他隐私保护FL范式比较

（1） Distributed/Centralized Perturbation

（2） User-Level/Record-Level Privacy Protection

4种类别
1）More Actions1) user-level privacy protection with distributed perturbation (ULDP)
2) record-level privacy protection with distributed perturbation (RLDP)
3) record-level privacy protection with centralized perturbation (RLCP)
4) user-level privacy protection with centralized perturbation (ULCP)

表三 4种类别比较

privacy granularity and place of perturbation	privacy property	adversary model
$\epsilon$-LDP(defined for distributed perturbation	$\epsilon$-ULDP	defend against a honest-but-curious aggregator & external attacks after model publishing
$\epsilon$-DP with distributed perturbation	$\epsilon$-RLDP
$\epsilon$-DP with centralized perturbation	$\epsilon$-RLCP	trusted aggregator; defend against external attacks after model publishing
user-level privacy with centralized perturbation	$\epsilon$-ULCP

7. 论文PROBLEM FORMATION

8. MECHANISMS FOR ESTIMATION OF SINGLE NUMERIC ATTRIBUTE

提出4种LDP机制： Three-Outputs, PM-OPT, PM-SUB, and HM-TP.

A. Three-Outputs Mechanism

B. PM-OPT Mechanism

C. PM-SUB Mechanism

D. Discretization Postprocessing

E. HM-TP Mechanism

9. MECHANISMS FOR ESTIMATION OF MULTIPLE NUMERIC ATTRIBUTES

多属性收集方法

• straightforward approach，隐私预算为 $ϵ/d$

• Duchi et al.'s solution [22]

• Wang et al.'s solution[8]

部分参考文献

[1] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. Y. Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. 20th Int. Conf. Artif. Intell. Stat., 2017, pp. 1273–1282. FedAVG

[5] J. Chen, X. Pan, R. Monga, S. Bengio, and R. Jozefowicz, “Revisiting distributed synchronous SGD,” 2016. [Online]. Available: arXiv:1604.00981. FedSGD

[6] J. Duchi, M. J. Wainwright, and M. I. Jordan, “Local privacy and minimax bounds: Sharp rates for probability estimation,” in Advances in Neural Information Processing Systems. Red Hook, NY, USA: Curran,
2013, pp. 1529–1537.

[7] C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Proc. Theory Cryptogr. Conf. (TCC), 2006, pp. 265–284. Laplace

[8] N. Wang et al., “Collecting and analyzing multidimensional data with local differential privacy,” in Proc. IEEE Int. Conf. Data Eng. (ICDE), 2019, pp. 638–649.

[22] J. C. Duchi, M. I. Jordan, and M. J. Wainwright, “Minimax optimal procedures for locally private estimation,” J. Amer. Stat. Assoc., vol. 113, no. 521, pp. 182–201, 2018.