A New Federated Learning Framework Against Gradient Inversion Attacks
Pengxin Guo, Shuang Zeng, Wenhao Chen, Xiaodan Zhang, Weihong Ren, Yuyin Zhou, Liangqiong Qu
2024-12-11
Summary
This paper talks about a new federated learning framework designed to protect user privacy against gradient inversion attacks, which can expose sensitive data during the training of machine learning models.
What's the problem?
Federated Learning (FL) allows multiple users to train machine learning models without sharing their actual data, which helps keep their information private. However, recent studies show that attackers can exploit the information shared during this process through gradient inversion attacks. These attacks can potentially reconstruct private data from the gradients (updates) shared by the models, raising serious privacy concerns.
What's the solution?
The authors propose a new method called Hypernetwork Federated Learning (HyperFL), which changes how model parameters are shared. Instead of directly sharing the gradients that contain sensitive information, HyperFL uses hypernetworks to generate model parameters based on the user's data. Only these generated parameters are sent to the central server for aggregation, effectively breaking the direct link between shared data and private user information. This method requires less human feedback and maintains strong privacy protections while still allowing effective model training.
Why it matters?
This research is important because it addresses a critical vulnerability in federated learning systems. By developing a framework that enhances privacy without sacrificing performance, HyperFL helps ensure that users can benefit from machine learning technologies without risking their personal data. This advancement is crucial for building trust in AI systems and promoting wider adoption of federated learning in sensitive applications like healthcare and finance.
Abstract
Federated Learning (FL) aims to protect data privacy by enabling clients to collectively train machine learning models without sharing their raw data. However, recent studies demonstrate that information exchanged during FL is subject to Gradient Inversion Attacks (GIA) and, consequently, a variety of privacy-preserving methods have been integrated into FL to thwart such attacks, such as Secure Multi-party Computing (SMC), Homomorphic Encryption (HE), and Differential Privacy (DP). Despite their ability to protect data privacy, these approaches inherently involve substantial privacy-utility trade-offs. By revisiting the key to privacy exposure in FL under GIA, which lies in the frequent sharing of model gradients that contain private data, we take a new perspective by designing a novel privacy preserve FL framework that effectively ``breaks the direct connection'' between the shared parameters and the local private data to defend against GIA. Specifically, we propose a Hypernetwork Federated Learning (HyperFL) framework that utilizes hypernetworks to generate the parameters of the local model and only the hypernetwork parameters are uploaded to the server for aggregation. Theoretical analyses demonstrate the convergence rate of the proposed HyperFL, while extensive experimental results show the privacy-preserving capability and comparable performance of HyperFL. Code is available at https://github.com/Pengxin-Guo/HyperFL.