Indirect Information Leakage

Federated Learning is a machine learning approach where a model is trained across multiple decentralized devices (or servers) holding local data samples without exchanging them. Each device computes a local model update, and only this update is sent to a centralized server, where it is aggregated to construct an improved global model. Federated learning aims to achieve good model performance while preserving data privacy. However, federated learning is not immune to privacy risks, including "Indirect Information Leakage."

What is Indirect Information Leakage?

Indirect Information Leakage occurs when the updates sent from the local devices to the central server inadvertently contain information that could be exploited to infer something about the local dataset. Even though raw data is not shared, the statistical information embedded in the model updates can sometimes be reverse-engineered to gain insights into the data. For example, if a local dataset has a unique, rare feature, the local model updates may reflect that peculiarity. Sophisticated attackers can exploit these peculiarities to make educated guesses about the data in a specific local dataset.

Mechanisms of Indirect Information Leakage

1. Model Inversion. An attacker might generate inputs that produce unique outputs by analyzing the model updates from specific users, which could then be used to infer private information about those users.
2. Membership Inference. This attack aims to determine if a particular data point was part of the training dataset by observing the behavior of the updated model.
3. Differential Privacy Violations. Differential privacy aims to maximize the accuracy of queries from statistical databases while minimizing the chances of identifying its entries. Indirect information leakage might occur if differential privacy is not implemented correctly or the "privacy budget" is exhausted.