Vertical Federated Learning
Vertical Federated Learning, or feature-based federated learning, is a type of Federated Learning where multiple parties collaborate to train a shared machine learning model without directly exchanging their raw data. Unlike Horizontal Federated Learning, where different parties have different samples (or data points) but share the same feature set, in Vertical Federated Learning, various parties possess different subsets of features for the same collection of samples.
Example
Consider two companies, a bank, and a retail store, that want to collaborate on a machine-learning model to predict customer spending behavior. The bank has financial features like income, credit score, and loan history, while the retail store has behavioral features like purchase history, product preferences, and online engagement metrics. Both companies have data on the same set of customers (i.e., same sample set) but have collected different kinds of information (i.e., different feature sets).
How it Works
- Initialization. A global model is initialized, often on a centralized server, or one of the parties acts as a coordinator.
- Local Computation. Each party computes local model updates using its feature set and the shared model parameters. Since the parties have different features but share the same samples, the local computations can be aligned to the same group of individuals.
- Secure Aggregation. The local updates from each party are aggregated securely, often employing advanced cryptographic techniques like Secure Multi-Party Computation (SMPC) or Homomorphic Encryption to generate an updated global model.