# Federated Learning with Nonvacuous Generalisation Bounds

Source: [Federated Learning with Nonvacuous Generalisation Bounds](<https://www.sota2.com/research/paper/68fa70ee1e209131cea4e1ce>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-21T05:00:05.413Z

Authors: Pierre Jobic, Maxime Haddouche, Benjamin Guedj

Published: 2023-10-17T12:29:29.000Z

## Abstract

We introduce a novel strategy to train randomised predictors in federated learning, where each node of the network aims at preserving its privacy by releasing a local predictor but keeping secret its training dataset with respect to the other nodes.   We then build a global randomised predictor which inherits the properties of the local private predictors in the sense of a PAC-Bayesian generalisation bound. We consider the synchronous case where all nodes share the same training objective (derived from a generalisation bound), and the heterogenous and homogenous cases where each node may have its own personalised training objective. We show through a series of numerical experiments that our approach achieves a comparable predictive performance to that of the batch approach where all datasets are shared across nodes. Moreover the predictors are supported by numerically nonvacuous generalisation bounds while preserving privacy for each node. We explicitly compute the increment on predictive performance and generalisation bounds for our two federated settings, highlighting the price to pay to preserve privacy.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Classification on MNIST (test) (Bound Loss, Test Error, KL Divergence)](<https://www.sota2.com/research/sota/classification-on-mnist-test-bound-loss-test-error-kl-divergence.md>): Classification; MNIST (test); Bound Loss (0-1): 0.039.

- [Image Classification on CIFAR-10 (test) (Error Bounds and KL Divergence)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-test-error-bounds-and-kl-divergence.md>): Image Classification; CIFAR-10 (test); Error Bound: 0.345.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2310.11203>)

[PDF](<https://arxiv.org/pdf/2310.11203.pdf>)