# Who to Trust? Aggregating Client Predictions in Federated Distillation

Source: [Who to Trust? Aggregating Client Predictions in Federated Distillation](<https://www.sota2.com/research/paper/68fa7b631e209131cea881b0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-27T05:00:04.193Z

Authors: Viktor Kovalchuk, Denis Son, Arman Bolatov, Mohsen Guizani, Samuel Horv\\'ath, Maxim Panov, Martin Tak\\'a\\v{c}, Eduard Gorbunov, Nikita Kotelevskii

Published: 2025-09-18T16:54:23.000Z

## Abstract

Under data heterogeneity (e.g., $\\textit{class mismatch}$), clients may produce unreliable predictions for instances belonging to unfamiliar classes. An equally weighted combination of such predictions can corrupt the teacher signal used for distillation. In this paper, we provide a theoretical analysis of Federated Distillation and show that aggregating client predictions on a shared public dataset converges to a neighborhood of the optimum, where the neighborhood size is governed by the aggregation quality. We further propose two uncertainty-aware aggregation methods, $\\mathbf{UWA}$ and $\\mathbf{sUWA}$, which leverage density-based uncertainty estimates to down-weight unreliable client predictions. Experiments on image and text classification benchmarks demonstrate that our methods are particularly effective under high data heterogeneity, while matching standard averaging when heterogeneity is low.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on CIFAR-10 (test) (k-NN Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-test-k-nn-accuracy.md>): Image Classification; CIFAR-10 (test); Test Accuracy (k=2): 39.51.

- [Image Classification on CIFAR-100 (test) (k-Accuracy Profile)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-test-k-accuracy-profile.md>): Image Classification; CIFAR-100 (test); Accuracy (k=80): not recorded.

- [Topic Classification on Yahoo Answers (test)](<https://www.sota2.com/research/sota/topic-classification-on-yahoo-answers-test.md>): Topic Classification; Yahoo! Answers (test); F1 Score (Test): not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2509.15147>)

[PDF](<https://arxiv.org/pdf/2509.15147.pdf>)