# Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learning

Source: [Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learning](<https://www.sota2.com/research/paper/6940e7cb456146936284faad>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-17T05:00:05.807Z

Authors: Li Xia

Published: 2025-12-14T15:21:31.000Z

## Abstract

Decentralized federated learning (DFL) enables collaborative model training without a central server, but converges slowly under statistical heterogeneity. Recent work has shown that neural tangent kernel (NTK) methods achieve faster convergence than gradient-based updates in DFL, while momentum has proven effective for accelerating gradient-based FL. However, applying momentum to NTK updates can destabilize training under heterogeneous data. We propose SPARK, which addresses this instability with a stage-wise annealed soft-label regularizer evaluated on neighborhood-aggregated data, so that momentum can accelerate NTK updates stably. Under high heterogeneity, SPARK converges about 3$\\times$ faster than baselines and lowers the total communication to a target accuracy by up to about 70\\%, and it attains higher accuracy across heterogeneity levels. We further study random projection as an optional Jacobian-compression strategy for bandwidth-constrained settings. We validate the approach across multiple datasets, network topologies, and heterogeneity levels.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on Fashion-MNIST IID](<https://www.sota2.com/research/sota/image-classification-on-fashion-mnist-iid.md>): Image Classification; Fashion-MNIST IID; Communication Rounds: 6.

- [Image Classification on Fashion-MNIST alpha=0.5](<https://www.sota2.com/research/sota/image-classification-on-fashion-mnist-alpha-0-5.md>): Image Classification; Fashion-MNIST alpha = 0.5; Communication Rounds: 6.

- [Image Classification on Fashion-MNIST alpha=0.1](<https://www.sota2.com/research/sota/image-classification-on-fashion-mnist-alpha-0-1.md>): Image Classification; Fashion-MNIST alpha = 0.1; Communication Rounds: 6.

- [Image Classification on FashionMNIST (test)](<https://www.sota2.com/research/sota/image-classification-on-fashionmnist-test.md>): Image Classification; FashionMNIST (test); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2512.12737>)

[PDF](<https://arxiv.org/pdf/2512.12737.pdf>)