# DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learning

Source: [DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learning](<https://www.sota2.com/research/paper/68fa5ac81e209131ce9c2b7b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:56:06.082Z

Authors: Qinglun Li, Li Shen, Guanghao Li, Quanjun Yin, Dacheng Tao

Published: 2023-08-16T11:22:36.000Z

## Abstract

To address the communication burden issues associated with federated learning (FL), decentralized federated learning (DFL) discards the central server and establishes a decentralized communication network, where each client communicates only with neighboring clients. However, existing DFL methods still suffer from two major challenges: local inconsistency and local heterogeneous overfitting, which have not been fundamentally addressed by existing DFL methods. To tackle these issues, we propose novel DFL algorithms, DFedADMM and its enhanced version DFedADMM-SAM, to enhance the performance of DFL. The DFedADMM algorithm employs primal-dual optimization (ADMM) by utilizing dual variables to control the model inconsistency raised from the decentralized heterogeneous data distributions. The DFedADMM-SAM algorithm further improves on DFedADMM by employing a Sharpness-Aware Minimization (SAM) optimizer, which uses gradient perturbations to generate locally flat models and searches for models with uniformly low loss values to mitigate local heterogeneous overfitting. Theoretically, we derive convergence rates of $\\small \\mathcal{O}\\Big(\\frac{1}{\\sqrt{KT}}+\\frac{1}{KT(1-\\psi)^2}\\Big)$ and $\\small \\mathcal{O}\\Big(\\frac{1}{\\sqrt{KT}}+\\frac{1}{KT(1-\\psi)^2}+ \\frac{1}{T^{3/2}K^{1/2}}\\Big)$ in the non-convex setting for DFedADMM and DFedADMM-SAM, respectively, where $1 - \\psi$ represents the spectral gap of the gossip matrix. Empirically, extensive experiments on MNIST, CIFAR10 and CIFAR100 datesets demonstrate that our algorithms exhibit superior performance in terms of both generalization and convergence speed compared to existing state-of-the-art (SOTA) optimizers in DFL.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on CIFAR-10 Dirichlet](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-dirichlet.md>): Image Classification; CIFAR-10 Dirichlet; Final Accuracy: 64.8.

- [Image Classification on F-MNIST Dirichlet](<https://www.sota2.com/research/sota/image-classification-on-f-mnist-dirichlet.md>): Image Classification; F-MNIST Dirichlet; Final Accuracy (%): 81.5.

- [Image Classification on F-MNIST 500x2](<https://www.sota2.com/research/sota/image-classification-on-f-mnist-500x2.md>): Image Classification; F-MNIST 500x2; Final Accuracy: 40.3.

- [Image Classification on F-MNIST 1000x1](<https://www.sota2.com/research/sota/image-classification-on-f-mnist-1000x1.md>): Image Classification; F-MNIST 1000x1; Accuracy: 30.2.

- [Image Classification on CIFAR-10 2500x2](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-2500x2.md>): Image Classification; CIFAR-10 2500x2; Final Accuracy: 26.3.

- [Image Classification on CIFAR-10 (5000x1)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-5000x1.md>): Image Classification; CIFAR-10 (5000x1); Final Accuracy: 11.6.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2308.08290>)

[PDF](<https://arxiv.org/pdf/2308.08290.pdf>)