# Taming the Instability: A Robust Second-Order Optimizer for Federated Learning over Non-IID Data Source: [Taming the Instability: A Robust Second-Order Optimizer for Federated Learning over Non-IID Data]() Collection index: [llms.txt]() Data updated: 2026-04-01T05:00:12.613Z Authors: Yuanqiao Zhang, Tiantian He, Yuan Gao, Yixin Wang, Yew-Soon Ong, Maoguo Gong, A.K. Qin, Hui Li Published: 2026-03-30T11:37:46.000Z ## Abstract In this paper, we present Federated Robust Curvature Optimization (FedRCO), a novel second-order optimization framework designed to improve convergence speed and reduce communication cost in Federated Learning systems under statistical heterogeneity. Existing second-order optimization methods are often computationally expensive and numerically unstable in distributed settings. In contrast, FedRCO addresses these challenges by integrating an efficient approximate curvature optimizer with a provable stability mechanism. Specifically, FedRCO incorporates three key components: (1) a Gradient Anomaly Monitor that detects and mitigates exploding gradients in real-time, (2) a Fail-Safe Resilience protocol that resets optimization states upon numerical instability, and (3) a Curvature-Preserving Adaptive Aggregation strategy that safely integrates global knowledge without erasing the local curvature geometry. Theoretical analysis shows that FedRCO can effectively mitigate instability and prevent unbounded updates while preserving optimization efficiency. Extensive experiments show that FedRCO achieves superior robustness against diverse non-IID scenarios while achieving higher accuracy and faster convergence than both state-of-the-art first-order and second-order methods. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on CIFAR-10 (Robustness under Dirichlet and Pathological Corruption)](): Image Classification; CIFAR-10; Acc (Dirichlet, alpha=0.1): 78.8. - [Image Classification on EMNIST (Robustness Metrics)](): Image Classification; EMNIST; Accuracy (Dirichlet, alpha=0.1): 90.2. ## Original sources and implementation [Original paper]() [PDF]()