# High Dimensional Distributed Gradient Descent with Arbitrary Number of Byzantine Attackers

Source: [High Dimensional Distributed Gradient Descent with Arbitrary Number of Byzantine Attackers](<https://www.sota2.com/research/paper/68fa72b41e209131cea580c8>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-11-20T05:01:37.411Z

Authors: Wenyu Liu, Tianqiang Huang, Pengfei Zhang, Zong Ke, Minghui Min, Puning Zhao

Published: 2023-07-25T09:14:45.000Z

## Abstract

Adversarial attacks pose a major challenge to distributed learning systems, prompting the development of numerous robust learning methods. However, most existing approaches suffer from the curse of dimensionality, i.e. the error increases with the number of model parameters. In this paper, we make a progress towards high dimensional problems, under arbitrary number of Byzantine attackers. The cornerstone of our design is a direct high dimensional semi-verified mean estimation method. The idea is to identify a subspace with large variance. The components of the mean value perpendicular to this subspace are estimated using corrupted gradient vectors uploaded from worker machines, while the components within this subspace are estimated using auxiliary dataset. As a result, a combination of large corrupted dataset and small clean dataset yields significantly better performance than using them separately. We then apply this method as the aggregator for distributed learning problems. The theoretical analysis shows that compared with existing solutions, our method gets rid of $\\sqrt{d}$ dependence on the dimensionality, and achieves minimax optimal statistical rates. Numerical results validate our theory as well as the effectiveness of the proposed method.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Object Detection on Locomotive coil spring defect dataset (test)](<https://www.sota2.com/research/sota/object-detection-on-locomotive-coil-spring-defect-dataset-test.md>): Object Detection; Locomotive coil spring defect dataset (test); mAP@0.5: 85.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2307.13352>)

[PDF](<https://arxiv.org/pdf/2307.13352.pdf>)