# Distributed Training of Deep Neural Networks with Theoretical Analysis: Under SSP Setting

Source: [Distributed Training of Deep Neural Networks with Theoretical Analysis: Under SSP Setting](<https://www.sota2.com/research/paper/68fa53371e209131ce981af5>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T12:20:09.474Z

Authors: Abhimanu Kumar, Pengtao Xie, Junming Yin, Eric P. Xing

Published: 2015-12-09T02:46:28.000Z

## Abstract

We propose a distributed approach to train deep neural networks (DNNs), which has guaranteed convergence theoretically and great scalability empirically: close to 6 times faster on instance of ImageNet data set when run with 6 machines. The proposed scheme is close to optimally scalable in terms of number of machines, and guaranteed to converge to the same optima as the undistributed setting. The convergence and scalability of the distributed setting is shown empirically across different datasets (TIMIT and ImageNet) and machine learning tasks (image classification and phoneme extraction). The convergence analysis provides novel insights into this complex learning scheme, including: 1) layerwise convergence, and 2) convergence of the weights in probability.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1512.02728>)

[PDF](<https://arxiv.org/pdf/1512.02728.pdf>)