# Optimizing Distributed Training Approaches for Scaling Neural Networks

Source: [Optimizing Distributed Training Approaches for Scaling Neural Networks](<https://www.sota2.com/research/paper/68fa77bf1e209131cea75159>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:25:31.619Z

Authors: Vishnu Vardhan Baligodugula, Fathi Amsaad

Published: 2025-03-29T18:51:56.000Z

## Abstract

This paper presents a comparative analysis of distributed training strategies for large-scale neural networks, focusing on data parallelism, model parallelism, and hybrid approaches. We evaluate these strategies on image classification tasks using the CIFAR-100 dataset, measuring training time, convergence rate, and model accuracy. Our experimental results demonstrate that hybrid parallelism achieves a 3.2x speedup compared to single-device training while maintaining comparable accuracy. We propose an adaptive scheduling algorithm that dynamically switches between parallelism strategies based on network characteristics and available computational resources, resulting in an additional 18% improvement in training efficiency.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2503.23186>)

[PDF](<https://arxiv.org/pdf/2503.23186.pdf>)