# Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutes

Source: [Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutes](<https://www.sota2.com/research/paper/68fa63d31e209131cea00ffe>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T05:53:50.473Z

Authors: Takuya Akiba, Shuji Suzuki, Keisuke Fukuda

Published: 2017-11-12T17:36:46.000Z

## Abstract

We demonstrate that training ResNet-50 on ImageNet for 90 epochs can be achieved in 15 minutes with 1024 Tesla P100 GPUs. This was made possible by using a large minibatch size of 32k. To maintain accuracy with this large minibatch size, we employed several techniques such as RMSprop warm-up, batch normalization without moving averages, and a slow-start learning rate schedule. This paper also describes the details of the hardware and software of the system used to achieve the above performance.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1711.04325>)

[PDF](<https://arxiv.org/pdf/1711.04325.pdf>)