# Nonlinear Acceleration of CNNs

Source: [Nonlinear Acceleration of CNNs](<https://www.sota2.com/research/paper/68fa54e91e209131ce98fa3a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:55:46.184Z

Authors: Damien Scieur, Edouard Oyallon, Alexandre d'Aspremont, Francis Bach

Published: 2018-06-01T14:26:57.000Z

## Abstract

The Regularized Nonlinear Acceleration (RNA) algorithm is an acceleration method capable of improving the rate of convergence of many optimization schemes such as gradient descend, SAGA or SVRG. Until now, its analysis is limited to convex problems, but empirical observations shows that RNA may be extended to wider settings. In this paper, we investigate further the benefits of RNA when applied to neural networks, in particular for the task of image recognition on CIFAR10 and ImageNet. With very few modifications of exiting frameworks, RNA improves slightly the optimization process of CNNs, after training.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1806.00370>)

[PDF](<https://arxiv.org/pdf/1806.00370.pdf>)