# A Critical Review of Recurrent Neural Networks for Sequence Learning

Source: [A Critical Review of Recurrent Neural Networks for Sequence Learning](<https://www.sota2.com/research/paper/68fa622b1e209131ce9f7d9a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:14:30.423Z

Authors: Zachary C. Lipton, John Berkowitz, Charles Elkan

Published: 2015-05-29T20:16:51.000Z

## Abstract

Countless learning tasks require dealing with sequential data. Image captioning, speech synthesis, and music generation all require that a model produce outputs that are sequences. In other domains, such as time series prediction, video analysis, and musical information retrieval, a model must learn from inputs that are sequences. Interactive tasks, such as translating natural language, engaging in dialogue, and controlling a robot, often demand both capabilities. Recurrent neural networks (RNNs) are connectionist models that capture the dynamics of sequences via cycles in the network of nodes. Unlike standard feedforward neural networks, recurrent networks retain a state that can represent information from an arbitrarily long context window. Although recurrent neural networks have traditionally been difficult to train, and often contain millions of parameters, recent advances in network architectures, optimization techniques, and parallel computation have enabled successful large-scale learning with them. In recent years, systems based on long short-term memory (LSTM) and bidirectional (BRNN) architectures have demonstrated ground-breaking performance on tasks as varied as image captioning, language translation, and handwriting recognition. In this survey, we review and synthesize the research that over the past three decades first yielded and then made practical these powerful learning models. When appropriate, we reconcile conflicting notation and nomenclature. Our goal is to provide a self-contained explication of the state of the art together with a historical perspective and references to primary research.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [fMRI Time Series Forecasting on HCP-t-fMRI Shen functional atlas (test)](<https://www.sota2.com/research/sota/fmri-time-series-forecasting-on-hcp-t-fmri-shen-functional-atlas-test.md>): fMRI time series forecasting; HCP-t-fMRI Shen functional atlas (test); MSE: 0.663.

- [fMRI time series forecasting on HCP-rs-fMRI Shen functional atlas (test)](<https://www.sota2.com/research/sota/fmri-time-series-forecasting-on-hcp-rs-fmri-shen-functional-atlas-test.md>): fMRI time series forecasting; HCP-rs-fMRI Shen functional atlas (test); MSE: 0.592.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1506.00019>)

[PDF](<https://arxiv.org/pdf/1506.00019.pdf>)
