# The Cascaded Forward Algorithm for Neural Network Training

Source: [The Cascaded Forward Algorithm for Neural Network Training](<https://www.sota2.com/research/paper/68fa717b1e209131cea51845>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:07:38.985Z

Authors: Gongpei Zhao, Tao Wang, Yidong Li, Yi Jin, Congyan Lang, Haibin Ling

Published: 2023-03-17T02:01:11.000Z

## Abstract

Backpropagation algorithm has been widely used as a mainstream learning procedure for neural networks in the past decade, and has played a significant role in the development of deep learning. However, there exist some limitations associated with this algorithm, such as getting stuck in local minima and experiencing vanishing/exploding gradients, which have led to questions about its biological plausibility. To address these limitations, alternative algorithms to backpropagation have been preliminarily explored, with the Forward-Forward (FF) algorithm being one of the most well-known. In this paper we propose a new learning framework for neural networks, namely Cascaded Forward (CaFo) algorithm, which does not rely on BP optimization as that in FF. Unlike FF, our framework directly outputs label distributions at each cascaded block, which does not require generation of additional negative samples and thus leads to a more efficient process at both training and testing. Moreover, in our framework each block can be trained independently, so it can be easily deployed into parallel acceleration systems. The proposed method is evaluated on four public image classification benchmarks, and the experimental results illustrate significant improvement in prediction accuracy in comparison with the baseline.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on MNIST standard (test)](<https://www.sota2.com/research/sota/image-classification-on-mnist-standard-test.md>): Image Classification; MNIST standard (test); Accuracy: 99.04.

- [Node Classification on Flickr (test)](<https://www.sota2.com/research/sota/node-classification-on-flickr-test.md>): Node Classification; Flickr (test); Accuracy: 50.02.

- [Image Classification on MNIST (test) (Accuracy Only)](<https://www.sota2.com/research/sota/image-classification-on-mnist-test-accuracy-only.md>): Image Classification; MNIST (test); Accuracy: 98.8.

- [Node Classification on Reddit (test)](<https://www.sota2.com/research/sota/node-classification-on-reddit-test.md>): Node Classification; Reddit (test); Accuracy: 91.55.

- [Node Classification on Computers](<https://www.sota2.com/research/sota/node-classification-on-computers.md>): Node Classification; Computers; Mean Accuracy: 83.02.

- [Image Classification on CIFAR-100 standard (test)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-standard-test.md>): Image Classification; CIFAR-100 standard (test); Top-1 Accuracy: 43.48.

- [Image Classification on CIFAR-100 (ACC, NLL, ECE, ESS)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-acc-nll-ece-ess.md>): Image Classification; CIFAR-100; Accuracy: 40.76.

- [Node Classification on Photo](<https://www.sota2.com/research/sota/node-classification-on-photo.md>): Node Classification; Photo; Mean Accuracy: 90.59.

- [Node Classification on CiteSeer (accuracy)](<https://www.sota2.com/research/sota/node-classification-on-citeseer-accuracy.md>): Node Classification; Citeseer; Accuracy: 65.43.

- [Node Classification on ogbn-arxiv (test)](<https://www.sota2.com/research/sota/node-classification-on-ogbn-arxiv-test.md>): Node Classification; ogbn-arxiv (test); Accuracy: 60.57.

- [Node Classification on PubMed (accuracy)](<https://www.sota2.com/research/sota/node-classification-on-pubmed-accuracy.md>): Node Classification; Pubmed; Accuracy: 78.29.

- [Image Classification on CIFAR-10 (Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-accuracy.md>): Image Classification; CIFAR-10; Accuracy: 67.43.

- [Node Classification on Actor](<https://www.sota2.com/research/sota/node-classification-on-actor.md>): Node Classification; Actor; Accuracy: 23.83.

- [Node Classification on Texas](<https://www.sota2.com/research/sota/node-classification-on-texas.md>): Node Classification; Texas; Accuracy: 0.5639.

- [Node Classification on Squirrel](<https://www.sota2.com/research/sota/node-classification-on-squirrel.md>): Node Classification; Squirrel; Accuracy: 31.

- [Node Classification on Cornell](<https://www.sota2.com/research/sota/node-classification-on-cornell.md>): Node Classification; Cornell; Accuracy: 36.72.

- [Node Classification on Chameleon](<https://www.sota2.com/research/sota/node-classification-on-chameleon.md>): Node Classification; Chameleon; Accuracy: 37.36.

- [Node Classification on Cora](<https://www.sota2.com/research/sota/node-classification-on-cora.md>): Node Classification; Cora; Accuracy: 71.79.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2303.09728>)

[PDF](<https://arxiv.org/pdf/2303.09728.pdf>)
