# Selectivity and Shape in the Design of Forward-Forward Goodness Functions

Source: [Selectivity and Shape in the Design of Forward-Forward Goodness Functions](<https://www.sota2.com/research/paper/69e06ce37d0172140ec7c880>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-21T05:00:04.784Z

Authors: Talha Ruzgar Akkus, Suayp Talha Kocabay, Kamer Ali Yuksel, Hassan Sawaf

Published: 2026-03-28T23:11:21.000Z

## Abstract

The Forward-Forward (FF) algorithm trains networks layer-by-layer using a local "goodness function," yet sum-of-squares (SoS) has remained the only choice studied. We systematically explore the goodness-function design space and identify a unifying principle: the goodness function must be sensitive to the shape of neural activity, not its total energy. This principle is motivated by the observation that deep network activations follow heavy-tailed distributions and that discriminative information is often concentrated in peak activities. We propose two complementary families: selective functions (top-k, entmax-weighted energy) that measure only peak activity, and shape-sensitive functions (excess kurtosis / "burstiness" and higher-order moments) that reward heavy-tailed distributions via scale-invariant statistics. Combined with separate label-feature forwarding (FFCL), controlled experiments across 13 goodness functions, 5 activations, 6 datasets, and three continuous sweeps, each tracing a characteristic inverted-U, yield 89.0% on Fashion-MNIST and 98.2+-0.1% on MNIST (4x2000), a +32.6pp gain over SoS, with consistent improvements across all benchmarks (+72pp USPS, +52pp SVHN). The scale-invariant nature of burstiness makes it particularly robust to magnitude shifts across layers and datasets.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on Fashion-MNIST 4x2000 (test)](<https://www.sota2.com/research/sota/image-classification-on-fashion-mnist-4x2000-test.md>): Image Classification; Fashion-MNIST 4x2000 (test); Accuracy: 87.12.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.13081>)

[PDF](<https://arxiv.org/pdf/2604.13081.pdf>)