# Learning Trajectories are Generalization Indicators

Source: [Learning Trajectories are Generalization Indicators](<https://www.sota2.com/research/paper/68fa710f1e209131cea4ef43>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-02T04:07:48.393Z

Authors: Jingwen Fu, Zhizheng Zhang, Dacheng Yin, Yan Lu, Nanning Zheng

Published: 2023-04-25T05:08:57.000Z

## Abstract

This paper explores the connection between learning trajectories of Deep Neural Networks (DNNs) and their generalization capabilities when optimized using (stochastic) gradient descent algorithms. Instead of concentrating solely on the generalization error of the DNN post-training, we present a novel perspective for analyzing generalization error by investigating the contribution of each update step to the change in generalization error. This perspective allows for a more direct comprehension of how the learning trajectory influences generalization error. Building upon this analysis, we propose a new generalization bound that incorporates more extensive trajectory information. Our proposed generalization bound depends on the complexity of learning trajectory and the ratio between the bias and diversity of training set. Experimental findings reveal that our method effectively captures the generalization error throughout the training process. Furthermore, our approach can also track changes in generalization error when adjustments are made to learning rates and label noise levels. These results demonstrate that learning trajectory information is a valuable indicator of a model's generalization capabilities.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Generalization Bound Estimation on Synthetic Gaussian Distribution Dataset](<https://www.sota2.com/research/sota/generalization-bound-estimation-on-synthetic-gaussian-distribution-dataset.md>): Generalization Bound Estimation; Synthetic Gaussian Distribution Dataset; Generalization Bound Value: 3.62.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2304.12579>)

[PDF](<https://arxiv.org/pdf/2304.12579.pdf>)
