# Diversity Is All You Need for Contrastive Learning: Spectral Bounds on Gradient Magnitudes

Source: [Diversity Is All You Need for Contrastive Learning: Spectral Bounds on Gradient Magnitudes](<https://www.sota2.com/research/paper/68fa7bf01e209131cea8b74a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:52:16.996Z

Authors: Peter Ochieng

Published: 2025-10-07T10:35:58.000Z

## Abstract

We derive non-asymptotic spectral bands that bound the squared InfoNCE gradient norm via alignment, temperature, and batch spectrum, recovering the \\(1/\\tau^{2}\\) law and closely tracking batch-mean gradients on synthetic data and ImageNet. Using effective rank \\(R\_{\\mathrm{eff}}\\) as an anisotropy proxy, we design spectrum-aware batch selection, including a fast greedy builder. On ImageNet-100, Greedy-64 cuts time-to-67.5\\% top-1 by 15\\% vs.\\ random (24\\% vs.\\ Pool--P3) at equal accuracy; CIFAR-10 shows similar gains. In-batch whitening promotes isotropy and reduces 50-step gradient variance by \\(1.37\\times\\), matching our theoretical upper bound.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2510.05767>)

[PDF](<https://arxiv.org/pdf/2510.05767.pdf>)