ResearchBenchmarksSelf-attention inverse temperature scaling analysis on SlimPajamaFollow6Tie PercentagenanoGPT2.6723.5364.45.264May 12, 2026Evaluation ResultsMethodMethodLinksTie PercentageXi * BetaHatted Xi LambdaHatted Xi AlphaHatted Xi DeltananoGPTneval/iter=100kneval/iter=100k2026.0560.350.520.860.38nanoGPTneval/iter=25kneval/iter=25k2026.052.80.40.540.90.44