ResearchBenchmarksInstruction-tuning on Alpaca-style 500 samples (held-out)Instruction-tuning on Alpaca-style 500 samples (held-out)Follow1.347Cross-Entropy Loss (Std)AdamW1.279651.3133251.3471.380675May 20, 2026Evaluation resultsMethodMethodLinksCross-Entropy Loss (Std)Cross-Entropy Loss (Full BC)Loss DeltaAdamWStandard Setting=b512,...Standard Setting=b512, lr 2e−5, Corrected Setting=b512, lr 1e−42026.051.3471.346-0.001ShampooStandard Setting=attn...Standard Setting=attn only, b512, lr 2e−5, Corrected Setting=attn only, b512, lr 2e−52026.051.3471.3470