List your company

Text Classification on AG News short protocol (600 steps)

82.8Accuracyvanilla Adam
22.68838.29453.969.506Jul 4, 2026

Evaluation results

MethodLinks
vanilla AdamLearning Rate=3×10⁻⁴, Warmup=None, Architecture=Transformer (12-layer), Tuning=best tuned2026.0782.8
Adam-InitOnlyLearning Rate=10⁻³, Initialization Cap (kappa)=0.005, Warmup=None, Architecture=Transformer (12-layer)2026.0781
vanilla AdamLearning Rate=10⁻³, Warmup=None, Architecture=Transformer (12-layer)2026.0725