List your company
ResearchFormerly WizWand

Language Modeling on (val) using Validation Loss

2.524Validation LossPowerStep
2.423163.103833.78454.46517May 11, 2026
Updated 5mo ago

Evaluation results

MethodLinks
PowerStepModel=Qwen3-235B-A22B, Precision=int82026.052.524
AdamWModel=Qwen3-235B-A22B, Precision=fp322026.052.525
AdamWModel=Qwen3-32B, Precision=fp322026.052.572
PowerStepModel=Qwen3-32B, Precision=int82026.052.582
PowerStepModel=DeepSeek-V2-Lite (16B), Precision=int82026.052.618
AdamWModel=Qwen3-30B-A3B, Precision=fp322026.052.62
AdamWModel=DeepSeek-V2-Lite (16B), Precision=fp322026.052.623
PowerStepModel=Qwen3-30B-A3B, Precision=int82026.052.624
AdamWModel=Qwen3-8B, Optimizer state=fp322026.052.66
AdamSModel=Qwen3-4B, Optimizer state=fp322026.052.672
SignSGDModel=Qwen3-4B, Optimizer state=fp322026.052.68
PowerStepModel=Qwen3-8B, Optimizer state=fp322026.052.688
AdamWModel=Qwen3-4B, Optimizer state=fp322026.052.69
AdamWModel=GPT-2-Medium (350M), Optimizer state=fp322026.052.711
pbSGDMModel=Qwen3-4B, Optimizer state=fp322026.052.712
PowerStepModel=Qwen3-4B, Optimizer state=fp322026.052.712
PowerStepModel=GPT-2-Medium (350M), Optimizer state=fp322026.052.717
AdamSModel=GPT-2-Medium (350M), Optimizer state=fp322026.052.725
SignSGDModel=GPT-2-Medium (350M), Optimizer state=fp322026.052.736
AdamWModel=Qwen3-1.7B, Optimizer state=fp322026.052.78
AdamSModel=Qwen3-1.7B, Optimizer state=fp322026.052.782
SignSGDModel=Qwen3-1.7B, Optimizer state=fp322026.052.794
PowerStepModel=Qwen3-1.7B, Optimizer state=fp322026.052.799
pbSGDMModel=Qwen3-8B, Optimizer state=fp322026.052.853
SignSGDModel=Qwen3-0.6B, Optimizer state=fp322026.052.897
AdamWModel=Qwen3-0.6B, Optimizer state=fp322026.052.912
AdamSModel=Qwen3-0.6B, Optimizer state=fp322026.052.912
PowerStepModel=Qwen3-0.6B, Optimizer state=fp322026.052.923
SignSGDModel=GPT-2-Small (124M), Optimizer state=fp322026.052.942
PowerStepModel=GPT-2-Small (124M), Optimizer state=fp322026.052.949
AdamWModel=GPT-2-Small (124M), Optimizer state=fp322026.052.95
AdamSModel=GPT-2-Small (124M), Optimizer state=fp322026.052.951
pbSGDMModel=Qwen3-1.7B, Optimizer state=fp322026.053.034
pbSGDMModel=GPT-2-Medium (350M), Optimizer state=fp322026.053.11
pbSGDMModel=Qwen3-0.6B, Optimizer state=fp322026.053.198
pbSGDMModel=GPT-2-Small (124M), Optimizer state=fp322026.053.377
AdamSModel=Qwen3-8B, Optimizer state=fp322026.054.689
SignSGDModel=Qwen3-8B, Optimizer state=fp322026.055.045