List your company
ResearchFormerly WizWand

Language Modeling on C4 Llama-160M scratch (val)

3.0908Validation LossGPA-AdamW
3.0888523.1020013.115153.128299Dec 18, 2025
Updated 8mo ago

Evaluation results

MethodLinks
GPA-AdamWNumber of inner steps (H)=322025.123.0908
GPA-AdamWNumber of inner steps (H)=1282025.123.0951
GPA-AdamWNumber of inner steps (H)=642025.123.0974
SF-AdamWNumber of inner steps (H)=642025.123.1051
DiLoCo-AdamWNumber of inner steps (H)=642025.123.1051
DiLoCo-AdamWNumber of inner steps (H)=322025.123.1066
SF-AdamWNumber of inner steps (H)=82025.123.1089
SF-AdamWNumber of inner steps (H)=162025.123.1089
SF-AdamWNumber of inner steps (H)=322025.123.1089
SF-AdamWNumber of inner steps (H)=1282025.123.1089
GPA-AdamWNumber of inner steps (H)=162025.123.1089
DiLoCo-AdamWNumber of inner steps (H)=162025.123.1133
AdamWNumber of inner steps (H)=82025.123.1141
AdamWNumber of inner steps (H)=162025.123.1141
AdamWNumber of inner steps (H)=322025.123.1141
AdamWNumber of inner steps (H)=642025.123.1141
AdamWNumber of inner steps (H)=1282025.123.1141
DiLoCo-AdamWNumber of inner steps (H)=1282025.123.128
GPA-AdamWNumber of inner steps (H)=82025.123.1353
DiLoCo-AdamWNumber of inner steps (H)=82025.123.1395