List your company
ResearchFormerly WizWand

Language Modeling on C4 Qwen2.5 (val)

15.8Perplexity (PPL)FOAM-2
15.103219.806624.5129.2134Dec 8, 2025
Updated 8mo ago

Evaluation results

MethodLinks
FOAM-2Model size=350M2025.1215.8
FOAM-MiniModel size=350M2025.1215.99
MUONModel size=350M2025.1216.78
APOLLO-1/4Model size=350M2025.1216.83
Full-AdamModel size=350M2025.1216.97
APOLLO-MiniModel size=350M2025.1216.97
Adam-miniModel size=350M2025.1217.55
GWT-MiniModel size=350M2025.1217.6
GaLore-1/4Model size=350M2025.1219.58
FOAM-2Model size=130M2025.1221.25
FOAM-MiniModel size=130M2025.1221.75
MUONModel size=130M2025.1222.16
Full-AdamModel size=130M2025.1222.43
GWT-MiniModel size=130M2025.1223.31
APOLLO-MiniModel size=130M2025.1223.34
APOLLO-1/4Model size=130M2025.1223.43
Adam-miniModel size=130M2025.1224.12
GaLore-1/4Model size=130M2025.1225.63
FOAM-MiniModel size=60M2025.1227.98
FOAM-2Model size=60M2025.1228.57
MUONModel size=60M2025.1228.93
Full-AdamModel size=60M2025.1229.71
APOLLO-1/4Model size=60M2025.1230
Adam-miniModel size=60M2025.1230.04
GWT-MiniModel size=60M2025.1231.88
APOLLO-MiniModel size=60M2025.1232.05
GaLore-1/4Model size=60M2025.1233.22