List your company
ResearchFormerly WizWand

Fine-tuning on GSM8K (train)

0.3347Training LossAdamW16
0.3096280.4788640.64810.817336May 6, 2026
Updated 6mo ago

Evaluation results

MethodLinks
AdamW16Model=Llama_3.2_3B, State (MB)=12256.3125 ± 0.0000, Time (s)=4085.5169 ± 168.23402026.050.33471.2031
GaLore-AW32Model=Llama_3.2_3B, State (MB)=24511.3125 ± 0.0000, Time (s)=4125.9537 ± 165.64562026.050.38041.2343
AdamW8Model=Llama_3.2_3B, State (MB)=6224.9395 ± 0.0000, Time (s)=3945.8413 ± 160.40612026.050.38861.2522
BAOC0.5Model=Llama_3.2_3B, State (MB)=6127.4092 ± 0.0000, Time (s)=4030.8570 ± 157.74222026.050.39041.2501
AdamW16Model=Llama_3.2_1B, State (MB)=4714.5000 ± 0.0000, Time (s)=339.7833 ± 1.30112026.050.46361.3168
GaLore-AW32Model=Llama_3.2_1B, State (MB)=9428.5000 ± 0.0000, Time (s)=339.3417 ± 1.72662026.050.48381.3357
BAOC0.5Model=Llama_3.2_1B, State (MB)=2140.1338 ± 0.0000, Time (s)=336.2259 ± 0.27602026.050.48761.3444
AdamW8Model=Llama_3.2_1B, State (MB)=2394.5488 ± 0.0000, Time (s)=335.7520 ± 1.35822026.050.48951.3419
Adam-miniModel=Llama_3.2_3B, State (MB)=12258.1823 ± 0.0000, Time (s)=4135.7124 ± 140.34102026.050.52161.3837
Adam-miniModel=Llama_3.2_1B, State (MB)=4715.5338 ± 0.0000, Time (s)=338.5162 ± 0.79522026.050.74681.695
GaLore-AFModel=Llama_3.2_3B, State (MB)=6.9551 ± 0.0000, Time (s)=4191.0108 ± 162.96932026.050.80652.1099
GaLore-AFModel=Llama_3.2_1B, State (MB)=3.4351 ± 0.0000, Time (s)=344.5018 ± 0.41292026.050.96152.5308