List your company
ResearchFormerly WizWand

Fine-tuning on Alpaca (train)

0.5622LossAdamW16
0.5299680.7475340.96511.182666May 6, 2026
Updated 5mo ago

Evaluation results

MethodLinks
AdamW16Model=Llama_3.2_3B, State (MB)=12256.3125 ± 0.0000, Time (s)=895.5152 ± 4.45522026.050.56221.4073
AdamW16Model=Llama_3.2_1B, State (MB)=4714.5000 ± 0.0000, Time (s)=1731.4853 ± 22.33832026.050.72071.5548
GaLore-AW32Model=Llama_3.2_3B, State (MB)=24511.3125 ± 0.0000, Time (s)=881.6705 ± 4.40142026.050.76211.6311
BAOC0.5Model=Llama_3.2_3B, State (MB)=6127.4092 ± 0.0000, Time (s)=824.0374 ± 4.12012026.050.79441.6526
GaLore-AW32Model=Llama_3.2_1B, State (MB)=9428.5000 ± 0.0000, Time (s)=1733.7957 ± 19.62312026.050.81111.6861
AdamW8Model=Llama_3.2_3B, State (MB)=6224.9395 ± 0.0000, Time (s)=876.4541 ± 4.42652026.050.8151.6423
AdamW8Model=Llama_3.2_1B, State (MB)=2394.5488 ± 0.0000, Time (s)=1720.7466 ± 28.29412026.050.87211.7859
BAOC0.5Model=Llama_3.2_1B, State (MB)=2140.1338 ± 0.0000, Time (s)=1717.8212 ± 17.05482026.050.87391.7823
Adam-miniModel=Llama_3.2_1B, State (MB)=4715.5338 ± 0.0000, Time (s)=1729.2948 ± 30.95042026.051.12982.2214
GaLore-AFModel=Llama_3.2_3B, State (MB)=6.9551 ± 0.0000, Time (s)=890.5379 ± 4.25832026.051.18542.9719
Adam-miniModel=Llama_3.2_3B, State (MB)=12258.1823 ± 0.0000, Time (s)=869.8237 ± 4.34502026.051.2072.2555
GaLore-AFModel=Llama_3.2_1B, State (MB)=3.4351 ± 0.0000, Time (s)=1768.7772 ± 17.25822026.051.3683.7171