Reasoning on GPQA (Accuracy, Loss)
32.8AccuracyAdamW
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| AdamWOptimizer=AdamW, Model Size=3B2026.04 | 32.8 | 2.005 | |
| Top-HModel=Phi-3-Mini, Temperature=1.02025.09 | 32.37 | — | |
| MuonOptimizer=Muon, Model Size=3B2026.04 | 32 | 1.968 | |
| Top-HModel=Phi-3-Mini, Temperature=1.52025.09 | 30.8 | — | |
| Top-HModel=Phi-3-Mini, Temperature=2.02025.09 | 30.8 | — | |
| Top-HModel=LLaMA3.1-8B-Instruct, Temperature=1.52025.09 | 30.58 | — | |
| MirostatModel=LLaMA3.1-8B-Instruct, Temperature=1.02025.09 | 30.36 | — | |
| MirostatModel=Phi-3-Mini, Temperature=1.02025.09 | 30.13 | — | |
| MirostatModel=Phi-3-Mini, Temperature=2.02025.09 | 29.91 | — | |
| Adam+NexusOptimizer=Adam+Nexus, Model Size=3B2026.04 | 29.6 | 1.981 | |
| Top-HModel=LLaMA3.1-8B-Instruct, Temperature=1.02025.09 | 29.24 | — | |
| MirostatModel=Phi-3-Mini, Temperature=1.52025.09 | 29.02 | — | |
| Top-HModel=Qwen2.5 3B, Temperature=1.02025.09 | 28.79 | — | |
| Top-HModel=LLaMA3.1-8B-Instruct, Temperature=2.02025.09 | 28.79 | — | |
| MirostatModel=LLaMA3.1-8B-Instruct, Temperature=2.02025.09 | 28.79 | — | |
| MirostatModel=Qwen2.5 3B, Temperature=1.02025.09 | 28.35 | — | |
| Top-HModel=Qwen2.5 3B, Temperature=2.02025.09 | 28.12 | — | |
| Top-HModel=Qwen2.5 3B, Temperature=1.52025.09 | 27.9 | — | |
| MirostatModel=Qwen2.5 3B, Temperature=2.02025.09 | 27.79 | — | |
| MirostatModel=Qwen2.5 3B, Temperature=1.52025.09 | 26.34 | — | |
| MirostatModel=LLaMA3.1-8B-Instruct, Temperature=1.52025.09 | 25.67 | — |