Mathematical Reasoning on Mathematics out-of-domain (test)
75.9AccuracyMathCoder-CL
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MathCoder-CLBase=CodeLlama, Size=34B2023.10 | 75.9 | — | — | |
| MathCoder-LBase=Llama-2, Size=70B2023.10 | 74.4 | — | — | |
| FARLTraining Time=4h 22m 53s2025.09 | 72.4 | — | 1,633.5 | |
| RL (GRPO)Training Time=4h 09m 35s2025.09 | 71.2 | — | 1,486.4 | |
| R1-Qwen-7B (Base)Training Time=/2025.09 | 69 | — | 1,237.6 | |
| SFTTraining Time=9m 36s2025.09 | 68.7 | — | 1,412.2 | |
| MathCoder-CLBase=CodeLlama, Size=13B2023.10 | 62.5 | — | — | |
| MathCoder-CLBase=CodeLlama, Size=7B2023.10 | 55.8 | — | — | |
| MathCoder-LBase=Llama-2, Size=13B2023.10 | 54.7 | — | — | |
| MathCoder-LBase=Llama-2, Size=7B2023.10 | 46.9 | — | — | |
| WizardMathBase=Llama-2, Size=70B2023.10 | 17.1 | — | — | |
| WizardMathBase=Llama-2, Size=13B2023.10 | 14.1 | — | — | |
| GalacticaSize=30B2023.10 | 11.8 | — | — | |
| WizardMathBase=Llama-2, Size=7B2023.10 | 9.3 | — | — | |
| Llama-1 RFTBase=Llama-1, Size=34B2023.10 | 7.6 | — | — | |
| Llama-1 RFTBase=Llama-1, Size=13B2023.10 | 6.7 | — | — | |
| Llama-1 RFTBase=Llama-1, Size=7B2023.10 | 5.1 | — | — | |
| GalacticaSize=6.7B2023.10 | 4.6 | — | — | |
| VAE (query_add)Injection=query_add, Train Setup=weight=1.02026.01 | 2.14 | 0.1223 | — | |
| VAE (kv_add)Injection=kv_add, Train Setup=prior=False2026.01 | 2 | 0.1055 | — | |
| VAE (kv_mem)Injection=kv_mem, Train Setup=prior=False2026.01 | 2 | 0.0985 | — | |
| VAE (query_add)Injection=query_add, Train Setup=prior=False2026.01 | 2 | 0.113 | — | |
| Qwen2.5-0.5B (Base model)Injection=Base model, Train Setup=Zero-shot2026.01 | 1.85 | 0.1293 | — | |
| VAE (kv_add)Injection=kv_add, Train Setup=weight=0.52026.01 | 1.85 | 0.0504 | — | |
| VAE (kv_mem)Injection=kv_mem, Train Setup=weight=1.02026.01 | 1.85 | 0.0956 | — | |
| VAE (kv_mem)Injection=kv_mem, Train Setup=weight=0.52026.01 | 1.85 | 0.113 | — | |
| VAE (query_add)Injection=query_add, Train Setup=weight=0.52026.01 | 1.85 | 0.1441 | — | |
| VAE (kv_add)Injection=kv_add, Train Setup=weight=1.02026.01 | 1.57 | 0.0869 | — | |
| Qwen2.5-0.5B (Base model)Injection=Base model, Train Setup=Few-shot2026.01 | 0.57 | 0.1011 | — | |
| Qwen2.5-0.5B (Base model)Injection=Base model, Train Setup=Few-shot (FT)2026.01 | 0.28 | 0.1022 | — |