Mathematical Reasoning on MSVAMP
80.21Average AccuracyTAPO-ChrF++
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TAPO-ChrF++Model=Qwen2026.03 | 80.21 | — | — | — | — | — | — | — | — | — | — | — | — | |
| MERLINDecoder Model=Gemma 2 9B, Multilingual Encoder=NLLB-600M-distilled2025.09 | 79.2 | 72.1 | 76.1 | 77.4 | 78.7 | 79.8 | 81.5 | 81.8 | 79.8 | 81.8 | 83.3 | 75.2 | 81 | |
| QAlign-LoRADecoder Model=Gemma 2 9B, Multilingual Encoder=NLLB-600M-distilled2025.09 | 78.3 | 69.9 | 72 | 75.5 | 77.9 | 77.1 | 82.3 | 80.8 | 80.7 | 81.8 | 84.6 | 72.5 | 80.7 | |
| MindMergerDecoder Model=Gemma 2 9B, Multilingual Encoder=NLLB-600M-distilled2025.09 | 78.3 | 69.1 | 73.1 | 76.8 | 79.3 | 78.3 | 82.6 | 79.5 | 79.5 | 82.5 | 82.3 | 73 | 80.6 | |
| QwenModel=Qwen2026.03 | 77.55 | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPO-EnCoTModel=Qwen2026.03 | 77.28 | — | — | — | — | — | — | — | — | — | — | — | — | |
| TAPO-ChrF++Model=Llama2026.03 | 77.25 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma 2 9BDecoder Model=Gemma 2 9B, Group=Open Models2025.09 | 76.5 | 63.9 | 72 | 71.2 | 78.7 | 81.8 | 78.7 | 79.3 | 78.3 | 80.3 | 80.7 | 69 | 79.7 | |
| GRPO-TransTestModel=Qwen2026.03 | 76.39 | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPO-TransTestModel=Llama2026.03 | 76.02 | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPO-EnCoTModel=Llama2026.03 | 73.75 | — | — | — | — | — | — | — | — | — | — | — | — | |
| SLAMDecoder Model=Gemma 2 9B, Multilingual Encoder=NLLB-600M-distilled2025.09 | 72 | 60.5 | 70.2 | 68.8 | 74.7 | 74.5 | 74.5 | 74.7 | 73.6 | 75.7 | 72.9 | 66.5 | 74.4 | |
| MERLINDecoder Model=Metamath 7B, Multilingual Encoder=mT5-XL2025.09 | 61.8 | 53.1 | 55.6 | 58.9 | 62 | 62.3 | 65.5 | 65.7 | 62.2 | 64.9 | 68.2 | 55.9 | 64.4 | |
| LlamaModel=Llama2026.03 | 61.43 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ST-Merge2026.06 | 61.4 | 52.8 | 56.3 | 57.6 | 59.7 | 61.7 | 63.8 | 65.7 | 62.2 | 66.1 | 67.6 | 55.6 | 63.9 | |
| SLAMDecoder Model=Metamath 7B, Multilingual Encoder=mT5-XL2025.09 | 60.5 | 49.1 | 50.6 | 55.4 | 60.6 | 63.1 | 64.6 | 65.4 | 64.1 | 66.6 | 65.3 | 51.7 | 64.2 | |
| MindMerger2026.06 | 60.4 | 52 | 53.4 | 54 | 59 | 61.7 | 64.1 | 64 | 63.3 | 65 | 67.7 | 53.1 | 63.5 | |
| LayAlign2026.06 | 59.1 | 51.8 | 55.1 | 56.9 | 59.3 | 58.7 | 62.5 | 62.1 | 58.8 | 62 | 64 | 54.6 | 61.1 | |
| QAlign-LoRADecoder Model=Metamath 7B, Multilingual Encoder=mT5-XL2025.09 | 57.4 | 41.7 | 47.7 | 54.8 | 58 | 55.7 | 62.8 | 63.2 | 61.1 | 63.3 | 65.3 | 48.1 | 61.3 | |
| QAlign2026.06 | 57.2 | 41.7 | 47.7 | 54.8 | 58 | 55.7 | 62.8 | 63.2 | 61.1 | 63.3 | 65.3 | 48.1 | 61.3 | |
| MindMergerDecoder Model=Metamath 7B, Multilingual Encoder=mT5-XL2025.09 | 56.2 | 51.1 | 47.9 | 53 | 54.7 | 53.6 | 58.4 | 60.7 | 57.7 | 60.5 | 64.1 | 50.7 | 58.5 | |
| LangBridgeDecoder Model=Metamath 7B, Multilingual Encoder=mT5-XL2025.09 | 52 | 46.8 | 46.3 | 42.1 | 45.5 | 50.4 | 58.1 | 57 | 55.8 | 56.9 | 60.6 | 45.1 | 54.9 | |
| LangBridge2026.06 | 52 | 46.8 | 46.3 | 42.1 | 45.5 | 50.4 | 58.1 | 57 | 55.8 | 56.9 | 60.6 | 45.1 | 54.9 | |
| Translate-En2026.06 | 50.9 | 47.9 | 51.3 | 43.1 | 50.4 | 55.8 | 43.9 | 50.9 | 53.4 | 51.4 | 60.6 | 47.4 | 52.3 | |
| MetaMath2026.06 | 47.8 | 14.4 | 19.5 | 16.8 | 53.4 | 55 | 63.5 | 64.1 | 60.3 | 64.9 | 66.3 | 16.9 | 61.1 | |
| MultiReason2026.06 | 42.8 | 34.8 | 38.1 | 39.8 | 43.4 | 42.9 | 45.6 | 45.8 | 45 | 46.1 | 46.8 | 37.6 | 45.1 | |
| Metamath 7BDecoder Model=Metamath 7B, Group=Open Models2025.09 | 41.7 | 11.8 | 13.1 | 11.8 | 40.9 | 48.2 | 56.1 | 58 | 54.8 | 59.5 | 63 | 12.2 | 54.4 | |
| LayAlignDecoder Model=Metamath 7B, Multilingual Encoder=mT5-XL2025.09 | 40.6 | 7.3 | 15.4 | 16.9 | 41.7 | 45.9 | 56.4 | 55.3 | 52.9 | 55 | 59.3 | 13.2 | 52.4 |