Mathematical Reasoning on MultiArith (Emotional/Neutralized Evaluation)
99Original AccuracyQwen-2.5-7B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Qwen-2.5-7BModel Tier=Small, Prompting Strategy=Zero-shot CoT2026.04 | 99 | 96.9 | 98.7 | |
| Gemma-2-27BModel Tier=Medium, Prompting Strategy=Base2026.04 | 99 | 96.9 | 98.8 | |
| Gemma-2-27BModel Tier=Medium, Prompting Strategy=Zero-shot CoT2026.04 | 99 | 97.8 | 98.9 | |
| Llama-3.3-70BModel Tier=Medium, Prompting Strategy=Base2026.04 | 99 | 97.9 | 98.6 | |
| Llama-3.3-70BModel Tier=Medium, Prompting Strategy=Zero-shot CoT2026.04 | 99 | 97.8 | 98.5 | |
| Qwen-2.5-72BModel Tier=Medium, Prompting Strategy=Base2026.04 | 99 | 97.8 | 98.4 | |
| Qwen-2.5-72BModel Tier=Medium, Prompting Strategy=Zero-shot CoT2026.04 | 99 | 98 | 98.4 | |
| GPT-4oModel Tier=Frontier, Prompting Strategy=Base2026.04 | 99 | 97.6 | 98.6 | |
| GPT-4oModel Tier=Frontier, Prompting Strategy=Zero-shot CoT2026.04 | 99 | 97.2 | 98.3 | |
| GPT-5Model Tier=Frontier, Prompting Strategy=Base2026.04 | 99 | 97.6 | 98.5 | |
| GPT-5Model Tier=Frontier, Prompting Strategy=Zero-shot CoT2026.04 | 99 | 97.3 | 98.5 | |
| GPT-5.4Model Tier=Frontier, Prompting Strategy=Base2026.04 | 99 | 98 | 98.7 | |
| GPT-5.4Model Tier=Frontier, Prompting Strategy=Zero-shot CoT2026.04 | 99 | 98.1 | 98.7 | |
| o3Model Tier=Frontier, Prompting Strategy=Zero-shot CoT2026.04 | 98.7 | 97.5 | 98.6 | |
| Frontier avgPrompting Strategy=Base2026.04 | 98.6 | 97.5 | 98.4 | |
| Frontier avgPrompting Strategy=Zero-shot CoT2026.04 | 98.6 | 97.3 | 98.4 | |
| Mistral-Sm-24BModel Tier=Medium, Prompting Strategy=Zero-shot CoT2026.04 | 98.3 | 97.1 | 97.9 | |
| DeepSeek-V3Model Tier=Frontier, Prompting Strategy=Base2026.04 | 98.3 | 97.4 | 98.1 | |
| GPT-4o-miniModel Tier=Frontier, Prompting Strategy=Base2026.04 | 98.3 | 96.7 | 98.2 | |
| Gemma-2-9BModel Tier=Small, Prompting Strategy=Base2026.04 | 98 | 96.9 | 99.6 | |
| DeepSeek-V3Model Tier=Frontier, Prompting Strategy=Zero-shot CoT2026.04 | 98 | 97.2 | 98.1 | |
| o3Model Tier=Frontier, Prompting Strategy=Base2026.04 | 98 | 97.6 | 98.4 | |
| Qwen-2.5-3BModel Tier=Tiny, Prompting Strategy=Zero-shot CoT2026.04 | 97.7 | 95 | 97.3 | |
| Llama-3.1-8BModel Tier=Small, Prompting Strategy=Zero-shot CoT2026.04 | 97.7 | 92.7 | 96.2 | |
| Gemma-2-9BModel Tier=Small, Prompting Strategy=Zero-shot CoT2026.04 | 97.7 | 97.4 | 99.4 | |
| Mistral-Sm-24BModel Tier=Medium, Prompting Strategy=Base2026.04 | 97.7 | 96.4 | 97.1 | |
| GPT-4o-miniModel Tier=Frontier, Prompting Strategy=Zero-shot CoT2026.04 | 97.7 | 96.4 | 97.9 | |
| Llama-3.2-3BModel Tier=Tiny, Prompting Strategy=Zero-shot CoT2026.04 | 97 | 88.4 | 96.9 | |
| Llama-3.1-8BModel Tier=Small, Prompting Strategy=Base2026.04 | 95.7 | 91.4 | 95.9 | |
| Qwen-2.5-3BModel Tier=Tiny, Prompting Strategy=Base2026.04 | 95 | 92.4 | 94.1 | |
| Llama-3.2-3BModel Tier=Tiny, Prompting Strategy=Base2026.04 | 94 | 83 | 92 | |
| Open-src avgPrompting Strategy=Zero-shot CoT2026.04 | 93 | 90.8 | 93.6 | |
| Gemma-2-2BModel Tier=Tiny, Prompting Strategy=Base2026.04 | 92 | 88.4 | 92.4 | |
| Gemma-2-2BModel Tier=Tiny, Prompting Strategy=Zero-shot CoT2026.04 | 89.7 | 91.1 | 93.3 | |
| Open-src avgPrompting Strategy=Base2026.04 | 86.6 | 83.7 | 87.8 | |
| Mistral-7BModel Tier=Small, Prompting Strategy=Zero-shot CoT2026.04 | 81.3 | 80.6 | 83.5 | |
| Qwen-2.5-7BModel Tier=Small, Prompting Strategy=Base2026.04 | 68.3 | 68.8 | 73.6 | |
| Llama-3.2-1BModel Tier=Tiny, Prompting Strategy=Zero-shot CoT2026.04 | 60 | 56.6 | 63.8 | |
| Mistral-7BModel Tier=Small, Prompting Strategy=Base2026.04 | 52 | 53.5 | 61.3 | |
| Llama-3.2-1BModel Tier=Tiny, Prompting Strategy=Base2026.04 | 49 | 41.4 | 51.3 |