Mathematical Reasoning on GSM8K (Original, Emotional, Neutralized Accuracy)
98Original AccuracyGPT-4o
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4oModel Tier=Frontier, Prompting Strategy=Base, Decoding=Greedy2026.04 | 98 | 95 | 95.3 | |
| GPT-4oModel Tier=Frontier, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 97.8 | 95.2 | 95.2 | |
| Llama-3.3-70BModel Tier=Medium, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 97.5 | 93.2 | 94.7 | |
| GPT-5Model Tier=Frontier, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 97.5 | 94.5 | 94.9 | |
| GPT-5.4Model Tier=Frontier, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 97.5 | 96 | 95.7 | |
| o3Model Tier=Frontier, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 97.5 | 94.6 | 95.2 | |
| GPT-5.4Model Tier=Frontier, Prompting Strategy=Base, Decoding=Greedy2026.04 | 97 | 95.8 | 95.8 | |
| o3Model Tier=Frontier, Prompting Strategy=Base, Decoding=Greedy2026.04 | 97 | 94.4 | 95.7 | |
| Llama-3.3-70BModel Tier=Medium, Prompting Strategy=Base, Decoding=Greedy2026.04 | 96.8 | 93.7 | 94.6 | |
| Qwen-2.5-72BModel Tier=Medium, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 96.8 | 93.9 | 94.3 | |
| GPT-5Model Tier=Frontier, Prompting Strategy=Base, Decoding=Greedy2026.04 | 96.5 | 94.9 | 95.2 | |
| Frontier avgPrompting Strategy=Base2026.04 | 96.5 | 94.2 | 95 | |
| Frontier avgPrompting Strategy=Zero-shot CoT2026.04 | 96.4 | 94 | 94.5 | |
| DeepSeek-V3Model Tier=Frontier, Prompting Strategy=Base, Decoding=Greedy2026.04 | 95.8 | 94.1 | 94.5 | |
| DeepSeek-V3Model Tier=Frontier, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 95.8 | 93.7 | 93.9 | |
| Mistral-Sm-24BModel Tier=Medium, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 95.5 | 91.7 | 93.1 | |
| Qwen-2.5-72BModel Tier=Medium, Prompting Strategy=Base, Decoding=Greedy2026.04 | 95.2 | 93.7 | 94.4 | |
| Mistral-Sm-24BModel Tier=Medium, Prompting Strategy=Base, Decoding=Greedy2026.04 | 95 | 91.8 | 93.8 | |
| GPT-4o-miniModel Tier=Frontier, Prompting Strategy=Base, Decoding=Greedy2026.04 | 94.8 | 90.9 | 93.3 | |
| Qwen-2.5-7BModel Tier=Small, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 92.8 | 88 | 89.7 | |
| GPT-4o-miniModel Tier=Frontier, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 92.5 | 90.2 | 92.3 | |
| Gemma-2-27BModel Tier=Medium, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 92 | 89.2 | 90.5 | |
| Gemma-2-27BModel Tier=Medium, Prompting Strategy=Base, Decoding=Greedy2026.04 | 91.8 | 87.8 | 90.6 | |
| Gemma-2-9BModel Tier=Small, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 88.8 | 85.4 | 88.5 | |
| Gemma-2-9BModel Tier=Small, Prompting Strategy=Base, Decoding=Greedy2026.04 | 87 | 83.8 | 86.5 | |
| Qwen-2.5-3BModel Tier=Tiny, Prompting Strategy=Base, Decoding=Greedy2026.04 | 86.2 | 79.1 | 83.9 | |
| Qwen-2.5-3BModel Tier=Tiny, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 86 | 81.1 | 85 | |
| Llama-3.1-8BModel Tier=Small, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 86 | 80.9 | 84.4 | |
| Llama-3.1-8BModel Tier=Small, Prompting Strategy=Base, Decoding=Greedy2026.04 | 84 | 78.5 | 82.6 | |
| Open-src avgPrompting Strategy=Zero-shot CoT2026.04 | 80.9 | 76.7 | 79.6 | |
| Qwen-2.5-7BModel Tier=Small, Prompting Strategy=Base, Decoding=Greedy2026.04 | 80.2 | 73.7 | 78.8 | |
| Open-src avgPrompting Strategy=Base2026.04 | 78.6 | 73.5 | 77.4 | |
| Llama-3.2-3BModel Tier=Tiny, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 76.2 | 67.6 | 74.2 | |
| Llama-3.2-3BModel Tier=Tiny, Prompting Strategy=Base, Decoding=Greedy2026.04 | 74.5 | 64.5 | 71.5 | |
| Gemma-2-2BModel Tier=Tiny, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 64 | 59.2 | 62.5 | |
| Mistral-7BModel Tier=Small, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 61.2 | 57 | 58.7 | |
| Gemma-2-2BModel Tier=Tiny, Prompting Strategy=Base, Decoding=Greedy2026.04 | 60.8 | 56.9 | 61.7 | |
| Mistral-7BModel Tier=Small, Prompting Strategy=Base, Decoding=Greedy2026.04 | 56 | 49.5 | 53.8 | |
| Llama-3.2-1BModel Tier=Tiny, Prompting Strategy=Base, Decoding=Greedy2026.04 | 35.2 | 29 | 36.8 | |
| Llama-3.2-1BModel Tier=Tiny, Prompting Strategy=Zero-shot CoT, Decoding=Greedy2026.04 | 34 | 33.4 | 39.7 |