Mathematical Reasoning on GSM-Hard (Acc, Pass@5)
48.57Accuracy (GSM-Hard)ReLAR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ReLARShot Setting=5-shot2026.06 | 48.57 | 52.12 | |
| Qwen2.5-7BShot Setting=5-shot2026.06 | 47.64 | 54.83 | |
| ReLARShot Setting=0-shot2026.06 | 45.02 | 47.58 | |
| Med42-Llama3-8BShot Setting=5-shot2026.06 | 43.49 | 51.63 | |
| Llama-3-8B-InstructShot Setting=5-shot2026.06 | 43.28 | 51.84 | |
| Qwen2.5-Med-7BShot Setting=5-shot2026.06 | 42.82 | 50.03 | |
| Qwen2.5-7BShot Setting=0-shot2026.06 | 39.83 | 47.28 | |
| Med42-Mistral-7BShot Setting=5-shot2026.06 | 38.84 | 47.23 | |
| Mistral-7B-v0.3Shot Setting=5-shot2026.06 | 38.64 | 46.28 | |
| Qwen2.5-Med-7BShot Setting=0-shot2026.06 | 36.47 | 43.82 | |
| Mistral-7B-InstructShot Setting=5-shot2026.06 | 36.28 | 44.83 | |
| Med42-Llama3-8BShot Setting=0-shot2026.06 | 35.62 | 42.93 | |
| MedGemma-4BShot Setting=5-shot2026.06 | 34.63 | 42.08 | |
| Llama-3-8B-InstructShot Setting=0-shot2026.06 | 34.47 | 42.83 | |
| Gemma-7BShot Setting=5-shot2026.06 | 31.84 | 39.47 | |
| Med42-Mistral-7BShot Setting=0-shot2026.06 | 30.63 | 38.47 | |
| LLaMA-2-7BShot Setting=5-shot2026.06 | 29.47 | 38.57 | |
| Mistral-7B-InstructShot Setting=0-shot2026.06 | 27.83 | 34.47 | |
| MedGemma-4BShot Setting=0-shot2026.06 | 26.03 | 33.2 | |
| Mistral-7B-v0.3Shot Setting=0-shot2026.06 | 24.83 | 31.47 | |
| Gemma-7BShot Setting=0-shot2026.06 | 22.47 | 29.63 | |
| LLaMA-2-7BShot Setting=0-shot2026.06 | 22.34 | 27.83 | |
| Falcon-7BShot Setting=5-shot2026.06 | 21.28 | 27.64 | |
| Falcon-7BShot Setting=0-shot2026.06 | 14.47 | 19.83 |