Multi-hop Numeric Reasoning QA on MuSiQue 200 samples
638.03Average TokensDeepSeek-R1-0528
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| DeepSeek-R1-0528Adversarial Setting=No-adv2026.01 | 638.03 | 1 | 35 | — | — | — | — | |
| GPT-5.1Adversarial Setting=No-adv2026.01 | 969.9 | 1 | 51 | — | — | — | — | |
| Gemini 2.5 FlashAdversarial Setting=No-adv2026.01 | 1,275.71 | 1 | 41 | — | — | — | — | |
| DeepSeek-V3.2Adversarial Setting=No-adv2026.01 | 2,274.12 | 1 | 42 | — | — | — | — | |
| Qwen-PlusAdversarial Setting=No-adv2026.01 | 3,269.71 | 1 | 39 | — | — | — | — | |
| GPT-5.1Adversarial Setting=Adv(token level)2026.01 | 5,163.72 | 5.32 | 70 | — | — | — | — | |
| DeepSeek-R1-0528Adversarial Setting=Adv(token level)2026.01 | 8,720.65 | 13.67 | 69 | — | — | — | — | |
| Gemini 2.5 FlashAdversarial Setting=Adv(token level)2026.01 | 10,275.31 | 8.05 | 56 | — | — | — | — | |
| DeepSeek-V3.2Adversarial Setting=Adv(token level)2026.01 | 12,524.86 | 5.51 | 70 | — | — | — | — | |
| Qwen-PlusAdversarial Setting=Adv(token level)2026.01 | 70,574.68 | 21.58 | 70 | — | — | — | — | |
| DeepSeek-R1-0528Adversarial Setting=Adv(task level)2026.01 | — | — | — | 97.49 | 93.47 | 88.94 | 23.995 | |
| DeepSeek-V3.2Adversarial Setting=Adv(task level)2026.01 | — | — | — | 94.5 | 69.5 | 45.4 | 16.332 | |
| Gemini 2.5 FlashAdversarial Setting=Adv(task level)2026.01 | — | — | — | 98.5 | 80 | 57.5 | 16.255 | |
| GPT-5.1Adversarial Setting=Adv(task level)2026.01 | — | — | — | 92.5 | 75 | 49 | 12.698 | |
| Qwen-PlusAdversarial Setting=Adv(task level)2026.01 | — | — | — | 98.47 | 93.88 | 86.22 | 40.23 |