Theory-of-Mind reasoning on MuMA-ToM
95.89AccuracyUserHarness
Evaluation results
| Method | Links | ||
|---|---|---|---|
| UserHarnessBackbone=GPT-5.42026.05 | 95.89 | — | |
| UserHarnessBackbone=Gemini-3.1-Pro2026.05 | 95.89 | — | |
| UserHarnessBackbone=Claude-Opus-4.72026.05 | 95.89 | — | |
| UserHarnessBackbone=Qwen3-8B2026.05 | 95.78 | — | |
| UserHarnessBackbone=Qwen3-32B2026.05 | 95.78 | — | |
| UserHarnessBackbone=GPT-OSS-120B2026.05 | 95.78 | — | |
| UserHarnessBackbone=Claude-Sonnet-4.62026.05 | 95.78 | — | |
| UserHarnessBackbone=Qwen3-14B2026.05 | 95.67 | — | |
| UserHarnessBackbone=GPT-OSS-20B2026.05 | 95.67 | — | |
| UserHarnessBackbone=GPT-5.4-mini2026.05 | 95.67 | — | |
| UserHarnessBackbone=Llama-3.1-8B2026.05 | 95.56 | — | |
| UserHarnessBackbone=Gemini-3-Flash2026.05 | 94.89 | — | |
| AutoToMBackbone=GPT-4o2026.05 | 81.44 | — | |
| Thinking-RFTModality=Lan.+Vis.2026.06 | 81.1 | — | |
| Direct promptingBackbone=GPT-OSS-120B2026.05 | 80.78 | — | |
| Paradox promptingBackbone=GPT-OSS-120B2026.05 | 80.11 | — | |
| Ledger promptingBackbone=GPT-OSS-120B2026.05 | 78.89 | — | |
| LIMPSetting=reported setting2026.05 | 76.6 | — | |
| Direct promptingBackbone=GPT-5.42026.05 | 75.67 | — | |
| SFTModality=Lan.+Vis.2026.06 | 75.2 | — | |
| Paradox promptingBackbone=GPT-OSS-20B2026.05 | 71.78 | — | |
| Direct promptingBackbone=GPT-OSS-20B2026.05 | 69.67 | — | |
| Paradox promptingBackbone=GPT-5.42026.05 | 68.44 | — | |
| Ledger promptingBackbone=GPT-OSS-20B2026.05 | 67.89 | — | |
| Ledger promptingBackbone=GPT-5.42026.05 | 64.78 | — | |
| Direct promptingBackbone=Qwen3-32B2026.05 | 60.44 | — | |
| AutoToMModality=Lan.+Vis.2026.06 | 59.1 | — | |
| Direct promptingBackbone=Qwen3-14B2026.05 | 56.44 | — | |
| Paradox promptingBackbone=Qwen3-14B2026.05 | 56.44 | — | |
| Paradox promptingBackbone=GPT-5.4-mini2026.05 | 56 | — | |
| Paradox promptingBackbone=Qwen3-32B2026.05 | 53.33 | — | |
| Paradox promptingBackbone=Qwen3-8B2026.05 | 53 | — | |
| SimToMModality=Lan.+Vis.2026.06 | 49.6 | — | |
| Ledger promptingBackbone=GPT-5.4-mini2026.05 | 49.44 | — | |
| SimToMSetting=reported setting2026.05 | 47.63 | — | |
| Direct promptingBackbone=GPT-5.4-mini2026.05 | 46.33 | — | |
| Ledger promptingBackbone=Qwen3-14B2026.05 | 45.89 | — | |
| Direct promptingBackbone=Qwen3-8B2026.05 | 44.56 | — | |
| Paradox promptingBackbone=Llama-3.1-8B2026.05 | 43.44 | — | |
| Zero-shotModality=Lan.+Vis.2026.06 | 43.3 | — | |
| Direct promptingBackbone=Llama-3.1-8B2026.05 | 42.33 | — | |
| Ledger promptingBackbone=Qwen3-32B2026.05 | 36.56 | — | |
| BIP-ALMSetting=reported setting2026.05 | 33.9 | — | |
| Ledger promptingBackbone=Llama-3.1-8B2026.05 | 29 | — | |
| Ledger promptingBackbone=Qwen3-8B2026.05 | 13.56 | — |