Theory of Mind on FANToM
95AccuracyDITTO
Evaluation results
| Method | Links | |
|---|---|---|
| DITTOBackbone=Qwen3-VL-8B-Instruct2026.05 | 95 | |
| GRPOBackbone=Qwen3-VL-8B-Instruct2026.05 | 94 | |
| GPT-5.42026.05 | 90 | |
| HumanLM-8B2026.05 | 78 | |
| Qwen3-VL-8B-InstructRole=Base2026.05 | 78 | |
| OSCToM-8BParams=8B, Inference Protocol=direct model inference2026.05 | 76 | |
| GPT-5-nano2026.05 | 72 | |
| Llama-3.1-8B-BaseParams=8B, Inference Protocol=direct model inference2026.05 | 66 | |
| Qwen2.5-14BParams=14B, Inference Protocol=direct model inference2026.05 | 54.5 | |
| Phi-3-Medium-14BParams=14B, Inference Protocol=direct model inference2026.05 | 51 | |
| Qwen2.5-32BParams=32B, Inference Protocol=direct model inference2026.05 | 46.5 | |
| Gemma-2-27BParams=27B, Inference Protocol=direct model inference2026.05 | 38.5 | |
| Mistral-NeMo-12BParams=12B, Inference Protocol=direct model inference2026.05 | 37.5 | |
| ExploreToMParams=8B, Inference Protocol=iterative A* heuristic search2026.05 | 0.2 |