Theory of Mind on TomBench OOD
75.24EmotionGPT-4o
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| GPT-4oModel=GPT-4o2026.01 | 75.24 | 62.78 | 82.35 | 57.79 | 85.94 | 78.61 | 73.78 | — | |
| HumanLLMBackbone=Qwen3-8B2026.01 | 66.14 | 61.63 | 72.3 | 48.48 | 66.24 | 80.54 | 65.89 | 3.08 | |
| BE.FMModel=BE.FM2026.01 | 65.14 | 53.92 | 66.58 | 41.86 | 61.9 | 68.7 | 59.69 | — | |
| Qwen3-8BModel=Qwen3-8B2026.01 | 64.68 | 60.02 | 66.39 | 50.39 | 62.82 | 79.22 | 63.92 | — | |
| HumanLLM (t1)Backbone=Qwen3-8B, Ablation=Trained on Task Group 12026.01 | 63.17 | 60.27 | 67.83 | 46.19 | 66.71 | 80.43 | 64.1 | 0.28 | |
| HumanLLMBackbone=Qwen2.5-3B-Instruct2026.01 | 62.89 | 56.71 | 57.21 | 36.76 | 74.68 | 79.05 | 61.22 | 0.97 | |
| HumanLLM (t2)Backbone=Qwen3-8B, Ablation=Trained on Task Group 22026.01 | 62.78 | 60.79 | 66 | 47.81 | 66.15 | 80.8 | 64.06 | 0.22 | |
| HumanLLMBackbone=Qwen2.5-7B-Instruct2026.01 | 62.35 | 56.55 | 68.26 | 38.73 | 83.29 | 75.68 | 64.14 | 3.54 | |
| HumanLLMBackbone=Llama-3.1-8B-Instruct2026.01 | 61.35 | 58.06 | 69.61 | 46.64 | 61.78 | 66.08 | 60.59 | 0.97 | |
| Qwen2.5-3B-InstructModel=Qwen2.5-3B-Instruct2026.01 | 60.89 | 52.92 | 52.11 | 46.26 | 77.58 | 74 | 60.63 | — | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct2026.01 | 60.67 | 58.64 | 59.42 | 41.98 | 79.62 | 71.35 | 61.95 | — | |
| Llama-3.1-8B-InstructModel=Llama-3.1-8B-Instruct2026.01 | 59.28 | 54.48 | 64.61 | 52.4 | 64.06 | 65.26 | 60.01 | — | |
| HumanLLMBackbone=Phi-3-mini-128k-instruct2026.01 | 57.25 | 53.02 | 57.06 | 43.1 | 53.47 | 61.17 | 54.18 | 4.74 | |
| Phi-3-mini-128k-instructModel=Phi-3-mini-128k-instruct2026.01 | 55.85 | 54.55 | 52.45 | 46.26 | 46.62 | 54.68 | 51.73 | — | |
| HumanLLM (raw)Backbone=Qwen2.5-7B-Instruct, Ablation=Trained on raw corpora text completion2026.01 | 48.88 | 48.12 | 60 | 44.44 | 81.66 | 70 | 58.85 | -5 | |
| HumanLLM (gen)Backbone=Qwen2.5-7B-Instruct, Ablation=General-purpose instruction tuning mix2026.01 | 45.55 | 52.5 | 60.83 | 37.77 | 58.05 | 70 | 54.12 | -12.64 | |
| CentaurModel=Centaur2026.01 | 42.46 | 36.86 | 40.72 | 28.2 | 35.13 | 41.64 | 37.5 | — |