Social Reasoning on SimpleToM
99.24AccuracyGPT-5
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-5Config=MS2026.03 | 99.24 | |
| SocialR1-8BAblation=Full2026.03 | 96.75 | |
| O3Config=MS2026.03 | 96.72 | |
| SocialR1-8BAblation=w/o Rlen2026.03 | 96.19 | |
| SocialR1-4BAblation=w/o Rstruct2026.03 | 94.94 | |
| SocialR1-8BAblation=only Rout2026.03 | 94.33 | |
| SocialR1-8BAblation=w/o Rcont2026.03 | 93.52 | |
| SocialR1-4BAblation=Full2026.03 | 92.85 | |
| SocialR1-4BAblation=w/o Rlen2026.03 | 91.98 | |
| SocialR1-8BAblation=w/o Rstruct2026.03 | 90.41 | |
| SocialR1-4BAblation=only Rout2026.03 | 89.63 | |
| SocialR1-4BAblation=w/o Rcont2026.03 | 89.6 | |
| O32026.03 | 76.4 | |
| Qwen3-32B2026.03 | 76.34 | |
| Distill-Llama-70B2026.03 | 76.29 | |
| O3Mode=COT2026.03 | 75.01 | |
| GPT-4oConfig=MS2026.03 | 73.58 | |
| GPT-52026.03 | 73.55 | |
| LLaMa3.1-70B2026.03 | 71.9 | |
| DeepSeek-R12026.03 | 71.87 | |
| GPT-5Mode=COT2026.03 | 71.11 | |
| GPT-4o2026.03 | 66.61 | |
| Qwen3-32BThinking=Disabled2026.03 | 64.78 | |
| Qwen3-8B2026.03 | 61.76 | |
| GPT-4oMode=COT2026.03 | 61.26 | |
| LLaMa3.1-70BMode=COT2026.03 | 59.08 | |
| Qwen3-4B2026.03 | 54.11 | |
| Qwen3-4BThinking=Disabled2026.03 | 51.09 | |
| Qwen3-8BThinking=Disabled2026.03 | 51.09 |