Social Reasoning on ToMBench
78.34AccuracyO3
Evaluation results
| Method | Links | |
|---|---|---|
| O32026.03 | 78.34 | |
| GPT-5Mode=COT2026.03 | 78.3 | |
| GPT-52026.03 | 78.23 | |
| GPT-4oMode=COT2026.03 | 74.06 | |
| O3Mode=COT2026.03 | 74.02 | |
| Distill-Llama-70B2026.03 | 73.99 | |
| GPT-4o2026.03 | 73.78 | |
| Qwen3-32B2026.03 | 69.75 | |
| LLaMa3.1-70BMode=COT2026.03 | 69.56 | |
| Qwen3-32BThinking=Disabled2026.03 | 69.17 | |
| SocialR1-8BAblation=Full2026.03 | 68.81 | |
| SocialR1-8BAblation=w/o Rlen2026.03 | 68.62 | |
| SocialR1-8BAblation=only Rout2026.03 | 68.5 | |
| SocialR1-8BAblation=w/o Rcont2026.03 | 68.48 | |
| DeepSeek-R12026.03 | 68.34 | |
| SocialR1-4BAblation=Full2026.03 | 65.74 | |
| SocialR1-4BAblation=w/o Rstruct2026.03 | 64.49 | |
| SocialR1-4BAblation=w/o Rcont2026.03 | 64.49 | |
| SocialR1-8BAblation=w/o Rstruct2026.03 | 64.49 | |
| SocialR1-4BAblation=w/o Rlen2026.03 | 63.76 | |
| Qwen3-4B2026.03 | 63.56 | |
| Qwen3-8B2026.03 | 61.79 | |
| LLaMa3.1-70B2026.03 | 60.36 | |
| SocialR1-4BAblation=only Rout2026.03 | 59.29 | |
| Qwen3-4BThinking=Disabled2026.03 | 57.7 | |
| Qwen3-8BThinking=Disabled2026.03 | 53.49 |