Social Reasoning on ToMBench Hard (val)
62.79AccuracySocialR1-8B
Evaluation results
| Method | Links | |
|---|---|---|
| SocialR1-8BAblation=Full2026.03 | 62.79 | |
| SocialR1-8BAblation=w/o Rstruct2026.03 | 61.28 | |
| SocialR1-8BAblation=w/o Rcont2026.03 | 55.09 | |
| SocialR1-8BAblation=w/o Rlen2026.03 | 52.64 | |
| SocialR1-8BAblation=only Rout2026.03 | 50.42 | |
| SocialR1-4BAblation=Full2026.03 | 48.46 | |
| O3Mode=COT2026.03 | 48.2 | |
| SocialR1-4BAblation=w/o Rlen2026.03 | 47.28 | |
| O32026.03 | 44.78 | |
| LLaMa3.1-70B2026.03 | 44.71 | |
| DeepSeek-R12026.03 | 44.36 | |
| SocialR1-4BAblation=w/o Rstruct2026.03 | 44.23 | |
| SocialR1-4BAblation=w/o Rcont2026.03 | 44.23 | |
| Qwen3-32B2026.03 | 43.64 | |
| LLaMa3.1-70BMode=COT2026.03 | 43.35 | |
| GPT-5Mode=COT2026.03 | 42.72 | |
| Qwen3-32BThinking=Disabled2026.03 | 41.19 | |
| GPT-52026.03 | 40.92 | |
| Distill-Llama-70B2026.03 | 40.33 | |
| GPT-4oMode=COT2026.03 | 40.12 | |
| Qwen3-8B2026.03 | 39.82 | |
| SocialR1-4BAblation=only Rout2026.03 | 39.46 | |
| Qwen3-8BThinking=Disabled2026.03 | 38.62 | |
| GPT-4o2026.03 | 37.33 | |
| Qwen3-4B2026.03 | 34.03 | |
| Qwen3-4BThinking=Disabled2026.03 | 31.87 |