Social Reasoning on Sotopia hard
2.4Rel ScoreFT+MS
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FT+MSBackbone=Qwen2.5-7B2025.09 | 2.4 | 4.33 | 6.3 | 4.34 | |
| TOMABackbone=Qwen2.5-7B, Training Strategy=FT+MS+Uttr2025.09 | 2.33 | 4.78 | 6.32 | 4.48 | |
| Base+MSBackbone=Qwen2.5-7B2025.09 | 2.17 | 4.51 | 5.86 | 4.18 | |
| TOMABackbone=Qwen2.5-3B, Training Strategy=FT+MS+Uttr2025.09 | 1.9 | 4.22 | 5.88 | 4 | |
| FT+MSBackbone=Qwen2.5-3B2025.09 | 1.7 | 4.08 | 5.42 | 3.73 | |
| Base+MSModel=GPT-5-nano, Conditioning=Mental State (MS)2025.09 | 1.51 | 5.21 | 6.67 | 4.46 | |
| FT+UttrBackbone=Qwen2.5-7B2025.09 | 1.36 | 4.43 | 5.7 | 3.83 | |
| TOMABackbone=Llama3.1-8B, Training Strategy=FT+MS+Uttr2025.09 | 1.27 | 5.36 | 5.68 | 4.1 | |
| FT+UttrBackbone=Qwen2.5-3B2025.09 | 1.22 | 4.1 | 5.23 | 3.52 | |
| Base+MSBackbone=Qwen2.5-3B2025.09 | 1.04 | 4.05 | 5.27 | 3.45 | |
| BaseModel=GPT-5-nano2025.09 | 0.77 | 4.39 | 6.24 | 3.8 | |
| BaseBackbone=Qwen2.5-7B2025.09 | 0.58 | 4.21 | 5.26 | 3.35 | |
| FT+MSBackbone=Llama3.1-8B2025.09 | 0.33 | 5.04 | 5.06 | 3.48 | |
| BaseBackbone=Qwen2.5-3B2025.09 | 0.18 | 4.2 | 4.96 | 3.11 | |
| FT+UttrBackbone=Llama3.1-8B2025.09 | -0.35 | 4.91 | 4.85 | 3.13 | |
| Base+MSBackbone=Llama3.1-8B2025.09 | -0.52 | 5.16 | 4.8 | 3.15 | |
| BaseBackbone=Llama3.1-8B2025.09 | -1.59 | 5.1 | 4.22 | 2.58 |