# Social Reasoning on Sotopia hard Source: [Social Reasoning on Sotopia hard]() Collection index: [llms.txt]() Data updated: 2026-04-14T08:30:53.501Z Evaluation of model performance on Social Reasoning using the Sotopia hard dataset, measured by Rel, Know, Goal, and Avg scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Social Reasoning | | Dataset (including any stated split) | Sotopia hard | | Metric | Rel Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Rel Score](): Higher is better; unit: SCALAR. - [Know Score](): Higher is better; unit: SCALAR. - [Goal Score](): Higher is better; unit: SCALAR. - [Avg Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | Rel Score | Result date | Report paper | | --- | --- | --- | --- | | FT+MS | 2.4 | Not recorded | 68fa7ba61e209131cea89a30 | | TOMA | 2.33 | Not recorded | 68fa7ba61e209131cea89a30 | | Base+MS | 2.17 | Not recorded | 68fa7ba61e209131cea89a30 | | TOMA | 1.9 | Not recorded | 68fa7ba61e209131cea89a30 | | FT+MS | 1.7 | Not recorded | 68fa7ba61e209131cea89a30 | | Base+MS | 1.51 | Not recorded | 68fa7ba61e209131cea89a30 | | FT+Uttr | 1.36 | Not recorded | 68fa7ba61e209131cea89a30 | | TOMA | 1.27 | Not recorded | 68fa7ba61e209131cea89a30 | | FT+Uttr | 1.22 | Not recorded | 68fa7ba61e209131cea89a30 | | Base+MS | 1.04 | Not recorded | 68fa7ba61e209131cea89a30 | | Base | 0.77 | Not recorded | 68fa7ba61e209131cea89a30 | | Base | 0.58 | Not recorded | 68fa7ba61e209131cea89a30 | | FT+MS | 0.33 | Not recorded | 68fa7ba61e209131cea89a30 | | Base | 0.18 | Not recorded | 68fa7ba61e209131cea89a30 | | FT+Uttr | -0.35 | Not recorded | 68fa7ba61e209131cea89a30 | | Base+MS | -0.52 | Not recorded | 68fa7ba61e209131cea89a30 | | Base | -1.59 | Not recorded | 68fa7ba61e209131cea89a30 | ### FT+MS Rel Score: 2.4. Result ID: 69ddfa1247f34ce924c55ea3. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### TOMA Rel Score: 2.33. Result ID: 69ddfa1247f34ce924c55ea4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Training Strategy":"FT+MS+Uttr"}. This record does not assert independent reproduction. ### Base+MS Rel Score: 2.17. Result ID: 69ddfa1247f34ce924c55ea1. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### TOMA Rel Score: 1.9. Result ID: 69ddfa1247f34ce924c55e9f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Training Strategy":"FT+MS+Uttr"}. This record does not assert independent reproduction. ### FT+MS Rel Score: 1.7. Result ID: 69ddfa1247f34ce924c55e9e. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### Base+MS Rel Score: 1.51. Result ID: 69ddfa1247f34ce924c55e9a. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-5-nano","Conditioning":"Mental State (MS)"}. This record does not assert independent reproduction. ### FT+Uttr Rel Score: 1.36. Result ID: 69ddfa1247f34ce924c55ea2. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### TOMA Rel Score: 1.27. Result ID: 69ddfa1247f34ce924c55ea9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama3.1-8B","Training Strategy":"FT+MS+Uttr"}. This record does not assert independent reproduction. ### FT+Uttr Rel Score: 1.22. Result ID: 69ddfa1247f34ce924c55e9d. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### Base+MS Rel Score: 1.04. Result ID: 69ddfa1247f34ce924c55e9c. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### Base Rel Score: 0.77. Result ID: 69ddfa1247f34ce924c55e99. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-5-nano"}. This record does not assert independent reproduction. ### Base Rel Score: 0.58. Result ID: 69ddfa1247f34ce924c55ea0. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### FT+MS Rel Score: 0.33. Result ID: 69ddfa1247f34ce924c55ea8. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama3.1-8B"}. This record does not assert independent reproduction. ### Base Rel Score: 0.18. Result ID: 69ddfa1247f34ce924c55e9b. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### FT+Uttr Rel Score: -0.35. Result ID: 69ddfa1247f34ce924c55ea7. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama3.1-8B"}. This record does not assert independent reproduction. ### Base+MS Rel Score: -0.52. Result ID: 69ddfa1247f34ce924c55ea6. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama3.1-8B"}. This record does not assert independent reproduction. ### Base Rel Score: -1.59. Result ID: 69ddfa1247f34ce924c55ea5. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama3.1-8B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.