SOTOPIA
Benchmarks
TaskDatasetSOTA resultTrendResults
SOTOPIA all social scenarios
8.95Goal Score
47
SOTOPIA Hard
7.93GOAL Score
35
SOTOPIA Interaction with GPT-4o
8.21Goal Score
28
SOTOPIA Self-Chat
8.56GOAL
28
SOTOPIA-Hard GPT-4o-as-Partner
7.68Goal Score
24
SOTOPIA GPT-4o-as-Partner
8.75Goal Score
24
SOTOPIA-Hard (Self-Play)
8.06GOAL Score
24
SOTOPIA (Self-Play)
9.08Goal Score
24
Sotopia hard
2.4Rel Score
17
SOTOPIA hard episodes (test)
7.21GOA Score
16
Sotopia (all)
2.73Rel Score
15
SOTOPIA Overall (AVG)
5.63AVG Score
11
SOTOPIA Interaction with GPT-4o-mini
7.53GOAL Score
11
Sotopia Hard
49.2Primary Metric
9
Sotopia hard (test)
7.59Goal Score
8
Sotopia standard (test)
8.92Goal Score
8
Sotopia
47Primary Metric
6
SOTOPIA ALL
8.51Goal Score
5
Sotopia subset
7.85Goal Score
3
SOTOPIA AURATown
3.71Human Rating Score
2
SOTOPIA AURATown
3.75Mean Human Rating
2
SOTOPIA AURATown
3.82Human Rating Score
2
SOTOPIA AURATown
3.83Average Human Rating
2
SOTOPIA
9Believability Score
1