Social Interaction on SOTOPIA all social scenarios
8.95Goal ScoreGPT-5
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| GPT-52026.03 | 8.95 | — | — | |
| SAVOIREvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 8.43 | — | 3.85 | |
| GRPO iStarBackbone=Qwen2.5-7B-Instruct2026.03 | 8.42 | — | — | |
| Claude-3.5-SonnetEvaluation Setting=GPT-4o-as-Partner2026.04 | 8.42 | — | 3.77 | |
| SAVOIREvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 8.42 | — | 3.94 | |
| RAPOBackbone=Qwen2.5-7B-Instruct2026.03 | 8.41 | 3.87 | — | |
| EPOEvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 8.41 | — | 3.86 | |
| DSIEvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 8.35 | — | 3.75 | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.03 | 8.31 | — | — | |
| Sotopia-RLEvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 8.31 | — | 3.9 | |
| Claude-3.5-Sonnet2026.03 | 8.29 | 3.71 | — | |
| Claude-3.5-SonnetEvaluation Setting=Self-Play2026.04 | 8.29 | — | 3.71 | |
| PPOBackbone=Qwen2.5-7B-Instruct2026.03 | 8.25 | — | — | |
| GPT-4o2026.03 | 8.19 | 3.76 | — | |
| GPT-4oEvaluation Setting=Self-Play2026.04 | 8.19 | — | 3.76 | |
| GPT-4oEvaluation Setting=GPT-4o-as-Partner2026.04 | 8.19 | — | 3.76 | |
| DeepSeek-V3Evaluation Setting=Self-Play2026.04 | 8.15 | — | 3.62 | |
| DSIEvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 8.15 | — | 3.7 | |
| DeepSeek-V3Evaluation Setting=GPT-4o-as-Partner2026.04 | 8.14 | — | 3.72 | |
| CPOBackbone=Qwen2.5-7B-Instruct2026.03 | 8.13 | 3.77 | — | |
| Doubao-Pro-32k2026.03 | 8.12 | 3.7 | — | |
| Gemini-2.5-ProEvaluation Setting=GPT-4o-as-Partner2026.04 | 8.12 | — | 3.59 | |
| DATEvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 8.11 | — | 3.7 | |
| EPOEvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 8.09 | — | 3.51 | |
| OpenAI-o1Evaluation Setting=GPT-4o-as-Partner2026.04 | 8.09 | — | 3.69 | |
| PPDPPEvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 8.07 | — | 3.71 | |
| DeepSeek-R12026.03 | 7.97 | 3.4 | — | |
| DeepSeek-R1Evaluation Setting=Self-Play2026.04 | 7.97 | — | 3.4 | |
| PPDPPEvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 7.97 | — | 3.65 | |
| DATEvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 7.97 | — | 3.59 | |
| OpenAI-o3-miniEvaluation Setting=GPT-4o-as-Partner2026.04 | 7.96 | — | 3.61 | |
| OpenAI-o1Evaluation Setting=Self-Play2026.04 | 7.93 | — | 3.58 | |
| DeepSeek-R1Evaluation Setting=GPT-4o-as-Partner2026.04 | 7.92 | — | 3.49 | |
| Qwen2.5-7B-Instruct2026.03 | 7.91 | 3.55 | — | |
| Qwen2.5-7B-InstructEvaluation Setting=Self-Play2026.04 | 7.91 | — | 3.55 | |
| Gemini-2.5-ProEvaluation Setting=Self-Play2026.04 | 7.85 | — | 3.43 | |
| Sotopia-RLEvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 7.8 | — | 3.55 | |
| QwQ-32BEvaluation Setting=GPT-4o-as-Partner2026.04 | 7.8 | — | 3.47 | |
| QwQ-32BEvaluation Setting=Self-Play2026.04 | 7.7 | — | 3.3 | |
| Expert (GPT-4)Backbone=GPT-42024.03 | 7.62 | 3.31 | — | |
| BC+SRBackbone=Mistral-7B, Method=SOTOPIA-π2024.03 | 7.62 | 3.44 | — | |
| OpenAI-o3-miniEvaluation Setting=Self-Play2026.04 | 7.38 | — | 3.3 | |
| Behavior Cloning (BC)Backbone=Mistral-7B, Method=SOTOPIA-π2024.03 | 7.27 | 3.41 | — | |
| RLOOBackbone=Qwen2.5-7B-Instruct2026.03 | 7.13 | — | — | |
| Qwen2.5-7B-InstructEvaluation Setting=GPT-4o-as-Partner2026.04 | 6.71 | — | 3.13 | |
| Self-Reinforcement (SR)Backbone=Mistral-7B, Method=SOTOPIA-π2024.03 | 5.83 | 2.57 | — | |
| Base (Mistral-7B)Backbone=Mistral-7B2024.03 | 5.07 | 2.33 | — |