Social Interaction on SOTOPIA Hard
7.93GOAL ScoreSAVOIR
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| SAVOIREvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 7.93 | 3.97 | — | — | |
| Sotopia-RLEvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 7.81 | 3.8 | — | — | |
| DSIEvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 7.31 | 3.51 | — | — | |
| SAVOIREvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 7.18 | 3.51 | — | — | |
| ALSO2026.05 | 7.11 | 3.53 | 2.43 | 5.47 | |
| GPT-4oEvaluation Setting=Self-Play2026.04 | 6.97 | 3.46 | — | — | |
| GPT-4oEvaluation Setting=GPT-4o-as-Partner2026.04 | 6.97 | 3.46 | — | — | |
| INSTINCT2026.05 | 6.92 | 3.43 | 2.16 | 5.44 | |
| DSIEvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 6.87 | 3.42 | — | — | |
| EPOEvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 6.82 | 3.12 | — | — | |
| EPOEvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 6.81 | 3.51 | — | — | |
| DATEvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 6.78 | 3.36 | — | — | |
| EvoPrompt2026.05 | 6.77 | 3.29 | 1.93 | 5.15 | |
| PPDPPEvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 6.76 | 3.35 | — | — | |
| OPRO2026.05 | 6.71 | 3.24 | 1.89 | 4.63 | |
| Gemini-2.5-ProEvaluation Setting=GPT-4o-as-Partner2026.04 | 6.7 | 3.09 | — | — | |
| DeepSeek-V3Evaluation Setting=GPT-4o-as-Partner2026.04 | 6.69 | 3.31 | — | — | |
| Sotopia-RLEvaluation Setting=GPT-4o-as-Partner, Base Model=Qwen2.5-7B-Instruct2026.04 | 6.68 | 3.29 | — | — | |
| OpenAI-o1Evaluation Setting=GPT-4o-as-Partner2026.04 | 6.65 | 3.2 | — | — | |
| Claude-3.5-SonnetEvaluation Setting=GPT-4o-as-Partner2026.04 | 6.64 | 3.3 | — | — | |
| PPDPPEvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 6.63 | 3.31 | — | — | |
| Vanilla2026.05 | 6.52 | 3.02 | 1.32 | 4.37 | |
| DATEvaluation Setting=Self-Play, Base Model=Qwen2.5-7B-Instruct2026.04 | 6.39 | 3.1 | — | — | |
| DeepSeek-V3Evaluation Setting=Self-Play2026.04 | 6.34 | 3.09 | — | — | |
| Claude-3.5-SonnetEvaluation Setting=Self-Play2026.04 | 6.33 | 3.09 | — | — | |
| OpenAI-o3-miniEvaluation Setting=GPT-4o-as-Partner2026.04 | 6.33 | 2.98 | — | — | |
| Qwen2.5-7B-InstructEvaluation Setting=Self-Play2026.04 | 6.21 | 3.01 | — | — | |
| DeepSeek-R1Evaluation Setting=GPT-4o-as-Partner2026.04 | 6.2 | 2.95 | — | — | |
| QwQ-32BEvaluation Setting=GPT-4o-as-Partner2026.04 | 6.19 | 2.91 | — | — | |
| Qwen2.5-7B-InstructEvaluation Setting=GPT-4o-as-Partner2026.04 | 5.9 | 2.9 | — | — | |
| DeepSeek-R1Evaluation Setting=Self-Play2026.04 | 5.86 | 2.73 | — | — | |
| OpenAI-o1Evaluation Setting=Self-Play2026.04 | 5.69 | 2.71 | — | — | |
| Gemini-2.5-ProEvaluation Setting=Self-Play2026.04 | 5.67 | 2.55 | — | — | |
| QwQ-32BEvaluation Setting=Self-Play2026.04 | 5.35 | 2.41 | — | — | |
| OpenAI-o3-miniEvaluation Setting=Self-Play2026.04 | 5.14 | 2.36 | — | — |