Social Interaction Evaluation on SOTOPIA (Self-Play)
9.08Goal ScoreAMPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AMPOBackbone=Llama3.1-8B-Instruct, Variant=BC+AMPO2025.05 | 9.08 | 3.95 | |
| AMPOBackbone=Qwen2.5-7B-Instruct, Variant=BC+AMPO2025.05 | 8.95 | 3.87 | |
| ASL w/ BC+GRPOBackbone=Qwen2.5-7B-Instruct, Variant=BC+GRPO2025.05 | 8.87 | 3.85 | |
| ASL w/ BC+GRPOBackbone=Llama3.1-8B-Instruct, Variant=BC+GRPO2025.05 | 8.86 | 3.84 | |
| ASL w/ BCBackbone=Llama3.1-8B-Instruct, Variant=Behavioral Cloning2025.05 | 8.43 | 3.76 | |
| Qwen2.5-7B-Instruct w/ DSIBackbone=Qwen2.5-7B-Instruct, Social Intelligence Method=DSI2025.05 | 8.35 | 3.75 | |
| Claude-3.5-SonnetModel Category=Proprietary LLMs2025.05 | 8.29 | 3.71 | |
| ASL w/ BCBackbone=Qwen2.5-7B-Instruct, Variant=Behavioral Cloning2025.05 | 8.22 | 3.67 | |
| GPT-4oModel Category=Proprietary LLMs2025.05 | 8.19 | 3.76 | |
| DeepSeek-V3Model Category=Proprietary LLMs2025.05 | 8.15 | 3.62 | |
| Qwen2.5-7B-Instruct w/ EPOBackbone=Qwen2.5-7B-Instruct, Social Intelligence Method=EPO2025.05 | 8.09 | 3.51 | |
| Llama3.1-8B-Instruct w/ DSIBackbone=Llama3.1-8B-Instruct, Social Intelligence Method=DSI2025.05 | 8.08 | 3.64 | |
| DeepSeek-R1Model Category=Large Reasoning Models2025.05 | 7.97 | 3.4 | |
| Qwen2.5-7B-Instruct w/ PPDPPBackbone=Qwen2.5-7B-Instruct, Social Intelligence Method=PPDPP2025.05 | 7.97 | 3.65 | |
| Qwen2.5-7B-Instruct w/ DATBackbone=Qwen2.5-7B-Instruct, Social Intelligence Method=DAT2025.05 | 7.97 | 3.59 | |
| Llama3.1-8B-Instruct w/ EPOBackbone=Llama3.1-8B-Instruct, Social Intelligence Method=EPO2025.05 | 7.94 | 3.78 | |
| OpenAI-o1Model Category=Large Reasoning Models2025.05 | 7.93 | 3.58 | |
| Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct2025.05 | 7.91 | 3.55 | |
| Gemini-2.5-ProModel Category=Large Reasoning Models2025.05 | 7.85 | 3.43 | |
| QwQ-32BModel Category=Large Reasoning Models2025.05 | 7.7 | 3.3 | |
| OpenAI-o3-miniModel Category=Large Reasoning Models2025.05 | 7.38 | 3.3 | |
| Llama3.1-8B-Instruct w/ DATBackbone=Llama3.1-8B-Instruct, Social Intelligence Method=DAT2025.05 | 7.34 | 3.44 | |
| Llama3.1-8B-Instruct w/ PPDPPBackbone=Llama3.1-8B-Instruct, Social Intelligence Method=PPDPP2025.05 | 7.2 | 3.4 | |
| Llama3.1-8B-InstructBackbone=Llama3.1-8B-Instruct2025.05 | 6.99 | 3.23 |