Multi-Player Social Reasoning and Strategy on Multi-Player Game Categories Avalon, Diplomacy
65Avalon Win RateGPT-5.4
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-5.42026.04 | 65 | 4.7 | |
| GEMINI-3.1-PRO2026.04 | 42 | 2.72 | |
| CLAUDE-4.6-SONNET2026.04 | 40 | 3.16 | |
| GPT-OSS-120B2026.04 | 40 | 2.46 | |
| COS-PLAYBase Model=Qwen3-8B2026.04 | 39 | 2.96 | |
| GRPOBase Model=Qwen3-8B, Skill Bank Configuration=W/O Skill2026.04 | 36.2 | 2.76 | |
| GRPOBase Model=Qwen3-8B, Skill Bank Configuration=1ST Skill2026.04 | 31.2 | 2.56 | |
| QWEN3-8BTraining Protocol=Base2026.04 | 30 | 2.64 | |
| SFTBase Model=Qwen3-8B, Skill Bank Configuration=W/O Skill2026.04 | 28.7 | 2.75 | |
| SFTBase Model=Qwen3-8B, Skill Bank Configuration=Final Skill2026.04 | 25 | 2.65 | |
| SFTBase Model=Qwen3-8B, Skill Bank Configuration=1ST Skill2026.04 | 21.2 | 2.89 |