Multi-turn Strategic Gameplay on Multi-turn Game Suite
54.9SimpleNegotiation Win RateMEMO
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| MEMOBackbone=GPT-4o-mini, Type=OURS2026.03 | 54.9 | 52.4 | 55.6 | 42.7 | 41.8 | 49.5 | 6.4 | |
| MEMOBackbone=Qwen2.5-7B-Instruct, Type=OURS2026.03 | 48 | 48.4 | 60 | 31.1 | 34 | 44.3 | 6.1 | |
| SPIRALBackbone=Qwen2.5-7B-Instruct, Type=RL2026.03 | 45.7 | — | 56.7 | — | 32.7 | — | — | |
| MIPROBackbone=Qwen2.5-7B-Instruct, Type=PROMPT2026.03 | 42.4 | 47.5 | 53.8 | 2.2 | 20.9 | 33.4 | 7.3 | |
| TextGradBackbone=GPT-4o-mini, Type=PROMPT2026.03 | 42 | 44.6 | 55.6 | 7.1 | 23.6 | 34.6 | 18.4 | |
| UnstableBaselineBackbone=Qwen2.5-7B-Instruct, Type=RL2026.03 | 41.1 | 30.4 | 52.7 | 53.3 | 47.3 | 45 | 43.3 | |
| MIPROBackbone=GPT-4o-mini, Type=PROMPT2026.03 | 38.4 | 50.9 | 55.1 | 19.7 | 19.1 | 36.7 | 12.4 | |
| TextGradBackbone=Qwen2.5-7B-Instruct, Type=PROMPT2026.03 | 37.1 | 29.3 | 52.8 | 7.1 | 22.4 | 29.9 | 21.7 | |
| GEPABackbone=GPT-4o-mini, Type=PROMPT2026.03 | 36.8 | 40.4 | 52.2 | 3.3 | 26.9 | 32 | 11.3 | |
| GEPABackbone=Qwen2.5-7B-Instruct, Type=PROMPT2026.03 | 34.4 | 31.7 | 55.8 | 3.3 | 19.3 | 28.8 | 14.8 | |
| baselineBackbone=GPT-4o-mini, Type=STATIC2026.03 | 31.3 | 32.2 | 39.1 | 0.3 | 21.4 | 25.1 | 44.9 | |
| CoTBackbone=GPT-4o-mini, Type=STATIC2026.03 | 27.8 | 25.7 | 46.5 | 30.4 | 24.8 | 31.1 | 28.7 | |
| ToTBackbone=Qwen2.5-7B-Instruct, Type=STATIC2026.03 | 27.1 | 20.7 | 42.2 | 22.7 | 15.1 | 25.6 | 40.2 | |
| ToTBackbone=GPT-4o-mini, Type=STATIC2026.03 | 26.3 | 27 | 51.7 | 45.1 | 23.8 | 34.8 | 36.5 | |
| baselineBackbone=Qwen2.5-7B-Instruct, Type=STATIC2026.03 | 24 | 17.1 | 45.3 | 2.8 | 15.1 | 20.9 | 30.1 | |
| CoTBackbone=Qwen2.5-7B-Instruct, Type=STATIC2026.03 | 23.8 | 18.7 | 42 | 25.8 | 13.6 | 24.8 | 43.4 |