Strategic Game Playing on TextArena SPIN-Bench
54.9Win Rate (Simple Negotiation)MEMO
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| MEMOBase Model=GPT-4o-mini2026.03 | 54.9 | 52.4 | 55.6 | 42.7 | 41.8 | 49.5 | 6.4 | |
| MEMOBase Model=Qwen2.5-7B-Instruct2026.03 | 48 | 48.4 | 60 | 31.1 | 34 | 44.3 | 6.1 | |
| baselineBase Model=GPT-4o-mini2026.03 | 31.3 | 32.2 | 39.1 | 0.3 | 21.4 | 25.1 | 44.9 | |
| MEMOBase Model=Gemini-2.5-Flash2026.03 | 30 | 35 | 58 | 49 | 32 | 40.8 | — | |
| baselineBase Model=Qwen2.5-7B-Instruct2026.03 | 24 | 17.1 | 45.3 | 2.8 | 15.1 | 20.9 | 30.1 | |
| baselineBase Model=Gemini-2.5-Flash2026.03 | 14 | 15 | 50 | 32 | 26 | 27.4 | — |