Multi-Agent Strategic Reasoning on ConnectFour OOD
99.93First-mover Normalized ScoreGPT-5-mini
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-5-miniModel Source=Closed-source, Opponent=MCTS Bot, Simulations=102026.05 | 99.93 | 97.95 | |
| GPT-5-miniModel Source=Closed-source, Opponent=MCTS Bot, Simulations=1002026.05 | 96.97 | 72.1 | |
| Gemini-2.5-flashModel Source=Closed-source, Opponent=MCTS Bot, Simulations=102026.05 | 78.74 | 82.7 | |
| Strat-Reasoner-4BModel Source=Open-source, Opponent=MCTS Bot, Simulations=1002026.05 | 75.93 | 69.26 | |
| MARSHAL-4BModel Source=Open-source, Opponent=MCTS Bot, Simulations=1002026.05 | 70.67 | 67 | |
| Strat-Reasoner-4BModel Source=Open-source, Opponent=MCTS Bot, Simulations=102026.05 | 70.39 | 70.26 | |
| SPIRAL-4BModel Source=Open-source, Opponent=MCTS Bot, Simulations=1002026.05 | 68.71 | 59.24 | |
| SPIRAL-4BModel Source=Open-source, Opponent=MCTS Bot, Simulations=102026.05 | 68 | 60.85 | |
| MARSHAL-4BModel Source=Open-source, Opponent=MCTS Bot, Simulations=102026.05 | 66.98 | 69.94 | |
| Gemini-2.5-flashModel Source=Closed-source, Opponent=MCTS Bot, Simulations=1002026.05 | 64.9 | 66.14 | |
| Qwen3-8BModel Source=Open-source, Opponent=MCTS Bot, Simulations=102026.05 | 64.1 | 62.34 | |
| Qwen3-4BModel Source=Open-source, Opponent=MCTS Bot, Simulations=102026.05 | 63.2 | 57.01 | |
| Qwen3-4BModel Source=Open-source, Opponent=MCTS Bot, Simulations=1002026.05 | 62.77 | 58.39 | |
| Gemma3-12BModel Source=Open-source, Opponent=MCTS Bot, Simulations=1002026.05 | 61.14 | 58.66 | |
| Qwen3-32BModel Source=Open-source, Opponent=MCTS Bot, Simulations=1002026.05 | 58.11 | 58.12 | |
| Qwen3-8BModel Source=Open-source, Opponent=MCTS Bot, Simulations=1002026.05 | 58 | 64.88 | |
| Qwen3-32BModel Source=Open-source, Opponent=MCTS Bot, Simulations=102026.05 | 56.62 | 63.63 | |
| Gemma3-12BModel Source=Open-source, Opponent=MCTS Bot, Simulations=102026.05 | 50.5 | 57.01 |