Multi-level Strategic Reasoning on 2-player Beauty Contest Game (BCG) (test)
35.57Mean GuessHuman Students
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Human StudentsGroup=Humans2025.02 | 35.57 | 0.84 | — | — | |
| Claude 3.5 HaikuPersona=Default2025.02 | 29.7 | 1.28 | 2.17 | 2 | |
| Claude 3.5 HaikuPersona=Student2025.02 | 28.9 | 1.35 | 1.97 | 2 | |
| Human ProfessionalsGroup=Humans2025.02 | 27.73 | 1.45 | — | — | |
| Claude 3.5 HaikuPersona=Professional2025.02 | 25.2 | 1.69 | 2.5 | 2.5 | |
| Claude 3.5 SonnetPersona=Default2025.02 | 22.9 | 1.92 | 2.17 | 2 | |
| GPT-4o-miniPersona=Default2025.02 | 15.3 | 2.92 | 2.57 | 3 | |
| GPT-4oPersona=Student2025.02 | 14 | 3.14 | 1.8 | 2 | |
| GPT-4oPersona=Default2025.02 | 11.2 | 3.69 | 2.27 | 2 | |
| Self-tuning EWAType=Non-LLM baseline2025.02 | 11.2 | 3.69 | — | — | |
| GPT-4oPersona=Professional2025.02 | 10.5 | 3.85 | 2.4 | 3 |