Theory of Mind reasoning on BigTOM (All)
95.5Accuracygpt-4
Evaluation Results
| Method | Links | |
|---|---|---|
| gpt-4Prompting strategy=0-shot CoT2023.11 | 95.5 | |
| gpt-4Prompting strategy=SIMTOM2023.11 | 95 | |
| gpt-4Prompting strategy=0-Shot2023.11 | 92.5 | |
| gpt-4version=0125-preview2025.06 | 85.2 | |
| gpt-4version=1106-preview2025.06 | 84.8 | |
| PUGCobjective=DPO2025.06 | 84.4 | |
| gpt-4-turboversion=2024-04-092025.06 | 82.4 | |
| Claude-3-Opus2025.06 | 81.8 | |
| gpt-3.5-turboPrompting strategy=SIMTOM2023.11 | 81.62 | |
| gpt-4oversion=2024-05-132025.06 | 81.6 | |
| LLama-3-70B-Instructparameters=70B, type=Instruct2025.06 | 80.8 | |
| PUGCobjective=SimPO2025.06 | 80.6 | |
| Meta-Llama-3-8B-Instructparameters=8B, type=Instruct2025.06 | 76.6 | |
| Mixtral-8x7B-Instruct-v0.1parameters=8x7B, type=Instruct, version=v0.12025.06 | 76.2 | |
| gpt-3.5-turboPrompting strategy=0-shot CoT2023.11 | 75.88 | |
| Mistral-7B-Instruct-v0.2parameters=7B, type=Instruct, version=v0.22025.06 | 75.4 | |
| Nous-Hermes-Mistral-7B-DPObase_model=Mistral-7B, objective=DPO2025.06 | 73.8 | |
| gpt-3.5-turboPrompting strategy=0-Shot2023.11 | 66.38 | |
| Llama2-13b-chatPrompting strategy=SIMTOM2023.11 | 58 | |
| Llama2-7b-chatPrompting strategy=SIMTOM2023.11 | 57.25 | |
| Llama2-13b-chatPrompting strategy=0-shot CoT2023.11 | 56 | |
| Llama2-7b-chatPrompting strategy=0-Shot2023.11 | 53.62 | |
| Llama2-13b-chatPrompting strategy=0-Shot2023.11 | 51.38 | |
| Llama2-7b-chatPrompting strategy=0-shot CoT2023.11 | 48.62 |