List your company
ResearchFormerly WizWand

Theory of Mind reasoning on ToMI False Belief

98.2AccuracyMeTHanol
13.23235.29157.3579.409Nov 16, 2023Jan 6, 2024Feb 26, 2024Apr 17, 2024Jun 7, 2024Jul 28, 2024Sep 18, 2024
Updated 6mo ago

Evaluation results

MethodLinks
MeTHanolBase=Llama3-8B2024.0998.2
COTBase=GPT-42024.0995.5
SimTomBase=GPT-42024.0995
directBase=GPT-42024.0992.5
gpt-4Prompting strategy=SIMTOM2023.1187.75
gpt-3.5-turboPrompting strategy=SIMTOM2023.1181
gpt-4Prompting strategy=0-shot CoT2023.1174.25
gpt-3.5-turboPrompting strategy=0-Shot2023.1167.25
SFTBase=Llama3-8B2024.0943.2
Llama2-7b-chatPrompting strategy=SIMTOM2023.1140
Llama2-13b-chatPrompting strategy=0-Shot2023.1139.25
Llama2-13b-chatPrompting strategy=SIMTOM2023.1135.5
gpt-3.5-turboPrompting strategy=0-shot CoT2023.1134
Llama2-7b-chatPrompting strategy=0-Shot2023.1128.25
gpt-4Prompting strategy=0-Shot2023.1125.5
Llama2-7b-chatPrompting strategy=0-shot CoT2023.1124
directBase=Llama3-8B2024.0922.2
Llama2-13b-chatPrompting strategy=0-shot CoT2023.1116.5