List your company
ResearchFormerly WizWand

Theory of Mind reasoning on BigTOM False Belief

99.4AccuracyMeTHanol
28.78447.11765.4583.783Nov 16, 2023Jan 6, 2024Feb 26, 2024Apr 17, 2024Jun 7, 2024Jul 28, 2024Sep 18, 2024
Updated 6mo ago

Evaluation results

MethodLinks
MeTHanolBase=Llama3-8B2024.0999.4
gpt-4Prompting strategy=0-shot CoT2023.1193.25
gpt-4Prompting strategy=SIMTOM2023.1192
gpt-4Prompting strategy=0-Shot2023.1189
SimTomBase=GPT-42024.0987.8
SFTBase=Llama3-8B2024.0977.7
COTBase=GPT-42024.0974.4
directBase=Llama3-8B2024.0971.3
Llama2-7b-chatPrompting strategy=SIMTOM2023.1170.5
gpt-3.5-turboPrompting strategy=SIMTOM2023.1170.5
directBase=GPT-42024.0966.5
Llama2-13b-chatPrompting strategy=SIMTOM2023.1161.75
gpt-3.5-turboPrompting strategy=0-shot CoT2023.1156.25
Llama2-13b-chatPrompting strategy=0-shot CoT2023.1152.25
Llama2-7b-chatPrompting strategy=0-Shot2023.1147.5
Llama2-13b-chatPrompting strategy=0-Shot2023.1141.25
gpt-3.5-turboPrompting strategy=0-Shot2023.1141
Llama2-7b-chatPrompting strategy=0-shot CoT2023.1131.5