List your company
ResearchFormerly WizWand

Theory of Mind on ToMi

100AccuracyUserHarness
1.7227.23552.7578.265May 19, 2026May 20, 2026May 21, 2026May 22, 2026May 23, 2026May 24, 2026May 26, 2026
Updated 5mo ago

Evaluation results

MethodLinks
UserHarnessBackbone=Qwen3-8B2026.05100
UserHarnessBackbone=Qwen3-14B2026.05100
UserHarnessBackbone=Qwen3-32B2026.05100
UserHarnessBackbone=Llama-3.1-8B2026.05100
UserHarnessBackbone=GPT-OSS-20B2026.05100
UserHarnessBackbone=GPT-OSS-120B2026.05100
UserHarnessBackbone=GPT-5.4-mini2026.05100
UserHarnessBackbone=GPT-5.42026.05100
UserHarnessBackbone=Gemini-3-Flash2026.05100
UserHarnessBackbone=Gemini-3.1-Pro2026.05100
UserHarnessBackbone=Claude-Sonnet-4.62026.05100
UserHarnessBackbone=Claude-Opus-4.72026.05100
SymbolicToMSetting=reported setting2026.0598.6
ExploreToMParams=8B, Inference Protocol=iterative A* heuristic search2026.0595
DITTOBackbone=Qwen3-VL-8B-Instruct2026.0593
Paradox promptingBackbone=GPT-5.42026.0589.1
AutoToMBackbone=GPT-4o2026.0588.3
GPT-5.42026.0588
Ledger promptingBackbone=GPT-5.42026.0585.72
GPT-5-nano2026.0585
Paradox promptingBackbone=GPT-5.4-mini2026.0583.97
GRPOBackbone=Qwen3-VL-8B-Instruct2026.0582
Direct promptingBackbone=GPT-OSS-120B2026.0580.22
SimToMSetting=reported setting2026.0579.9
Direct promptingBackbone=GPT-5.42026.0579.72
OSCToM-8BParams=8B, Inference Protocol=direct model inference2026.0579.5
Direct promptingBackbone=GPT-5.4-mini2026.0578.36
Ledger promptingBackbone=GPT-OSS-120B2026.0577.38
Phi-3-Medium-14BParams=14B, Inference Protocol=direct model inference2026.0576
Paradox promptingBackbone=GPT-OSS-120B2026.0575.05
Qwen2.5-32BParams=32B, Inference Protocol=direct model inference2026.0574.7
Ledger promptingBackbone=GPT-5.4-mini2026.0573.03
HumanLM-8B2026.0572
Direct promptingBackbone=GPT-OSS-20B2026.0570.08
Paradox promptingBackbone=GPT-OSS-20B2026.0569.56
Qwen3-VL-8B-InstructRole=Base2026.0568
Mistral-NeMo-12BParams=12B, Inference Protocol=direct model inference2026.0566.8
Llama-3.1-8B-BaseParams=8B, Inference Protocol=direct model inference2026.0564.3
Ledger promptingBackbone=GPT-OSS-20B2026.0562.37
BIP-ALMSetting=reported setting2026.0555.6
Direct promptingBackbone=Qwen3-8B2026.0554.52
Qwen2.5-14BParams=14B, Inference Protocol=direct model inference2026.0553.8
Gemma-2-27BParams=27B, Inference Protocol=direct model inference2026.0553.3
Direct promptingBackbone=Llama-3.1-8B2026.0552.48
Paradox promptingBackbone=Qwen3-32B2026.0550.38
Paradox promptingBackbone=Qwen3-14B2026.0548.96
Ledger promptingBackbone=Qwen3-32B2026.0547.34
LIMPSetting=reported setting2026.0544.6
Ledger promptingBackbone=Qwen3-8B2026.0526.44
Ledger promptingBackbone=Qwen3-14B2026.0525.28
Paradox promptingBackbone=Qwen3-8B2026.0517.32
Direct promptingBackbone=Qwen3-32B2026.0514.02
Ledger promptingBackbone=Llama-3.1-8B2026.0511.24
Direct promptingBackbone=Qwen3-14B2026.056.2
Paradox promptingBackbone=Llama-3.1-8B2026.055.5