List your company
ResearchFormerly WizWand

Theory of Mind on BigToM

99.5AccuracyRECTOM
15.2637.135980.87May 19, 2026May 22, 2026May 26, 2026May 30, 2026Jun 2, 2026Jun 6, 2026Jun 10, 2026
Updated 4mo ago

Evaluation results

MethodLinks
RECTOMModel=GPT-5.42026.0699.5
CoTModel=GPT-5.42026.0699
SIMTOMModel=GPT-5.42026.0699
RECTOMModel=Gemini-32026.0699
RECTOMModel=Gemma-42026.0699
CoTModel=Gemini-32026.0698.75
UserHarnessBackbone=Claude-Opus-4.72026.0598.67
RECTOMModel=Qwen3.52026.0698.5
Ledger promptingBackbone=GPT-5.42026.0598.25
UserHarnessBackbone=Gemini-3.1-Pro2026.0598.25
UserHarnessBackbone=Claude-Sonnet-4.62026.0598.17
SIMTOMModel=Gemini-32026.0698
TIMETOMModel=GPT-5.42026.0697.75
UserHarnessBackbone=GPT-5.42026.0597.67
Ledger promptingBackbone=Qwen3-14B2026.0597.25
Ledger promptingBackbone=GPT-OSS-120B2026.0597.08
Ledger promptingBackbone=GPT-5.4-mini2026.0596.67
Ledger promptingBackbone=Qwen3-32B2026.0596.5
CoTModel=Qwen3.52026.0696.25
Ledger promptingBackbone=GPT-OSS-20B2026.0596.17
UserHarnessBackbone=Qwen3-32B2026.0595.42
UserHarnessBackbone=GPT-5.4-mini2026.0595.25
SIMTOMModel=Qwen3.52026.0694.75
UserHarnessBackbone=Qwen3-14B2026.0594.17
Paradox promptingBackbone=GPT-5.42026.0593.83
UserHarnessBackbone=GPT-OSS-120B2026.0593.83
Ledger promptingBackbone=Qwen3-8B2026.0593.5
CoTModel=Gemma-42026.0693.25
TIMETOMModel=Gemini-32026.0692.5
Ledger promptingBackbone=Llama-3.1-8B2026.0592.17
Direct promptingBackbone=GPT-OSS-120B2026.0592
Paradox promptingBackbone=GPT-OSS-120B2026.0591.83
TIMETOMModel=Qwen3.52026.0691
Mistral-NeMo-12BParams=12B, Inference Protocol=direct model inference2026.0590.5
UserHarnessBackbone=GPT-OSS-20B2026.0590.17
OSCToM-8BParams=8B, Inference Protocol=direct model inference2026.0589.8
UserHarnessBackbone=Qwen3-8B2026.0589.67
Direct promptingBackbone=Qwen3-32B2026.0588.5
SIMTOMModel=Gemma-42026.0688
Paradox promptingBackbone=GPT-OSS-20B2026.0587.83
Paradox promptingBackbone=Qwen3-32B2026.0587.33
Direct promptingBackbone=GPT-5.42026.0586.92
AutoToMBackbone=GPT-4o2026.0586.92
Direct promptingBackbone=GPT-OSS-20B2026.0586.75
UserHarnessBackbone=Gemini-3-Flash2026.0586.25
Llama-3.1-8B-BaseParams=8B, Inference Protocol=direct model inference2026.0586
Phi-3-Medium-14BParams=14B, Inference Protocol=direct model inference2026.0585.5
UserHarnessBackbone=Llama-3.1-8B2026.0582.83
ExploreToMParams=8B, Inference Protocol=iterative A* heuristic search2026.0581
TIMETOMModel=Gemma-42026.0680
Paradox promptingBackbone=GPT-5.4-mini2026.0577.92
SimToMSetting=reported setting2026.0577.5
Paradox promptingBackbone=Qwen3-14B2026.0577.08
Direct promptingBackbone=GPT-5.4-mini2026.0576.42
Paradox promptingBackbone=Qwen3-8B2026.0576.42
Direct promptingBackbone=Llama-3.1-8B2026.0574.42
Paradox promptingBackbone=Llama-3.1-8B2026.0574
Direct promptingBackbone=Qwen3-14B2026.0573.5
Direct promptingBackbone=Qwen3-8B2026.0572.33
LIMPSetting=reported setting2026.0561.67
BIP-ALMSetting=reported setting2026.0550.33
Qwen2.5-32BParams=32B, Inference Protocol=direct model inference2026.0547.5
Qwen2.5-14BParams=14B, Inference Protocol=direct model inference2026.0529.5
Gemma-2-27BParams=27B, Inference Protocol=direct model inference2026.0518.5