Theory of Mind reasoning on MMToM-QA Text-only

1Belief Inference 1.1SymbolicToM

Evaluation results

Method
SymbolicToM2024.01	1	0.61	0.74	0.783	0.733	0.667	0	0.507	0.477	0.63
SymbolicToM w/ GPT-42025.06	1	0.61	0.74	0.783	0.733	0.667	0	0.507	0.477	0.63
GPT-42024.01	0.97	0.12	0.77	0.62	0.48	0.427	0.027	0.427	0.34	0.48
GPT-42025.06	0.97	0.12	0.77	0.62	0.48	0.427	0.027	0.427	0.34	0.48
Human2024.01	0.96	0.958	0.813	0.91	0.858	0.767	0.65	0.683	0.74	0.825
SimToM2024.01	0.96	0.15	0.82	0.643	0.613	0.44	0.027	0.547	0.407	0.525
Human2025.06	0.96	0.958	0.813	0.91	0.858	0.767	0.65	0.683	0.74	0.825
SimToM w/ GPT-42025.06	0.96	0.15	0.82	0.643	0.613	0.44	0.027	0.547	0.407	0.525
DeepSeek-R1-671B2025.06	0.92	0.502	0.724	0.715	0.683	0.44	0.45	0.482	0.514	0.615
Scalable Bayesian ToM Planner2025.06	0.901	0.705	0.874	0.827	0.688	0.755	0.753	0.718	0.729	0.778
BIP-ALM2024.01	0.89	0.68	0.9	0.823	0.547	0.667	0.507	0.627	0.587	0.705
BIP-ALM2024.01	0.88	0.69	0.88	0.817	0.773	0.68	0.307	0.707	0.617	0.717
BIPALM w/ GPT-J-6B2025.06	0.88	0.69	0.88	0.817	0.773	0.68	0.307	0.707	0.617	0.717
OpenAI-o3-mini2025.06	0.831	0.476	0.623	0.643	0.64	0.386	0.387	0.46	0.468	0.556
GPT-3.52024.01	0.81	0.11	0.39	0.437	0.467	0.16	0.213	0.48	0.33	0.383
LLAMA 22024.01	0.64	0.55	0.5	0.563	0.493	0.48	0.413	0.387	0.443	0.503
GPT-J2024.01	0.56	0.53	0.38	0.49	0.52	0.507	0.507	0.56	0.523	0.507