Mental State Inference on MMToM-QA human 1.0 (test) benchmark leaderboard · SOTA2 Research

Mental State Inference on MMToM-QA human 1.0 (test)

100Sub-score 1.1GPT-4

Evaluation Results

Method	Links
GPT-42024.01		100	50.9	65.3	36.7	31.5	64	40	40	13.3	53.3
SimToM2024.01		100	56.7	70	43.4	31.6	80	46.7	40	20	66.7
SymbolicToM2024.01		100	53.3	70	36.7	78.9	44	40	73.3	0	33.3
BIP-ALM2024.01		93.8	76.6	78.1	75	52.6	88	73.3	86.7	73.3	66.7
GPT-3.52024.01		93.8	45.7	58.1	33.3	52.6	28	53.3	0	33.3	46.7
GPT-4V2024.01		93.4	48.9	52.7	45	36.8	28	33.3	73.3	13.3	60
BIP-ALM2024.01		87.5	77	85.7	68.3	73.7	96	60	86.7	60	66.7
LLaMA 22024.01		75	53.6	57.2	50	52.6	44	46.7	40	60	53.3
BIP-ALM2024.01		75	62.6	71.9	53.3	52.6	88	40	73.3	33.3	66.7
BIP-ALM2024.01		75	67.9	75.9	60	52.6	100	66.7	80	53.3	40
BIP-ALM2024.01		75	63.1	69.6	56.7	57.9	76	60	53.3	60	53.3
GPT-J2024.01		62.5	45.6	49.5	41.7	42.1	44	33.3	53.3	26.7	53.3
BIP-ALM2024.01		62.5	59.2	60.1	58.3	57.9	60	46.7	53.3	73.3	60
GPT-4V2024.01		56.3	44.8	41.2	48.3	47.4	20	33.3	40	60	60
Video-LLaMA 22024.01		50	36.9	40.5	33.3	47.4	24	33.3	33.3	20	46.7
InstructBLIP2024.01		43.8	49.1	49.9	48.3	57.9	48	53.3	46.7	40	53.3
InstructBLIP2024.01		43.8	49.1	48.1	50	52.6	48	46.7	46.7	46.7	60
Video-LLaMA 22024.01		31.3	45.2	42	48.3	26.8	68	53.3	60	40	40
LLaVA2024.01		24	50.9	60.1	41.7	56.3	100	40	33.3	40	53.3
LLaVA2024.01		4	39.6	40.9	38.3	18.8	100	46.7	26.7	40	40