Murder Mystery Game Evaluation on Murder Mystery Games benchmark leaderboard · SOTA2 Research

Murder Mystery Game Evaluation on Murder Mystery Games

58.75MMRGPT-4V

Evaluation Results

Method	Links
GPT-4V2026.04		58.75	26.43	6.43	24.2	92.4	51.88	69.25
Ours (Full Model)2026.04		58.42	36.18	6.82	36.87	89.15	77.28	62.53
Claude2026.04		57.78	22.07	7.89	19.2	88.8	35.31	55.02
Gemini-1.5-Pro2026.04		57.39	19.2	7.22	16.9	88.8	57.78	57.84
Ours (Full Model)2026.04		55.01	34.25	6.35	35	87.4	74.56	61.09
w/o Image Clues Match2026.04		50.22	34.91	5.44	32.54	78.66	72.48	45.23
w/o Supervised Fine-tuning2026.04		50.12	29.36	5.68	35.48	83.27	71.02	61.42
Gemma3-12B-it2026.04		49.96	33.22	7.34	19.5	81.25	65.43	55.01
w/o Image Clues Match2026.04		48.75	33.25	5.15	31.25	77.13	71.01	44.05
w/o Supervised Fine-tuning2026.04		48.56	27.84	5.32	34.32	82.04	69.76	60.1
Gemma3-27B-it2026.04		48.28	26.91	7.61	15.83	76.34	64.24	56.42
w/o Reinforcement Learning2026.04		47.4	18.5	5.12	26.13	86.41	63.85	59.73
w/o Reinforcement Learning2026.04		45.83	17.02	4.76	24.62	85.27	61.97	58.88
Mistral-small3.1-24B2026.04		44.92	40.43	7.53	33.09	83.07	50.74	48.41
Gemma3-4B-it2026.04		40.53	18.21	7.16	17.09	34.7	23.5	22.38
Qwen2.5-VL-7B-Instruct (baseline)2026.04		37.63	30.7	7.11	25.61	83.97	40.63	38.74
LLaMA3.2-Vision-11B2026.04		33.71	26.39	3.95	11.69	57.2	24.13	22.6
Qwen2.5-VL-3B-Instruct (baseline)2026.04		30.92	23.93	4.69	20.14	72.88	35.09	32.16
LLaVA-13B2026.04		19.01	20.78	2.17	15.83	23.92	21.35	18.7