Medical Decision-Making on MMLUPH (test) benchmark leaderboard · SOTA2 Research

Medical Decision-Making on MMLUPH (test)

78.48AccuracyMAC (Ours w/ GPTs)

Evaluation Results

Method
MAC (Ours w/ GPTs)2025.07	78.48	78.47	78.72	97.6	76.05
Gemini 2.5 Flash2025.07	77.34	77.34	77.55	97.47	74.77
GPT-5.12025.07	71.35	71.45	72.25	96.79	68.09
Masked Agent Collaboration (MAC)2025.07	71.35	71.45	72.25	96.79	68.09
Phi4 14B2025.07	70.46	70.4	71.17	96.69	67.07
Claude Haiku 4.52025.07	68.77	68.74	68.99	96.62	65.18
Debate2025.07	67.93	67.95	69.49	96.4	64.33
Openthinker 32B2025.07	67.81	67.91	68.49	96.52	64.14
Qwen2.5 32B2025.07	67.77	67.8	68.08	96.4	64.08
QWQ 32B2025.07	67.62	67.76	68.37	96.5	63.94
Llama3 instruct 70B2025.07	66.06	66.02	66.3	96.22	62.18
MoA2025.07	65.31	65.57	66.98	96.12	61.38
Deepseek-R1 32B2025.07	62.71	63.29	65.01	95.99	58.64
Qwen2.5 14B2025.07	62	62	62.4	95.75	57.62
WizardLM 8x22 141B2025.07	60.68	61.19	64.11	95.61	56.57
SelfMoA2025.07	60.19	60.86	63.34	95.56	55.99
Mixtral 8x22 141B2025.07	59.05	59.1	60.31	95.43	54.39
Qwen1.5 Chat 110B2025.07	45.36	48.73	61.13	93.94	42.06
Med42 8B2025.07	44.84	45.44	48.82	94.03	38.82
dbrx-instruct 132B2025.07	44.42	46.14	51.93	93.83	39.11
Medllama2 7B2025.07	38.96	41.45	53.08	93.4	34.44
HuatuoGPT-o1 8B2025.07	32.49	32.91	44.03	92.68	27
Qwen1.5 Chat 72B2025.07	15.29	13.19	48.32	90.64	12.05