Multilingual Multiple-Choice Question Answering on HeadQA 1.0 (test)

88.88Chinese Acco4-mini

Evaluation Results

Method
o4-mini2026.01	88.88	90.28	90.22	89.97	89.48	87.08	90.77	90.5	86.74	47.23	76.07	82.47	84.14
Gemini-2.5-Flash2026.01	88.83	88.11	90.35	89.6	88.84	88.96	90.27	90.61	88.23	78.26	80.43	83.53	87.17
GPT-4.12026.01	88.57	90.05	90.19	89.59	89.51	86.14	90.52	89.28	86.67	58.72	67.49	78.58	83.78
GPT-52026.01	88.3	90.44	89.99	89.61	89.11	86.01	90.41	90.86	86.71	57.47	71.58	79.55	84.17
DeepSeek-R12026.01	88.28	89.64	89.63	89.25	87.85	86.69	90.1	89.85	84.13	65.64	69.79	77.36	84.02
Claude-4.0-Sonnet2026.01	87.78	89.56	89.97	89.32	88.42	87.83	90.32	90.72	80.47	55.56	70.72	76.12	83.07
LLaMA-4-Maverick2026.01	87.73	89.55	90.02	89.57	87.02	86	89.98	90.05	84.15	54.47	69.27	77.79	82.97
DeepSeek-V32026.01	87.64	89.67	88.59	88.74	87.44	84.1	88.48	89.02	77.07	51.05	58.45	64.69	79.58
GPT-4o2026.01	86.96	87.87	88.95	88.29	87.69	85.07	88.81	89.18	83.97	37.49	61.44	73.68	79.95
gpt-oss-120B2026.01	86.91	88	88.48	88.13	86.81	85.46	88.13	88.26	77.86	52.62	68.26	76.32	81.27
GPT-4.1-mini2026.01	86.13	88.59	87.75	87.61	86.53	83.05	88.89	88.41	77.55	27.2	61.77	72.59	78.01
GPT-5-mini2026.01	85.53	88.36	87.9	87.37	86.53	82.71	88.22	88.34	77.87	33.48	59.4	73.02	78.23
QwQ-32B2026.01	85.03	86.24	86.57	85.38	82.29	79.6	86.68	86	62.21	23.75	29.49	29.87	68.59
Baichuan-M2-32B2026.01	84.76	86.1	83.98	82.91	81.52	78.42	83.88	83.72	34.33	25.98	33.35	29.88	65.74
HuatuoGPT-o1-72B2026.01	84.34	85.54	86.2	84.46	83.03	78.55	86.84	86.85	62.18	44.44	44.91	29.51	71.41
LLaMA-4-Scout2026.01	84.12	87.1	86.31	86.83	84.23	82.29	86.43	87	77.98	50.25	50.64	69.34	77.71
Qwen3-14B-thinking2026.01	83.59	86.03	84.71	84.81	83.42	80.83	84.66	85.02	64.36	17.63	24.89	27.53	67.29
Qwen2.5-72B2026.01	83.21	85.62	84.48	84.17	81.56	78.96	85.1	84.97	54.45	43.9	41.74	43.98	71.01
gpt-oss-20B2026.01	82.64	78.47	84.91	84.63	82.95	81.27	85.03	84.74	68.96	33.72	61.98	69.13	74.87
MedGemma-27B2026.01	82.2	86.11	84.85	84.64	81.91	80.72	85.48	85.96	77.75	18.04	50.94	65.88	73.71
Qwen3-8B-thinking2026.01	81.15	83.88	82.17	82.52	79.96	78.42	81.9	82.18	36.87	14.73	17.74	12.06	61.13
Qwen3-14B2026.01	80.88	83.93	80.96	80.53	76.98	73.88	80.59	80.92	46.86	15.01	24.17	27.17	62.66
Phi-4-Reasoning2026.01	79.89	85.5	84.47	84.67	83.64	79.93	78.34	85.09	71.75	17.47	48.91	37.77	69.79
DeepSeek-R1-Qwen3-8B2026.01	79.34	82.46	77.59	77.49	75.66	73.91	78.48	78.63	36.65	11.08	10.83	13.17	57.94
LLaMA-3.1-70B2026.01	78.36	85.67	84.26	83.43	77.57	67.09	83.76	84.28	70.4	41.28	45.09	49.39	70.88
Qwen2.5-14B2026.01	77.17	80.3	76.86	75.79	72.86	68.43	75.99	75.73	37.04	26.76	35.73	35.38	61.5
Claude-3.5-Haiku2026.01	76.96	83.14	82.79	81.82	79.36	74.99	83.34	83.04	59.26	19.45	42.53	42.95	67.47
Qwen3-8B2026.01	76.95	80.75	75.54	76.37	72.29	67.65	75.36	75.22	31.28	10.53	22.73	11.42	56.34
GPT-4o-mini2026.01	76.93	83.54	82.68	81.97	78.1	72.34	82.37	82.18	69.38	35.69	48.67	58.1	71
Gemma-3-27B2026.01	76.73	82.14	80.95	80.88	77.86	76.83	80.71	81.34	71.33	23.93	52.41	62.27	70.62
Qwen3-4B-thinking2026.01	76.64	80.4	78.85	78.31	75.53	72.08	77.34	78.18	29.54	17.14	13.21	8.99	57.18
GPT-4.1-nano2026.01	76.59	84.59	81.86	80.96	74.56	72.63	82.2	81.2	59.13	33.51	44.72	53.54	68.79
Mistral-Small-3.1-24B2026.01	76.22	83.46	80.23	80.87	74.67	68.63	81.02	81.13	38.85	11.82	13.73	20.57	59.27
HuatuoGPT-o1-7B2026.01	73.8	69.96	72.11	71.8	65.12	62.83	72.32	70.88	7.32	7.04	7.19	6.26	48.89
Qwen3-4B2026.01	72.91	76.47	70.4	70.43	66.57	61.1	70	69.67	19.33	14.16	16.25	9.66	51.41
HuatuoGPT-o1-70B2026.01	72.87	85.19	84.22	84.27	72.69	77.43	84.77	84.83	74.9	43.44	50.15	56.66	72.62
Qwen2.5-7B2026.01	71.29	74.45	68.09	65.88	63.1	57.78	66.76	66.51	32.25	9.13	24.63	18.55	51.54
Gemma-3-12B2026.01	70.8	78.31	75.85	75.65	71.93	69.44	75.22	75.47	62.87	11.15	43.19	54.35	63.69
Phi-42026.01	70.17	84.6	81.31	81.12	72.99	66.18	80.93	80.43	52.61	23.43	38.22	34.67	63.89
GPT-5-nano2026.01	67.63	80.66	78.25	77.47	74.14	66.74	75.31	77.05	57.85	17.02	41.53	50.74	63.7
HuatuoGPT-o1-8B2026.01	64.48	73.33	70.71	70.5	62.24	54.89	69.3	69.15	50.34	9.62	9.04	8.46	51.01
LLaMA-3.3-70B2026.01	61.85	86.28	85.89	84.91	65.94	76.32	85.67	85.93	72.72	41.81	47.93	51.22	70.54
Qwen2.5-3B2026.01	61.05	66.95	58.48	55.81	50.25	44.89	56.09	55.37	16.09	9.17	13.37	11.75	41.61
Qwen3-1.7B2026.01	57.75	65.09	55.49	52.98	45.46	42.18	52.99	52.82	23.75	22.52	26.3	23.17	43.38
LLaMA-3.1-8B2026.01	56.79	74.27	63.01	62.34	50.87	42.49	62.7	62.16	37.13	23.72	19.75	14.36	47.47
MedGemma-4B2026.01	55	69.39	63.51	61.79	55.02	50.91	62.94	62.57	45.48	20.66	21.98	30.95	50.02
Gemma-3-4B2026.01	52.83	64.58	59.58	58.23	53.69	48.51	59.74	59.42	42.89	12.9	25.55	20.01	46.49
Bio-Medical-LLaMA-3-8B2026.01	50.07	69.06	59.37	58.85	49.4	44.72	57.13	54.77	38.93	35.79	34.01	33.82	48.83
LLaMA-3.2-3B2026.01	49.2	64.07	53.22	49.62	41.37	31.63	50.68	43.97	33.28	14.19	18.89	17.28	38.95
MedReason-8B2026.01	47.48	63.47	18.4	14.44	48.26	41.14	26.6	34.88	12.92	8.11	13.19	11.66	28.38
Phi-4-mini2026.01	42.01	71.17	52.71	55.27	39.4	31.97	50.59	50.39	30.29	14.39	19.51	17.02	39.56
MediPhi2026.01	38.13	68.04	62.01	61.21	31.25	32.76	57.91	53.8	16.53	14.77	15.83	17.52	39.15
Phi-4-mini-Reasoning2026.01	36.42	74.05	52.82	60.22	26.83	14.56	49.83	44.06	28.71	20.84	18.5	19.8	37.22
OpenBioLLM-70B2026.01	31.62	73.19	74.86	52.14	32.53	31.09	58.81	70.78	31.04	10.13	11.79	23.73	41.81
Mistral-7B-v0.32026.01	28.39	36.61	21.04	40.89	24.09	27.67	30.07	37.35	18.13	14.69	10.4	9.78	24.93
OpenBioLLM-8B2026.01	22.73	35.28	24.55	32.01	8.99	12.06	34.79	35.48	11.49	6.95	10.1	11.1	20.46