Medical Reasoning on RareDis Sub (test)

80Symptoms AccuracyLlama-3.1-8B-Instruct + MedSSR

Evaluation results

Method
Llama-3.1-8B-Instruct + MedSSR2026.04	80	80.3	64.96	71.99	82.89	59.79	86.38	75.19
Qwen3-8B-Base + MedSSR2026.04	79.71	80.89	68.19	72.22	76.64	57.51	78.62	73.4
Llama-3.1-8B-Instruct + Fully-Supervised2026.04	76.27	77.72	60.18	67.13	81.09	54.55	83.45	71.48
HuatuoGPT-o1-8B2026.04	74.9	78.47	61.64	70.1	79.53	52.89	78.68	70.89
m1-7B-23K2026.04	74.51	74.64	57.82	69.44	68.59	52.37	72.76	67.16
UltraMedical-3.1-8B2026.04	73.53	73.01	57.28	65.28	77.47	50.83	73.79	67.31
MedReason-8B2026.04	73.53	74.39	62.85	70.43	76.43	57.1	75.33	70.01
Qwen3-8B-Base + Fully-Supervised2026.04	73.24	75.41	60.65	65.97	70.39	54.41	72.24	67.47
MedPRM-8B2026.04	73.14	76	61.79	66.2	78.45	52.37	78.97	69.56
Llama-3.1-8B-Instruct2026.04	71.67	74.28	60.85	64.81	79.28	52.97	76.55	68.63
Qwen3-8B2026.04	68.63	72.01	57.95	66.44	70.39	52.18	68.28	65.13
Qwen3-8B-Base2026.04	67.16	70.74	54.92	68.75	59.05	49.3	67.41	62.48
R1-Distill-Llama-8B2026.04	53.58	63.04	45.49	60.19	55.52	46.57	63.28	55.38