Medical Knowledge Evaluation on Medical Knowledge Item Bank CAT benchmark leaderboard · SOTA2 Research

Medical Knowledge Evaluation on Medical Knowledge Item Bank CAT

73Test Lengtho1

Evaluation Results

Method	Links
o12026.02		73	2.42	90.41	75,008	790.05
gemini-2.5-pro2026.02		70	2.34	87.14	108,122	1,116.99
gemini-pro2026.02		69	2.36	89.86	107,350	1,043.98
deepseek-r12026.02		59	2.25	89.83	107,159	3,241.27
gemini-2.5-flash2026.02		55	2.09	89.09	68,188	429.8
claude-4-sonnet-thinking2026.02		54	2.17	85.19	39,635	1,068.06
gpt-4o2026.02		52	1.7	86.54	13,003	48.47
gpt-oss-120b2026.02		50	1.97	80	39,114	932.48
glm-4.5-air2026.02		49	1.57	83.67	94,582	1,756.69
minimax-m12026.02		48	1.83	75	133,680	6,566.4
qwen3-30b-a3b2026.02		45	1.55	75.56	111,964	886.19
o1-mini2026.02		42	1.43	88.1	30,748	158.63
kimi-k22026.02		41	1.43	75.61	10,809	101.51
gemini-2.0-flash2026.02		41	1.43	75.61	10,811	36.51
gpt-42026.02		38	1.31	78.95	9,688	65.76
gpt-oss-20b2026.02		37	1.37	78.38	27,882	138.6
qwen2.5-72b-instruct2026.02		32	0.84	56.25	8,448	13.64
qwen-max2026.02		31	0.73	64.52	8,076	22.05
glm-3-turbo2026.02		30	1.03	73.33	7,693	32.33
yi-lightning2026.02		30	1	73.33	9,045	40.46
yi-spark2026.02		30	0.85	63.33	9,045	43.31
yi-medium2026.02		29	0.66	68.97	8,521	47.81