Natural Language Inference on BioNLI

72.46Accuracy (Chinese)LLaMA-3.2-3B

Evaluation results

Method
LLaMA-3.2-3B2026.01	72.46	45.05	36.17	43.47	60.4	53.49	54.48	55.69	31.2	20.09	27.16	20.14	43.32
o4-mini2026.01	68.97	73.34	67.13	66.9	68.42	67.08	70.24	71.06	61.42	32.3	49.2	55.24	62.61
gpt-oss-20B2026.01	68.88	62.77	61.55	60.24	64.22	64.6	62.81	63.91	55.16	24.47	46.64	54.23	57.46
Claude-4.0-Sonnet2026.01	68.31	75.46	63.88	59.72	62.98	58.72	68.48	67.95	54.74	45.47	47.38	53.82	60.58
GPT-52026.01	65.7	73.41	57.73	56.7	63	67.47	60.32	62.98	51.64	48.43	45.86	45.43	58.22
MediPhi2026.01	64.56	72.46	68.53	73.89	60.16	41.85	72.57	74.08	25.73	44.63	21.35	42.7	55.21
gpt-oss-120B2026.01	64.07	70.93	59.62	60.13	63.76	64.62	63.1	63.29	67.2	41.49	49.74	58.14	60.51
GPT-4o-mini2026.01	63.52	63.71	49.23	46.17	53.56	57.05	49.98	50.12	42.23	27.65	40.99	31.44	47.97
Phi-4-Reasoning2026.01	63.14	69.16	59.47	55.84	61.87	59.33	57.68	61.65	57.74	15.25	48.78	37.89	53.98
Baichuan-M2-32B2026.01	62.71	57.5	50.36	59.86	58.83	59.71	54.7	55.67	36.48	30.79	41.07	43.31	50.92
DeepSeek-R12026.01	62.59	74.17	60.19	60.48	56.87	55.42	64.32	62.49	51.09	50.52	54.59	49.14	58.49
Gemini-2.5-Flash2026.01	62.44	69.06	58.13	59.4	58.74	61.82	61.29	60.66	53.76	55.54	63.93	54.59	59.95
GPT-5-mini2026.01	60.99	68.93	60.2	58.04	59.24	59.09	63.31	63.86	53.63	26.32	33.52	52.49	54.97
DeepSeek-R1-Qwen3-8B2026.01	59.35	64.43	51.34	48.34	52.42	48.93	52.63	54.97	44.67	11.54	18.2	13.92	43.39
GPT-4.1-mini2026.01	59.1	64.68	55.96	52.49	55.87	54.89	58.95	59.87	46.81	21.39	36.37	37.96	50.36
Qwen2.5-3B2026.01	58.53	63.52	49.35	46.79	58.77	58.81	55.05	54.3	20.77	1.72	12.86	10.49	40.91
Qwen3-14B2026.01	58.21	68.72	57.62	52.86	58.35	57.16	63.26	61.61	49.15	2.41	32.63	29.04	49.25
Qwen3-14B-thinking2026.01	57.79	69.96	59.83	56.71	61.92	60.43	65.14	62.68	55.64	2.6	27.19	33.88	51.15
Mistral-Small-3.1-24B2026.01	53.63	65.01	48.41	43.97	47.28	48.82	43.9	48.11	42.66	24	21.3	32.49	43.3
Phi-4-mini2026.01	53.49	54.39	51.75	48.41	54.65	43.9	43.6	49.32	38.82	31.5	31.71	41.1	45.22
HuatuoGPT-o1-72B2026.01	52.81	64.92	59.08	57.54	53.93	55.59	61.61	63.33	44.26	14.88	40.32	25.41	49.47
Phi-42026.01	52.26	59.4	52.55	51.69	53.78	52.24	54.52	54.96	44.45	34.99	44.64	35.97	49.29
Qwen2.5-14B2026.01	52.19	67.4	48.87	45.29	45.68	48.63	51.31	53.24	45.47	59.04	70.53	38.01	52.14
Qwen3-4B2026.01	52.17	59.04	40.76	38.76	49.69	43.75	50.2	49.51	11.72	3.21	24.3	14.56	36.47
LLaMA-3.1-8B2026.01	52.1	38.37	36.39	35.42	68.51	45.4	43.84	40.94	39.99	49.94	38.47	12.67	41.84
QwQ-32B2026.01	51.84	59.03	49.35	47.15	48.46	47.31	50.77	50.86	47.19	39.39	43.35	45.3	48.33
DeepSeek-V32026.01	51.37	63.91	49.8	51.9	49.28	48.66	52.21	50.81	43.59	42.33	37.78	32.85	47.87
Qwen3-1.7B2026.01	51.29	54.77	45.41	55.56	49.77	41.56	45.1	47.07	49.64	37.22	47.39	42.44	47.27
LLaMA-4-Maverick2026.01	50.31	61.59	46.13	45.74	50.17	53.14	48.98	49.04	39.67	63.19	42.39	37.39	48.98
Qwen3-4B-thinking2026.01	50.26	61.98	50.45	47.11	51.75	49.65	54.19	54.57	26.59	3.43	27.4	11.66	40.75
Qwen2.5-7B2026.01	50.22	62.04	53.87	47.88	49.61	44.7	54.62	50.11	45.43	28.77	38.67	29.16	46.26
GPT-4.12026.01	49.67	64.53	51.2	49.07	49.68	52.27	51.45	51.39	40.29	45.09	41.09	34.23	48.33
HuatuoGPT-o1-7B2026.01	49.02	57.66	52.03	51.43	54.46	49.95	52.49	55.11	16.97	2.74	6.52	4.01	37.7
Qwen3-8B-thinking2026.01	48.72	62.34	52.52	48.36	56.03	50.5	56.11	56.56	29.98	8.27	29.72	23.39	43.54
LLaMA-4-Scout2026.01	48.14	56.46	42.52	42.47	43.55	45.14	47.65	44.04	40.84	54.19	62.82	44.04	47.65
Qwen2.5-72B2026.01	47.66	65	55.39	52.62	50.07	53.59	58.92	60.07	38.96	71.4	55.46	29.4	53.21
Qwen3-8B2026.01	47.02	61.2	41.1	38.88	51.16	45.59	45.29	48.19	35.02	3.79	52.74	17.08	40.59
LLaMA-3.3-70B2026.01	46.8	58.4	45.35	41.29	42.4	44.2	46.68	46.01	39	69.33	48.76	29.38	46.47
MedGemma-27B2026.01	46.39	54.81	45.3	51.22	48.42	52.05	51.33	50.76	48.75	24.67	41.94	46.1	46.81
GPT-4.1-nano2026.01	46.35	55.66	48.35	45.15	53.86	52.72	50.24	49.21	48.22	46.26	43.89	39.74	48.31
GPT-5-nano2026.01	45.9	41.14	34.17	45.48	51.23	42.11	49.31	51.51	58.19	31.55	58.94	55.01	47.04
HuatuoGPT-o1-70B2026.01	44.62	58.75	47.65	46.3	43.58	46.88	48.04	49.35	37.81	55.35	39.05	33.87	45.94
Phi-4-mini-Reasoning2026.01	43.18	59.28	48.11	44.69	31.63	20.37	37.43	37.06	35.98	45.14	35.04	23.85	38.48
GPT-4o2026.01	38.77	45.35	42.09	40.1	41.2	47.08	42.96	42.76	34.31	15.88	33.78	34.17	38.2
Claude-3.5-Haiku2026.01	37.75	50.18	39.77	40.47	40.11	44.79	40.49	40.68	37.09	47.12	70.89	25.76	42.93
Gemma-3-27B2026.01	36.93	49.26	36.72	39.24	39.79	44	39.22	39.61	35.47	12.51	31.7	37.02	36.79
LLaMA-3.1-70B2026.01	35.17	58.56	35.9	35.95	36.19	40.67	37.7	35.69	34.54	58.88	32.91	26.15	39.03
Gemma-3-4B2026.01	35.05	39.87	35.29	34.47	36.01	33.38	34.4	33.8	34.21	2.32	26.58	32.98	31.53
Bio-Medical-LLaMA-3-8B2026.01	35.03	40.84	25.97	31.76	34.83	25.73	32.19	29.83	26.07	36.28	30.89	35.01	32.04
Mistral-7B-v0.32026.01	33.77	2.96	15.37	32.37	26.52	27.15	22.79	19.82	14.17	9.84	10.87	10.78	18.87
HuatuoGPT-o1-8B2026.01	33.68	34.41	36.26	31.37	44.02	35.35	37.86	37.12	27.49	1.78	21.64	7.56	29.04
Gemma-3-12B2026.01	32.39	38.32	34.52	32.29	35.82	37.37	36	35.28	39.35	4.83	55.15	35.69	34.75
MedGemma-4B2026.01	30.72	36.53	39.85	35.28	32.59	36.4	35.67	36.33	32.33	16.32	28.52	28.07	32.38
MedReason-8B2026.01	27.6	6.41	4.32	4.15	25.53	30.91	24.1	1.69	8.43	0.49	37.09	23.86	16.22
OpenBioLLM-70B2026.01	12.77	32.56	26.86	15.55	10.26	10.92	22.36	27.53	11.47	11.47	16.16	15.93	17.82
OpenBioLLM-8B2026.01	9.19	11.68	13.83	17.5	8.92	8.73	17.15	19.13	10.21	14.51	11.25	12.28	12.86