Question Answering on Probe 2 1.0 (test)

65AccuracyQwen2.5-32B-Instruct

Evaluation results

Method
Qwen2.5-32B-Instruct2026.01	65	65	61	63	60
Qwen2.5-32B-Instruct2026.01	64	64	62	65	61
Anthropic.Claude-3-Haiku2026.01	62	62	60	60	57
Anthropic.Claude-3-Haiku2026.01	61	61	56	55	54
Anthropic.Claude-3-Haiku2026.01	61	61	54	55	55
Llama-3.1-8B-Instruct2026.01	61	61	51	50	50
Anthropic.Claude-3-Haiku2026.01	60	60	57	57	55
Llama-3.3-70B-Instruct2026.01	59	59	53	52	52
Anthropic.Claude-3-Haiku2026.01	58	58	53	52	50
Llama-3.3-70B-Instruct2026.01	58	58	52	51	51
Mixtral-8x7B-v0.12026.01	58	58	55	54	54
Mixtral-8x7B-v0.12026.01	57	57	51	50	50
Llama-3.1-8B-Instruct2026.01	56	56	52	44	45
Llama-3.3-70B-Instruct2026.01	56	56	52	49	49
Mixtral-8x7B-v0.12026.01	56	56	52	51	51
Qwen2.5-32B-Instruct2026.01	56	56	60	60	54
Qwen2.5-32B-Instruct2026.01	56	56	59	61	55
Qwen2.5-32B-Instruct2026.01	55	55	53	55	52
Llama-3.1-8B-Instruct2026.01	54	54	45	41	40
Llama-3.3-70B-Instruct2026.01	54	54	47	47	47
Llama-3.3-70B-Instruct2026.01	54	54	51	50	49
Anthropic.Claude-3-Haiku2026.01	53	53	50	49	47
Llama-3.1-8B-Instruct2026.01	53	53	44	41	41
Llama-3.3-70B-Instruct2026.01	52	52	48	49	48
Llama-3.1-8B-Instruct2026.01	51	51	47	41	42
Qwen2.5-32B-Instruct2026.01	51	51	51	53	48
GPT-OSS-20B2026.01	50	50	56	47	46
GPT-OSS-20B2026.01	50	50	63	51	48
GPT-OSS-20B2026.01	48	48	57	47	46
Llama-3.1-8B-Instruct2026.01	48	48	40	38	37
Mixtral-8x7B-v0.12026.01	47	47	44	43	42
Mixtral-8x7B-v0.12026.01	47	47	45	44	43
GPT-OSS-20B2026.01	46	46	56	45	45
Mixtral-8x7B-v0.12026.01	46	46	47	39	39
GPT-OSS-20B2026.01	44	44	54	46	41
Mistral-7B-Instruct-v0.32026.01	44	44	46	34	33
GPT-OSS-20B2026.01	43	43	49	42	39
Mistral-7B-Instruct-v0.32026.01	40	40	46	31	29
Mistral-7B-Instruct-v0.32026.01	39	39	36	28	25
Mistral-7B-Instruct-v0.32026.01	38	38	40	31	29
Mistral-7B-Instruct-v0.32026.01	36	36	32	26	23
Mistral-7B-Instruct-v0.32026.01	35	35	38	27	24