Knowledge-aware refusal on SimpleQA benchmark leaderboard · SOTA2 Research

Knowledge-aware refusal on SimpleQA

0.4Delta AccuracyMistral-123B

Evaluation Results

Method	Links
Mistral-123B2025.10		0.4	0.93	0.37	0.16	0.83	0.06	16	35	14	6	31	4
Qwen2-35B2025.10		0.47	0.95	0.12	0.31	0.62	0.19	22	47	6	14	32	9
Average2025.10		0.8	0.98	0.49	0.54	0.48	0.07	31	39	19	22	47	9
Qwen2.5-72B2025.10		0.84	0.43	0.5	0.6	1.32	0.07	35	17	19	26	53	3
Qwen3-32B2025.10		0.96	0.54	0.48	0.71	1.42	0.14	35	19	17	28	51	7
Gemma-3-12B2025.10		1.31	2.04	0.96	0.93	1.79	0.42	49	76	39	36	66	23