Knowledge-aware refusal on SimpleQA
0.4Delta AccuracyMistral-123B
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mistral-123Bp (Weighted Score)=0.22025.10 | 0.4 | 0.93 | 0.37 | 0.16 | 0.83 | 0.06 | 16 | 35 | 14 | 6 | 31 | 4 | |
| Qwen2-35Bp (Weighted Score)=0.22025.10 | 0.47 | 0.95 | 0.12 | 0.31 | 0.62 | 0.19 | 22 | 47 | 6 | 14 | 32 | 9 | |
| Averagep (Weighted Score)=0.22025.10 | 0.8 | 0.98 | 0.49 | 0.54 | 0.48 | 0.07 | 31 | 39 | 19 | 22 | 47 | 9 | |
| Qwen2.5-72Bp (Weighted Score)=0.22025.10 | 0.84 | 0.43 | 0.5 | 0.6 | 1.32 | 0.07 | 35 | 17 | 19 | 26 | 53 | 3 | |
| Qwen3-32Bp (Weighted Score)=0.22025.10 | 0.96 | 0.54 | 0.48 | 0.71 | 1.42 | 0.14 | 35 | 19 | 17 | 28 | 51 | 7 | |
| Gemma-3-12Bp (Weighted Score)=0.22025.10 | 1.31 | 2.04 | 0.96 | 0.93 | 1.79 | 0.42 | 49 | 76 | 39 | 36 | 66 | 23 |