Refusal on Irrelevant Queries on LongBench
97.52WikiMQA ScoreDoc2Atom
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Doc2AtomBase Model=Gemma-2-2B-It2026.06 | 97.5 | 90.17 | 95.22 | 95.95 | 95.33 | 84.33 | 61.5 | 64 | |
| D2LatomBase Model=Gemma-2-2B-It2026.06 | 22 | 15.83 | 11.33 | 11.5 | 12.5 | 24.5 | 22.33 | 5.5 | |
| Doc2AtomBase Model=Qwen3-4B-Instruct2026.06 | 17.64 | 19.08 | 64 | 95 | 28.58 | 92.5 | 93.25 | 20 | |
| D2LrawBase Model=Gemma-2-2B-It2026.06 | 15.83 | 16 | 18 | 40.5 | 8.5 | 7 | 21.15 | 8.5 | |
| D2LckptBase Model=Qwen3-4B-Instruct2026.06 | 13.2 | 15.28 | 17.96 | 21.47 | 14.38 | 75.04 | 38.35 | 6.54 | |
| D2LrawBase Model=Qwen3-4B-Instruct2026.06 | 11 | 10.5 | 17.33 | 33 | 6.5 | 5.5 | 50.5 | 4 | |
| D2LckptBase Model=Gemma-2-2B-It2026.06 | 8.5 | 8 | 2.67 | 14.5 | 3 | 1 | 1.5 | 1 | |
| D2LatomBase Model=Qwen3-4B-Instruct2026.06 | 7.5 | 10.5 | 20.67 | 21 | 6.5 | 7 | 40.33 | 3 |