Question Answering with Abstention on ABSTAIN (test)
78.6AccuracyDeepSeek-R1
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| DeepSeek-R1Model=DeepSeek-R12026.04 | 78.6 | 8.5 | 85.9 | 52.2 | 46.5 | 89.1 | |
| DeepSeek-V3Model=DeepSeek-V32026.04 | 77.8 | 10.9 | 87.3 | 58.4 | 50.1 | 85.8 | |
| Qwen2.5 32B InstructModel=Qwen2.5 32B Instruct2026.04 | 71.9 | 11.5 | 81.2 | 56.9 | 30.7 | 54 | |
| Qwen2.5 7B InstructModel=Qwen2.5 7B Instruct2026.04 | 62.4 | 12.7 | 71.5 | 47.2 | 2.3 | 4.9 | |
| Llama3.1 8B InstructModel=Llama3.1 8B Instruct2026.04 | 58.3 | 8.5 | 63.7 | 33.8 | 0 | 0 | |
| Abstain-R1Model=Abstain-R12026.04 | 57.2 | 20.4 | 71.9 | 68.1 | 55.1 | 80.9 | |
| Qwen2.5 3B InstructModel=Qwen2.5 3B Instruct2026.04 | 48.8 | 18.8 | 60.1 | 9.4 | 0.6 | 6.4 |