Selective Answering on integrated benchmark
43.9ScoreSABER
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SABERBackbone=Llama-3.1-8B, τ (abstention threshold)=0.52026.05 | 43.9 | 65 | 16.2 | 72.5 | |
| SABERBackbone=Qwen-2.5-7B, τ (abstention threshold)=0.52026.05 | 43.4 | 58.5 | 12.9 | 78 | |
| SABERBackbone=Qwen-2.5-3B, τ (abstention threshold)=0.52026.05 | 39.6 | 60 | 17 | 76.6 | |
| SABERBackbone=Llama-3.2-3B, τ (abstention threshold)=0.52026.05 | 38.5 | 58.6 | 17.2 | 76.8 | |
| Implicit-SCRBackbone=Llama-3.1-8B, τ (abstention threshold)=0.52026.05 | 25.4 | 59.8 | 28.8 | 56.2 | |
| Vanilla RAGBackbone=Llama-3.1-8B, τ (abstention threshold)=0.52026.05 | 19.8 | 72.9 | 36.4 | 49.6 | |
| Implicit-SCRBackbone=Qwen-2.5-7B, τ (abstention threshold)=0.52026.05 | 18.8 | 81.6 | 38.5 | 39.8 | |
| TACS-LRBackbone=Llama-3.1-8B, τ (abstention threshold)=0.52026.05 | 13.6 | 64.4 | 39.5 | 49.9 | |
| Explicit-SCRBackbone=Qwen-2.5-7B, τ (abstention threshold)=0.52026.05 | 11.4 | 43.7 | 36.9 | 55.4 | |
| Implicit-SCRBackbone=Llama-3.2-3B, τ (abstention threshold)=0.52026.05 | 11.4 | 70.7 | 41.9 | 51 | |
| Vanilla RAGBackbone=Qwen-2.5-7B, τ (abstention threshold)=0.52026.05 | 11.1 | 87.1 | 43.6 | 27.9 | |
| TACS-LRBackbone=Qwen-2.5-7B, τ (abstention threshold)=0.52026.05 | 8.7 | 68.3 | 43.7 | 42.8 | |
| Vanilla RAGBackbone=Llama-3.2-3B, τ (abstention threshold)=0.52026.05 | 8.2 | 79.4 | 44.9 | 43.1 | |
| Implicit-SCRBackbone=Qwen-2.5-3B, τ (abstention threshold)=0.52026.05 | 7.6 | 73 | 44.8 | 42 | |
| Vanilla RAGBackbone=Qwen-2.5-3B, τ (abstention threshold)=0.52026.05 | 6.2 | 79.5 | 46.1 | 35.8 | |
| TACS-LRBackbone=Llama-3.2-3B, τ (abstention threshold)=0.52026.05 | 5.4 | 68.7 | 46.1 | 47.5 | |
| TACS-LRBackbone=Qwen-2.5-3B, τ (abstention threshold)=0.52026.05 | 5.1 | 71.2 | 46.4 | 40.6 | |
| Closed-BookBackbone=Qwen-2.5-7B, τ (abstention threshold)=0.52026.05 | 1.7 | 23.2 | 46.4 | 58 | |
| Internal-EvalBackbone=Qwen-2.5-7B, τ (abstention threshold)=0.52026.05 | 0.8 | 45.3 | 49.2 | 50.4 | |
| Internal-EvalBackbone=Llama-3.1-8B, τ (abstention threshold)=0.52026.05 | 0.5 | 89.2 | 49.7 | 20 | |
| Closed-BookBackbone=Qwen-2.5-3B, τ (abstention threshold)=0.52026.05 | -0.3 | 6.3 | 52.4 | 62.3 | |
| Context-EvalBackbone=Qwen-2.5-7B, τ (abstention threshold)=0.52026.05 | -1.8 | 77.6 | 51.1 | 27.3 | |
| Closed-BookBackbone=Llama-3.1-8B, τ (abstention threshold)=0.52026.05 | -3 | 50.6 | 53 | 50.1 | |
| Closed-BookBackbone=Llama-3.2-3B, τ (abstention threshold)=0.52026.05 | -5 | 37 | 56.8 | 57.3 | |
| Explicit-SCRBackbone=Qwen-2.5-3B, τ (abstention threshold)=0.52026.05 | -5.9 | 43.2 | 56.8 | 54.5 | |
| Internal-EvalBackbone=Qwen-2.5-3B, τ (abstention threshold)=0.52026.05 | -6.8 | 68.4 | 54.9 | 40.8 | |
| Context-EvalBackbone=Llama-3.1-8B, τ (abstention threshold)=0.52026.05 | -7.6 | 99.2 | 53.8 | 3.3 | |
| Explicit-SCRBackbone=Llama-3.1-8B, τ (abstention threshold)=0.52026.05 | -8.7 | 97 | 54.5 | 8.3 | |
| Internal-EvalBackbone=Llama-3.2-3B, τ (abstention threshold)=0.52026.05 | -11.9 | 72.5 | 58.2 | 38.3 | |
| Context-EvalBackbone=Qwen-2.5-3B, τ (abstention threshold)=0.52026.05 | -33.5 | 99.3 | 66.9 | 1.9 | |
| Explicit-SCRBackbone=Llama-3.2-3B, τ (abstention threshold)=0.52026.05 | -34.2 | 97.6 | 67.5 | 4.9 | |
| Context-EvalBackbone=Llama-3.2-3B, τ (abstention threshold)=0.52026.05 | -34.6 | 100 | 67.3 | 0.1 |