Uncertainty Estimation on NaturalQA
77AUROCSAR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SARBackbone=LLaMA-3.2-3B-Instruct2026.04 | 77 | 74 | |
| Geometry-Calibrated Conformal AbstentionBackbone=LLaMA-3.2-3B-Instruct2026.04 | 77 | 75 | |
| SE + MARSBackbone=Mistral-7b2024.02 | 75.5 | — | |
| SEBackbone=Mistral-7b2024.02 | 75.12 | — | |
| SE + MARSBackbone=Llama2-13b2024.02 | 73.89 | — | |
| SEBackbone=Llama2-13b2024.02 | 73.56 | — | |
| SE + MARSBackbone=Llama2-7b2024.02 | 72.75 | — | |
| SEBackbone=Llama2-7b2024.02 | 72.47 | — | |
| Confidence + MARSBackbone=Mistral-7b2024.02 | 71.36 | — | |
| SE + MARSBackbone=Falcon-7b2024.02 | 71.24 | — | |
| PerplexityBackbone=LLaMA-3.2-3B-Instruct2026.04 | 71 | 67 | |
| Confidence + MARSBackbone=Llama2-13b2024.02 | 70.88 | — | |
| Entropy + MARSBackbone=Mistral-7b2024.02 | 70.71 | — | |
| Entropy + MARSBackbone=Llama2-13b2024.02 | 70.63 | — | |
| SEBackbone=Falcon-7b2024.02 | 70.41 | — | |
| Confidence + MARSBackbone=Llama2-7b2024.02 | 69.81 | — | |
| ConfidenceBackbone=Mistral-7b2024.02 | 69.54 | — | |
| SE + MARSBackbone=Llama2-7b-chat2024.02 | 69.43 | — | |
| Entropy + MARSBackbone=Llama2-7b2024.02 | 69.32 | — | |
| SEBackbone=Llama2-7b-chat2024.02 | 68.66 | — | |
| ConfidenceBackbone=Llama2-7b2024.02 | 68.56 | — | |
| ConfidenceBackbone=Llama2-13b2024.02 | 68.56 | — | |
| EntropyBackbone=Llama2-13b2024.02 | 68.34 | — | |
| Confidence + MARSBackbone=Falcon-7b2024.02 | 68.3 | — | |
| EntropyBackbone=Mistral-7b2024.02 | 68.05 | — | |
| Confidence + MARSBackbone=Llama2-7b-chat2024.02 | 67.86 | — | |
| Entropy + MARSBackbone=Falcon-7b2024.02 | 67.51 | — | |
| Entropy + MARSBackbone=Llama2-7b-chat2024.02 | 67.41 | — | |
| EntropyBackbone=Llama2-7b2024.02 | 67.08 | — | |
| ConfidenceBackbone=Llama2-7b-chat2024.02 | 65.98 | — | |
| EntropyBackbone=Llama2-7b-chat2024.02 | 65.23 | — | |
| SemanticBackbone=LLaMA-3.2-3B-Instruct2026.04 | 64 | 62 | |
| ConfidenceBackbone=Falcon-7b2024.02 | 63.78 | — | |
| EntropyBackbone=Falcon-7b2024.02 | 63.28 | — | |
| EigenBackbone=LLaMA-3.2-3B-Instruct2026.04 | 63 | 62 | |
| FocusBackbone=LLaMA-3.2-3B-Instruct2026.04 | 62 | 64 | |
| ATRMDBackbone=LLaMA-3.2-3B-Instruct2026.04 | 59 | 58 | |
| P(True)Backbone=LLaMA-3.2-3B-Instruct2026.04 | 55 | 55 | |
| AttentionBackbone=LLaMA-3.2-3B-Instruct2026.04 | 50 | 53 |