Uncertainty Estimation on TriviaQA
88AUROCGeometry-Calibrated Conformal Abstention
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Geometry-Calibrated Conformal AbstentionBackbone=LLaMA-3.2-3B-Instruct2026.04 | 88 | — | — | — | 87 | |
| VCBackbone=Llama-3.2-3B, Evaluation Setup=Vanilla2026.04 | 85.56 | — | 17.78 | — | — | |
| VCBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 85.56 | — | 20.38 | 2.6 | — | |
| EntropyBackbone=Llama-3.2-3B, Evaluation Setup=Vanilla2026.04 | 84.6 | — | 37.83 | — | — | |
| EntropyBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 84.6 | — | 18.88 | -18.95 | — | |
| GLU-Sα-SPModel=Qwen2026.06 | 83.7 | — | — | — | — | |
| SE + MARSBackbone=Mistral-7b2024.02 | 83.63 | — | — | — | — | |
| GLUModel=Qwen2026.06 | 83.4 | — | — | — | — | |
| BSDETECTORLLM=Text-Davinci-0032023.08 | 82.8 | — | — | — | — | |
| GLU-Sα-AUModel=Fanar2026.06 | 82.8 | — | — | — | — | |
| SE + MARSBackbone=Llama2-7b2024.02 | 82.22 | — | — | — | — | |
| SEBackbone=Mistral-7b2024.02 | 82.17 | — | — | — | — | |
| RAUQModel=Fanar2026.06 | 82.1 | — | — | — | — | |
| GLUModel=Fanar2026.06 | 82.1 | — | — | — | — | |
| GLU-Sα-AUModel=Gemma2026.06 | 81.9 | — | — | — | — | |
| BSDETECTORLLM=GPT-3.5 Turbo2023.08 | 81.7 | — | — | — | — | |
| GLUModel=Gemma2026.06 | 81.7 | — | — | — | — | |
| TotalBackbone=Mistral-7B2026.04 | 81.5 | — | — | — | — | |
| SEBackbone=Llama2-7b2024.02 | 81.1 | — | — | — | — | |
| SE + MARSBackbone=Llama2-13b2024.02 | 81 | — | — | — | — | |
| P(true)Model=Qwen2026.06 | 80.5 | — | — | — | — | |
| LogProbModel=Gemma2026.06 | 80.4 | — | — | — | — | |
| ProposedBackbone Model=LLaMA-3-8B-Instruct2026.04 | 80 | — | — | — | 67 | |
| LogProbModel=Fanar2026.06 | 79.7 | — | — | — | — | |
| AleatoricBackbone=Mistral-7B2026.04 | 79.6 | — | — | — | — | |
| SEBackbone=Llama2-13b2024.02 | 79.49 | — | — | — | — | |
| EnergyBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 79.12 | — | 5.39 | -11.68 | — | |
| EnergyBackbone=Llama-3.2-3B, Evaluation Setup=Vanilla2026.04 | 78.92 | — | 17.07 | — | — | |
| RAUQModel=Qwen2026.06 | 78.9 | — | — | — | — | |
| LogTokUModel=Qwen2026.06 | 78.6 | — | — | — | — | |
| Entropy + MARSBackbone=Mistral-7b2024.02 | 78.51 | — | — | — | — | |
| EigenscoreBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 78.05 | — | 12.15 | -15.56 | — | |
| Confidence + MARSBackbone=Mistral-7b2024.02 | 77.97 | — | — | — | — | |
| SE + MARSBackbone=Llama2-7b-chat2024.02 | 77.67 | — | — | — | — | |
| LogProbModel=Qwen2026.06 | 77.5 | — | — | — | — | |
| SE + MARSBackbone=Falcon-7b2024.02 | 77.48 | — | — | — | — | |
| EntropyBackbone=Llama-3.1-8B, Evaluation Setup=Vanilla2026.04 | 76.91 | — | 20.53 | — | — | |
| EntropyBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 76.91 | — | 11.66 | -8.87 | — | |
| Temperature SamplingLLM=Text-Davinci-0032023.08 | 76.9 | — | — | — | — | |
| SEBackbone=Falcon-7b2024.02 | 76.78 | — | — | — | — | |
| EigenscoreBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 76.76 | — | 4.75 | -4.88 | — | |
| EigenscoreBackbone=Llama-3.1-8B, Evaluation Setup=Vanilla2026.04 | 76.42 | — | 9.63 | — | — | |
| SEBackbone=Llama2-7b-chat2024.02 | 76.19 | — | — | — | — | |
| MSPBackbone=Llama-3.2-3B, Evaluation Setup=Vanilla2026.04 | 76.15 | — | 48.25 | — | — | |
| MSPBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 76.15 | — | 13.41 | -34.84 | — | |
| Entropy + MARSBackbone=Llama2-7b2024.02 | 75.94 | — | — | — | — | |
| MSPBackbone=Llama-3.1-8B, Evaluation Setup=Vanilla2026.04 | 75.79 | — | 30.31 | — | — | |
| MSPBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 75.79 | — | 8.53 | -21.78 | — | |
| CoE-RBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 75.3 | — | 4.07 | -18.64 | — | |
| Confidence + MARSBackbone=Llama2-7b2024.02 | 75.06 | — | — | — | — | |
| Entropy + MARSBackbone=Llama2-13b2024.02 | 74.95 | — | — | — | — | |
| CoE-RBackbone=Llama-3.2-3B, Evaluation Setup=Vanilla2026.04 | 74.43 | — | 22.71 | — | — | |
| Confidence + MARSBackbone=Llama2-7b-chat2024.02 | 74.23 | — | — | — | — | |
| Confidence + MARSBackbone=Llama2-13b2024.02 | 73.99 | — | — | — | — | |
| PerplexityBackbone=Llama-3.1-8B, Evaluation Setup=Vanilla2026.04 | 73.87 | — | 29.34 | — | — | |
| PerplexityBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 73.87 | — | 7.96 | -21.37 | — | |
| Entropy + MARSBackbone=Llama2-7b-chat2024.02 | 73.82 | — | — | — | — | |
| CoE-CBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 73.38 | — | 7.63 | -13.27 | — | |
| SC + VCBackbone=Mistral-7B2026.04 | 73.1 | — | — | — | — | |
| Confidence + MARSBackbone=Falcon-7b2024.02 | 72.95 | — | — | — | — | |
| P(true)Model=Fanar2026.06 | 72.9 | — | — | — | — | |
| Entropy + MARSBackbone=Falcon-7b2024.02 | 72.87 | — | — | — | — | |
| EntropyBackbone=Mistral-7b2024.02 | 72.57 | — | — | — | — | |
| ConfidenceBackbone=Mistral-7b2024.02 | 72.55 | — | — | — | — | |
| Kernel Lang. Ent.Backbone=Mistral-7B2026.04 | 72.3 | — | — | — | — | |
| Likelihood Based UncertaintyLLM=Text-Davinci-0032023.08 | 70.8 | — | — | — | — | |
| SC Based VCBackbone=Mistral-7B2026.04 | 70.6 | — | — | — | — | |
| LogTokUModel=Fanar2026.06 | 70.6 | — | — | — | — | |
| EnergyBackbone=Llama-3.1-8B, Evaluation Setup=Vanilla2026.04 | 70.53 | — | 10.21 | — | — | |
| EnergyBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 70.53 | — | 8.52 | -1.69 | — | |
| ConfidenceBackbone=Llama2-7b-chat2024.02 | 70.4 | — | — | — | — | |
| Closeness CentralityBackbone=Mistral-7B2026.04 | 70.2 | — | — | — | — | |
| ConfidenceBackbone=Llama2-7b2024.02 | 70.18 | — | — | — | — | |
| SARBackbone=LLaMA-3.2-3B-Instruct2026.04 | 70 | — | — | — | 52 | |
| RAUQModel=Gemma2026.06 | 70 | — | — | — | — | |
| EntropyBackbone=Llama2-7b-chat2024.02 | 69.94 | — | — | — | — | |
| EntropyBackbone=Llama2-7b2024.02 | 69.7 | — | — | — | — | |
| Mean Token EntropyBackbone=Mistral-7B2026.04 | 69.6 | — | — | — | — | |
| PerplexityBackbone=Llama-3.2-3B, Evaluation Setup=Vanilla2026.04 | 69.19 | — | 58.46 | — | — | |
| EntropyBackbone=Falcon-7b2024.02 | 69.1 | — | — | — | — | |
| EntropyBackbone=Llama2-13b2024.02 | 69.04 | — | — | — | — | |
| Token EntropyBackbone=Mistral-7B2026.04 | 69 | — | — | — | — | |
| Temperature SamplingLLM=GPT-3.5 Turbo2023.08 | 68.9 | — | — | — | — | |
| SEBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 68.68 | — | 3.85 | -36.54 | — | |
| ConfidenceBackbone=Falcon-7b2024.02 | 68.47 | — | — | — | — | |
| EigenscoreBackbone=Llama-3.2-3B, Evaluation Setup=Vanilla2026.04 | 68.37 | — | 27.7 | — | — | |
| ConfidenceBackbone=Llama2-13b2024.02 | 68.19 | — | — | — | — | |
| Max Token Prob.Backbone=Mistral-7B2026.04 | 67.4 | — | — | — | — | |
| SC ScoreBackbone=Mistral-7B2026.04 | 67.3 | — | — | — | — | |
| PerplexityBackbone=Mistral-7B2026.04 | 67.2 | — | — | — | — | |
| PerplexityBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 67.11 | — | 6.51 | -51.95 | — | |
| P(true)Model=Gemma2026.06 | 67.1 | — | — | — | — | |
| SARBackbone Model=LLaMA-3-8B-Instruct2026.04 | 67 | — | — | — | 60 | |
| SelfCheckGPTBackbone=Mistral-7B2026.04 | 66.7 | — | — | — | — | |
| Max Sequence Prob.Backbone=Mistral-7B2026.04 | 66 | — | — | — | — | |
| PerplexityBackbone Model=LLaMA-3-8B-Instruct2026.04 | 66 | — | — | — | 51 | |
| FocusBackbone Model=LLaMA-3-8B-Instruct2026.04 | 66 | — | — | — | 52 | |
| Semantic EntropyBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 65.69 | — | 6.58 | -52.86 | — | |
| Self-reflection CertaintyLLM=GPT-3.5 Turbo2023.08 | 65.5 | — | — | — | — | |
| Self-reflection CertaintyLLM=Text-Davinci-0032023.08 | 65.3 | — | — | — | — |