Uncertainty Estimation on GSM8K
0.951AUROCBSDETECTOR
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| BSDETECTORLLM=GPT-3.5 Turbo2023.08 | 0.951 | — | — | — | — | |
| BSDETECTORLLM=Text-Davinci-0032023.08 | 0.867 | — | — | — | — | |
| Self-reflection CertaintyLLM=GPT-3.5 Turbo2023.08 | 0.831 | — | — | — | — | |
| Geometry-Calibrated Conformal AbstentionBackbone=LLaMA-3.2-3B-Instruct2026.04 | 0.78 | — | — | — | 0.42 | |
| SelfCheckGPTBackbone=Mistral-7B2026.04 | 0.77 | — | — | — | — | |
| PerplexityBackbone=LLaMA-3.2-3B-Instruct2026.04 | 0.77 | — | — | — | 0.4 | |
| SARBackbone=LLaMA-3.2-3B-Instruct2026.04 | 0.73 | — | — | — | 0.39 | |
| SemanticBackbone=LLaMA-3.2-3B-Instruct2026.04 | 0.73 | — | — | — | 0.32 | |
| ProposedBackbone Model=LLaMA-3-8B-Instruct2026.04 | 0.68 | — | — | — | 0.77 | |
| Temperature SamplingLLM=GPT-3.5 Turbo2023.08 | 0.66 | — | — | — | — | |
| EigenBackbone=LLaMA-3.2-3B-Instruct2026.04 | 0.65 | — | — | — | 0.31 | |
| Likelihood Based UncertaintyLLM=Text-Davinci-0032023.08 | 0.647 | — | — | — | — | |
| Kernel Lang. Ent.Backbone=Mistral-7B2026.04 | 0.637 | — | — | — | — | |
| SARBackbone Model=LLaMA-3-8B-Instruct2026.04 | 0.62 | — | — | — | 0.7 | |
| Temperature SamplingLLM=Text-Davinci-0032023.08 | 0.614 | — | — | — | — | |
| TotalBackbone=Mistral-7B2026.04 | 0.61 | — | — | — | — | |
| SemanticBackbone Model=LLaMA-3-8B-Instruct2026.04 | 0.61 | — | — | — | 0.66 | |
| EigenBackbone Model=LLaMA-3-8B-Instruct2026.04 | 0.61 | — | — | — | 0.69 | |
| PerplexityBackbone Model=LLaMA-3-8B-Instruct2026.04 | 0.6 | — | — | — | 0.67 | |
| SemanticEntropyBackbone=Mistral-7B2026.04 | 0.584 | — | — | — | — | |
| ATRMDBackbone Model=LLaMA-3-8B-Instruct2026.04 | 0.58 | — | — | — | 0.64 | |
| AleatoricBackbone=Mistral-7B2026.04 | 0.577 | — | — | — | — | |
| Closeness CentralityBackbone=Mistral-7B2026.04 | 0.56 | — | — | — | — | |
| FocusBackbone=LLaMA-3.2-3B-Instruct2026.04 | 0.56 | — | — | — | 0.2 | |
| ATRMDBackbone=LLaMA-3.2-3B-Instruct2026.04 | 0.56 | — | — | — | 0.16 | |
| SC Based VCBackbone=Mistral-7B2026.04 | 0.555 | — | — | — | — | |
| SC ScoreBackbone=Mistral-7B2026.04 | 0.551 | — | — | — | — | |
| SC + VCBackbone=Mistral-7B2026.04 | 0.544 | — | — | — | — | |
| Self-reflection CertaintyLLM=Text-Davinci-0032023.08 | 0.521 | — | — | — | — | |
| FocusBackbone Model=LLaMA-3-8B-Instruct2026.04 | 0.52 | — | — | — | 0.63 | |
| Max Sequence Prob.Backbone=Mistral-7B2026.04 | 0.496 | — | — | — | — | |
| Token EntropyBackbone=Mistral-7B2026.04 | 0.465 | — | — | — | — | |
| Max Token Prob.Backbone=Mistral-7B2026.04 | 0.464 | — | — | — | — | |
| PerplexityBackbone=Mistral-7B2026.04 | 0.462 | — | — | — | — | |
| Mean Token EntropyBackbone=Mistral-7B2026.04 | 0.461 | — | — | — | — | |
| P(True)Backbone=LLaMA-3.2-3B-Instruct2026.04 | 0.46 | — | — | — | 0.14 | |
| AttentionBackbone Model=LLaMA-3-8B-Instruct2026.04 | 0.46 | — | — | — | 0.55 | |
| Self CertaintyBackbone=Mistral-7B2026.04 | 0.459 | — | — | — | — | |
| P(True)Backbone Model=LLaMA-3-8B-Instruct2026.04 | 0.44 | — | — | — | 0.55 | |
| AttentionBackbone=LLaMA-3.2-3B-Instruct2026.04 | 0.39 | — | — | — | 0.11 | |
| PTrueBackbone=Mistral-7B2026.04 | 0.371 | — | — | — | — | |
| DiSEBackbone=LLaDA-Instruct-8B2026.03 | — | 0.633 | 0.644 | 0.658 | — | |
| DiSEBackbone=LLaDA-1.5-8B2026.03 | — | 0.61 | 0.616 | 0.613 | — | |
| LLaMA perplexityBackbone=LLaDA-Instruct-8B, Evaluation Model=LLaMA3-Instruct-8B2026.03 | — | 0.675 | 0.605 | 0.577 | — | |
| LLaMA perplexityBackbone=LLaDA-1.5-8B, Evaluation Model=LLaMA3-Instruct-8B2026.03 | — | 0.635 | 0.631 | 0.546 | — | |
| MCBackbone=LLaDA-Instruct-8B, Nmc=12026.03 | — | 0.539 | 0.513 | 0.54 | — | |
| MCBackbone=LLaDA-Instruct-8B, Nmc=322026.03 | — | 0.59 | 0.552 | 0.595 | — | |
| MCBackbone=LLaDA-1.5-8B, Nmc=12026.03 | — | 0.516 | 0.559 | 0.525 | — | |
| MCBackbone=LLaDA-1.5-8B, Nmc=322026.03 | — | 0.559 | 0.578 | 0.608 | — |