Uncertainty Estimation on SVAMP
68.8ROC-AUC (128)DiSE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DiSEBackbone=LLaDA-Instruct-8B2026.03 | 68.8 | — | 69.2 | 75.5 | |
| LLaMA perplexityBackbone=LLaDA-Instruct-8B, Evaluation Model=LLaMA3-Instruct-8B2026.03 | 68.6 | — | 65 | 59 | |
| LLaMA perplexityBackbone=LLaDA-1.5-8B, Evaluation Model=LLaMA3-Instruct-8B2026.03 | 63.9 | — | 58.7 | 62 | |
| MCBackbone=LLaDA-Instruct-8B, Nmc=322026.03 | 61.6 | — | 55.1 | 64.7 | |
| DiSEBackbone=LLaDA-1.5-8B2026.03 | 59.9 | — | 62.9 | 67.7 | |
| MCBackbone=LLaDA-1.5-8B, Nmc=322026.03 | 58.5 | — | 51.3 | 59.7 | |
| MCBackbone=LLaDA-Instruct-8B, Nmc=12026.03 | 56.3 | — | 57.5 | 50.9 | |
| MCBackbone=LLaDA-1.5-8B, Nmc=12026.03 | 56.2 | — | 46.6 | 55.4 | |
| BSDETECTORLLM=Text-Davinci-0032023.08 | — | 93.6 | — | — | |
| BSDETECTORLLM=GPT-3.5 Turbo2023.08 | — | 92.7 | — | — | |
| Likelihood Based UncertaintyLLM=Text-Davinci-0032023.08 | — | 66.8 | — | — | |
| Self-reflection CertaintyLLM=Text-Davinci-0032023.08 | — | 61.9 | — | — | |
| Self-reflection CertaintyLLM=GPT-3.5 Turbo2023.08 | — | 83.9 | — | — | |
| Temperature SamplingLLM=Text-Davinci-0032023.08 | — | 65.3 | — | — | |
| Temperature SamplingLLM=GPT-3.5 Turbo2023.08 | — | 67.1 | — | — |