Uncertainty Estimation on CoQA
0.857AUROCLogit Magnitude
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Logit MagnitudeLLM=Qwen3.5, Size=35B*2026.05 | 0.857 | 0.685 | 78.5 | 24.4 | |
| Logit MagnitudeLLM=Gemma4, Size=31B2026.05 | 0.855 | 0.721 | 78.1 | 13.5 | |
| Logit MagnitudeLLM=Gemma4, Size=26B*2026.05 | 0.83 | 0.699 | 75.6 | 13.8 | |
| LogTokULLM=Gemma4, Size=26B*2026.05 | 0.822 | 0.695 | 73.6 | 13.8 | |
| TotalBackbone=Mistral-7B2026.04 | 0.819 | — | — | — | |
| LogTokULLM=Gemma4, Size=31B2026.05 | 0.819 | 0.704 | 73.5 | 13.5 | |
| LogTokULLM=Qwen3.5, Size=35B*2026.05 | 0.816 | 0.663 | 73.5 | 24.4 | |
| MetaUELLM=Gemma4, Size=31B2026.05 | 0.807 | 0.692 | 73.9 | 0 | |
| MetaUELLM=Gemma4, Size=26B*2026.05 | 0.794 | 0.676 | 72.2 | 0 | |
| Logit MagnitudeLLM=Qwen3.5-4B, generation=Full (τ = T)2026.05 | 0.793 | 0.604 | 72.8 | 19.4 | |
| Logit MagnitudeLLM=Qwen3.5-4B, generation=Early-stopped2026.05 | 0.791 | 0.603 | 72.7 | 16.3 | |
| Logit MagnitudeLLM=Qwen3.5, Size=9B2026.05 | 0.791 | 0.656 | 72.3 | 18.6 | |
| LogTokULLM=Qwen3.5, Size=9B2026.05 | 0.781 | 0.652 | 70.4 | 18.6 | |
| LogTokUBase Model=Llama3, LLM Size=8B2026.05 | 0.773 | 0.516 | 69.6 | 23.1 | |
| LogTokULLM=Qwen3.5-4B2026.05 | 0.771 | 0.587 | 69.4 | 19.4 | |
| MetaUELLM=Qwen3.5, Size=35B*2026.05 | 0.765 | 0.638 | 70.1 | 0 | |
| Logit MagnitudeBase Model=Llama3, LLM Size=8B2026.05 | 0.756 | 0.504 | 70.3 | 23.1 | |
| Semantic EntropyLLM=Qwen3.5-4B2026.05 | 0.754 | 0.575 | 70.2 | 195.9 | |
| LogTokUBase Model=Llama3, LLM Size=3B2026.05 | 0.751 | 0.499 | 67.5 | 18.9 | |
| Logit MagnitudeLLM=Gemma4-4B, generation=Full (τ = T)2026.05 | 0.733 | 0.488 | 68.2 | 17.4 | |
| Logit MagnitudeBase Model=Llama3, LLM Size=3B2026.05 | 0.731 | 0.482 | 67.7 | 18.9 | |
| Logit MagnitudeLLM=Gemma4-4B, generation=Early-stopped2026.05 | 0.73 | 0.486 | 67.5 | 15 | |
| MetaUELLM=Qwen3.5-4B2026.05 | 0.724 | 0.564 | 67.3 | 0 | |
| LogTokULLM=Gemma4-4B2026.05 | 0.723 | 0.484 | 68.1 | 17.4 | |
| Self-CertaintyBase Model=Llama3, LLM Size=8B2026.05 | 0.717 | 0.474 | 65.8 | 23.1 | |
| Self-ConsistencyLLM=Qwen3.5-4B2026.05 | 0.714 | 0.561 | 65.1 | 195.9 | |
| Self-CertaintyLLM=Gemma4, Size=31B2026.05 | 0.71 | 0.627 | 64.9 | 13.5 | |
| MetaUELLM=Gemma4-4B2026.05 | 0.709 | 0.475 | 65.6 | 0 | |
| Hidden-State ScoreLLM=Qwen3.5-4B2026.05 | 0.704 | 0.543 | 65.8 | 19.4 | |
| Self-CertaintyBase Model=Llama3, LLM Size=3B2026.05 | 0.703 | 0.471 | 64.7 | 18.9 | |
| MetaUELLM=Qwen3.5, Size=9B2026.05 | 0.701 | 0.601 | 65.2 | 0 | |
| Sequence EntropyLLM=Qwen3.5-4B2026.05 | 0.68 | 0.542 | 63.1 | 19.4 | |
| MetaUEBase Model=Llama3, LLM Size=8B2026.05 | 0.679 | 0.453 | 63.5 | 0 | |
| Semantic EntropyLLM=Gemma4-4B2026.05 | 0.676 | 0.443 | 63.8 | 173 | |
| Self-CertaintyLLM=Gemma4, Size=26B*2026.05 | 0.675 | 0.599 | 62.5 | 13.8 | |
| Self-ConsistencyLLM=Gemma4-4B2026.05 | 0.667 | 0.445 | 62 | 173 | |
| Self-CertaintyLLM=Gemma4-4B2026.05 | 0.66 | 0.445 | 66.1 | 17.4 | |
| MetaUEBase Model=Llama3, LLM Size=3B2026.05 | 0.656 | 0.43 | 62.4 | 0 | |
| Hidden-State ScoreLLM=Gemma4-4B2026.05 | 0.654 | 0.445 | 66.5 | 17.4 | |
| Self-CertaintyLLM=Qwen3.5-4B2026.05 | 0.651 | 0.52 | 61.4 | 19.4 | |
| Sequence EntropyLLM=Gemma4-4B2026.05 | 0.649 | 0.436 | 60.5 | 17.4 | |
| Kernel Lang. Ent.Backbone=Mistral-7B2026.04 | 0.642 | — | — | — | |
| Closeness CentralityBackbone=Mistral-7B2026.04 | 0.612 | — | — | — | |
| Self-CertaintyLLM=Qwen3.5, Size=9B2026.05 | 0.608 | 0.544 | 57.8 | 18.6 | |
| Self-CertaintyLLM=Qwen3.5, Size=35B*2026.05 | 0.591 | 0.528 | 56.3 | 24.4 | |
| SemanticEntropyBackbone=Mistral-7B2026.04 | 0.561 | — | — | — | |
| Max Sequence Prob.Backbone=Mistral-7B2026.04 | 0.561 | — | — | — | |
| PTrueBackbone=Mistral-7B2026.04 | 0.555 | — | — | — | |
| SC ScoreBackbone=Mistral-7B2026.04 | 0.553 | — | — | — | |
| PerplexityBackbone=Mistral-7B2026.04 | 0.513 | — | — | — | |
| Max Token Prob.Backbone=Mistral-7B2026.04 | 0.51 | — | — | — | |
| Self CertaintyBackbone=Mistral-7B2026.04 | 0.51 | — | — | — | |
| Token EntropyBackbone=Mistral-7B2026.04 | 0.502 | — | — | — | |
| Mean Token EntropyBackbone=Mistral-7B2026.04 | 0.498 | — | — | — | |
| SC Based VCBackbone=Mistral-7B2026.04 | 0.483 | — | — | — | |
| AleatoricBackbone=Mistral-7B2026.04 | 0.458 | — | — | — | |
| SelfCheckGPTBackbone=Mistral-7B2026.04 | 0.454 | — | — | — | |
| SC + VCBackbone=Mistral-7B2026.04 | 0.411 | — | — | — |