Uncertainty Estimation on emrQA
77.5AUROCLogit Magnitude
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Logit MagnitudeLLM=Gemma4-4B, generation=Full (τ = T)2026.05 | 77.5 | 76.4 | 71.1 | 21.5 | |
| Logit MagnitudeLLM=Qwen3.5-4B, generation=Full (τ = T)2026.05 | 77.5 | 60.5 | 71.3 | 43.3 | |
| Self-CertaintyLLM=Qwen3.5-4B2026.05 | 77.1 | 60.3 | 70.7 | 43.3 | |
| Logit MagnitudeLLM=Qwen3.5-4B, generation=Early-stopped2026.05 | 77 | 60.4 | 71.1 | 38.2 | |
| Self-CertaintyLLM=Gemma4-4B2026.05 | 76.8 | 75.5 | 70.8 | 21.5 | |
| Sequence EntropyLLM=Qwen3.5-4B2026.05 | 76.5 | 60 | 69.8 | 43.3 | |
| Logit MagnitudeLLM=Gemma4, Size=31B2026.05 | 76.1 | 71 | 70.3 | 19.5 | |
| Self-CertaintyLLM=Gemma4, Size=31B2026.05 | 75.3 | 70.5 | 71.4 | 19.5 | |
| Sequence EntropyLLM=Gemma4-4B2026.05 | 74.5 | 74.5 | 68.9 | 21.5 | |
| Self-CertaintyBase Model=Llama3, LLM Size=8B2026.05 | 74.2 | 60.2 | 68.1 | 20.4 | |
| Logit MagnitudeLLM=Gemma4-4B, generation=Early-stopped2026.05 | 74 | 75 | 68.7 | 17.8 | |
| MetaUELLM=Gemma4-4B2026.05 | 73.3 | 58.8 | 66.7 | 0 | |
| MetaUELLM=Qwen3.5-4B2026.05 | 73.3 | 58.8 | 66.7 | 0 | |
| Logit MagnitudeLLM=Qwen3.5, Size=9B2026.05 | 73.3 | 64.1 | 68.3 | 41.1 | |
| Logit MagnitudeLLM=Gemma4, Size=26B*2026.05 | 73.1 | 72.2 | 67.5 | 21.5 | |
| Self-CertaintyLLM=Gemma4, Size=26B*2026.05 | 72.9 | 71.4 | 68.7 | 21.5 | |
| Self-CertaintyLLM=Qwen3.5, Size=9B2026.05 | 72.9 | 63.8 | 68 | 41.1 | |
| Self-CertaintyBase Model=Llama3, LLM Size=3B2026.05 | 72.5 | 60 | 66.6 | 27.4 | |
| Logit MagnitudeLLM=Qwen3.5, Size=35B*2026.05 | 72 | 55.4 | 68.7 | 38 | |
| Hidden-State ScoreLLM=Qwen3.5-4B2026.05 | 71.9 | 56.9 | 67.1 | 43.3 | |
| LogTokULLM=Qwen3.5-4B2026.05 | 71.8 | 56.5 | 67 | 43.3 | |
| MetaUELLM=Gemma4, Size=31B2026.05 | 71.7 | 68.5 | 66.2 | 0 | |
| Self-CertaintyLLM=Qwen3.5, Size=35B*2026.05 | 71.5 | 55.8 | 68.2 | 38 | |
| MetaUELLM=Qwen3.5, Size=9B2026.05 | 71 | 64.2 | 64.8 | 0 | |
| MetaUELLM=Qwen3.5, Size=35B*2026.05 | 70.2 | 55.1 | 64.6 | 0 | |
| MetaUELLM=Gemma4, Size=26B*2026.05 | 69.6 | 70.7 | 64.8 | 0 | |
| LogTokULLM=Qwen3.5, Size=9B2026.05 | 69.6 | 61.2 | 66.4 | 41.1 | |
| LogTokULLM=Gemma4, Size=26B*2026.05 | 67.2 | 67.1 | 65.8 | 21.5 | |
| Semantic EntropyLLM=Gemma4-4B2026.05 | 67.1 | 70.8 | 62.7 | 216.1 | |
| Semantic EntropyLLM=Qwen3.5-4B2026.05 | 66.7 | 54.7 | 61.8 | 430.2 | |
| Logit MagnitudeBase Model=Llama3, LLM Size=3B2026.05 | 65.5 | 55.2 | 61.8 | 27.4 | |
| Self-ConsistencyLLM=Gemma4-4B2026.05 | 65.5 | 70.3 | 61.5 | 216.1 | |
| Logit MagnitudeBase Model=Llama3, LLM Size=8B2026.05 | 65.1 | 54.8 | 61.2 | 20.4 | |
| MetaUEBase Model=Llama3, LLM Size=8B2026.05 | 64.3 | 54.1 | 60.2 | 0 | |
| LogTokUBase Model=Llama3, LLM Size=8B2026.05 | 63.1 | 51.8 | 60.7 | 20.4 | |
| LogTokUBase Model=Llama3, LLM Size=3B2026.05 | 61.6 | 51.8 | 60 | 27.4 | |
| MetaUEBase Model=Llama3, LLM Size=3B2026.05 | 60.9 | 53.1 | 57.7 | 0 | |
| Self-ConsistencyLLM=Qwen3.5-4B2026.05 | 60.1 | 50.6 | 56.7 | 430.2 | |
| LogTokULLM=Qwen3.5, Size=35B*2026.05 | 58.3 | 43 | 61.3 | 38 | |
| LogTokULLM=Gemma4-4B2026.05 | 52.4 | 60.7 | 54.4 | 21.5 | |
| LogTokULLM=Gemma4, Size=31B2026.05 | 51.2 | 51.4 | 57.2 | 19.5 | |
| Hidden-State ScoreLLM=Gemma4-4B2026.05 | 37.3 | 51.5 | 50.6 | 21.5 |