Uncertainty Estimation on SciQA
0.8269AUROCEigenscore
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| EigenscoreBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 0.8269 | 0.0981 | — | -3.08 | |
| EntropyBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 0.8243 | 0.0638 | — | -27.26 | |
| MSPBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 0.8239 | 0.0649 | — | -40.35 | |
| PerplexityBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 0.8157 | 0.0991 | — | -34.38 | |
| EntropyBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 0.7628 | 0.1556 | — | -19.49 | |
| MSPBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 0.7579 | 0.1323 | — | -34.73 | |
| SAR-sTarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.05 | 0.7572 | 0.3618 | 0.76 | — | |
| SARTarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.05 | 0.7519 | 0.256 | 1.31 | — | |
| SAR-sTarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.05 | 0.7496 | 0.3854 | — | — | |
| SEBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 0.7463 | 0.0365 | — | -54.78 | |
| EnergyBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 0.7409 | 0.1384 | — | -1.71 | |
| SARTarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.05 | 0.7388 | 0.2897 | — | — | |
| SAR-tTarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.05 | 0.7387 | 0.2681 | 1.61 | — | |
| SETarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.05 | 0.7293 | 0.2523 | 1.34 | — | |
| SAR-tTarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.05 | 0.7226 | 0.3037 | — | — | |
| PerplexityBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 0.7225 | 0.087 | — | -41.77 | |
| VCTarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.05 | 0.7215 | 0.1947 | 3.93 | — | |
| SETarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.05 | 0.7159 | 0.3054 | — | — | |
| LN-PETarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.05 | 0.7156 | 0.2376 | 2.08 | — | |
| P(True)Target Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.05 | 0.7141 | 0.3192 | 5.78 | — | |
| CoE-CBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 0.7036 | 0.0728 | — | -10.33 | |
| LN-PETarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.05 | 0.6948 | 0.2938 | — | — | |
| VCBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 0.6867 | 0.042 | — | -53.42 | |
| VCTarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.05 | 0.6822 | 0.3188 | — | — | |
| PETarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.05 | 0.6798 | 0.3407 | 1.44 | — | |
| EigenscoreBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 0.6694 | 0.072 | — | -13.37 | |
| CoE-CBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 0.6679 | 0.062 | — | -24.84 | |
| PETarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.05 | 0.6654 | 0.4067 | — | — | |
| VCBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 0.6641 | 0.1402 | — | -33.71 | |
| SETarget Model=OPT-6.7B, Setting=+Corrector2025.05 | 0.6615 | 0.4223 | 1.63 | — | |
| P(True)Backbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 0.659 | 0.0418 | — | -49.81 | |
| P(True)Target Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.05 | 0.6563 | 0.3456 | — | — | |
| LSTarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.05 | 0.6538 | 0.1819 | 11.71 | — | |
| EnergyBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 0.6514 | 0.0428 | — | -13.36 | |
| SETarget Model=OPT-6.7B, Setting=Vanilla2025.05 | 0.6452 | 0.5266 | — | — | |
| LN-PETarget Model=OPT-6.7B, Setting=+Corrector2025.05 | 0.6423 | 0.3468 | 3.35 | — | |
| Semantic EntropyBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 0.6407 | 0.0616 | — | -59.87 | |
| CoE-RBackbone=Llama-3.1-8B, Evaluation Setup=TAC2026.04 | 0.6398 | 0.0419 | — | -16.46 | |
| SAR-tTarget Model=OPT-6.7B, Setting=+Corrector2025.05 | 0.6375 | 0.4419 | 3.74 | — | |
| SARTarget Model=OPT-6.7B, Setting=+Corrector2025.05 | 0.6272 | 0.3899 | 2.68 | — | |
| P(True)Backbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 0.61 | 0.0233 | — | -58.72 | |
| LN-PETarget Model=OPT-6.7B, Setting=Vanilla2025.05 | 0.6088 | 0.4952 | — | — | |
| CoE-RBackbone=Llama-3.2-3B, Evaluation Setup=TAC2026.04 | 0.6077 | 0.0499 | — | -35.05 | |
| SARTarget Model=OPT-6.7B, Setting=Vanilla2025.05 | 0.6004 | 0.494 | — | — | |
| SAR-tTarget Model=OPT-6.7B, Setting=Vanilla2025.05 | 0.6 | 0.4533 | — | — | |
| P(True)Target Model=OPT-6.7B, Setting=+Corrector2025.05 | 0.5949 | 0.3452 | 3.37 | — | |
| VCTarget Model=OPT-6.7B, Setting=+Corrector2025.05 | 0.5655 | 0.3899 | 4.86 | — | |
| P(True)Target Model=OPT-6.7B, Setting=Vanilla2025.05 | 0.5612 | 0.5879 | — | — | |
| PETarget Model=OPT-6.7B, Setting=+Corrector2025.05 | 0.5602 | 0.3692 | 5.95 | — | |
| SAR-sTarget Model=OPT-6.7B, Setting=+Corrector2025.05 | 0.5415 | 0.3483 | 0.95 | — | |
| LSTarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.05 | 0.5367 | 0.3864 | — | — | |
| SAR-sTarget Model=OPT-6.7B, Setting=Vanilla2025.05 | 0.532 | 0.7621 | — | — | |
| VCTarget Model=OPT-6.7B, Setting=Vanilla2025.05 | 0.5169 | 0.6265 | — | — | |
| PETarget Model=OPT-6.7B, Setting=Vanilla2025.05 | 0.5007 | 0.6205 | — | — | |
| LSTarget Model=OPT-6.7B, Setting=+Corrector2025.05 | 0.494 | 0.3278 | 5.29 | — | |
| LSTarget Model=OPT-6.7B, Setting=Vanilla2025.05 | 0.4412 | 0.7638 | — | — |