Error prediction on NCQA (test)
99.2AUROCLlama-3.1-8B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama-3.1-8BFeatures=Semantic energy, Strategy=oracle gated2026.06 | 99.2 | — | |
| Llama-3.1-8BFeatures=SAR, Strategy=oracle gated2026.06 | 99.2 | — | |
| Llama-3.1-8BFeatures=all UQ, Strategy=oracle gated2026.06 | 99.1 | — | |
| Llama-3.1-8BFeatures=MI, Strategy=oracle gated2026.06 | 99.1 | — | |
| Llama-3.1-8BFeatures=Semantic entropy, Strategy=oracle gated2026.06 | 98.9 | — | |
| Llama-3.1-8BFeatures=CoCoA, Strategy=oracle gated2026.06 | 98.8 | — | |
| Llama-3.1-8BFeatures=MSP, Strategy=oracle gated2026.06 | 98.5 | — | |
| Qwen3-14BFeatures=SAR, Strategy=oracle gated2026.06 | 98.4 | — | |
| Spectrum-Llama-3.1-8BFeatures=all UQ, Strategy=oracle gated2026.06 | 98.3 | — | |
| Spectrum-Qwen3-14BFeatures=all UQ, Strategy=oracle gated2026.06 | 98.2 | — | |
| Qwen3-14BFeatures=all UQ, Strategy=oracle gated2026.06 | 97 | — | |
| A2Search-7BFeatures=MSP, Strategy=oracle gated2026.06 | 93.8 | — | |
| A2Search-7BFeatures=all UQ, Strategy=oracle gated2026.06 | 93.1 | — | |
| Qwen3-14B (all UQ - latent gated)Model=Qwen3-14B, Features=all UQ, Strategy=latent gated, Supervision=LLM-judge2026.06 | 92.7 | — | |
| A2Search-7BFeatures=SAR, Strategy=oracle gated2026.06 | 91.8 | — | |
| Llama-3.1-8B (all UQ - latent gated)Model=Llama-3.1-8B, Features=all UQ, Strategy=latent gated, Supervision=LLM-judge2026.06 | 84.6 | — | |
| Spectrum-Llama-3.1-8B (SAR - baseline)Model=Spectrum-Llama-3.1-8B, Features=SAR, Strategy=baseline, Supervision=LLM-judge2026.06 | 83.7 | — | |
| Qwen3-14BFeatures=SAR, Strategy=baseline2026.06 | 79.4 | — | |
| Qwen3-14BFeatures=SAR, Strategy=latent gated2026.06 | 77.6 | — | |
| Qwen3-14BFeatures=SAR, Strategy=latent selective2026.06 | 76.8 | — | |
| A2Search-7BFeatures=MSP, Strategy=latent gated2026.06 | 76.8 | — | |
| A2Search-7BFeatures=SAR, Strategy=latent gated2026.06 | 76.6 | — | |
| A2Search-7BFeatures=SAR, Strategy=baseline2026.06 | 76.5 | — | |
| Spectrum-Qwen3-14BFeatures=all UQ, Strategy=baseline2026.06 | 75.2 | — | |
| A2Search-7BFeatures=SAR, Strategy=oracle selective2026.06 | 74.8 | — | |
| Spectrum-Llama-3.1-8B (all UQ - latent selective)Model=Spectrum-Llama-3.1-8B, Features=all UQ, Strategy=latent selective, Supervision=LLM-judge2026.06 | 73.9 | — | |
| A2Search-7B (all UQ - latent gated)Model=A2Search-7B, Features=all UQ, Strategy=latent gated, Supervision=LLM-judge2026.06 | 73.7 | — | |
| Qwen3-14BFeatures=SAR, Strategy=oracle selective2026.06 | 73 | — | |
| Qwen3-14BFeatures=all UQ, Strategy=latent gated2026.06 | 72.3 | — | |
| A2Search-7BFeatures=MSP, Strategy=latent selective2026.06 | 72.2 | — | |
| Llama-3.1-8B (all UQ)Model=Llama-3.1-8B, Features=all UQ, Strategy=baseline, Supervision=LLM-judge2026.06 | 71.8 | — | |
| Qwen3-14B (Semantic energy - latent selective)Model=Qwen3-14B, Features=Semantic energy, Strategy=latent selective, Supervision=LLM-judge2026.06 | 70.8 | — | |
| Qwen3-14BFeatures=all UQ, Strategy=latent selective2026.06 | 70.4 | — | |
| A2Search-7BFeatures=SAR, Strategy=latent selective2026.06 | 69.9 | — | |
| A2Search-7BFeatures=MSP, Strategy=baseline2026.06 | 69 | — | |
| A2Search-7B (MSP - oracle selective)Model=A2Search-7B, Features=MSP, Strategy=oracle selective, Supervision=LLM-judge2026.06 | 68.7 | — | |
| Spectrum-Qwen3-14B (SAR - latent gated)Model=Spectrum-Qwen3-14B, Features=SAR, Strategy=latent gated, Supervision=LLM-judge2026.06 | 68.6 | — | |
| A2Search-7BFeatures=MSP, Strategy=oracle selective2026.06 | 68.3 | — | |
| Spectrum-Qwen3-14BFeatures=all UQ, Strategy=latent gated2026.06 | 68 | — | |
| Llama-3.1-8B (CoCoA - baseline)Model=Llama-3.1-8B, Features=CoCoA, Strategy=baseline, Supervision=LLM-judge2026.06 | 67.6 | — | |
| Spectrum-Llama-3.1-8BFeatures=all UQ, Strategy=baseline2026.06 | 67.2 | — | |
| Qwen3-14BFeatures=all UQ, Strategy=baseline2026.06 | 66.7 | — | |
| A2Search-7BFeatures=all UQ, Strategy=oracle selective2026.06 | 66.7 | — | |
| Llama-3.1-8BFeatures=MI, Strategy=latent gated2026.06 | 66.5 | — | |
| A2Search-7BFeatures=all UQ, Strategy=baseline2026.06 | 66.2 | — | |
| Qwen3-14BFeatures=all UQ, Strategy=oracle selective2026.06 | 65.5 | — | |
| Llama-3.1-8BFeatures=SAR, Strategy=latent gated2026.06 | 65.2 | — | |
| Llama-3.1-8BFeatures=all UQ, Strategy=latent selective2026.06 | 64.9 | — | |
| A2Search-7BFeatures=all UQ, Strategy=latent selective2026.06 | 64.9 | — | |
| Llama-3.1-8BFeatures=MI, Strategy=latent selective2026.06 | 64.6 | — | |
| Llama-3.1-8BFeatures=MSP, Strategy=latent gated2026.06 | 63.9 | — | |
| Llama-3.1-8BFeatures=MSP, Strategy=latent selective2026.06 | 63.7 | — | |
| Llama-3.1-8BFeatures=MSP, Strategy=baseline2026.06 | 63.6 | — | |
| Llama-3.1-8BFeatures=SAR, Strategy=latent selective2026.06 | 63.2 | — | |
| Llama-3.1-8BFeatures=Semantic entropy, Strategy=baseline2026.06 | 62 | — | |
| Llama-3.1-8BFeatures=MI, Strategy=oracle selective2026.06 | 61.8 | — | |
| Llama-3.1-8BFeatures=all UQ, Strategy=baseline2026.06 | 61.5 | — | |
| Llama-3.1-8BFeatures=CoCoA, Strategy=oracle selective2026.06 | 61 | — | |
| Llama-3.1-8BFeatures=SAR, Strategy=baseline2026.06 | 60.9 | — | |
| Llama-3.1-8BFeatures=SAR, Strategy=oracle selective2026.06 | 60.5 | — | |
| A2Search-7BFeatures=all UQ, Strategy=latent gated2026.06 | 60.4 | — | |
| Llama-3.1-8BFeatures=all UQ, Strategy=latent gated2026.06 | 59.9 | — | |
| Llama-3.1-8BFeatures=CoCoA, Strategy=baseline2026.06 | 59.8 | — | |
| Llama-3.1-8BFeatures=Semantic energy, Strategy=latent gated2026.06 | 59.1 | — | |
| Llama-3.1-8BFeatures=MSP, Strategy=oracle selective2026.06 | 59 | — | |
| Llama-3.1-8BFeatures=CoCoA, Strategy=latent gated2026.06 | 58.4 | — | |
| Llama-3.1-8BFeatures=Semantic entropy, Strategy=latent gated2026.06 | 58.1 | — | |
| Llama-3.1-8BFeatures=all UQ, Strategy=oracle selective2026.06 | 57.8 | — | |
| Spectrum-Llama-3.1-8BFeatures=all UQ, Strategy=latent gated2026.06 | 57.8 | — | |
| Llama-3.1-8BFeatures=Semantic energy, Strategy=baseline2026.06 | 56.9 | — | |
| Llama-3.1-8BFeatures=Semantic energy, Strategy=latent selective2026.06 | 56.9 | — | |
| Llama-3.1-8BFeatures=Semantic entropy, Strategy=latent selective2026.06 | 54.7 | — | |
| Llama-3.1-8BFeatures=CoCoA, Strategy=latent selective2026.06 | 54.2 | — | |
| Llama-3.1-8BFeatures=MI, Strategy=baseline2026.06 | 52.9 | — | |
| Llama-3.1-8BFeatures=Semantic energy, Strategy=oracle selective2026.06 | 50.8 | — | |
| Llama-3.1-8BFeatures=Semantic entropy, Strategy=oracle selective2026.06 | 49 | — | |
| baselineFeatures=all UQ, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.499 | |
| baselineFeatures=CoCoA, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.476 | |
| baselineFeatures=Sem. energy, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.468 | |
| baselineFeatures=Sem. entropy, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.086 | |
| baselineFeatures=SAR, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.684 | |
| baselineFeatures=MSP, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.444 | |
| baselineFeatures=MI, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | -0.16 | |
| latent gatedFeatures=all UQ, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.577 | |
| latent gatedFeatures=CoCoA, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.606 | |
| latent gatedFeatures=Sem. energy, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.489 | |
| latent gatedFeatures=Sem. entropy, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.533 | |
| latent gatedFeatures=SAR, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.644 | |
| latent gatedFeatures=MSP, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.623 | |
| latent gatedFeatures=MI, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.576 | |
| latent selectiveFeatures=all UQ, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.534 | |
| latent selectiveFeatures=CoCoA, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.52 | |
| latent selectiveFeatures=Sem. energy, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.436 | |
| latent selectiveFeatures=Sem. entropy, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | -0.545 | |
| latent selectiveFeatures=SAR, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | -0.319 | |
| latent selectiveFeatures=MSP, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.645 | |
| latent selectiveFeatures=MI, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.576 | |
| oracle gatedFeatures=all UQ, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.967 | |
| oracle gatedFeatures=CoCoA, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.963 | |
| oracle gatedFeatures=Sem. energy, Backbone=Qwen3-14B, Supervision=AlignScore2026.06 | — | 0.971 |