Question Answering on MMLU (AUC, F95, AUP)
64.07AUCSIVR (Ours)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SIVR (Ours)Backbone=Llama-3.1-8B-Instruct2026.04 | 64.07 | 91.3 | 88.6 | |
| SIVRBackbone=Llama-3.1-8B-Instruct2026.04 | 64.07 | 91.3 | 88.6 | |
| SIVRModel=Ministral-8B-Instruct2026.04 | 55.54 | 91.46 | 77.69 | |
| SIVRBackbone=Ministral-8B-Instruct2026.04 | 55.54 | 91.46 | 77.69 | |
| TADModel=Ministral-8B-Instruct2026.04 | 54.89 | 94.74 | 81.29 | |
| SAPLMAModel=Ministral-8B-Instruct2026.04 | 54.66 | 90.28 | 80.01 | |
| SAPLMABackbone=Llama-3.1-8B-Instruct2026.04 | 54.64 | 97.06 | 74.8 | |
| Temp-ScaledModel=Ministral-8B-Instruct2026.04 | 52.86 | 94.69 | 76.48 | |
| Lookback LensModel=Ministral-8B-Instruct2026.04 | 52.66 | 92.54 | 77.79 | |
| P(True)model=Llama-3.2-3B-Instruct2026.04 | 52.63 | 93.53 | 74.28 | |
| Max ProbModel=Ministral-8B-Instruct2026.04 | 52.56 | — | 76.37 | |
| PerplexityModel=Ministral-8B-Instruct2026.04 | 51.9 | 94.97 | 76.33 | |
| EntropyModel=Ministral-8B-Instruct2026.04 | 50.96 | 94.13 | 76.23 | |
| SIVRmodel=Llama-3.2-3B-Instruct2026.04 | 50.89 | 84.42 | 74.97 | |
| SIVRBackbone=Llama-3.2-3B-Instruct2026.04 | 50.89 | 84.42 | 74.97 | |
| SEBackbone=Llama-3.1-8B-Instruct2026.04 | 50.56 | 96.83 | 71.36 | |
| CoE-CBackbone=Llama-3.1-8B-Instruct2026.04 | 50.53 | 93.45 | 70.47 | |
| SAPLMAmodel=Llama-3.2-3B-Instruct2026.04 | 50.49 | 95 | 75.42 | |
| CoE-Cmodel=Llama-3.2-3B-Instruct2026.04 | 50.42 | 94.53 | 73.1 | |
| TADBackbone=Llama-3.1-8B-Instruct2026.04 | 50.38 | 95.89 | 75.34 | |
| TADmodel=Llama-3.2-3B-Instruct2026.04 | 50.19 | 91.76 | 71.39 | |
| Temp-ScaledBackbone=Llama-3.1-8B-Instruct2026.04 | 50.02 | 97.02 | 70.17 | |
| CoE-Rmodel=Llama-3.2-3B-Instruct2026.04 | 49.87 | 95.52 | 72.66 | |
| Max ProbBackbone=Llama-3.1-8B-Instruct2026.04 | 49.79 | 97.02 | 70.02 | |
| PerplexityBackbone=Llama-3.1-8B-Instruct2026.04 | 49.67 | 97.02 | 69.66 | |
| SARBackbone=Llama-3.1-8B-Instruct2026.04 | 49.54 | 92.06 | 69.51 | |
| SATMD + MSPModel=Ministral-8B-Instruct2026.04 | 49.46 | 97.22 | 76.97 | |
| SATMD + MSPBackbone=Ministral-8B-Instruct2026.04 | 49.46 | 97.22 | 76.97 | |
| SARModel=Ministral-8B-Instruct2026.04 | 49.34 | 95.83 | 76.16 | |
| EnergyModel=Ministral-8B-Instruct2026.04 | 49.25 | 93.85 | 75.35 | |
| EntropyBackbone=Llama-3.1-8B-Instruct2026.04 | 49.1 | 96.43 | 69.16 | |
| P(True)Model=Ministral-8B-Instruct2026.04 | 48.85 | 95.53 | 74.96 | |
| CoE-RBackbone=Llama-3.1-8B-Instruct2026.04 | 48.34 | 93.45 | 69.83 | |
| SEModel=Ministral-8B-Instruct2026.04 | 48.03 | 87.5 | 74.23 | |
| Entropymodel=Llama-3.2-3B-Instruct2026.04 | 48 | 96.02 | 71.72 | |
| P(True)Backbone=Llama-3.1-8B-Instruct2026.04 | 47.98 | 96.43 | 68.41 | |
| SATMD + MSPBackbone=Llama-3.1-8B-Instruct2026.04 | 47.91 | 96.43 | 73.15 | |
| SATMD + MSPBackbone=Llama-3.1-8B-Instruct2026.04 | 47.91 | 96.43 | 73.15 | |
| CoE-RModel=Ministral-8B-Instruct2026.04 | 47.88 | 96.37 | 74.33 | |
| SEmodel=Llama-3.2-3B-Instruct2026.04 | 47.82 | 92.31 | 72.09 | |
| Perplexitymodel=Llama-3.2-3B-Instruct2026.04 | 47.76 | 96.52 | 71.57 | |
| Max Probmodel=Llama-3.2-3B-Instruct2026.04 | 47.67 | 96.27 | 71.42 | |
| Temp-Scaledmodel=Llama-3.2-3B-Instruct2026.04 | 47.66 | 96.27 | 71.38 | |
| EnergyBackbone=Llama-3.1-8B-Instruct2026.04 | 47.48 | 92.86 | 67.25 | |
| Energymodel=Llama-3.2-3B-Instruct2026.04 | 47.33 | 96.02 | 72.44 | |
| Lookback Lensmodel=Llama-3.2-3B-Instruct2026.04 | 47.14 | 96.1 | 69.58 | |
| SATMD + MSPmodel=Llama-3.2-3B-Instruct2026.04 | 47.03 | 94.25 | 68.08 | |
| SATMD + MSPBackbone=Llama-3.2-3B-Instruct2026.04 | 47.03 | 94.25 | 68.08 | |
| CoE-CModel=Ministral-8B-Instruct2026.04 | 46.2 | 96.93 | 73.75 | |
| Lookback LensBackbone=Llama-3.1-8B-Instruct2026.04 | 46.13 | 93.44 | 72.04 | |
| SARmodel=Llama-3.2-3B-Instruct2026.04 | 45.39 | 98.08 | 73.01 |