Question Answering on MGSM
90.79AUCSATMD + MSP
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SATMD + MSPBackbone=Llama-3.1-8B-Instruct2026.04 | 90.79 | 35.09 | 69.12 | |
| SIVR (Ours)Backbone=Llama-3.1-8B-Instruct2026.04 | 89.04 | 17.54 | 58.13 | |
| SIVRBackbone=Llama-3.1-8B-Instruct2026.04 | 89.04 | 17.54 | 58.13 | |
| SATMD + MSPModel=Ministral-8B-Instruct2026.04 | 86.38 | 53.97 | 74.94 | |
| SIVRModel=Ministral-8B-Instruct2026.04 | 84.7 | 58.78 | 73.12 | |
| SIVRBackbone=Ministral-8B-Instruct2026.04 | 84.7 | 58.78 | 73.12 | |
| EnergyBackbone=Llama-3.1-8B-Instruct2026.04 | 83.73 | 58.82 | 54.66 | |
| SAPLMABackbone=Llama-3.1-8B-Instruct2026.04 | 81.03 | 41.38 | 48.44 | |
| SAPLMABackbone=Llama-3.1-8B-Instruct2026.04 | 81.03 | 41.38 | 48.44 | |
| Lookback LensBackbone=Llama-3.1-8B-Instruct2026.04 | 79.56 | 64.56 | 71.44 | |
| TADModel=Ministral-8B-Instruct2026.04 | 79.11 | 62.96 | 63.11 | |
| Lookback LensModel=Ministral-8B-Instruct2026.04 | 78.06 | 66.67 | 43.78 | |
| TADBackbone=Llama-3.1-8B-Instruct2026.04 | 77.37 | 56.03 | 55.88 | |
| SAPLMAmodel=Llama-3.2-3B-Instruct2026.04 | 76.48 | 72.53 | 70.33 | |
| SAPLMABackbone=Llama-3.2-3B-Instruct2026.04 | 76.48 | 72.53 | 70.33 | |
| TADmodel=Llama-3.2-3B-Instruct2026.04 | 76.28 | 41.51 | 51.31 | |
| SIVRmodel=Llama-3.2-3B-Instruct2026.04 | 75.65 | 74.42 | 69.23 | |
| SIVRBackbone=Llama-3.2-3B-Instruct2026.04 | 75.65 | 74.42 | 69.23 | |
| SEModel=Ministral-8B-Instruct2026.04 | 73.94 | 63.08 | 60.17 | |
| SATMD + MSPmodel=Llama-3.2-3B-Instruct2026.04 | 72.27 | 77.89 | 55.4 | |
| Lookback Lensmodel=Llama-3.2-3B-Instruct2026.04 | 71.05 | 83.33 | 62.66 | |
| SARModel=Ministral-8B-Instruct2026.04 | 70.44 | 69.84 | 53.94 | |
| SAPLMAModel=Ministral-8B-Instruct2026.04 | 70.34 | 88.89 | 57.77 | |
| SAPLMABackbone=Ministral-8B-Instruct2026.04 | 70.34 | 88.89 | 57.77 | |
| EntropyModel=Ministral-8B-Instruct2026.04 | 69.72 | 78.14 | 45.82 | |
| EnergyModel=Ministral-8B-Instruct2026.04 | 69.46 | 80.65 | 43.5 | |
| PerplexityModel=Ministral-8B-Instruct2026.04 | 68.97 | 80.35 | 47.65 | |
| Max ProbModel=Ministral-8B-Instruct2026.04 | 68.87 | 82.27 | 47.84 | |
| Temp-ScaledModel=Ministral-8B-Instruct2026.04 | 67.67 | 83.46 | 47.03 | |
| Energymodel=Llama-3.2-3B-Instruct2026.04 | 67.14 | 67.85 | 55.47 | |
| SEmodel=Llama-3.2-3B-Instruct2026.04 | 66.34 | 88.89 | 50.44 | |
| CoE-RBackbone=Llama-3.1-8B-Instruct2026.04 | 66.09 | 95.85 | 32.2 | |
| SEBackbone=Llama-3.1-8B-Instruct2026.04 | 66 | 76.67 | 56.88 | |
| Entropymodel=Llama-3.2-3B-Instruct2026.04 | 64.65 | 93.57 | 55.57 | |
| SARBackbone=Llama-3.1-8B-Instruct2026.04 | 63.71 | 66.67 | 52.09 | |
| EntropyBackbone=Llama-3.1-8B-Instruct2026.04 | 61.45 | 96.19 | 38.87 | |
| Perplexitymodel=Llama-3.2-3B-Instruct2026.04 | 61.42 | 94.9 | 52.78 | |
| CoE-CModel=Ministral-8B-Instruct2026.04 | 60.88 | 90.1 | 38.13 | |
| Max Probmodel=Llama-3.2-3B-Instruct2026.04 | 60.49 | 95.34 | 51.94 | |
| PerplexityBackbone=Llama-3.1-8B-Instruct2026.04 | 58.82 | 96.19 | 35.32 | |
| Max ProbBackbone=Llama-3.1-8B-Instruct2026.04 | 58.54 | 96.19 | 35.27 | |
| Temp-Scaledmodel=Llama-3.2-3B-Instruct2026.04 | 57.33 | 94.46 | 48.79 | |
| Temp-ScaledBackbone=Llama-3.1-8B-Instruct2026.04 | 57.2 | 96.54 | 31.35 | |
| P(True)Backbone=Llama-3.1-8B-Instruct2026.04 | 56.43 | 95.85 | 21.02 | |
| CoE-Rmodel=Llama-3.2-3B-Instruct2026.04 | 54.84 | 92.9 | 45.15 | |
| CoE-RModel=Ministral-8B-Instruct2026.04 | 53.07 | 96.01 | 34.22 | |
| P(True)Model=Ministral-8B-Instruct2026.04 | 52.28 | 96.16 | 31.9 | |
| CoE-CBackbone=Llama-3.1-8B-Instruct2026.04 | 52.18 | 94.46 | 30.61 | |
| P(True)model=Llama-3.2-3B-Instruct2026.04 | 50.79 | 96.67 | 42.17 | |
| CoE-Cmodel=Llama-3.2-3B-Instruct2026.04 | 46.24 | 90.24 | 37.03 | |
| SARmodel=Llama-3.2-3B-Instruct2026.04 | 39.67 | 97.62 | 33.7 |