Uncertainty Quantification on MMLU-pro (test)
77.74AUROCCAGE-CAL
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CAGE-CALRollouts=3, Topologies=52026.05 | 77.74 | — | — | — | 61.54 | |
| BaseModel=LLAMA-8B, Position=After2026.05 | 74.1 | 36.6 | 0.423 | 0.455 | — | |
| BaseModel=GEMMA-12B, Position=After2026.05 | 73.2 | 52.6 | 0.387 | 0.406 | — | |
| UnlearnedModel=PHI-4, Position=After2026.05 | 71.9 | 68 | 0.206 | 0.144 | — | |
| DropoutModel=LLAMA-8B, Position=After2026.05 | 70.8 | 36.6 | 0.219 | 0.1 | — | |
| DropoutModel=GEMMA-12B, Position=After2026.05 | 69.4 | 52.6 | 0.301 | 0.274 | — | |
| BaseModel=PHI-4, Position=After2026.05 | 68.3 | 68 | 0.265 | 0.261 | — | |
| UnlearnedModel=GEMMA-12B, Position=After2026.05 | 67.6 | 52.6 | 0.269 | 0.176 | — | |
| DropoutModel=PHI-4, Position=After2026.05 | 65.5 | 68 | 0.206 | 0.086 | — | |
| BaseModel=PHI-4, Position=Before2026.05 | 63.6 | 68 | 0.219 | 0.106 | — | |
| UnlearnedModel=LLAMA-8B, Position=After2026.05 | 63.1 | 36.6 | 0.281 | 0.212 | — | |
| UnlearnedModel=PHI-4, Position=Before2026.05 | 62.1 | 68 | 0.225 | 0.12 | — | |
| Plurality voteRollouts=3, Topologies=52026.05 | 61.78 | — | — | — | 53.02 | |
| MATURollouts=3, Topologies=52026.05 | 61.39 | — | — | — | 52.2 | |
| Answer entropyRollouts=3, Topologies=52026.05 | 57.92 | — | — | — | 51.01 | |
| DAERollouts=3, Topologies=52026.05 | 57.91 | — | — | — | 51.01 | |
| DropoutModel=LLAMA-8B, Position=Before2026.05 | 56.3 | 36.6 | 0.252 | 0.125 | — | |
| BaseModel=LLAMA-8B, Position=Before2026.05 | 55.8 | 36.6 | 0.552 | 0.565 | — | |
| UnlearnedModel=GEMMA-12B, Position=Before2026.05 | 55.4 | 52.6 | 0.409 | 0.399 | — | |
| DropoutModel=PHI-4, Position=Before2026.05 | 54.2 | 68 | 0.241 | 0.141 | — | |
| UnlearnedModel=LLAMA-8B, Position=Before2026.05 | 53.4 | 36.6 | 0.376 | 0.347 | — | |
| DropoutModel=GEMMA-12B, Position=Before2026.05 | 49.2 | 52.6 | 0.365 | 0.324 | — | |
| BaseModel=GEMMA-12B, Position=Before2026.05 | 46.7 | 52.6 | 0.453 | 0.448 | — | |
| Avg-logprobRollouts=3, Topologies=52026.05 | 43.65 | — | — | — | 27.33 |