Selective Generation on MMLU Out-of-domain
0.77PRR (Accuracy)HUQ-SATRMD
Evaluation results
| Method | Links | |
|---|---|---|
| HUQ-SATRMDModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 0.77 | |
| SATRMD+MSPModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 0.681 | |
| Maximum Sequence ProbabilityModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 0.405 | |
| SARModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 0.284 | |
| DegMat NLI Score Entail.Model=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 0.224 | |
| Semantic EntropyModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 0.22 | |
| FactoscopeModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 0.071 | |
| SAPLMAModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | -0.097 |