Selective Generation on MedQUAD Out-of-domain
30.4PRR (ROUGE-L)SATRMD+MSP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SATRMD+MSPModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 30.4 | 30.4 | |
| Maximum Sequence ProbabilityModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 29.7 | 35.6 | |
| HUQ-SATRMDModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 29.3 | 35.5 | |
| SARModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 28.6 | 19.2 | |
| FactoscopeModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 8.4 | 10.1 | |
| SAPLMAModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 7.9 | 12.9 | |
| Semantic EntropyModel=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 7.5 | 0.7 | |
| DegMat NLI Score Entail.Model=Llama 8b v3.1, Evaluation Setting=Out-of-domain, Training Protocol=Leave-one-out2025.02 | 6.6 | 16.2 |