Selective Generation on MedQUAD (PRR Metrics)
46.6PRR (ROUGE-L)SATRMD+MSP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SATRMD+MSPModel=Llama 8b v3.12025.02 | 46.6 | 57.5 | |
| PerplexityModel=Llama 8b v3.12025.02 | 42.5 | 43.8 | |
| SAPLMAModel=Llama 8b v3.12025.02 | 40.7 | 49 | |
| HUQ-SATRMDModel=Llama 8b v3.12025.02 | 38.6 | 50.6 | |
| FactoscopeModel=Llama 8b v3.12025.02 | 35.8 | 42.8 | |
| Maximum Sequence ProbabilityModel=Llama 8b v3.12025.02 | 29.7 | 35.6 | |
| SARModel=Llama 8b v3.12025.02 | 28.6 | 19.2 | |
| Lexical Similarity ROUGE-LModel=Llama 8b v3.12025.02 | 25.2 | 13.2 | |
| Semantic EntropyModel=Llama 8b v3.12025.02 | 7.5 | 0.7 | |
| Eccentricity NLI Score Entail.Model=Llama 8b v3.12025.02 | 7 | 6 | |
| DegMat NLI Score Entail.Model=Llama 8b v3.12025.02 | 6.6 | 16.2 | |
| EigValLaplacian NLI Score Entail.Model=Llama 8b v3.12025.02 | 5.6 | 16 | |
| EigenScoreModel=Llama 8b v3.12025.02 | 5 | 4.3 | |
| SentenceSARModel=Llama 8b v3.12025.02 | 1.5 | 3.3 |