Selective Generation on SamSum (PRR metrics)
48.6PRR (ROUGE-L)HUQ-SATRMD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HUQ-SATRMDModel=Llama 8b v3.12025.02 | 48.6 | 0.297 | |
| Maximum Sequence ProbabilityModel=Llama 8b v3.12025.02 | 45.2 | 0.218 | |
| SATRMD+MSPModel=Llama 8b v3.12025.02 | 38.3 | 0.408 | |
| Lexical Similarity ROUGE-LModel=Llama 8b v3.12025.02 | 32 | 0.209 | |
| Semantic EntropyModel=Llama 8b v3.12025.02 | 31.1 | 0.206 | |
| SARModel=Llama 8b v3.12025.02 | 30 | 0.217 | |
| SentenceSARModel=Llama 8b v3.12025.02 | 26.4 | 0.215 | |
| EigValLaplacian NLI Score Entail.Model=Llama 8b v3.12025.02 | 25.1 | 0.224 | |
| DegMat NLI Score Entail.Model=Llama 8b v3.12025.02 | 25 | 0.226 | |
| SAPLMAModel=Llama 8b v3.12025.02 | 24.3 | 0.313 | |
| EigenScoreModel=Llama 8b v3.12025.02 | 19.6 | 0.15 | |
| Eccentricity NLI Score Entail.Model=Llama 8b v3.12025.02 | 16 | 0.117 | |
| PerplexityModel=Llama 8b v3.12025.02 | 8 | 0.308 | |
| FactoscopeModel=Llama 8b v3.12025.02 | 3.4 | -0.024 |