Long-form generation factuality and uncertainty estimation on LongFact (test)
91.5Factuality ScoreLOGU-DPO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LOGU-DPOBackbone=Llama3-8B-Instruct, Method Category=Training-Based2024.10 | 91.5 | 47.5 | 2.36 | |
| LOGU-DPOBackbone=Mistral-7B-Instruct, Method Category=Training-Based2024.10 | 91.3 | 54.6 | 2.09 | |
| Unc-ZeroBackbone=Llama3-8B-Instruct, Method Category=Prompt-Based2024.10 | 89.3 | 30.4 | 2.67 | |
| LOGU-SFTBackbone=Mistral-7B-Instruct, Method Category=Training-Based2024.10 | 88.6 | 43.5 | 3.21 | |
| Unc-ZeroBackbone=Mistral-7B-Instruct, Method Category=Prompt-Based2024.10 | 88.5 | 29.9 | 1.85 | |
| Unc-FewBackbone=Llama3-8B-Instruct, Method Category=Prompt-Based2024.10 | 88.3 | 40.6 | 4.01 | |
| LOGU-SFTBackbone=Llama3-8B-Instruct, Method Category=Training-Based2024.10 | 87.5 | 44.8 | 4.53 | |
| Pair-FewBackbone=Llama3-8B-Instruct, Method Category=Prompt-Based2024.10 | 86.7 | 42.2 | 4.78 | |
| Orig.Backbone=Mistral-7B-Instruct, Method Category=Original Model2024.10 | 86.2 | — | 4.55 | |
| Self-RefineBackbone=Mistral-7B-Instruct, Method Category=Prompt-Based2024.10 | 86.2 | 42.4 | 4.61 | |
| Pair-FewBackbone=Mistral-7B-Instruct, Method Category=Prompt-Based2024.10 | 86.1 | 35.1 | 5.08 | |
| Unc-FewBackbone=Mistral-7B-Instruct, Method Category=Prompt-Based2024.10 | 85.6 | 32.4 | 5.58 | |
| Orig.Backbone=Llama3-8B-Instruct, Method Category=Original Model2024.10 | 85.5 | — | 7.45 | |
| Self-RefineBackbone=Llama3-8B-Instruct, Method Category=Prompt-Based2024.10 | 85 | 27.2 | 6.38 |