Long-form generation factuality and uncertainty estimation on WildHallu (test)
0.86Factuality ScoreLOGU-DPO
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| LOGU-DPOBackbone=Llama3-8B-Instruct, Method Category=Training-Based2024.10 | 0.86 | 52.8 | 2.68 | |
| LOGU-DPOBackbone=Mistral-7B-Instruct, Method Category=Training-Based2024.10 | 0.844 | 61.8 | 3.49 | |
| Pair-FewBackbone=Llama3-8B-Instruct, Method Category=Prompt-Based2024.10 | 0.84 | 48.1 | 2.33 | |
| Unc-FewBackbone=Llama3-8B-Instruct, Method Category=Prompt-Based2024.10 | 0.802 | 48.6 | 3.44 | |
| LOGU-SFTBackbone=Mistral-7B-Instruct, Method Category=Training-Based2024.10 | 0.792 | 51.1 | 5.73 | |
| LOGU-SFTBackbone=Llama3-8B-Instruct, Method Category=Training-Based2024.10 | 0.789 | 44.6 | 5.37 | |
| Unc-ZeroBackbone=Llama3-8B-Instruct, Method Category=Prompt-Based2024.10 | 0.787 | 45.8 | 3.29 | |
| Self-RefineBackbone=Llama3-8B-Instruct, Method Category=Prompt-Based2024.10 | 0.77 | 28.7 | 5.13 | |
| Unc-ZeroBackbone=Mistral-7B-Instruct, Method Category=Prompt-Based2024.10 | 0.754 | 48.6 | 5.2 | |
| Orig.Backbone=Llama3-8B-Instruct, Method Category=Original Model2024.10 | 0.744 | — | 6.24 | |
| Pair-FewBackbone=Mistral-7B-Instruct, Method Category=Prompt-Based2024.10 | 0.734 | 51.6 | 9.8 | |
| Unc-FewBackbone=Mistral-7B-Instruct, Method Category=Prompt-Based2024.10 | 0.726 | 57.3 | 10.1 | |
| Self-RefineBackbone=Mistral-7B-Instruct, Method Category=Prompt-Based2024.10 | 0.724 | 50.6 | 8.29 | |
| Orig.Backbone=Mistral-7B-Instruct, Method Category=Original Model2024.10 | 0.715 | — | 8.31 |