Factuality Hallucination Evaluation on LongFact (test)
100Response ScoreVanilla
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| VanillaInstruction Dataset=Alpaca, Sample Ratio=100%2025.02 | 100 | 17.1 | 85.8 | 80.3 | 83.1 | — | — | — | — | — | |
| FLAME-DPO factInstruction Dataset=Alpaca, Sample Ratio=100%2025.02 | 100 | 15.6 | 88.3 | 81.2 | 84.8 | — | — | — | — | — | |
| SELF-EVALInstruction Dataset=Alpaca, Sample Ratio=100%2025.02 | 100 | 16.9 | 87.8 | 81 | 84.4 | — | — | — | — | — | |
| IFDInstruction Dataset=Alpaca, Sample Ratio=5%2025.02 | 100 | 21 | 87.2 | 80.5 | 83.9 | — | — | — | — | — | |
| CaRInstruction Dataset=Alpaca, Sample Ratio=5%2025.02 | 100 | 16.2 | 86.6 | 79.1 | 82.9 | — | — | — | — | — | |
| NuggetsInstruction Dataset=Alpaca, Sample Ratio=5%2025.02 | 100 | 18.3 | 88.6 | 81.2 | 84.9 | — | — | — | — | — | |
| NOVAInstruction Dataset=Alpaca, Sample Ratio=5%2025.02 | 100 | 17.9 | 92.4 | 82.7 | 87.6 | — | — | — | — | — | |
| IFDInstruction Dataset=Alpaca, Sample Ratio=10%2025.02 | 100 | 20.5 | 86.3 | 79.2 | 82.8 | — | — | — | — | — | |
| CaRInstruction Dataset=Alpaca, Sample Ratio=10%2025.02 | 100 | 24.3 | 87.1 | 81.3 | 84.2 | — | — | — | — | — | |
| NuggetsInstruction Dataset=Alpaca, Sample Ratio=10%2025.02 | 100 | 27.1 | 86.7 | 80.4 | 83.6 | — | — | — | — | — | |
| NOVAInstruction Dataset=Alpaca, Sample Ratio=10%2025.02 | 100 | 18.4 | 89.1 | 81.6 | 85.4 | — | — | — | — | — | |
| IFDInstruction Dataset=Alpaca, Sample Ratio=15%2025.02 | 100 | 19.4 | 84.7 | 80.7 | 82.7 | — | — | — | — | — | |
| CaRInstruction Dataset=Alpaca, Sample Ratio=15%2025.02 | 100 | 20.9 | 86.4 | 78 | 82.2 | — | — | — | — | — | |
| NuggetsInstruction Dataset=Alpaca, Sample Ratio=15%2025.02 | 100 | 23.4 | 86.5 | 80.1 | 83.3 | — | — | — | — | — | |
| NOVAInstruction Dataset=Alpaca, Sample Ratio=15%2025.02 | 100 | 18.7 | 88.1 | 82.1 | 85.1 | — | — | — | — | — | |
| VanillaInstruction Dataset=Alpaca - GPT4, Sample Ratio=100%2025.02 | 100 | 32 | 84.7 | 80.4 | 82.6 | — | — | — | — | — | |
| FLAME-DPO factInstruction Dataset=Alpaca - GPT4, Sample Ratio=100%2025.02 | 100 | 27.6 | 87.3 | 84.1 | 85.7 | — | — | — | — | — | |
| SELF-EVALInstruction Dataset=Alpaca - GPT4, Sample Ratio=100%2025.02 | 100 | 31.6 | 86.7 | 83.7 | 85.2 | — | — | — | — | — | |
| IFDInstruction Dataset=Alpaca - GPT4, Sample Ratio=5%2025.02 | 100 | 39.2 | 84.4 | 79.6 | 82 | — | — | — | — | — | |
| CaRInstruction Dataset=Alpaca - GPT4, Sample Ratio=5%2025.02 | 100 | 41.1 | 86.2 | 81.1 | 83.7 | — | — | — | — | — | |
| NuggetsInstruction Dataset=Alpaca - GPT4, Sample Ratio=5%2025.02 | 100 | 42.3 | 87 | 82.3 | 84.7 | — | — | — | — | — | |
| NOVAInstruction Dataset=Alpaca - GPT4, Sample Ratio=5%2025.02 | 100 | 33.8 | 90.1 | 85.2 | 87.7 | — | — | — | — | — | |
| IFDInstruction Dataset=Alpaca - GPT4, Sample Ratio=10%2025.02 | 100 | 39.2 | 86.5 | 77.8 | 82.2 | — | — | — | — | — | |
| CaRInstruction Dataset=Alpaca - GPT4, Sample Ratio=10%2025.02 | 100 | 38 | 87.1 | 78.3 | 82.7 | — | — | — | — | — | |
| NuggetsInstruction Dataset=Alpaca - GPT4, Sample Ratio=10%2025.02 | 100 | 35.7 | 88.2 | 80.1 | 84.2 | — | — | — | — | — | |
| NOVAInstruction Dataset=Alpaca - GPT4, Sample Ratio=10%2025.02 | 100 | 32.3 | 90.6 | 81.8 | 86.2 | — | — | — | — | — | |
| IFDInstruction Dataset=Alpaca - GPT4, Sample Ratio=15%2025.02 | 100 | 32.2 | 85.2 | 80.3 | 82.8 | — | — | — | — | — | |
| CaRInstruction Dataset=Alpaca - GPT4, Sample Ratio=15%2025.02 | 100 | 33.6 | 85.8 | 81.5 | 83.7 | — | — | — | — | — | |
| NuggetsInstruction Dataset=Alpaca - GPT4, Sample Ratio=15%2025.02 | 100 | 34.5 | 86.1 | 80.7 | 83.4 | — | — | — | — | — | |
| NOVAInstruction Dataset=Alpaca - GPT4, Sample Ratio=15%2025.02 | 100 | 32.1 | 88 | 82.5 | 85.3 | — | — | — | — | — | |
| Entropy2025.12 | — | — | — | — | — | 0.5124 | 10.76 | — | — | — | |
| Linear Probe2025.12 | — | — | — | — | — | 0.9022 | 68.91 | 90.22 | 88.29 | 52.8 | |
| MLP Probe2025.12 | — | — | — | — | — | 0.9528 | 70.24 | 95.28 | 89.02 | 57.15 | |
| Perplexity2025.12 | — | — | — | — | — | 0.4623 | 8.79 | — | — | — |