Hallucination Detection on FABLES+
84.5Macro F1VeriTrail
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| VeriTrailq=3, Base Model=gpt-4o-2024-08-062025.05 | 84.5 | 83.6 | 95.4 | 96.4 | 75.6 | 70.8 | |
| VeriTrailq=1, Base Model=gpt-4o-2024-08-062025.05 | 74 | 84.6 | 97.5 | 82.8 | 44.1 | 86.5 | |
| VeriTrailModel=Gemini 2.5 Flash, q=12025.05 | 70 | 80.2 | 96.2 | 79.2 | 39.4 | 81.2 | |
| RAGk=15, Base Model=gpt-4o-2024-08-062025.05 | 69.6 | 76.5 | 94.6 | 83.3 | 39.6 | 69.8 | |
| RAGModel=Gemini 2.5 Flash, k=52025.05 | 66.6 | 73.2 | 93.4 | 80.7 | 36.2 | 65.6 | |
| Bespoke-MiniCheck-7Bk=1, tau=0.5, full_name=Llama-3.1-Bespoke-MiniCheck-7B2025.05 | 62.2 | 69 | 92.6 | 77.7 | 29.9 | 60.4 | |
| VeriTrailModel=DeepSeek, q=12025.05 | 61.7 | 70.9 | 93.4 | 73.1 | 29.7 | 68.8 | |
| Gemini 1.5 Pro2025.05 | 61.1 | 60.8 | 89.3 | 90.3 | 33.7 | 31.2 | |
| GPT-4.1 Mini2025.05 | 60.7 | 58.2 | 88.4 | 98.7 | 68 | 17.7 | |
| AlignScorek=1, tau=0.62025.05 | 59.6 | 67.5 | 92.4 | 73.6 | 26.8 | 61.5 | |
| RAGModel=DeepSeek, k=32025.05 | 59.4 | 63.1 | 90 | 79.3 | 27.3 | 46.9 | |
| RAGModel=Mistral, k=52025.05 | 55.2 | 62.1 | 90.8 | 70.4 | 21.9 | 53.8 | |
| VeriTrailModel=Mistral, q=12025.05 | 49.2 | 67.2 | 95.5 | 49.7 | 20.6 | 84.6 | |
| INFUSEtau=0.52025.05 | 40.5 | 59.5 | 92.9 | 36.8 | 17 | 82.3 |