Hallucination Detection on DiverseSumm+
79.5Macro F1VeriTrail
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| VeriTrailq=3, Base Model=gpt-4o-2024-08-062025.05 | 79.5 | 76.3 | 87.1 | 96.7 | 84.5 | 55.9 | |
| VeriTrailq=1, Base Model=gpt-4o-2024-08-062025.05 | 76.6 | 83 | 95.8 | 76.2 | 55.1 | 89.8 | |
| RAGk=15, Base Model=gpt-4o-2024-08-062025.05 | 75.1 | 74 | 86.7 | 90.5 | 66.4 | 57.5 | |
| Bespoke-MiniCheck-7Bk=2, tau=0.5, full_name=Llama-3.1-Bespoke-MiniCheck-7B2025.05 | 72.1 | 69.4 | 83.8 | 95.4 | 75.3 | 43.3 | |
| VeriTrailModel=Gemini 2.5 Flash, q=12025.05 | 69.9 | 72.3 | 87.2 | 77.3 | 50.8 | 67.3 | |
| RAGModel=Gemini 2.5 Flash, k=102025.05 | 68.9 | 67.5 | 82.5 | 90.1 | 61.1 | 44.9 | |
| VeriTrailModel=DeepSeek, q=12025.05 | 68 | 73.2 | 89.8 | 68.8 | 46.3 | 77.6 | |
| RAGModel=Mistral, k=102025.05 | 67.6 | 66.5 | 82.2 | 88.2 | 56.8 | 44.7 | |
| VeriTrailModel=Mistral, q=12025.05 | 67 | 73.8 | 91.7 | 64.7 | 44.8 | 83 | |
| RAGModel=DeepSeek, k=302025.05 | 66 | 63.9 | 80.1 | 97.2 | 78.9 | 30.6 | |
| GPT-4.1 Mini2025.05 | 62.9 | 61.5 | 80.3 | 93.8 | 60.7 | 29.1 | |
| AlignScorek=1, tau=0.92025.05 | 60.4 | 62.7 | 82.8 | 70.3 | 37.6 | 55.1 | |
| Gemini 1.5 Pro2025.05 | 49.8 | 57.6 | 82.2 | 45.1 | 29.4 | 70.1 | |
| INFUSEtau=0.52025.05 | 20 | 50.1 | 100 | 0.3 | 24.6 | 100 |