Hallucination Detection on HaluEval QA (test)
78.9TPRHHEM (Non-Fabrication Checking)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HHEM (Non-Fabrication Checking)Time=1h judge, Non-Fabrication Checking=true, Base Model=Starling-LM-7B-alpha2025.12 | 78.9 | 85.5 | 82.2 | |
| PAPERFormat=Unstructured, Base Model=Starling-LM-7B-alpha2025.12 | 72.4 | 85.9 | 79.15 | |
| Ours (PAPER replication)Format=Unstructured, Time=8h query and 3h judge, Base Model=Starling-LM-7B-alpha, Test Set Size=1,000 samples2025.12 | 69.5 | 84.2 | 76.85 | |
| PAPERFormat=Structured, Base Model=Starling-LM-7B-alpha2025.12 | 67.8 | 83.1 | 75.45 | |
| HHEMTime=10 min judge, Base Model=Starling-LM-7B-alpha2025.12 | 67.2 | 86.6 | 76.9 | |
| Ours (PAPER replication)Format=Structured, Time=8h query and 3h judge, Base Model=Starling-LM-7B-alpha, Test Set Size=1,000 samples2025.12 | 66 | 84.5 | 75.25 | |
| Knowhalu + HHEMFormat=Structured, Time=8h query and 10 min judge, Base Model=Starling-LM-7B-alpha2025.12 | 54.5 | 92.9 | 73.7 | |
| Knowhalu + HHEMFormat=Unstructured, Time=8h query and 10 min judge, Base Model=Starling-LM-7B-alpha2025.12 | 54.1 | 93.3 | 73.7 |