Hallucination Detection on RAGTruth summarization task
77PrecisionRAG-HAT (Llama-3-8B-Instruct)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RAG-HAT (Llama-3-8B-Instruct)Backbone=Llama-3-8B-Instruct2025.12 | 77 | 59 | 67 | |
| Finetuned Llama-2-13bBackbone=Llama-2, Parameters=13b, Mode=Finetuned2025.12 | 64 | 54 | 59 | |
| TrueBrief.DetectionModel Size=0.5B, Task-Specific Finetuning=false, Classifier=LogisticRegression, Pooling Strategy=mean2025.12 | 34 | 45 | 39 | |
| TrueBrief.DetectionModel Size=1.5B, Task-Specific Finetuning=false, Classifier=LogisticRegression, Pooling Strategy=mean2025.12 | 32 | 72 | 44 | |
| TrueBrief.DetectionModel Size=3B, Task-Specific Finetuning=TrueBrief.Finetuning, Classifier=LogisticRegression, Pooling Strategy=mean2025.12 | 32 | 77 | 45 | |
| Prompt_gpt-4-turboPrompting=true2025.12 | 31 | 97 | 47 | |
| SelfCheckGPT_gpt-3.5-turboBackbone=gpt-3.5-turbo2025.12 | 31 | 56 | 40 | |
| TrueBrief.DetectionModel Size=0.5B, Task-Specific Finetuning=TrueBrief.Finetuning, Classifier=LogisticRegression, Pooling Strategy=mean2025.12 | 31 | 75 | 44 | |
| TrueBrief.DetectionModel Size=1.5B, Task-Specific Finetuning=TrueBrief.Finetuning, Classifier=LogisticRegression, Pooling Strategy=mean2025.12 | 31 | 79 | 45 | |
| TrueBrief.DetectionModel Size=3B, Task-Specific Finetuning=false, Classifier=LogisticRegression, Pooling Strategy=mean2025.12 | 31 | 72 | 43 | |
| Longformer-base-4096Variant=base-40962025.12 | 27 | 65 | 38 | |
| Prompt_gpt-3.5-turboPrompting=true2025.12 | 23 | 89 | 37 | |
| LMvLM_gpt-4-turboBackbone=gpt-4-turbo2025.12 | 23 | 81 | 36 | |
| BERT-base-uncasedVariant=base-uncased2025.12 | 23 | 94 | 37 |