Hallucination Detection on SciQ (Accuracy, F1)
96AccuracyAbnorDetector-Full
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| AbnorDetector-FullBase LLM=Llama-3.1-70B-Instruct, alpha (α)=0.125, beta (β)=0.1252025.04 | 96 | 0 | 96 | |
| AbnorDetector-FullBase LLM=Llama-3.1-70B-Instruct, alpha (α)=0.25, beta (β)=0.252025.04 | 94.83 | 0.0556 | 94.83 | |
| AbnorDetector-FullBase LLM=Llama-3.1-8B-Instruct, alpha (α)=0.25, beta (β)=0.252025.04 | 93.5 | 0 | 93.5 | |
| Lynx2025.04 | 92.2 | — | 91.75 | |
| AbnorDetector-FullBase LLM=Llama-2-7b-chat, alpha (α)=0.25, beta (β)=0.252025.04 | 91.83 | 0.0556 | 91.83 | |
| AbnorDetector-FullBase LLM=Gemma-7b-it, alpha (α)=0.25, beta (β)=0.252025.04 | 91.67 | 0.0556 | 91.66 | |
| AbnorDetector-FullBase LLM=Llama-2-7b-chat, alpha (α)=0.125, beta (β)=0.1252025.04 | 91.5 | 0.1667 | 91.49 | |
| AbnorDetector-FullBase LLM=Llama-3.1-8B-Instruct, alpha (α)=0.125, beta (β)=0.1252025.04 | 88.83 | 0.0556 | 88.83 | |
| AbnorDetector-LiteBase LLM=Llama-3.1-70B-Instruct, alpha (α)=0.5, beta (β)=0.52025.04 | 88.33 | 1.5556 | 88.33 | |
| AbnorDetector-FullBase LLM=Gemma-7b-it, alpha (α)=0.125, beta (β)=0.1252025.04 | 88 | 0 | 87.99 | |
| AbnorDetector-LiteBase LLM=Llama-3.1-70B-Instruct, alpha (α)=0.25, beta (β)=0.252025.04 | 86.17 | 0.3889 | 86.14 | |
| AbnorDetector-LiteBase LLM=Llama-3.1-8B-Instruct, alpha (α)=0.5, beta (β)=0.52025.04 | 85.17 | 1.0556 | 85.16 | |
| AbnorDetector-LiteBase LLM=Llama-3.1-8B-Instruct, alpha (α)=0.25, beta (β)=0.252025.04 | 74.83 | 6.8889 | 74.72 | |
| AbnorDetector-LiteBase LLM=Gemma-7b-it, alpha (α)=0.5, beta (β)=0.52025.04 | 74.5 | 1.1667 | 74.48 | |
| AbnorDetector-LiteBase LLM=Llama-2-7b-chat, alpha (α)=0.5, beta (β)=0.52025.04 | 73 | 0.1667 | 72.86 | |
| AbnorDetector-LiteBase LLM=Gemma-7b-it, alpha (α)=0.25, beta (β)=0.252025.04 | 70.83 | 2.8889 | 70.5 | |
| AbnorDetector-LiteBase LLM=Llama-2-7b-chat, alpha (α)=0.25, beta (β)=0.252025.04 | 66.83 | 2.7222 | 66.81 |