Hallucination Detection on CF (test)
98F1 ScoreVSDB + top1(L2)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| VSDB + top1(L2)lo threshold=0.04, hi threshold=0.152026.05 | 98 | 10,917 | 4,045 | 293 | 148 | 97.3 | 98.7 | 0.929 | 0.986 | 0.037 | |
| VSDB + Fieldlo threshold=1.0, hi threshold=3.752026.05 | 97.8 | 10,479 | 3,974 | 316 | 152 | 97.1 | 98.6 | 0.923 | 0.984 | 0.039 | |
| VSDB + top1(cos)lo threshold=0.01, hi threshold=0.142026.05 | 96 | 7,557 | 7,218 | 66 | 566 | 99.1 | 93 | 0.92 | 0.973 | 0.04 | |
| PoolingEClassifier architecture=Ensembled, Training data distribution=Organic2023.12 | 94 | — | — | — | — | — | — | — | — | — | |
| PoolingSLClassifier architecture=Single-layer, Training data distribution=Organic2023.12 | 93 | — | — | — | — | — | — | — | — | — | |
| Expert Human2023.12 | 89 | — | — | — | — | — | — | — | — | — | |
| VDB + top1(cos)lo threshold=0.01, hi threshold=0.132026.05 | 85.7 | 3,566 | 10,600 | 6 | 1,185 | 99.8 | 75.1 | 0.821 | 0.919 | 0.036 | |
| PoolingE-SYNTHClassifier architecture=Ensembled, Training data distribution=Synthetic2023.12 | 83 | — | — | — | — | — | — | — | — | — | |
| Seq-Logprob2023.12 | 81 | — | — | — | — | — | — | — | — | — | |
| FActScore2023.12 | 80 | — | — | — | — | — | — | — | — | — | |
| PoolingSL-SYNTHClassifier architecture=Single-layer, Training data distribution=Synthetic2023.12 | 80 | — | — | — | — | — | — | — | — | — | |
| Optimized Coin (OC)Threshold optimization=Validation set tuned2023.12 | 79 | — | — | — | — | — | — | — | — | — | |
| ChatGPTPrompting strategy=Standard2023.12 | 74 | — | — | — | — | — | — | — | — | — | |
| ChatGPT CoTPrompting strategy=Chain-of-thought2023.12 | 51 | — | — | — | — | — | — | — | — | — |