Hallucination Detection on SugarCrepe 1.0 (test)
98.86Avg-M ScoreHuman
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| HumanNote=Taken from original papers2026.05 | 98.86 | 99 | 97 | 99 | 100 | 99 | 99 | 99 | |
| VQAScoreModel Category=LLM-based, Backbone=CLIP-FlanT5-XXL2026.05 | 92 | 91.6 | 95.4 | 94.5 | 97.6 | 88.3 | 93.5 | 82.9 | |
| EXPERTModel Category=LLM-based2026.05 | 89.7 | 89.2 | 89.7 | 95.6 | 89.7 | 86.5 | 90.5 | 89.8 | |
| BEiTScoreModel Category=Encoder-based, Backbone=BEiT-L/14, Stage 1=Included2026.05 | 88.7 | 82.8 | 86.9 | 89 | 93.8 | 85.9 | 94 | 88.6 | |
| BEiTScoreModel Category=Encoder-based, Backbone=BEiT-L/14, Stage 1=Excluded2026.05 | 88.3 | 72.8 | 84 | 84.1 | 92.7 | 76.7 | 89.6 | 83.3 | |
| FLEURModel Category=LLM-based2026.05 | 88 | 88 | 87.5 | 93.7 | 91 | 87.6 | 88.7 | 86.5 | |
| LongCLIP-SModel Category=Encoder-based, Backbone=ViT-L2026.05 | 82.3 | 86.4 | 91.5 | 86.9 | 96.4 | 76.8 | 68.2 | 70.2 | |
| PAC-S++Model Category=Encoder-based, Backbone=ViT-L2026.05 | 82.2 | 86.9 | 94.9 | 87.4 | 97.8 | 77.3 | 67.7 | 63.7 | |
| BEiTScoreModel Category=Encoder-based, Backbone=BEiT-B/32, Stage 1=Included2026.05 | 81.9 | 76.2 | 78.3 | 81.9 | 90.1 | 77.9 | 89 | 80 | |
| PAC-S++Model Category=Encoder-based, Backbone=ViT-B2026.05 | 80.5 | 80.9 | 90.6 | 85 | 95.3 | 74.5 | 70.6 | 66.5 | |
| PAC-SModel Category=Encoder-based, Backbone=ViT-L2026.05 | 80.4 | 83 | 91.8 | 86 | 96.3 | 76.3 | 68.5 | 60.8 | |
| CLIP-SModel Category=Encoder-based, Backbone=ViT-L2026.05 | 79.7 | 83 | 91.4 | 84.9 | 96.2 | 70.8 | 70 | 61.6 | |
| LongCLIP-SModel Category=Encoder-based, Backbone=ViT-B2026.05 | 79 | 76 | 84.2 | 85.7 | 94.5 | 74.8 | 70.6 | 67.4 | |
| PAC-SModel Category=Encoder-based, Backbone=ViT-B2026.05 | 77.2 | 75.3 | 88.3 | 82 | 93.5 | 72.6 | 68.2 | 60.8 | |
| CLIP-SModel Category=Encoder-based, Backbone=ViT-B2026.05 | 77.1 | 78 | 87.3 | 82.6 | 93.6 | 69.1 | 68 | 61.2 | |
| BEiTScoreModel Category=Encoder-based, Backbone=BEiT-B/32, Stage 1=Excluded2026.05 | 73.6 | 62.3 | 74.4 | 71.7 | 84.9 | 65.4 | 76.1 | 69.4 | |
| GPT4o (API)Model Category=LLM-based2026.05 | 66.7 | 60.6 | 60.1 | 66.2 | 61.3 | 79 | 77.3 | 78.4 | |
| SPECSModel Category=Encoder-based2026.05 | 63.9 | 82.1 | 38.4 | 70.7 | 79.7 | 57.3 | 60.1 | 58.8 |