LLM Hallucination Detection on ContextualJudgeBench
92Refusal Rate (Ans.)DeepSeek-R1
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-R1Category=Instruct + Reasoning Model2026.03 | 92 | 52 | 72 | 50.4 | 41.2 | 60.6 | 20.4 | 26.2 | 51.9 | |
| R1Distill-Llama-3.3-70BCategory=Instruct + Reasoning Model2026.03 | 89.6 | 50.4 | 74 | 48.4 | 42.4 | 57.4 | 19.2 | 29.5 | 51.4 | |
| SFRJudge-70BCategory=Large Judge Model2026.03 | 87.6 | 32.4 | 60.8 | 54.8 | 40.8 | 53.4 | 44.7 | 36.1 | 51.4 | |
| Skywork-70BCategory=Large Judge Model2026.03 | 82.4 | 11.2 | 48 | 47.6 | 36.8 | 41.4 | 21.6 | 27.9 | 39.6 | |
| Alta-Selene-8BCategory=Small Judge Model2026.03 | 74.4 | 26.4 | 40.8 | 32.8 | 32.4 | 34.7 | 23.1 | 32 | 37.1 | |
| Llama-3.3-70BCategory=Instruct + Reasoning Model2026.03 | 71.6 | 42.4 | 68 | 48.4 | 42 | 51.8 | 20.8 | 30.7 | 47 | |
| GPT-4o-miniCategory=Instruct + Reasoning Model2026.03 | 71.2 | 22.8 | 45.6 | 42.4 | 33.2 | 54.2 | 11.8 | 29.5 | 38.8 | |
| SFRJudge-8BCategory=Small Judge Model2026.03 | 70.8 | 22 | 40.4 | 38.8 | 40.4 | 43.4 | 27.5 | 31.1 | 39.3 | |
| SFRJudge-12BCategory=Small Judge Model2026.03 | 68.4 | 28.4 | 45.2 | 43.6 | 28 | 51 | 16.1 | 29.5 | 38.8 | |
| Llama-3-OffsetBias-8BCategory=Small Judge Model2026.03 | 64.8 | 11.2 | 34 | 26.4 | 33.2 | 21.1 | 46.3 | 23 | 32.6 | |
| GPT-4oCategory=Instruct + Reasoning Model2026.03 | 64 | 52 | 68 | 50.8 | 39.6 | 56.2 | 12.9 | 22.5 | 45.8 | |
| Skywork-8BCategory=Small Judge Model2026.03 | 60.8 | 12 | 38.8 | 31.6 | 38.4 | 26.7 | 29.4 | 21.3 | 32.4 | |
| Llama-3.1-70BCategory=Instruct + Reasoning Model2026.03 | 59.6 | 48 | 58 | 48.4 | 38 | 51.8 | 15.7 | 27.5 | 43.4 | |
| ST-Eval-70BCategory=Large Judge Model2026.03 | 50 | 42 | 51.2 | 45.6 | 40.8 | 39.4 | 36.1 | 29.9 | 41.9 | |
| MARCH-STEMCategory=OURS, Training Data=STEM data2026.03 | 38.1 | 64 | 53.6 | 58 | 44.9 | 65.5 | 44.7 | 49.5 | 52.3 | |
| MARCH-GeneralCategory=OURS, Training Data=General QA data2026.03 | 37.2 | 57.8 | 52.6 | 54.8 | 48 | 63.9 | 48.1 | 50.1 | 51.6 | |
| Llama-3.1-8BCategory=OURS (Backbone)2026.03 | 28 | 43.2 | 34.8 | 34.8 | 23.2 | 41 | 11.4 | 21.3 | 29.7 | |
| Promtheus-2-8x7bCategory=Large Judge Model2026.03 | 22 | 29.6 | 22.4 | 29.6 | 20.4 | 39.8 | 10.2 | 18.4 | 24.1 | |
| Promtheus-2-7bCategory=Small Judge Model2026.03 | 12.4 | 44 | 27.2 | 32 | 24 | 42.6 | 6.7 | 29.5 | 27.3 | |
| Glider-3.8BCategory=Small Judge Model2026.03 | 12 | 8.8 | 45.6 | 9.2 | 20.8 | 28.7 | 5.1 | 4.1 | 16.8 |