Hallucination Detection on Code-agent (test)
61.9PrecisionLettuceDetect-mmBERT-base
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LettuceDetect-mmBERT-baseEval=span2026.07 | 61.9 | 43 | 50.8 | 77 | |
| LettuceDetect-Qwen-2BEval=span2026.07 | 59.6 | 60.9 | 60.2 | 83.5 | |
| LFM2.5-8B-A1BEval=span, Variant=SFT2026.07 | 53.1 | 48.5 | 50.7 | 81.1 | |
| Granite-Guardian-4.1-8BEval=answer2026.07 | 53 | 90 | 66 | — | |
| Lynx-8BEval=answer2026.07 | 52 | 73 | 61 | — | |
| HHEM-2.1-OpenEval=answer2026.07 | 50 | 86 | 63 | — | |
| MiniCheck-7BEval=answer2026.07 | 50 | 100 | 67 | — | |
| gpt-oss-120bEval=span, Variant=task-aware2026.07 | 13.5 | 50.1 | 21.2 | 69.1 | |
| Nemotron-3-UltraEval=span, Variant=task-aware2026.07 | 13.2 | 58.9 | 21.6 | 70 | |
| LettuceDetect-largeEval=span, Version=v12026.07 | 11.2 | 37.3 | 17.2 | 68.4 | |
| Nemotron-3-UltraEval=span, Variant=generic2026.07 | 10.8 | 66.5 | 18.6 | 65.5 |