Frame-level glitch detection on RefGlitch (test)
77AccuracyGPT-5
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-5Setting=Ref2026.04 | 77 | 74 | 87 | 64 | |
| Qwen3-VL-8BSetting=Ref2026.04 | 73 | 74 | 72 | 76 | |
| GPT-5Setting=LastCleanFrame2026.04 | 70 | 62 | 85 | 48 | |
| Qwen3-VL-8BSetting=LastCleanFrame2026.04 | 70 | 69 | 70 | 69 | |
| Gemini 3 Flash PreviewSetting=IrRef2026.04 | 68 | 65 | 72 | 60 | |
| Ministral-3-8BSetting=Ref2026.04 | 68 | 58 | 83 | 45 | |
| Gemini 3 Flash PreviewSetting=NoRef2026.04 | 67 | 65 | 70 | 61 | |
| GPT-5Setting=IrRef2026.04 | 66 | 51 | 88 | 36 | |
| GPT-5Setting=NoRef2026.04 | 65 | 49 | 88 | 34 | |
| Gemini 3 Flash PreviewSetting=Ref2026.04 | 63 | 73 | 58 | 98 | |
| Ministral-3-8BSetting=LastCleanFrame2026.04 | 63 | 45 | 86 | 30 | |
| Gemma-3-12BSetting=Ref2026.04 | 63 | 63 | 63 | 63 | |
| Qwen3-VL-8BSetting=NoRef2026.04 | 62 | 42 | 88 | 28 | |
| Gemma-3-12BSetting=LastCleanFrame2026.04 | 61 | 60 | 62 | 58 | |
| Gemini 3 Flash PreviewSetting=LastCleanFrame2026.04 | 59 | 68 | 56 | 86 | |
| Gemma-3-12BSetting=NoRef2026.04 | 58 | 48 | 64 | 38 | |
| Ministral-3-8BSetting=NoRef2026.04 | 57 | 35 | 73 | 23 | |
| Qwen3-VL-8BSetting=IrRef2026.04 | 54 | 19 | 83 | 11 | |
| Gemma-3-12BSetting=IrRef2026.04 | 51 | 14 | 60 | 8 | |
| Ministral-3-8BSetting=IrRef2026.04 | 50 | 3 | 44 | 2 |