Mask-quality reviewer alignment on Corrupted masks (n=96) (test)
79.2Grade-Level Accuracy (Prec)Gem. 3 Pro
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gem. 3 ProRater category=VLM2026.06 | 79.2 | 87.5 | 70.8 | 89.4 | 91.5 | 7.49 | 4.82 | |
| Gem. 3 FlashRater category=VLM2026.06 | 78.1 | 87.5 | 66.7 | 89.6 | 95.1 | 6.09 | 3.99 | |
| Hum. (Gi)Rater category=Human2026.06 | 76 | 75 | 60.4 | 88.6 | 89.6 | 4.35 | 4.35 | |
| Gem. 3.1 ProRater category=VLM2026.06 | 74 | 88.5 | 63.5 | 89.2 | 95.3 | 7.55 | 4.58 | |
| Hum. (C)Rater category=Human2026.06 | 72.9 | 85.4 | 64.6 | 88.7 | 94.1 | 6.3 | 3.97 | |
| Hum. (V)Rater category=Human2026.06 | 68.8 | 84.4 | 59.4 | 86.2 | 93.5 | 6.09 | 3.26 |