Visual Question Answering on HR-Bench 8k
14.4C@1SIEVES
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SIEVESReasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 14.4 | 80.8 | 4.4 | 90.1 | |
| SIEVESReasoner=o3, Localization setting=w/ localization2026.04 | 8 | 67.7 | 9.3 | 82 | |
| SelectorReasoner=Gemini-3-Pro, Localization setting=w/o localization2026.04 | 5.8 | 74.5 | 5.4 | 85.6 | |
| SelectorReasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 4.5 | 80.1 | 3.5 | 90.1 | |
| SIEVESReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 2.3 | 41.1 | 18.1 | 68.4 | |
| Zero-shot logitReasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 1.4 | 79.9 | 4.9 | 90.1 | |
| SelectorReasoner=o3, Localization setting=w/ localization2026.04 | 1 | 61.7 | 10.5 | 82 | |
| SelectorReasoner=Pixel-Reasoner, Localization setting=w/o localization2026.04 | 0.9 | 30.8 | 20.3 | 65.6 | |
| LogprobsReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0.3 | 28.9 | 22.6 | 68.4 | |
| SelectorReasoner=o3, Localization setting=w/o localization2026.04 | 0.3 | 33.3 | 19.2 | 72.1 | |
| LogprobsReasoner=Pixel-Reasoner, Localization setting=w/o localization2026.04 | 0 | 6.2 | 30.3 | 65.6 | |
| Zero-shot verb.Reasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0 | 13.3 | 28.9 | 68.4 | |
| Zero-shot logitReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0 | 32.8 | 21.3 | 68.4 | |
| SelectorReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0 | 42.7 | 17.4 | 68.4 | |
| Zero-shot verb.Reasoner=o3, Localization setting=w/ localization2026.04 | 0 | 54.2 | 13.5 | 82 | |
| Zero-shot logitReasoner=o3, Localization setting=w/ localization2026.04 | 0 | 56.3 | 13.1 | 82 | |
| Zero-shot verb.Reasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 0 | 71.4 | 6.4 | 90.1 |