Visual Question Answering on MME RW-L
12.2C@5SIEVES
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SIEVESReasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 12.2 | 32.3 | 21.5 | 63.2 | |
| SelectorReasoner=Gemini-3-Pro, Localization setting=w/o localization2026.04 | 5.1 | 34.7 | 21.9 | 60.9 | |
| SIEVESReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 4 | 18.7 | 31 | 51.3 | |
| SIEVESReasoner=o3, Localization setting=w/ localization2026.04 | 1.7 | 17.5 | 28.4 | 58.7 | |
| SelectorReasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 1.2 | 28.2 | 22.8 | 63.2 | |
| SelectorReasoner=o3, Localization setting=w/o localization2026.04 | 1.1 | 13.4 | 29.3 | 56.9 | |
| LogprobsReasoner=Pixel-Reasoner, Localization setting=w/o localization2026.04 | 0.1 | 8.3 | 35.3 | 51.6 | |
| SelectorReasoner=o3, Localization setting=w/ localization2026.04 | 0.1 | 16.6 | 28.6 | 58.7 | |
| SelectorReasoner=Pixel-Reasoner, Localization setting=w/o localization2026.04 | 0 | 17.8 | 30.4 | 51.6 | |
| LogprobsReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0 | 0.7 | 43.5 | 51.3 | |
| Zero-shot verb.Reasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0 | 0 | 43.5 | 51.3 | |
| Zero-shot logitReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0 | 4.7 | 36.4 | 51.3 | |
| SelectorReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0 | 16.9 | 31.9 | 51.3 | |
| Zero-shot verb.Reasoner=o3, Localization setting=w/ localization2026.04 | 0 | 0 | 36.1 | 58.7 | |
| Zero-shot logitReasoner=o3, Localization setting=w/ localization2026.04 | 0 | 4.8 | 35.3 | 58.7 | |
| Zero-shot verb.Reasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 0 | 9.3 | 28.2 | 63.2 | |
| Zero-shot logitReasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 0 | 19.5 | 26.4 | 63.2 |