Visual Question Answering on AdVQA (C@10, C@r, AURC, Acc)
33.3C@10SIEVES
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SIEVESReasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 33.3 | 59.8 | 11.5 | 81.4 | |
| SelectorReasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 32.5 | 58.9 | 11.9 | 81.4 | |
| SelectorReasoner=Gemini-3-Pro, Localization setting=w/o localization2026.04 | 15 | 55 | 12.8 | 80.3 | |
| SIEVESReasoner=o3, Localization setting=w/ localization2026.04 | 8.2 | 36.5 | 19.6 | 72.4 | |
| SelectorReasoner=o3, Localization setting=w/ localization2026.04 | 3.8 | 33.3 | 20.1 | 72.4 | |
| SIEVESReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 2.8 | 9.3 | 32.2 | 58.1 | |
| SelectorReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0.2 | 6.3 | 33 | 58.1 | |
| LogprobsReasoner=Pixel-Reasoner, Localization setting=w/o localization2026.04 | 0.1 | 0.3 | 42.5 | 54.5 | |
| SelectorReasoner=Pixel-Reasoner, Localization setting=w/o localization2026.04 | 0 | 7.7 | 33.7 | 54.5 | |
| LogprobsReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0 | 0 | 40.3 | 58.1 | |
| Zero-shot verb.Reasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0 | 0 | 40.7 | 58.1 | |
| Zero-shot logitReasoner=Pixel-Reasoner, Localization setting=w/ localization2026.04 | 0 | 0 | 38.6 | 58.1 | |
| SelectorReasoner=o3, Localization setting=w/o localization2026.04 | 0 | 19.4 | 25.2 | 68 | |
| Zero-shot verb.Reasoner=o3, Localization setting=w/ localization2026.04 | 0 | 27.5 | 24.9 | 72.4 | |
| Zero-shot logitReasoner=o3, Localization setting=w/ localization2026.04 | 0 | 30.6 | 21.9 | 72.4 | |
| Zero-shot verb.Reasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 0 | 42.9 | 17 | 81.4 | |
| Zero-shot logitReasoner=Gemini-3-Pro, Localization setting=w/ localization2026.04 | 0 | 42.9 | 17.3 | 81.4 |