# Visual Question Answering on A-OKVQA

Source: [Visual Question Answering on A-OKVQA](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-02T08:50:04.227Z

Evaluation of model performance on the A-OKVQA dataset for visual question answering capabilities, reporting metrics including Accuracy, F1, ECE, and MECE.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Visual Question Answering |
| Dataset (including any stated split) | A-OKVQA |
| Metric | Acc |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 260 of 260 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Acc](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=Acc>): Higher is better; unit: PERCENT.

- [ECE](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=ECE>): Lower is better; unit: PERCENT.

- [MECE](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=MECE>): Higher is better; unit: PERCENT.

- [F1](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=F1>): Higher is better; unit: PERCENT.

- [AUROC](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=AUROC>): Higher is better; unit: SCALAR.

- [Rel. Avg](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=Rel.%20Avg>): Higher is better; unit: PERCENT.

- [MC](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=MC>): Higher is better; unit: PERCENT.

- [DA](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=DA>): Higher is better; unit: PERCENT.

- [Score](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=Score>): Higher is better; unit: PERCENT.

- [ASR](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=ASR>): Higher is better; unit: SCALAR.

- [Coh](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=Coh>): Higher is better; unit: SCALAR.

- [Rat](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=Rat>): Higher is better; unit: SCALAR.

- [ASR-C](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=ASR-C>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Acc | Result date | Report paper |
| --- | --- | --- | --- |
| Cont-Squeeze (128 -\> 1) | 92.68 | Not recorded | 698d5ed8c97a0b3e3b5e4020 |
| Cont-Squeeze (128 -\> 1) | 92.44 | Not recorded | 698d5ed8c97a0b3e3b5e4020 |
| Cont-Squeeze (128 -\> 1) | 92.37 | Not recorded | 698d5ed8c97a0b3e3b5e4020 |
| In-Squeeze (128 ... -\> 1) | 92 | Not recorded | 698d5ed8c97a0b3e3b5e4020 |
| Direct Fine-tuning (r=1) | 91.94 | Not recorded | 698d5ed8c97a0b3e3b5e4020 |
| In-Squeeze (128 ... -\> 1) | 91.7 | Not recorded | 698d5ed8c97a0b3e3b5e4020 |
| Direct Fine-tuning (r=1) | 91.33 | Not recorded | 698d5ed8c97a0b3e3b5e4020 |
| Direct Fine-tuning (r=1) | 91.27 | Not recorded | 698d5ed8c97a0b3e3b5e4020 |
| Base | 90 | Not recorded | 6a45f085dc71372d7795a21f |
| ReShift | 90 | Not recorded | 6a45f085dc71372d7795a21f |
| Gemma 3 4B IT | 89 | Not recorded | 698d5ed8c97a0b3e3b5e4020 |
| OpenVLThinker-7B | 88.73 | Not recorded | 6a0d405ff984739ed6737bdc |
| WeThink-7B | 88.65 | Not recorded | 6a0d405ff984739ed6737bdc |
| Qwen3-VL-8B | 87.86 | Not recorded | 6a0d405ff984739ed6737bdc |
| Qwen2.5-VL-7B | 87.6 | Not recorded | 6a0d405ff984739ed6737bdc |
| ALEAHallu | 87.24 | Not recorded | 69520adf53fdd5c36173d3f9 |
| Vanilla | 87.2 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| IVC-Prune | 87.2 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| ReShift | 87 | Not recorded | 6a45f085dc71372d7795a21f |
| PDrop | 86.9 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| Vanilla | 86.9 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| Qwen2.5-VL-7B | 86.81 | Not recorded | 6a0d405ff984739ed6737bdc |
| GThinker-7B | 86.72 | Not recorded | 6a0d405ff984739ed6737bdc |
| OneThinker-8B | 86.72 | Not recorded | 6a0d405ff984739ed6737bdc |
| IVC-Prune | 86.7 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| PDrop | 86.7 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| FastV | 86.6 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| IVC-Prune | 86.6 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| Vanilla | 86.5 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| FastV | 86.4 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| Qwen3-VL-8B | 86.29 | Not recorded | 6a0d405ff984739ed6737bdc |
| Nullu | 86 | Not recorded | 69520adf53fdd5c36173d3f9 |
| D.P. | 86 | Not recorded | 69c0c8db0636e5bf4afc397f |
| Base | 86 | Not recorded | 6a45f085dc71372d7795a21f |
| FastV | 85.9 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| Oracle | 85.8 | Not recorded | 69c0c8db0636e5bf4afc397f |
| PDrop | 85.6 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| MMR1-7B | 85.07 | Not recorded | 6a0d405ff984739ed6737bdc |
| Rewrite | 85 | Not recorded | 6a45f085dc71372d7795a21f |
| ICONS | 84.9 | Not recorded | 69c0c8db0636e5bf4afc397f |
| OPERA | 84.2 | Not recorded | 69520adf53fdd5c36173d3f9 |
| Beam Search | 84.12 | Not recorded | 69520adf53fdd5c36173d3f9 |
| Uniform | 83.7 | Not recorded | 69c0c8db0636e5bf4afc397f |
| R1-OneVision-RL-7B | 83.58 | Not recorded | 6a0d405ff984739ed6737bdc |
| BadToken | 83 | Not recorded | 6a45f085dc71372d7795a21f |
| DyCo-RL | 82.7 | Not recorded | 6a279e3c90966ade919bf828 |
| GRPO Baseline | 82.3 | Not recorded | 6a279e3c90966ade919bf828 |
| BadToken | 82 | Not recorded | 6a45f085dc71372d7795a21f |
| H-GIVR | 81.83 | Not recorded | 6984241aa8b01e76cc8ee75e |
| Full KV Caching | 81.4 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| AttentionPack | 81.25 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| VCD | 81.24 | Not recorded | 69520adf53fdd5c36173d3f9 |
| Minicache | 81.2 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| FastV | 81.17 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| InstructBLIP | 81 | Not recorded | 68fa73421e209131cea5b585 |
| Regular | 80.88 | Not recorded | 69520adf53fdd5c36173d3f9 |
| MMBoundary | 80.6 | Not recorded | 68fa79af1e209131cea7ee1c |
| AttentionPack | 80 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| Rewrite | 80 | Not recorded | 6a45f085dc71372d7795a21f |
| RCE | 79.3 | Not recorded | 68fa79af1e209131cea7ee1c |
| PDrop | 79.2 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| H2O | 79.1 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| FastV | 79 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| BadVision | 79 | Not recorded | 6a45f085dc71372d7795a21f |
| Vanilla | 78.9 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| IVC-Prune | 78.8 | Not recorded | 6982d2e4a8b01e76cc8e9893 |
| ScissorHands | 77.48 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| Conf-CSR | 77.4 | Not recorded | 68fa79af1e209131cea7ee1c |
| BadVision | 77 | Not recorded | 6a45f085dc71372d7795a21f |
| AttentionPack | 76.88 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| FastV | 76.72 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| Full KV Caching | 76.64 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| Minicache | 76.54 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| H-GIVR | 76.42 | Not recorded | 6984241aa8b01e76cc8ee75e |
| Prophet | 76.4 | Not recorded | 68fa73421e209131cea5b585 |
| H2O | 76.36 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| DRL | 76.2 | Not recorded | 68fa79af1e209131cea7ee1c |
| AttentionPack | 75.91 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| Full KV Caching | 75.67 | Not recorded | 69c4bd7f0636e5bf4afc7592 |
| ViCor | 75.6 | Not recorded | 68fa73421e209131cea5b585 |

### Cont-Squeeze (128 -\> 1)

Acc: 92.68. Result ID: 69963a7e5698665ec47d56c9.

[Reporting paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

Recorded evaluation configuration: {"Optimization steps":"+0 steps"}. This record does not assert independent reproduction.

### Cont-Squeeze (128 -\> 1)

Acc: 92.44. Result ID: 69963a7e5698665ec47d56ca.

[Reporting paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

Recorded evaluation configuration: {"Optimization steps":"+200 steps"}. This record does not assert independent reproduction.

### Cont-Squeeze (128 -\> 1)

Acc: 92.37. Result ID: 69963a7e5698665ec47d56cb.

[Reporting paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

Recorded evaluation configuration: {"Optimization steps":"+700 steps"}. This record does not assert independent reproduction.

### In-Squeeze (128 ... -\> 1)

Acc: 92. Result ID: 69963a7e5698665ec47d56cc.

[Reporting paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

Recorded evaluation configuration: {"Optimization mode":"Standard"}. This record does not assert independent reproduction.

### Direct Fine-tuning (r=1)

Acc: 91.94. Result ID: 69963a7e5698665ec47d56c6.

[Reporting paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b121e209131cea2bf59.md>)

[Implementation](<https://github.com/microsoft/LoRA>)

Recorded evaluation configuration: {"Optimization steps":"+0 steps"}. This record does not assert independent reproduction.

### In-Squeeze (128 ... -\> 1)

Acc: 91.7. Result ID: 69963a7e5698665ec47d56cd.

[Reporting paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

Recorded evaluation configuration: {"Optimization mode":"Min steps"}. This record does not assert independent reproduction.

### Direct Fine-tuning (r=1)

Acc: 91.33. Result ID: 69963a7e5698665ec47d56c7.

[Reporting paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b121e209131cea2bf59.md>)

[Implementation](<https://github.com/microsoft/LoRA>)

Recorded evaluation configuration: {"Optimization steps":"+200 steps"}. This record does not assert independent reproduction.

### Direct Fine-tuning (r=1)

Acc: 91.27. Result ID: 69963a7e5698665ec47d56c8.

[Reporting paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b121e209131cea2bf59.md>)

[Implementation](<https://github.com/microsoft/LoRA>)

Recorded evaluation configuration: {"Optimization steps":"+700 steps"}. This record does not assert independent reproduction.

### Base

Acc: 90. Result ID: 6a46263a60b88bb339856cc4.

[Reporting paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

Recorded evaluation configuration: {"Base model":"InternVL3.5-8B"}. This record does not assert independent reproduction.

### ReShift

Acc: 90. Result ID: 6a46263a60b88bb339856cc8.

[Reporting paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

Recorded evaluation configuration: {"Base model":"InternVL3.5-8B"}. This record does not assert independent reproduction.

### Gemma 3 4B IT

Acc: 89. Result ID: 69963a7e5698665ec47d56c5.

[Reporting paper](<https://www.sota2.com/research/paper/698d5ed8c97a0b3e3b5e4020.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa779f1e209131cea746cd.md>)

Recorded evaluation configuration: {"Zero-shot":true}. This record does not assert independent reproduction.

### OpenVLThinker-7B

Acc: 88.73. Result ID: 6a0d846ffbc373592c060725.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a251e209131cea814a7.md>)

Recorded evaluation configuration: {"Training":"Base"}. This record does not assert independent reproduction.

### WeThink-7B

Acc: 88.65. Result ID: 6a0d846ffbc373592c060727.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79391e209131cea7cb2c.md>)

Recorded evaluation configuration: {"Training":"Base"}. This record does not assert independent reproduction.

### Qwen3-VL-8B

Acc: 87.86. Result ID: 6a0d846ffbc373592c060729.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Implementation](<https://github.com/UCSC-VLAA/VLM-CapCurriculum>)

Recorded evaluation configuration: {"Training":"Base"}. This record does not assert independent reproduction.

### Qwen2.5-VL-7B

Acc: 87.6. Result ID: 6a0d846ffbc373592c060728.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Implementation](<https://github.com/UCSC-VLAA/VLM-CapCurriculum>)

Recorded evaluation configuration: {"Training":"Staged"}. This record does not assert independent reproduction.

### ALEAHallu

Acc: 87.24. Result ID: 69980a215698665ec463bb07.

[Reporting paper](<https://www.sota2.com/research/paper/69520adf53fdd5c36173d3f9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69520adf53fdd5c36173d3f9.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Vanilla

Acc: 87.2. Result ID: 699753b85698665ec4a5970d.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"InternVL 2.5 8B","Average Tokens":"100%"}. This record does not assert independent reproduction.

### IVC-Prune

Acc: 87.2. Result ID: 699753b85698665ec4a59710.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"InternVL 2.5 8B","Average Tokens":"50%"}. This record does not assert independent reproduction.

### ReShift

Acc: 87. Result ID: 6a46263a60b88bb339856cc3.

[Reporting paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

Recorded evaluation configuration: {"Base model":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction.

### PDrop

Acc: 86.9. Result ID: 699753b85698665ec4a5970f.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"InternVL 2.5 8B","Average Tokens":"56%"}. This record does not assert independent reproduction.

### Vanilla

Acc: 86.9. Result ID: 699753b85698665ec4a59711.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"DeepSeek-VL2 Small-16B","Average Tokens":"100%"}. This record does not assert independent reproduction.

### Qwen2.5-VL-7B

Acc: 86.81. Result ID: 6a0d846ffbc373592c060722.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Implementation](<https://github.com/UCSC-VLAA/VLM-CapCurriculum>)

Recorded evaluation configuration: {"Training":"Base"}. This record does not assert independent reproduction.

### GThinker-7B

Acc: 86.72. Result ID: 6a0d846ffbc373592c060723.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78ff1e209131cea7b90e.md>)

Recorded evaluation configuration: {"Training":"Base"}. This record does not assert independent reproduction.

### OneThinker-8B

Acc: 86.72. Result ID: 6a0d846ffbc373592c06072a.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc43dd6b0dd33429334dc.md>)

[Implementation](<https://github.com/tulerfeng/OneThinker>)

Recorded evaluation configuration: {"Training":"Base"}. This record does not assert independent reproduction.

### IVC-Prune

Acc: 86.7. Result ID: 699753b85698665ec4a5970c.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"Qwen2.5-VL 7B","Average Tokens":"50%"}. This record does not assert independent reproduction.

### PDrop

Acc: 86.7. Result ID: 699753b85698665ec4a59713.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"DeepSeek-VL2 Small-16B","Average Tokens":"57%"}. This record does not assert independent reproduction.

### FastV

Acc: 86.6. Result ID: 699753b85698665ec4a5970e.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74ae1e209131cea632b7.md>)

Recorded evaluation configuration: {"Models":"InternVL 2.5 8B","Average Tokens":"53%"}. This record does not assert independent reproduction.

### IVC-Prune

Acc: 86.6. Result ID: 699753b85698665ec4a59714.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"DeepSeek-VL2 Small-16B","Average Tokens":"52%"}. This record does not assert independent reproduction.

### Vanilla

Acc: 86.5. Result ID: 699753b85698665ec4a59709.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"Qwen2.5-VL 7B","Average Tokens":"100%"}. This record does not assert independent reproduction.

### FastV

Acc: 86.4. Result ID: 699753b85698665ec4a5970a.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74ae1e209131cea632b7.md>)

Recorded evaluation configuration: {"Models":"Qwen2.5-VL 7B","Average Tokens":"54%"}. This record does not assert independent reproduction.

### Qwen3-VL-8B

Acc: 86.29. Result ID: 6a0d846ffbc373592c06072b.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Implementation](<https://github.com/UCSC-VLAA/VLM-CapCurriculum>)

Recorded evaluation configuration: {"Training":"Staged"}. This record does not assert independent reproduction.

### Nullu

Acc: 86. Result ID: 69980a215698665ec463bb0c.

[Reporting paper](<https://www.sota2.com/research/paper/69520adf53fdd5c36173d3f9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa777c1e209131cea73837.md>)

[Implementation](<https://github.com/Ziwei-Zheng/Nullu>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### D.P.

Acc: 86. Result ID: 69c11a5e47f34ce924a954ae.

[Reporting paper](<https://www.sota2.com/research/paper/69c0c8db0636e5bf4afc397f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69c0c8db0636e5bf4afc397f.md>)

Recorded evaluation configuration: {"Data Source Type":"Real Data Reweighting","Sampling Strategy Category":"Ours","Compute Budget":"280K datapoints"}. This record does not assert independent reproduction.

### Base

Acc: 86. Result ID: 6a46263a60b88bb339856cbf.

[Reporting paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

Recorded evaluation configuration: {"Base model":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction.

### FastV

Acc: 85.9. Result ID: 699753b85698665ec4a59712.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74ae1e209131cea632b7.md>)

Recorded evaluation configuration: {"Models":"DeepSeek-VL2 Small-16B","Average Tokens":"54%"}. This record does not assert independent reproduction.

### Oracle

Acc: 85.8. Result ID: 69c11a5e47f34ce924a954ad.

[Reporting paper](<https://www.sota2.com/research/paper/69c0c8db0636e5bf4afc397f.md>)

Recorded evaluation configuration: {"Data Source Type":"Real Data Reweighting","Sampling Strategy Category":"Gold","Compute Budget":"280K datapoints"}. This record does not assert independent reproduction.

### PDrop

Acc: 85.6. Result ID: 699753b85698665ec4a5970b.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"Qwen2.5-VL 7B","Average Tokens":"61%"}. This record does not assert independent reproduction.

### MMR1-7B

Acc: 85.07. Result ID: 6a0d846ffbc373592c060724.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b8f1e209131cea8920f.md>)

Recorded evaluation configuration: {"Training":"Base"}. This record does not assert independent reproduction.

### Rewrite

Acc: 85. Result ID: 6a46263a60b88bb339856cc7.

[Reporting paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

Recorded evaluation configuration: {"Base model":"InternVL3.5-8B"}. This record does not assert independent reproduction.

### ICONS

Acc: 84.9. Result ID: 69c11a5e47f34ce924a954ac.

[Reporting paper](<https://www.sota2.com/research/paper/69c0c8db0636e5bf4afc397f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa795d1e209131cea7d48d.md>)

Recorded evaluation configuration: {"Data Source Type":"Real Data Reweighting","Sampling Strategy Category":"Baseline","Compute Budget":"280K datapoints"}. This record does not assert independent reproduction.

### OPERA

Acc: 84.2. Result ID: 69980a215698665ec463bb0a.

[Reporting paper](<https://www.sota2.com/research/paper/69520adf53fdd5c36173d3f9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72761e209131cea56869.md>)

[Implementation](<https://www.sota2.com/This%20link>)

Recorded evaluation configuration: {"Decoding Strategy":"OPERA"}. This record does not assert independent reproduction.

### Beam Search

Acc: 84.12. Result ID: 69980a215698665ec463bb09.

[Reporting paper](<https://www.sota2.com/research/paper/69520adf53fdd5c36173d3f9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa62d51e209131ce9fb707.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction.

### Uniform

Acc: 83.7. Result ID: 69c11a5e47f34ce924a954ab.

[Reporting paper](<https://www.sota2.com/research/paper/69c0c8db0636e5bf4afc397f.md>)

Recorded evaluation configuration: {"Data Source Type":"Real Data Reweighting","Sampling Strategy Category":"Baseline","Compute Budget":"280K datapoints"}. This record does not assert independent reproduction.

### R1-OneVision-RL-7B

Acc: 83.58. Result ID: 6a0d846ffbc373592c060726.

[Reporting paper](<https://www.sota2.com/research/paper/6a0d405ff984739ed6737bdc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77771e209131cea7357a.md>)

Recorded evaluation configuration: {"Training":"Base"}. This record does not assert independent reproduction.

### BadToken

Acc: 83. Result ID: 6a46263a60b88bb339856cc6.

[Reporting paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ea.md>)

Recorded evaluation configuration: {"Base model":"InternVL3.5-8B"}. This record does not assert independent reproduction.

### DyCo-RL

Acc: 82.7. Result ID: 6a27c571fbc373592c244920.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e3c90966ade919bf828.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a279e3c90966ade919bf828.md>)

[Implementation](<https://github.com/Sammy20207109/DyCo-RL>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-VL-3B","Zero-shot":"true"}. This record does not assert independent reproduction.

### GRPO Baseline

Acc: 82.3. Result ID: 6a27c571fbc373592c24491f.

[Reporting paper](<https://www.sota2.com/research/paper/6a279e3c90966ade919bf828.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-VL-3B","Zero-shot":"true"}. This record does not assert independent reproduction.

### BadToken

Acc: 82. Result ID: 6a46263a60b88bb339856cc1.

[Reporting paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77801e209131cea739ea.md>)

Recorded evaluation configuration: {"Base model":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction.

### H-GIVR

Acc: 81.83. Result ID: 69964d305698665ec491afa3.

[Reporting paper](<https://www.sota2.com/research/paper/6984241aa8b01e76cc8ee75e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6984241aa8b01e76cc8ee75e.md>)

Recorded evaluation configuration: {"Model":"Qwen2.5vl:7b","Prompt":"Ours"}. This record does not assert independent reproduction.

### Full KV Caching

Acc: 81.4. Result ID: 69c54c2747f34ce924af73f8.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-13B"}. This record does not assert independent reproduction.

### AttentionPack

Acc: 81.25. Result ID: 69c54c2747f34ce924af73fd.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-13B","Rkv":"64","Rvv":"64","Cache Memory Reduction":"5.17×","Average Throughput Change":"+43%"}. This record does not assert independent reproduction.

### VCD

Acc: 81.24. Result ID: 69980a215698665ec463bb0b.

[Reporting paper](<https://www.sota2.com/research/paper/69520adf53fdd5c36173d3f9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71461e209131cea5053d.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Visual Contrastive Decoding"}. This record does not assert independent reproduction.

### Minicache

Acc: 81.2. Result ID: 69c54c2747f34ce924af73fc.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74c61e209131cea63bba.md>)

[Implementation](<https://minicache.vmv.re/>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-13B","Cache Memory Reduction":"4.35×","Average Throughput Change":"+38%"}. This record does not assert independent reproduction.

### FastV

Acc: 81.17. Result ID: 69c54c2747f34ce924af73f9.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74ae1e209131cea632b7.md>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-13B","k":"5","e":"50%","Cache Memory Reduction":"1×","Average Throughput Change":"+20%"}. This record does not assert independent reproduction.

### InstructBLIP

Acc: 81. Result ID: 6992cd723a6c314849119bac.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b585.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6fb71e209131cea47021.md>)

[Implementation](<https://github.com/salesforce/LAVIS/tree/main/projects/instructblip>)

Recorded evaluation configuration: {"Paradigm":"Supervised"}. This record does not assert independent reproduction.

### Regular

Acc: 80.88. Result ID: 69980a215698665ec463bb08.

[Reporting paper](<https://www.sota2.com/research/paper/69520adf53fdd5c36173d3f9.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### MMBoundary

Acc: 80.6. Result ID: 698b58a27a967eddb76dd6d2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79af1e209131cea7ee1c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79af1e209131cea7ee1c.md>)

[Implementation](<https://github.com/Zhitao-He/MMBoundary>)

Recorded evaluation configuration: {"Base Model":"Qwen2VL 7B"}. This record does not assert independent reproduction.

### AttentionPack

Acc: 80. Result ID: 69c54c2747f34ce924af73fe.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-13B","Rkv":"32","Rvv":"32","Cache Memory Reduction":"7.30×","Average Throughput Change":"+50%"}. This record does not assert independent reproduction.

### Rewrite

Acc: 80. Result ID: 6a46263a60b88bb339856cc2.

[Reporting paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

Recorded evaluation configuration: {"Base model":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction.

### RCE

Acc: 79.3. Result ID: 698b58a27a967eddb76dd6d0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79af1e209131cea7ee1c.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2VL 7B"}. This record does not assert independent reproduction.

### PDrop

Acc: 79.2. Result ID: 699753b85698665ec4a59717.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"LLaVA-v1.5 7B","Average Tokens":"47%"}. This record does not assert independent reproduction.

### H2O

Acc: 79.1. Result ID: 69c54c2747f34ce924af73fb.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa718c1e209131cea51ead.md>)

[Implementation](<https://github.com/FMInference/H2O>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-13B","e":"50%","Cache Memory Reduction":"2×","Average Throughput Change":"+24%"}. This record does not assert independent reproduction.

### FastV

Acc: 79. Result ID: 699753b85698665ec4a59716.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74ae1e209131cea632b7.md>)

Recorded evaluation configuration: {"Models":"LLaVA-v1.5 7B","Average Tokens":"30%"}. This record does not assert independent reproduction.

### BadVision

Acc: 79. Result ID: 6a46263a60b88bb339856cc5.

[Reporting paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77a11e209131cea7479b.md>)

Recorded evaluation configuration: {"Base model":"InternVL3.5-8B"}. This record does not assert independent reproduction.

### Vanilla

Acc: 78.9. Result ID: 699753b85698665ec4a59715.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"LLaVA-v1.5 7B","Average Tokens":"100%"}. This record does not assert independent reproduction.

### IVC-Prune

Acc: 78.8. Result ID: 699753b85698665ec4a59718.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6982d2e4a8b01e76cc8e9893.md>)

Recorded evaluation configuration: {"Models":"LLaVA-v1.5 7B","Average Tokens":"28%"}. This record does not assert independent reproduction.

### ScissorHands

Acc: 77.48. Result ID: 69c54c2747f34ce924af73fa.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70781e209131cea4b48a.md>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-13B","e":"50%","Cache Memory Reduction":"2×","Average Throughput Change":"+24%"}. This record does not assert independent reproduction.

### Conf-CSR

Acc: 77.4. Result ID: 698b58a27a967eddb76dd6cf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79af1e209131cea7ee1c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75901e209131cea68a5c.md>)

[Implementation](<https://github.com/YiyangZhou/CSR>)

Recorded evaluation configuration: {"Base Model":"Qwen2VL 7B"}. This record does not assert independent reproduction.

### BadVision

Acc: 77. Result ID: 6a46263a60b88bb339856cc0.

[Reporting paper](<https://www.sota2.com/research/paper/6a45f085dc71372d7795a21f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77a11e209131cea7479b.md>)

Recorded evaluation configuration: {"Base model":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction.

### AttentionPack

Acc: 76.88. Result ID: 69c54c2747f34ce924af73f6.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-7B","Rkv":"64","Rvv":"64","Cache Memory Reduction":"5.09×","Average Throughput Change":"+54%"}. This record does not assert independent reproduction.

### FastV

Acc: 76.72. Result ID: 69c54c2747f34ce924af73f2.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74ae1e209131cea632b7.md>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-7B","k":"5","e":"50%","Cache Memory Reduction":"1×","Average Throughput Change":"+29%"}. This record does not assert independent reproduction.

### Full KV Caching

Acc: 76.64. Result ID: 69c54c2747f34ce924af73f1.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-7B"}. This record does not assert independent reproduction.

### Minicache

Acc: 76.54. Result ID: 69c54c2747f34ce924af73f5.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74c61e209131cea63bba.md>)

[Implementation](<https://minicache.vmv.re/>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-7B","Cache Memory Reduction":"4.51×","Average Throughput Change":"+44%"}. This record does not assert independent reproduction.

### H-GIVR

Acc: 76.42. Result ID: 69964d305698665ec491af9d.

[Reporting paper](<https://www.sota2.com/research/paper/6984241aa8b01e76cc8ee75e.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6984241aa8b01e76cc8ee75e.md>)

Recorded evaluation configuration: {"Model":"Llama3.2-vision:11b","Prompt":"Ours"}. This record does not assert independent reproduction.

### Prophet

Acc: 76.4. Result ID: 6992cd723a6c314849119bab.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b585.md>)

Recorded evaluation configuration: {"Paradigm":"Supervised","trained\_on\_dataset":true}. This record does not assert independent reproduction.

### H2O

Acc: 76.36. Result ID: 69c54c2747f34ce924af73f4.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa718c1e209131cea51ead.md>)

[Implementation](<https://github.com/FMInference/H2O>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-7B","e":"50%","Cache Memory Reduction":"2×","Average Throughput Change":"+32%"}. This record does not assert independent reproduction.

### DRL

Acc: 76.2. Result ID: 698b58a27a967eddb76dd6d1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79af1e209131cea7ee1c.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2VL 7B"}. This record does not assert independent reproduction.

### AttentionPack

Acc: 75.91. Result ID: 69c54c2747f34ce924af73f7.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

Recorded evaluation configuration: {"Model":"LLaVA1.5-7B","Rkv":"32","Rvv":"32","Cache Memory Reduction":"7.24×","Average Throughput Change":"+65%"}. This record does not assert independent reproduction.

### Full KV Caching

Acc: 75.67. Result ID: 69c54c2747f34ce924af73ff.

[Reporting paper](<https://www.sota2.com/research/paper/69c4bd7f0636e5bf4afc7592.md>)

Recorded evaluation configuration: {"Model":"QwenVL-Chat-7B"}. This record does not assert independent reproduction.

### ViCor

Acc: 75.6. Result ID: 6992cd723a6c314849119bb2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b585.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73421e209131cea5b585.md>)

Recorded evaluation configuration: {"Paradigm":"In-Context Learning","Backbone":"GPT-4"}. This record does not assert independent reproduction.

Page 1 of 3. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/sota/visual-question-answering-on-a-okvqa.md?metric=Acc&page=2>): Follow this link to discover more pages.
