Visual Question Answering on RealWorldQA (RWQA)
68.5ScoreVanilla
Evaluation results
| Method | Links | |
|---|---|---|
| VanillaMode=Upper bound2024.12 | 68.5 | |
| iLLaVAImage token reduction ratio=66.7%2024.12 | 68.2 | |
| iLLaVAImage token reduction ratio=77.8%2024.12 | 67.7 | |
| VisionZipImage token reduction ratio=66.7%2024.12 | 67.4 | |
| SparseVLMImage token reduction ratio=66.7%2024.12 | 67.2 | |
| PyramidDropImage token reduction ratio=66.7%2024.12 | 66.5 | |
| FasterVLMImage token reduction ratio=66.7%2024.12 | 66.4 | |
| FasterVLMImage token reduction ratio=77.8%2024.12 | 66 | |
| VisionZipImage token reduction ratio=88.9%2024.12 | 65.4 | |
| iLLaVAImage token reduction ratio=88.9%2024.12 | 65.3 | |
| VisionZipImage token reduction ratio=77.8%2024.12 | 64.8 | |
| SparseVLMImage token reduction ratio=77.8%2024.12 | 64.7 | |
| SparseVLMImage token reduction ratio=88.9%2024.12 | 64.7 | |
| PyramidDropImage token reduction ratio=77.8%2024.12 | 64.1 | |
| FasterVLMImage token reduction ratio=88.9%2024.12 | 63.7 | |
| PyramidDropImage token reduction ratio=88.9%2024.12 | 59.8 | |
| EOSRetain (visual tokens)=322026.06 | 52.94 | |
| SparseVLM-v2Retain (visual tokens)=322026.06 | 51.76 | |
| SparseVLM-v1Retain (visual tokens)=322026.06 | 50.2 | |
| EOSRetain (visual tokens)=162026.06 | 48.65 | |
| SparseVLM-v2Retain (visual tokens)=162026.06 | 47.46 | |
| ToMeRetain (visual tokens)=322026.06 | 47.21 | |
| SparseVLM-v1Retain (visual tokens)=162026.06 | 46.4 | |
| ToMeRetain (visual tokens)=162026.06 | 44.81 | |
| PoolRetain (visual tokens)=322026.06 | 38.5 | |
| PoolRetain (visual tokens)=162026.06 | 36.1 |