# FocusLLaVA: A Coarse-to-Fine Approach for Efficient and Effective Visual Token Compression

Source: [FocusLLaVA: A Coarse-to-Fine Approach for Efficient and Effective Visual Token Compression](<https://www.sota2.com/research/paper/68fa75da1e209131cea6a406>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:03:51.030Z

Authors: Yuke Zhu, Chi Xie, Shuang Liang, Bo Zheng, Sheng Guo

Published: 2024-11-21T15:37:52.000Z

## Abstract

Recent advances on Multi-modal Large Language Models have demonstrated that high-resolution image input is crucial for model capabilities, especially for fine-grained tasks. However, high-resolution images lead to a quadratic increase in the number of visual tokens input into LLMs, resulting in significant computational costs. Current work develop visual token compression methods to achieve efficiency improvements, often at the expense of performance. We argue that removing visual redundancy can simultaneously improve both efficiency and performance. We build a coarse-to-fine visual token compression method, with a vision-guided sampler for compressing redundant regions with low information density, and a text-guided sampler for selecting visual tokens that are strongly correlated with the user instructions.With these two modules, the proposed FocusLLaVA achieves improvements in both efficiency and performance. We validate the effectiveness of our approach on a wide range of evaluation datasets.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Multimodal Evaluation on MMBench CN](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mmbench-cn.md>): Multimodal Evaluation; MMBench CN; Accuracy: 70.3.

- [Question Answering on ScienceQA](<https://www.sota2.com/research/sota/question-answering-on-scienceqa.md>): Question Answering; ScienceQA; Accuracy: 79.

- [Multimodal Evaluation on LLaVA-bench in-the-wild](<https://www.sota2.com/research/sota/multimodal-evaluation-on-llava-bench-in-the-wild.md>): Multimodal Evaluation; LLaVA-Bench In-the-Wild; Score: 65.6.

- [Multimodal Model Evaluation on MMBench](<https://www.sota2.com/research/sota/multimodal-model-evaluation-on-mmbench.md>): Multimodal Model Evaluation; MMBench; Accuracy: 74.7.

- [Visual Question Answering on GQA](<https://www.sota2.com/research/sota/visual-question-answering-on-gqa.md>): Visual Question Answering; GQA; Accuracy: 66.

- [Multimodal Capability Evaluation on MM-Vet](<https://www.sota2.com/research/sota/multimodal-capability-evaluation-on-mm-vet.md>): Multimodal Capability Evaluation; MM-Vet; Score: 41.3.

- [Visual Question Answering on TextVQA](<https://www.sota2.com/research/sota/visual-question-answering-on-textvqa.md>): Visual Question Answering; TextVQA; Accuracy: 70.

- [Object Hallucination Evaluation on POPE](<https://www.sota2.com/research/sota/object-hallucination-evaluation-on-pope.md>): Object Hallucination Evaluation; POPE; Accuracy: 87.7.

- [Multimodal Evaluation on MME](<https://www.sota2.com/research/sota/multimodal-evaluation-on-mme.md>): Multimodal Evaluation; MME; Score: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2411.14228>)

[PDF](<https://arxiv.org/pdf/2411.14228.pdf>)
