# ParGo: Bridging Vision-Language with Partial and Global Views

Source: [ParGo: Bridging Vision-Language with Partial and Global Views](<https://www.sota2.com/research/paper/68fa777c1e209131cea7380c>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:19:49.174Z

Authors: An-Lan Wang, Bin Shan, Wei Shi, Kun-Yu Lin, Xiang Fei, Guozhi Tang, Lei Liao, Can Huang, Jingqun Tang, Wei-Shi Zheng

Published: 2024-08-23T09:14:58.000Z

## Abstract

This work presents ParGo, a novel Partial-Global projector designed to connect the vision and language modalities for Multimodal Large Language Models (MLLMs). Unlike previous works that rely on global attention-based projectors, our ParGo bridges the representation gap between the separately pre-trained vision encoders and the LLMs by integrating global and partial views, which alleviates the overemphasis on prominent regions. To facilitate the effective training of ParGo, we collect a large-scale detail-captioned image-text dataset named ParGoCap-1M-PT, consisting of 1 million images paired with high-quality captions. Extensive experiments on several MLLM benchmarks demonstrate the effectiveness of our ParGo, highlighting its superiority in aligning vision and language modalities. Compared to conventional Q-Former projector, our ParGo achieves an improvement of 259.96 in MME benchmark. Furthermore, our experiments reveal that ParGo significantly outperforms other projectors, particularly in tasks that emphasize detail perception ability.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Object Detection on Locomotive coil spring defect dataset (test)](<https://www.sota2.com/research/sota/object-detection-on-locomotive-coil-spring-defect-dataset-test.md>): Object Detection; Locomotive coil spring defect dataset (test); mAP@0.5: 88.9.

- [Driving Question Answering on DRIVEXQA 1.0 (test)](<https://www.sota2.com/research/sota/driving-question-answering-on-drivexqa-1-0-test.md>): Driving Question Answering; DRIVEXQA 1.0 (test); CL Score (GPT-4o-mini): 36.5.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2408.12928>)

[PDF](<https://arxiv.org/pdf/2408.12928.pdf>)