# MIVC: Multiple Instance Visual Component for Visual-Language Models

Source: [MIVC: Multiple Instance Visual Component for Visual-Language Models](<https://www.sota2.com/research/paper/68fa71a31e209131cea523a4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:09:33.631Z

Authors: Wenyi Wu, Qi Li, Wenliang Zhong, Junzhou Huang

Published: 2023-12-28T16:33:32.000Z

## Abstract

Vision-language models have been widely explored across a wide range of tasks and achieve satisfactory performance. However, it's under-explored how to consolidate entity understanding through a varying number of images and to align it with the pre-trained language models for generative tasks. In this paper, we propose MIVC, a general multiple instance visual component to bridge the gap between various image inputs with off-the-shelf vision-language models by aggregating visual representations in a permutation-invariant fashion through a neural network. We show that MIVC could be plugged into the visual-language models to improve the model performance consistently on visual question answering, classification and captioning tasks on a public available e-commerce dataset with multiple images per product. Furthermore, we show that the component provides insight into the contribution of each image to the downstream tasks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2312.17109>)

[PDF](<https://arxiv.org/pdf/2312.17109.pdf>)
