# Can CLIP Count Stars? An Empirical Study on Quantity Bias in CLIP

Source: [Can CLIP Count Stars? An Empirical Study on Quantity Bias in CLIP](<https://www.sota2.com/research/paper/68fa74e31e209131cea646fa>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:40:00.946Z

Authors: Zeliang Zhang, Zhuo Liu, Mingqian Feng, Chenliang Xu

Published: 2024-09-23T14:01:16.000Z

## Abstract

CLIP has demonstrated great versatility in adapting to various downstream tasks, such as image editing and generation, visual question answering, and video understanding. However, CLIP-based applications often suffer from misunderstandings regarding user intent, leading to discrepancies between the required number of objects and the actual outputs in image generation tasks. In this work, we empirically investigate the quantity bias in CLIP. By carefully designing different experimental settings and datasets, we comprehensively evaluate CLIP's understanding of quantity from text, image, and cross-modal perspectives. Our experimental results reveal a quantity bias in CLIP embeddings, impacting the reliability of downstream tasks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2409.15035>)

[PDF](<https://arxiv.org/pdf/2409.15035.pdf>)
