# Benchmarking and Improving Detail Image Caption Source: [Benchmarking and Improving Detail Image Caption]() Collection index: [llms.txt]() Data updated: 2025-10-26T03:00:03.231Z Authors: Hongyuan Dong, Jiawen Li, Bohong Wu, Jiacong Wang, Yuan Zhang, Haoyuan Guo Published: 2024-05-29T13:54:12.000Z ## Abstract Image captioning has long been regarded as a fundamental task in visual understanding. Recently, however, few large vision-language model (LVLM) research discusses model's image captioning performance because of the outdated short-caption benchmarks and unreliable evaluation metrics. In this work, we propose to benchmark detail image caption task by curating high-quality evaluation datasets annotated by human experts, GPT-4V and Gemini-1.5-Pro. We also design a more reliable caption evaluation metric called CAPTURE (CAPtion evaluation by exTracting and coUpling coRE information). CAPTURE extracts visual elements, e.g., objects, attributes and relations from captions, and then matches these elements through three stages, achieving the highest consistency with expert judgements over other rule-based or model-based caption metrics. The proposed benchmark and metric provide reliable evaluation for LVLM's detailed image captioning ability. Guided by this evaluation, we further explore to unleash LVLM's detail caption capabilities by synthesizing high-quality data through a five-stage data construction pipeline. Our pipeline only uses a given LVLM itself and other open-source tools, without any human or GPT-4V annotation in the loop. Experiments show that the proposed data construction strategy significantly improves model-generated detail caption data quality for LVLMs with leading performance, and the data quality can be further improved in a self-looping paradigm. All code and dataset will be publicly available at https://github.com/foundation-multimodal-models/CAPTURE. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Captioning on DID-Bench GT-{GPT4-V}](): Image Captioning; DID-Bench GT-{GPT4-V}; BLEU-1: 37.4. - [Multimodal Evaluation on DID-Bench](): Multimodal Evaluation; DID-Bench; CLIP-S Score: 41.13. - [Image Captioning on DID-Bench GT-{LLaVA}](): Image Captioning; DID-Bench GT-{LLaVA}; BLEU-1: 35.8. - [Image Captioning on DID-Bench GT-GPT4-V 1.0 (test)](): Image Captioning; DID-Bench GT-GPT4-V 1.0 (test); BLEU-1: 27.7. - [Image Captioning on DID-Bench GT-LLaVA (test)](): Image Captioning; DID-Bench GT-LLaVA (test); BLEU-1: 30.4. - [Linguistic Complexity Evaluation on LIN-Bench](): Linguistic Complexity Evaluation; LIN-Bench; ARI: 10.75. - [Image Reconstruction Similarity on D2I-Bench](): Image Reconstruction Similarity; D2I-Bench; CLIP Score: 75.31. - [Fine-grained Image Captioning on DetailCaps (test)](): Fine-grained Image Captioning; DetailCaps (test); CAPTURE: 62.53. ## Original sources and implementation [Original paper]() [PDF]()