# Image Difference Captioning with Pre-training and Contrastive Learning Source: [Image Difference Captioning with Pre-training and Contrastive Learning]() Collection index: [llms.txt]() Data updated: 2025-10-25T23:57:05.246Z Authors: Linli Yao, Weiying Wang, Qin Jin Published: 2022-02-09T06:14:22.000Z ## Abstract The Image Difference Captioning (IDC) task aims to describe the visual differences between two similar images with natural language. The major challenges of this task lie in two aspects: 1) fine-grained visual differences that require learning stronger vision and language association and 2) high-cost of manual annotations that leads to limited supervised data. To address these challenges, we propose a new modeling framework following the pre-training-finetuning paradigm. Specifically, we design three self-supervised tasks and contrastive learning strategies to align visual differences and text descriptions at a fine-grained level. Moreover, we propose a data expansion strategy to utilize extra cross-task supervision information, such as data for fine-grained image classification, to alleviate the limitation of available supervised IDC data. Extensive experiments on two IDC benchmark datasets, CLEVR-Change and Birds-to-Words, demonstrate the effectiveness of the proposed modeling framework. The codes and models will be released at https://github.com/yaolinli/IDC. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Change Captioning on CLEVR-Change](): Image Change Captioning; CLEVR-Change; BLEU-4: 51.2. ## Original sources and implementation [Original paper]() [PDF]()