# Paying Attention to Descriptions Generated by Image Captioning Models Source: [Paying Attention to Descriptions Generated by Image Captioning Models]() Collection index: [llms.txt]() Data updated: 2025-10-25T20:44:17.219Z Authors: Hamed R. Tavakoli, Rakshith Shetty, Ali Borji, Jorma Laaksonen Published: 2017-04-24T19:51:16.000Z ## Abstract To bridge the gap between humans and machines in image understanding and describing, we need further insight into how people describe a perceived scene. In this paper, we study the agreement between bottom-up saliency-based visual attention and object referrals in scene description constructs. We investigate the properties of human-written descriptions and machine-generated ones. We then propose a saliency-boosted image captioning model in order to investigate benefits from low-level cues in language models. We learn that (1) humans mention more salient objects earlier than less salient ones in their descriptions, (2) the better a captioning model performs, the better attention agreement it has with human descriptions, (3) the proposed saliency-boosted model, compared to its baseline form, does not improve significantly on the MS COCO database, indicating explicit bottom-up boosting does not help when the task is well learnt and tuned on a data, (4) a better generalization is, however, observed for the saliency-boosted model on unseen data. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Captioning on COCO (test)](): Image Captioning; COCO (test); CIDEr: 84.1. ## Original sources and implementation [Original paper]() [PDF]()