# Skeleton Key: Image Captioning by Skeleton-Attribute Decomposition

Source: [Skeleton Key: Image Captioning by Skeleton-Attribute Decomposition](<https://www.sota2.com/research/paper/68fa634a1e209131ce9fdb9f>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:37:15.413Z

Authors: Yufei Wang, Zhe Lin, Xiaohui Shen, Scott Cohen, Garrison W. Cottrell

Published: 2017-04-23T20:17:12.000Z

## Abstract

Recently, there has been a lot of interest in automatically generating descriptions for an image. Most existing language-model based approaches for this task learn to generate an image description word by word in its original word order. However, for humans, it is more natural to locate the objects and their relationships first, and then elaborate on each object, describing notable attributes. We present a coarse-to-fine method that decomposes the original image description into a skeleton sentence and its attributes, and generates the skeleton sentence and attribute phrases separately. By this decomposition, our method can generate more accurate and novel descriptions than the previous state-of-the-art. Experimental results on the MS-COCO and a larger scale Stock3M datasets show that our algorithm yields consistent improvements across different evaluation metrics, especially on the SPICE metric, which has much higher correlation with human ratings than the conventional metrics. Furthermore, our algorithm can generate descriptions with varied length, benefiting from the separate control of the skeleton and attributes. This enables image description generation that better accommodates user preferences.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Captioning on MS-COCO](<https://www.sota2.com/research/sota/image-captioning-on-ms-coco.md>): Image Captioning; MS-COCO; CIDEr: 0.966.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1704.06972>)

[PDF](<https://arxiv.org/pdf/1704.06972.pdf>)