# Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images

Source: [Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images](<https://www.sota2.com/research/paper/68fa621a1e209131ce9f7ac4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:13:30.351Z

Authors: Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, Alan Yuille

Published: 2015-04-25T06:45:35.000Z

## Abstract

In this paper, we address the task of learning novel visual concepts, and their interactions with other concepts, from a few images with sentence descriptions. Using linguistic context and visual features, our method is able to efficiently hypothesize the semantic meaning of new words and add them to its word dictionary so that they can be used to describe images which contain these novel concepts. Our method has an image captioning module based on m-RNN with several improvements. In particular, we propose a transposed weight sharing scheme, which not only improves performance on image captioning, but also makes the model more suitable for the novel concept learning task. We propose methods to prevent overfitting the new concepts. In addition, three novel concept datasets are constructed for this new task. In the experiments, we show that our method effectively learns novel visual concepts from a few examples without disturbing the previously learned concepts. The project page is http://www.stat.ucla.edu/~junhua.mao/projects/child\_learning.html

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on ILSVRC 2015 (val)](<https://www.sota2.com/research/sota/image-classification-on-ilsvrc-2015-val.md>): Image Classification; ILSVRC 2015 (val); Top-1 Acc (C\_large): 76.71.

- [Image Captioning on COCO 2014 (test)](<https://www.sota2.com/research/sota/image-captioning-on-coco-2014-test.md>): Image Captioning; COCO 2014 (test); CIDEr: 0.935.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1504.06692>)

[PDF](<https://arxiv.org/pdf/1504.06692.pdf>)