# Love Thy Neighbors: Image Annotation by Exploiting Image Metadata Source: [Love Thy Neighbors: Image Annotation by Exploiting Image Metadata]() Collection index: [llms.txt]() Data updated: 2025-10-25T20:12:29.814Z Authors: Justin Johnson, Lamberto Ballan, Fei-Fei Li Published: 2015-08-30T23:34:13.000Z ## Abstract Some images that are difficult to recognize on their own may become more clear in the context of a neighborhood of related images with similar social-network metadata. We build on this intuition to improve multilabel image annotation. Our model uses image metadata nonparametrically to generate neighborhoods of related images using Jaccard similarities, then uses a deep neural network to blend visual information from the image and its neighbors. Prior work typically models image metadata parametrically, in contrast, our nonparametric treatment allows our model to perform well even when the vocabulary of metadata changes between training and testing. We perform comprehensive experiments on the NUS-WIDE dataset, where we show that our model outperforms state-of-the-art methods for multilabel image annotation even when our model is forced to generalize to new types of metadata. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Multi-label Image Classification on NUS-WIDE 81 concept labels (test)](): Multi-label Image Classification; NUS-WIDE 81 concept labels (test); F1 (Class): 45.2. ## Original sources and implementation [Original paper]() [PDF]()