# Flaws of ImageNet, Computer Vision's Favourite Dataset

Source: [Flaws of ImageNet, Computer Vision's Favourite Dataset](<https://www.sota2.com/research/paper/68fa76031e209131cea6ae8e>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:45:34.560Z

Authors: Nikita Kisel, Illia Volkov, Katerina Hanzelkova, Klara Janouskova, Jiri Matas

Published: 2024-11-26T16:47:59.000Z

## Abstract

Since its release, ImageNet-1k dataset has become a gold standard for evaluating model performance. It has served as the foundation for numerous other datasets and training tasks in computer vision. As models have improved in accuracy, issues related to label correctness have become increasingly apparent. In this blog post, we analyze the issues in the ImageNet-1k dataset, including incorrect labels, overlapping or ambiguous class definitions, training-evaluation domain shifts, and image duplicates. The solutions for some problems are straightforward. For others, we hope to start a broader conversation about refining this influential dataset to better serve future research.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2412.00076>)

[PDF](<https://arxiv.org/pdf/2412.00076.pdf>)