# Neural Clustering based Visual Representation Learning Source: [Neural Clustering based Visual Representation Learning]() Collection index: [llms.txt]() Data updated: 2026-01-04T04:42:39.344Z Authors: Guikun Chen, Xia Li, Yi Yang, Wenguan Wang Published: 2024-03-26T06:04:50.000Z ## Abstract We investigate a fundamental aspect of machine vision: the measurement of features, by revisiting clustering, one of the most classic approaches in machine learning and data analysis. Existing visual feature extractors, including ConvNets, ViTs, and MLPs, represent an image as rectangular regions. Though prevalent, such a grid-style paradigm is built upon engineering practice and lacks explicit modeling of data distribution. In this work, we propose feature extraction with clustering (FEC), a conceptually elegant yet surprisingly ad-hoc interpretable neural clustering framework, which views feature extraction as a process of selecting representatives from data and thus automatically captures the underlying data distribution. Given an image, FEC alternates between grouping pixels into individual clusters to abstract representatives and updating the deep features of pixels with current representatives. Such an iterative working mechanism is implemented in the form of several neural layers and the final representatives can be used for downstream tasks. The cluster assignments across layers, which can be viewed and inspected by humans, make the forward process of FEC fully transparent and empower it with promising ad-hoc interpretability. Extensive experiments on various visual recognition models and tasks verify the effectiveness, generality, and interpretability of FEC. We expect this work will provoke a rethink of the current de facto grid-style paradigm. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on ImageNet Mini](): Image Classification; ImageNet mini; Top-1 Accuracy: 79.33. - [Image Classification on ImageNet-1K 1.0 (val)](): Image Classification; ImageNet-1K 1.0 (val); Top-1 Accuracy: 81.2. - [Semantic Segmentation on ADE20K (val)](): Semantic segmentation; ADE20K (val); mIoU: 40.5. - [Object Detection on COCO 2017 (val)](): Object Detection; COCO 2017 (val); AP: not recorded. - [Instance Segmentation on COCO 2017 (val)](): Instance Segmentation; COCO 2017 (val); APm: not recorded. - [Image Classification on CIFAR-100 (val)](): Image Classification; CIFAR-100 (val); Accuracy: not recorded. ## Original sources and implementation [Original paper]() [PDF]() [Implementation]()