# Visually Consistent Hierarchical Image Classification

Source: [Visually Consistent Hierarchical Image Classification](<https://www.sota2.com/research/paper/68fa78081e209131cea76717>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:30:26.676Z

Authors: Seulki Park, Youren Zhang, Stella X. Yu, Sara Beery, Jonathan Huang

Published: 2024-06-17T14:56:51.000Z

## Abstract

Hierarchical classification predicts labels across multiple levels of a taxonomy, e.g., from coarse-level 'Bird' to mid-level 'Hummingbird' to fine-level 'Green hermit', allowing flexible recognition under varying visual conditions. It is commonly framed as multiple single-level tasks, but each level may rely on different visual cues: Distinguishing 'Bird' from 'Plant' relies on global features like feathers or leaves, while separating 'Anna's hummingbird' from 'Green hermit' requires local details such as head coloration. Prior methods improve accuracy using external semantic supervision, but such statistical learning criteria fail to ensure consistent visual grounding at test time, resulting in incorrect hierarchical classification. We propose, for the first time, to enforce internal visual consistency by aligning fine-to-coarse predictions through intra-image segmentation. Our method outperforms zero-shot CLIP and state-of-the-art baselines on hierarchical classification benchmarks, achieving both higher accuracy and more consistent predictions. It also improves internal image segmentation without requiring pixel-level annotations.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Hierarchical Visual Recognition on ImageNet-F 20-127-505 free-grain (test)](<https://www.sota2.com/research/sota/hierarchical-visual-recognition-on-imagenet-f-20-127-505-free-grain-test.md>): Hierarchical Visual Recognition; ImageNet-F 20-127-505 free-grain (test); FPA: 57.59.

- [Hierarchical Visual Classification on CUB-F 13-38-200](<https://www.sota2.com/research/sota/hierarchical-visual-classification-on-cub-f-13-38-200.md>): Hierarchical Visual Classification; CUB-F 13-38-200; FPA: 45.1.

- [Hierarchical Visual Recognition on iNat mini-F 21 free-grain (test)](<https://www.sota2.com/research/sota/hierarchical-visual-recognition-on-inat-mini-f-21-free-grain-test.md>): Hierarchical Visual Recognition; iNat mini-F 21 free-grain (test); FPA: 25.63.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2406.11608>)

[PDF](<https://arxiv.org/pdf/2406.11608.pdf>)