# When Labels Have Structure: Improving Image Classification with Hierarchy-Aware Cross-Entropy

Source: [When Labels Have Structure: Improving Image Classification with Hierarchy-Aware Cross-Entropy](<https://www.sota2.com/research/paper/69fd6e205497b3b670c894c3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-09T05:00:12.752Z

Authors: April Chan, Davide D'Ascenzo, Sebastiano Cultrera di Montesano

Published: 2026-05-07T13:49:36.000Z

## Abstract

Standard cross-entropy is the default classification loss across virtually all of machine learning, yet it treats all misclassifications equally, ignoring the semantic distances that a class hierarchy encodes. We propose Hierarchy-Aware Cross-Entropy (HACE), a drop-in replacement for standard cross-entropy that incorporates a known class hierarchy directly into the loss. HACE combines two components: prediction aggregation, which propagates the model's probability mass upward through the class hierarchy to ensure that parent nodes accumulate the confidence of their children; and ancestral label smoothing, which distributes the ground-truth signal along the path from the true class to the root. We evaluate HACE on CIFAR-100, FGVC Aircraft, and NABirds in two regimes: end-to-end training across six architectures spanning convolutional and attention-based designs, and linear probing on frozen DINOv2-Large features. In end-to-end training, HACE improves accuracy over standard cross-entropy in 15 out of 18 architecture--dataset pairs, with a mean gain of 4.66\\%. In linear probing on frozen DINOv2-Large features, HACE outperforms all competing methods on all three datasets, with a mean improvement of 2.18\\% over the next best baseline.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on FGVC (Top-5 Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-fgvc-top-5-accuracy.md>): Image Classification; FGVC; Top-5 Accuracy: 95.23.

- [Image Classification on NABirds (test)](<https://www.sota2.com/research/sota/image-classification-on-nabirds-test.md>): Image Classification; NABirds (test); Top-5 Accuracy: 97.62.

- [Image Classification on NABirds (Top-5 Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-nabirds-top-5-accuracy.md>): Image Classification; NABirds; Top-5 Accuracy: 87.67.

- [Image Classification on NABirds](<https://www.sota2.com/research/sota/image-classification-on-nabirds.md>): Image Classification; NABirds; Accuracy: 84.96.

- [Image Classification on FGVC](<https://www.sota2.com/research/sota/image-classification-on-fgvc.md>): Image Classification; FGVC; Accuracy: 80.62.

- [Image Classification on CIFAR-100 (test) (Standard Metrics)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-test-standard-metrics.md>): Image Classification; CIFAR-100 (test); Top-1 Accuracy: not recorded.

- [Image Classification on CIFAR-100 (ACC, NLL, ECE, ESS)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-acc-nll-ece-ess.md>): Image Classification; CIFAR-100; Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.06274>)

[PDF](<https://arxiv.org/pdf/2605.06274.pdf>)