Digging Deeper: Learning Multi-Level Concept Hierarchies

About

Although concept-based models promise interpretability by explaining predictions with human-understandable concepts, they typically rely on exhaustive annotations and treat concepts as flat and independent. To circumvent this, recent work has introduced Hierarchical Concept Embedding Models (HiCEMs) to explicitly model concept relationships, and Concept Splitting to discover sub-concepts using only coarse annotations. However, both HiCEMs and Concept Splitting are restricted to shallow hierarchies. We overcome this limitation with Multi-Level Concept Splitting (MLCS), which discovers multi-level concept hierarchies from only top-level supervision, and Deep-HiCEMs, an architecture that represents these discovered hierarchies and enables interventions at multiple levels of abstraction. Experiments across multiple datasets show that MLCS discovers human-interpretable concepts absent during training and that Deep-HiCEMs maintain high accuracy while supporting test-time concept interventions that can improve task performance.

Oscar Hill, Mateo Espinosa Zarlenga, Mateja Jamnik · 2026

Related benchmarks

Task	Dataset	Result
Classification	CUB	Accuracy73	100
Classification	AWA2	Class Accuracy97	41
Classification	SHAPES	Accuracy87	8
Classification	MNIST ADD	Accuracy92	7
Classification	PseudoKitchens 2	Accuracy58	7
Discovered Concept Prediction	SHAPES	Mean ROC-AUC93	3
Discovered Concept Prediction	CUB	Mean ROC-AUC0.84	3
Discovered Concept Prediction	AWA2	Mean ROC-AUC85	3
Discovered Concept Prediction	MNIST ADD	Mean ROC-AUC94	2
Discovered Concept Prediction	PseudoKitchens 2	Mean ROC-AUC79	2