# HMID-Net: An Exploration of Masked Image Modeling and Knowledge Distillation in Hyperbolic Space

Source: [HMID-Net: An Exploration of Masked Image Modeling and Knowledge Distillation in Hyperbolic Space](<https://www.sota2.com/research/paper/68fa7a1c1e209131cea813fa>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:12:23.158Z

Authors: Changli Wang, Fang Yin, Jiafeng Liu, Rui Wu

Published: 2025-07-13T04:14:20.000Z

## Abstract

Visual and semantic concepts are often structured in a hierarchical manner. For instance, textual concept \`cat' entails all images of cats. A recent study, MERU, successfully adapts multimodal learning techniques from Euclidean space to hyperbolic space, effectively capturing the visual-semantic hierarchy. However, a critical question remains: how can we more efficiently train a model to capture and leverage this hierarchy? In this paper, we propose the Hyperbolic Masked Image and Distillation Network (HMID-Net), a novel and efficient method that integrates Masked Image Modeling (MIM) and knowledge distillation techniques within hyperbolic space. To the best of our knowledge, this is the first approach to leverage MIM and knowledge distillation in hyperbolic space to train highly efficient models. In addition, we introduce a distillation loss function specifically designed to facilitate effective knowledge transfer in hyperbolic space. Our experiments demonstrate that MIM and knowledge distillation techniques in hyperbolic space can achieve the same remarkable success as in Euclidean space. Extensive evaluations show that our method excels across a wide range of downstream tasks, significantly outperforming existing models like MERU and CLIP in both image classification and retrieval.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2507.09487>)

[PDF](<https://arxiv.org/pdf/2507.09487.pdf>)
