# Rapidly deploying on-device eye tracking by distilling visual foundation models Source: [Rapidly deploying on-device eye tracking by distilling visual foundation models]() Collection index: [llms.txt]() Data updated: 2026-07-04T05:01:04.041Z Authors: Cheng Jiang, Jogendra Kundu, David Colmenares, Fengting Yang, Joseph P Robinson, Ali Behrooz, Yatong An Published: 2026-04-02T21:07:17.000Z ## Abstract Eye tracking (ET) plays a critical role in augmented and virtual reality applications. However, rapidly deploying high-accuracy, on-device gaze estimation for new products remains challenging because hardware configurations (e.g., camera placement, camera pose, and illumination) often change across device generations. Visual foundation models (VFMs) excel on natural-image benchmarks and offer a promising path to rapid training and deployment; yet, we find that off-the-shelf VFMs still struggle to reach high accuracy on specialized near-eye infrared images. To close this gap, we introduce DistillGaze, a framework that distills a VFM using labeled synthetic data and unlabeled real data for rapid, high-accuracy on-device gaze estimation. DistillGaze proceeds in two stages. First, we adapt a VFM into a domain-specialized teacher using synthetic gaze labels and unlabeled real images. Synthetic data provide scalable, high-quality gaze supervision, while unlabeled real data bridges the synthetic-to-real domain gap. Second, we train an on-device student from both teacher guidance and self-training. Evaluated on a large-scale crowd-sourced dataset spanning more than 2,000 participants, DistillGaze reduces median gaze error by 58.6% relative to synthetic-only baselines while maintaining a lightweight 256K-parameter model suitable for real-time on-device deployment. More broadly, DistillGaze offers an efficient path to training and deploying ET models that adapt to hardware changes, and a recipe for combining synthetic supervision with unlabeled real data in on-device regression tasks. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Gaze Estimation on Project Aria](): Gaze Estimation; Project Aria; E50U50: 1.33. - [Gaze Estimation on Project Aria (test)](): Gaze Estimation; Project Aria (test); E50U50: 1.33. ## Original sources and implementation [Original paper]() [PDF]()