# Real-Time Onboard Object Detection for Augmented Reality: Enhancing Head-Mounted Display with YOLOv8

Source: [Real-Time Onboard Object Detection for Augmented Reality: Enhancing Head-Mounted Display with YOLOv8](<https://www.sota2.com/research/paper/68fa6fa31e209131cea46d87>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:24:25.039Z

Authors: Miko{\\l}aj {\\L}ysakowski, Kamil \\.Zywanowski, Adam Banaszczyk, Micha{\\l} R. Nowicki, Piotr Skrzypczy\\'nski, S{\\l}awomir K. Tadeja

Published: 2023-06-06T09:35:45.000Z

## Abstract

This paper introduces a software architecture for real-time object detection using machine learning (ML) in an augmented reality (AR) environment. Our approach uses the recent state-of-the-art YOLOv8 network that runs onboard on the Microsoft HoloLens 2 head-mounted display (HMD). The primary motivation behind this research is to enable the application of advanced ML models for enhanced perception and situational awareness with a wearable, hands-free AR platform. We show the image processing pipeline for the YOLOv8 model and the techniques used to make it real-time on the resource-limited edge computing platform of the headset. The experimental results demonstrate that our solution achieves real-time processing without needing offloading tasks to the cloud or any other external servers while retaining satisfactory accuracy regarding the usual mAP metric and measured qualitative performance

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2306.03537>)

[PDF](<https://arxiv.org/pdf/2306.03537.pdf>)