# World Mouse: Exploring Interactions with a Cross-Reality Cursor

Source: [World Mouse: Exploring Interactions with a Cross-Reality Cursor](<https://www.sota2.com/research/paper/69b24893d030084c22e2f93f>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-13T05:00:12.498Z

Authors: Esen K. T\\"ut\\"unc\\"u, Mar Gonzalez-Franco, Khushman Patel, Eric J. Gonzalez

Published: 2026-03-11T17:14:05.000Z

## Abstract

As Extended Reality (XR) systems increasingly map and understand the physical world, interacting with these blended representations remains challenging. The current push for "natural" inputs has its trade-offs: touch is limited by human reach and fatigue, while gaze often lacks the precision for fine interaction. To bridge this gap, we introduce World Mouse, a cross-reality cursor that reinterprets the familiar 2D desktop mouse for complex 3D scenes. The system is driven by two core mechanisms: within-object interaction, which uses surface normals for precise cursor placement, and between-object navigation, which leverages interpolation to traverse empty space. Unlike previous virtual-only approaches, World Mouse leverages semantic segmentation and mesh reconstruction to treat physical objects as interactive surfaces. Through a series of prototypes, including object manipulation and screen-to-world transitions, we illustrate how cross-reality cursors may enable seamless interactions across real and virtual environments.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2603.10984>)

[PDF](<https://arxiv.org/pdf/2603.10984.pdf>)