# Training-free Detection and 6D Pose Estimation of Unseen Surgical Instruments

Source: [Training-free Detection and 6D Pose Estimation of Unseen Surgical Instruments](<https://www.sota2.com/research/paper/69c60f020636e5bf4afc850f>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-28T05:00:04.914Z

Authors: Jonas Hein, Lilian Calvet, Matthias Seibold, Siyu Tang, Marc Pollefeys, Philipp F\\"urnstahl

Published: 2026-03-26T09:28:19.000Z

## Abstract

Purpose: Accurate detection and 6D pose estimation of surgical instruments are crucial for many computer-assisted interventions. However, supervised methods lack flexibility for new or unseen tools and require extensive annotated data. This work introduces a training-free pipeline for accurate multi-view 6D pose estimation of unseen surgical instruments, which only requires a textured CAD model as prior knowledge. Methods: Our pipeline consists of two main stages. First, for detection, we generate object mask proposals in each view and score their similarity to rendered templates using a pre-trained feature extractor. Detections are matched across views, triangulated into 3D instance candidates, and filtered using multi-view geometric consistency. Second, for pose estimation, a set of pose hypotheses is iteratively refined and scored using feature-metric scores with cross-view attention. The best hypothesis undergoes a final refinement using a novel multi-view, occlusion-aware contour registration, which minimizes reprojection errors of unoccluded contour points. Results: The proposed method was rigorously evaluated on real-world surgical data from the MVPSP dataset. The method achieves millimeter-accurate pose estimates that are on par with supervised methods under controlled conditions, while maintaining full generalization to unseen instruments. These results demonstrate the feasibility of training-free, marker-less detection and tracking in surgical scenes, and highlight the unique challenges in surgical environments. Conclusion: We present a novel and flexible pipeline that effectively combines state-of-the-art foundational models, multi-view geometry, and contour-based refinement for high-accuracy 6D pose estimation of surgical instruments without task-specific training. This approach enables robust instrument tracking and scene understanding in dynamic clinical environments.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [6D Pose Estimation on MVPSP Wetlab (test)](<https://www.sota2.com/research/sota/6d-pose-estimation-on-mvpsp-wetlab-test.md>): 6D Pose Estimation; MVPSP Wetlab (test); ADD(-S): 1.5.

- [6D Pose Estimation on MVPSP OR-X (test)](<https://www.sota2.com/research/sota/6d-pose-estimation-on-mvpsp-or-x-test.md>): 6D Pose Estimation; MVPSP OR-X (test); ADD(-S): 5.44.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2603.25228>)

[PDF](<https://arxiv.org/pdf/2603.25228.pdf>)
