# Deep learning approaches to surgical video segmentation and object detection: A Scoping Review

Source: [Deep learning approaches to surgical video segmentation and object detection: A Scoping Review](<https://www.sota2.com/research/paper/68fa77241e209131cea7159c>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:10:55.533Z

Authors: Devanish N. Kamtam, Joseph B. Shrager, Satya Deepya Malla, Nicole Lin, Juan J. Cardona, Jake J. Kim, Clarence Hu

Published: 2025-02-23T06:31:23.000Z

## Abstract

Introduction: Computer vision (CV) has had a transformative impact in biomedical fields such as radiology, dermatology, and pathology. Its real-world adoption in surgical applications, however, remains limited. We review the current state-of-the-art performance of deep learning (DL)-based CV models for segmentation and object detection of anatomical structures in videos obtained during surgical procedures.   Methods: We conducted a scoping review of studies on semantic segmentation and object detection of anatomical structures published between 2014 and 2024 from 3 major databases - PubMed, Embase, and IEEE Xplore. The primary objective was to evaluate the state-of-the-art performance of semantic segmentation in surgical videos. Secondary objectives included examining DL models, progress toward clinical applications, and the specific challenges with segmentation of organs/tissues in surgical videos.   Results: We identified 58 relevant published studies. These focused predominantly on procedures from general surgery \[20(34.4%)\], colorectal surgery \[9(15.5%)\], and neurosurgery \[8(13.8%)\]. Cholecystectomy \[14(24.1%)\] and low anterior rectal resection \[5(8.6%)\] were the most common procedures addressed. Semantic segmentation \[47(81%)\] was the primary CV task. U-Net \[14(24.1%)\] and DeepLab \[13(22.4%)\] were the most widely used models. Larger organs such as the liver (Dice score: 0.88) had higher accuracy compared to smaller structures such as nerves (Dice score: 0.49). Models demonstrated real-time inference potential ranging from 5-298 frames-per-second (fps).   Conclusion: This review highlights the significant progress made in DL-based semantic segmentation for surgical videos with real-time applicability, particularly for larger organs. Addressing challenges with smaller structures, data availability, and generalizability remains crucial for future advancements.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2502.16459>)

[PDF](<https://arxiv.org/pdf/2502.16459.pdf>)