# Multi-Modal Unsupervised Pre-Training for Surgical Operating Room Workflow Analysis

Source: [Multi-Modal Unsupervised Pre-Training for Surgical Operating Room Workflow Analysis](<https://www.sota2.com/research/paper/68fa6cfd1e209131cea37266>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:23:36.215Z

Authors: Muhammad Abdullah Jamal, Omid Mohareri

Published: 2022-07-16T10:32:27.000Z

## Abstract

Data-driven approaches to assist operating room (OR) workflow analysis depend on large curated datasets that are time consuming and expensive to collect. On the other hand, we see a recent paradigm shift from supervised learning to self-supervised and/or unsupervised learning approaches that can learn representations from unlabeled datasets. In this paper, we leverage the unlabeled data captured in robotic surgery ORs and propose a novel way to fuse the multi-modal data for a single video frame or image. Instead of producing different augmentations (or 'views') of the same image or video frame which is a common practice in self-supervised learning, we treat the multi-modal data as different views to train the model in an unsupervised manner via clustering. We compared our method with other state of the art methods and results show the superior performance of our approach on surgical video activity recognition and semantic segmentation.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2207.07894>)

[PDF](<https://arxiv.org/pdf/2207.07894.pdf>)