# Probing Intrinsic Medical Task Relationships: A Contrastive Learning Perspective

Source: [Probing Intrinsic Medical Task Relationships: A Contrastive Learning Perspective](<https://www.sota2.com/research/paper/69d5e1007d0172140ec75a0e>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-09T05:00:04.534Z

Authors: Jonas Muth, Zdravko Marinov, Simon Rei{\\ss}

Published: 2026-04-07T09:54:37.000Z

## Abstract

While much of the medical computer vision community has focused on advancing performance for specific tasks, the underlying relationships between tasks, i.e., how they relate, overlap, or differ on a representational level, remain largely unexplored. Our work explores these intrinsic relationships between medical vision tasks, specifically, we investigate 30 tasks, such as semantic tasks (e.g., segmentation and detection), image generative tasks (e.g., denoising, inpainting, or colorization), and image transformation tasks (e.g., geometric transformations). Our goal is to probe whether a data-driven representation space can capture an underlying structure of tasks across a variety of 39 datasets from wildly different medical imaging modalities, including computed tomography, magnetic resonance, electron microscopy, X-ray ultrasound and more. By revealing how tasks relate to one another, we aim to provide insights into their fundamental properties and interconnectedness. To this end, we introduce Task-Contrastive Learning (TaCo), a contrastive learning framework designed to embed tasks into a shared representation space. Through TaCo, we map these heterogeneous tasks from different modalities into a joint space and analyze their properties: identifying which tasks are distinctly represented, which blend together, and how iterative alterations to tasks are reflected in the embedding space. Our work provides a foundation for understanding the intrinsic structure of medical vision tasks, offering a deeper understanding of task similarities and their interconnected properties in embedding spaces.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Visual Task Classification on Medical vision seen](<https://www.sota2.com/research/sota/visual-task-classification-on-medical-vision-seen.md>): Visual Task Classification; Medical vision seen; F1 (seen tasks): 85.2.

- [Visual Task Classification on Medical vision unseen datasets](<https://www.sota2.com/research/sota/visual-task-classification-on-medical-vision-unseen-datasets.md>): Visual Task Classification; Medical vision unseen datasets; F1-score (seen tasks): 92.8.

- [Visual Task Classification on Medical vision seen and unseen datasets](<https://www.sota2.com/research/sota/visual-task-classification-on-medical-vision-seen-and-unseen-datasets.md>): Visual Task Classification; Medical vision seen and unseen datasets; F1 Score (Seen Tasks): 81.1.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.05651>)

[PDF](<https://arxiv.org/pdf/2604.05651.pdf>)
