# Robustness of transferability estimation metrics for medical imaging

Source: [Robustness of transferability estimation metrics for medical imaging](<https://www.sota2.com/research/paper/6a7bfe1c4b236a9bba1a18d5>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-08-13T05:00:05.216Z

Authors: Niclas Cla{\\ss}en, Th\\'eo Sourget, Dovile Juodelyte, Rob van der Goot, Veronika Cheplygina

Published: 2026-08-07T13:56:18.000Z

## Abstract

In transfer learning, the choice of source model largely influences the performance on a target dataset. Still, selecting a fitting source remains a challenging task, especially in medical imaging where one has to decide between models pre-trained on off-the-shelf options, such as ImageNet, and domain specific datasets. Transferability estimation (TE) metrics address this problem by aiming to predict the best performing source model in a computationally cost effective way. However, previous work has reported conflicting TE metric performances due to differences in experimental setups. Moreover, most TE metrics are designed for and evaluated on natural images, while being optimized for accuracy, whereas in medical imaging metrics that are more robust to class imbalance are typically used. We study the impact of varying the target dataset as an isolated factor, by constructing miniature populations of different sample sizes and random seeds. In addition, we investigate the influence of the evaluation metric used to obtain the reference ranking. We find that small modifications to the target dataset change the rankings. Furthermore, we show that the choice of evaluation metric affects the reference rankings and therefore the evaluation of TE metrics. Overall, we observe a low agreement between rankings from TE metrics and reference. The code, model checkpoints and data splits used in this work are available through https://github.com/niclasclassen/robustness-of-transferability-estimation-metrics-for-medical-imaging.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2608.09999>)

[PDF](<https://arxiv.org/pdf/2608.09999.pdf>)
