A Statistical Multi-Objective Framework for Assessing Sensitivity of Radiomic AI Models to Acquisition Parameters
About
A main barrier for the deployment of AI radiomic systems in clinical routine is their drop in performance under heterogeneous multicentre acquisition protocols. This work presents a performance-oriented framework for quantifying scan parameter sensitivity of radiomic AI models, while identifying clinically significant parameter regions associated with improved cross-dataset robustness. Mixed-effects modelling in combination with a hierarchical Pareto-based strategy is used to quantify the influence of acquisition parameters and select critical values associated to low performance of predictive models. We apply our framework to lung cancer diagnosis in CT scans using two independent multicentre datasets (a public dataset and own-collected private data) and several state-of-the-art architectures. To evaluate transferability of results, CT parameters were selected using the private data and validated on the public set. With the selected configurations, validation in low performance conditions has 0.72 ([0.59,0.86], 95% CI) accuracy and 0.71 ([0.58, 0. 84], 95% CI) F1Score, while in high performance ones, we obtain 0.88 ([0.7, 1.0], 95% CI) accuracy with 0.87 ([0.69, 1.0], 95% CI) F1Score.