Talk on AI in the Biomedical Imaging Analysis Domain

Title

Validation and performance uncertainty of AI in the Biomedical Imaging Analysis Domain: Recommendations and pitfalls one should avoid

 

Speaker

Evangelia Christodoulou

 

Abstract

In the era of widespread AI models in healthcare, promises are being made regarding their efficient support in clinical workflows and subsequent enhancement of life quality. However, these claims must be substantiated through rigorous and comprehensive model validation—an undertaking that is often underestimated in its importance.
My talk will focus on two critical pillars of model validation in the Biomedical Imaging Analysis domain: appropriate metric selection and the importance of reporting model performance uncertainty.

1. Appropriate Metric Selection: Ignoring aspects related to metric choice can lead to pitfalls. To address this, we introduce the Metrics Reloaded framework. Developed by an international consortium, this comprehensive framework guides users in selecting relevant validation metrics based on problem-specific aspects. It specifically targets image analysis problems and enhances validation practices across biomedical use cases.
2. Reporting Performance Uncertainty: Our analysis of 221 MICCAI segmentation papers (published in 2023) reveals that over 50% lack performance variability assessment, with only 0.5% reporting confidence intervals (CIs) around performance estimates. Systematic reporting, including CIs, is crucial for guiding model translation into clinical practice

 

Bio

Evangelia Christodoulou, with a BSc. in Mathematics, MSc. in Biostatistics, and PhD in clinical prediction modelling from KU Leuven, Belgium, collaborated with oncologists and statisticians during her doctoral studies, enhancing predictive algorithm validation methods. Joining the German Cancer Research Center (DKFZ Heidelberg, Germany) in February 2021, she secured a postdoctoral fellowship focusing on validating AI algorithms in medical imaging analysis within the AI Health Innovation Cluster, under the guidance of Prof. Dr. Lena Maier-Hein. Her current research covers aspects that include the development of reliable and robust AI-based models for clinical outcome prediction in the domain of Surgical Data Science alongside with methodological recommendations touching on critical bottlenecks of AI methods in the Biomedical Imaging Analysis domain, namely validation and uncertainty reporting of model performance.