(Bench)mark: Pitfalls in AI Validation

Watch the video on youtube

In medicine, the importance of Artificial Intelligence (AI)-based automatic analysis of medical images for applications such as tumor detection, classification and progression modeling is increasing at an enormous pace. There are, however, only a few algorithms that have successfully been translated into clinical practice. In this talk, several pitfalls related to the validation of AI algorithms will be presented as well as recommendations for overcoming them. In particular, I will discuss how we can make sure to select performance metrics that reflect our biomedical needs.

Annika Reinke joined the division of Intelligent Medical Systems at the German Cancer Research Center (DKFZ) to adapt mathematical concepts to societally relevant topics, like scientific benchmarking and validation. Having published disruptive findings on biomedical image analysis challenges in Nature Communications, she is a founding member of the initiative of Biomedical Image Analysis ChallengeS (BIAS) aiming for bringing biomedical image analysis challenges to the next level of quality. She serves as the secretary of the MICCAI special interest group on biomedical challenges and as an active member and taskforce lead of the MONAI working group on evaluation, reproducibility and benchmarking.