Image Denoising and the Generative Accumulation of Photons

video record

Shot noise is a fundamental property of many imaging applications, especially in fluorescence microscopy. Removing this noise is an ill-defined problem since many clean solutions exist for a single noisy image. Traditional approaches aiming to map a noisy image to its clean counterpart usually find the minimum mean square error (MMSE) solution,i.e. , they predict the expected value of the posterior distribution of possible clean images. We present a fresh perspective on shot noise corrupted images and noise removal. By viewing image formation as the sequential accumulation of photons on a detector grid, we show that a network trained to predict the where the next photon could arrive is in fact solving the traditional MMSE denoising task. This new perspective allows us to make three contributions:
(i) We present a new strategy for self-supervised denoising. (ii) We present a new method for sampling from the posterior of possible solutions by iteratively sampling and adding small numbers of photons to the image. (iii) We derive a full generative model by starting this process from an empty canvas."

Alexander Krull is a lecturer at the University of Birmingham in the United Kingdom. His research focuses on denoising of microscopy images, especially in the application of generative models in this context.
After his PhD in computer vision in Carsten Rother's lab in Dresden, Alexander took a postdoc position at the Max Planck Institute of Molecular Cell Biology and Genetics, in Florian Jug's lab.
He developed readily applicable self-supervised donoising methods (including Noise2Void and DivNoising) that are used research labs around the world.
In 2020 Alexander began his position as lecturer at the University of Birmingham.
He has published papers at computer vision and machine learning conferences (ECCV, ICCV, CVPR, ICLR), as well as in various scientific journals, focusing on life-science and imaging techniques.