[Back]

P. Zelger, K. Kaser, B. Rossboth, L. Velas, G. Schütz, A. Jesacher:
"Three-dimensional localization microscopy using deep learning";
Optics Express, 26 (2018), 25; 33166 - 33179.

Single molecule localization microscopy (SMLM) is one of the fastest evolving
and most broadly used super-resolving imaging techniques in the biosciences. While image
recordings could take up to hours only ten years ago, scientists are now reaching for real-time
imaging in order to follow the dynamics of biology. To this end, it is crucial to have data
processing strategies available that are capable of handling the vast amounts of data produced by
the microscope. In this article, we report on the use of a deep convolutional neural network (CNN)
for localizing particles in three dimensions on the basis of single images. In test experiments
conducted on fluorescent microbeads, we show that the precision obtained with a CNN can
be comparable to that of maximum likelihood estimation (MLE), which is the accepted gold
standard. Regarding speed, the CNN performs with about 22k localizations per second more
than three orders of magnitude faster than the MLE algorithm of ThunderSTORM. If only five
parameters are estimated (3D position, signal and background), our CNN implementation is
currently slower than the fastest, recently published GPU-based MLE algorithm. However, in this
comparison the CNN catches up with every additional parameter, with only a few percent extra
time required per additional dimension. Thus it may become feasible to estimate further variables
such as molecule orientation, aberration functions or color. We experimentally demonstrate that
jointly estimating Zernike mode magnitudes for aberration modeling can significantly improve
the accuracy of the estimates.