Scmos noise-correction algorithm for microscopy images

Access through your institution Buy or subscribe To the Editor: Scientific complementary metal-oxide semiconductor (sCMOS) cameras are rapidly gaining popularity in the biological sciences.

The sCMOS sensor provides significant advances in imaging speed, sensitivity and field of view over traditional detectors such as charge-coupled devices (CCD) or electron-multiplying CCDs

(EMCCD)1,2. However, this sensor introduces pixel-dependent noise; each pixel has its own noise statistics—primarily offset, gain and variance. Left uncorrected, this sCMOS-specific noise

generates imaging artifacts and biases in quantification3. A suite of algorithms was developed to characterize this noise in each pixel and incorporate the noise statistics in the likelihood

function for single-molecule localization3. However, these algorithms work exclusively on images with point objects such as in single-particle tracking or single-molecule-switching

nanoscopy. No general algorithm that works on conventional microscopy images exists. We developed such an algorithm that dramatically reduces sCMOS noise from microscopy images with

arbitrary structures. We show that our new method corrects pixel-dependent noise in fluorescence microscopy using an sCMOS sensor, and this allows the sensor's performance to approach

that of an ideal camera. This is a preview of subscription content, access via your institution RELEVANT ARTICLES Open Access articles citing this article. * ZERO-SHOT LEARNING ENABLES

INSTANT DENOISING AND SUPER-RESOLUTION IN OPTICAL FLUORESCENCE MICROSCOPY * Chang Qiao * , Yunmin Zeng *  … Qionghai Dai _Nature Communications_ Open Access 16 May 2024 * EXTENDING

RESOLUTION WITHIN A SINGLE IMAGING FRAME * Esley Torres-García * , Raúl Pinto-Cámara *  … Adán Guerrero _Nature Communications_ Open Access 02 December 2022 * PHOTON-FREE (S)CMOS CAMERA

CHARACTERIZATION FOR ARTIFACT REDUCTION IN HIGH- AND SUPER-RESOLUTION MICROSCOPY * Robin Diekmann * , Joran Deschamps *  … Jonas Ries _Nature Communications_ Open Access 11 June 2022 ACCESS

OPTIONS Access through your institution Access Nature and 54 other Nature Portfolio journals Get Nature+, our best-value online-access subscription $32.99 / 30 days cancel any time Learn

more Subscribe to this journal Receive 12 print issues and online access $259.00 per year only $21.58 per issue Learn more Buy this article * Purchase on SpringerLink * Instant access to

full article PDF Buy now Prices may be subject to local taxes which are calculated during checkout ADDITIONAL ACCESS OPTIONS: * Log in * Learn about institutional subscriptions * Read our

FAQs * Contact customer support REFERENCES * von Diezmann, A., Shechtman, Y. & Moerner, W.E. _Chem. Rev._ 117, 7244–7275 (2017). Article  CAS  PubMed  PubMed Central  Google Scholar  *

Huang, Z.-L. et al. _Opt. Express_ 19, 19156–19168 (2011). Article  PubMed  Google Scholar  * Huang, F. et al. _Nat. Methods_ 10, 653–658 (2013). Article  CAS  PubMed  PubMed Central  Google

Scholar  * Goodman, J.W. _Introduction to Fourier Optics_ (Roberts & Company, 2005). Google Scholar  * Liu, S., Kromann, E.B., Krueger, W.D., Bewersdorf, J. & Lidke, K.A. _Opt.

Express_ 21, 29462–29487 (2013). Article  PubMed  PubMed Central  Google Scholar  Download references ACKNOWLEDGEMENTS We thank C. Pellizzari for discussion on algorithm development; D.A.

Miller, K.F. Ziegler and P.M. Ivey for helping with the project and for their suggestions on the manuscript. D.M.S. was supported by an NSF grant (1146944-IOS). S.L., M.J.M. and F.H. were

supported by grants from the NIH (R35 GM119785) and DARPA (D16AP00093). AUTHOR INFORMATION AUTHORS AND AFFILIATIONS * Weldon School of Biomedical Engineering, Purdue University, West

Lafayette, Indiana, USA Sheng Liu, Michael J Mlodzianoski & Fang Huang * School of Electrical and Computer Engineering, Purdue University, West Lafayette, Indiana, USA Zhenhua Hu *

Department of Biological Sciences, Purdue University, West Lafayette, Indiana, USA Yuan Ren, Kristi McElmurry & Daniel M Suter * Purdue Institute for Integrative Neuroscience, Purdue

University, West Lafayette, Indiana, USA Daniel M Suter & Fang Huang * Bindley Bioscience Center, Purdue University, West Lafayette, Indiana, USA Daniel M Suter * Birck Nanotechnology

Center, Purdue University, West Lafayette, Indiana, USA Daniel M Suter * Purdue Institute of Inflammation, Immunology and Infectious Disease, Purdue University, West Lafayette, Indiana, USA

Fang Huang Authors * Sheng Liu View author publications You can also search for this author inPubMed Google Scholar * Michael J Mlodzianoski View author publications You can also search for

this author inPubMed Google Scholar * Zhenhua Hu View author publications You can also search for this author inPubMed Google Scholar * Yuan Ren View author publications You can also search

for this author inPubMed Google Scholar * Kristi McElmurry View author publications You can also search for this author inPubMed Google Scholar * Daniel M Suter View author publications You

can also search for this author inPubMed Google Scholar * Fang Huang View author publications You can also search for this author inPubMed Google Scholar CORRESPONDING AUTHOR Correspondence

to Fang Huang. ETHICS DECLARATIONS COMPETING INTERESTS S.L. and F.H. are co-inventors on a patent application related in part to the material presented here. INTEGRATED SUPPLEMENTARY

INFORMATION SUPPLEMENTARY FIGURE 1 DIAGRAM OF THE NOISE CORRECTION ALGORITHM Starting with a raw sCMOS frame, sequential steps include, pre-correction using offset and gain pixel maps,

calculating negative log-likelihood using variance and gain pixel maps and noise contribution in Fourier space and iterative update to minimize the pixel-wise sum of the two quantities (See

Supplementary Notes 2–7) SUPPLEMENTARY FIGURE 2 TEMPORAL FLUCTUATION COMPARISON OF FLUORESCENCE MICROSCOPY IMAGES. Peroxisome membrane proteins in COS-7 cells tagged with tdEos were imaged

on a conventional wide-field fluorescence microscope. (A) Temporal standard deviation (STD) map over 400 sCMOS frames (pre-corrected by gain and offset). The colormap scale is from min (STD

of 2.3, black) to max (STD of 12.3) in units of effective photon count. (B) Temporal STD map over 400 NCS (noise correction for sCMOS camera) frames. The terms sCMOS and NCS frame will be

used throughout the supplemental figures. (C) Zoom in regions i and ii from A and B show pixels with high variance are effectively removed after NCS. (D) The pixel intensity traces of

selected pixels from cropped regions i and ii over 50 frames. For the pixels with high readout noise (pixel 1 and 3), the value fluctuation decreases significantly after NCS, while for the

pixels with low readout noise (pixel 2 and 4), the pixel value fluctuation remains the same. And the mean pixel values stay the same in both high and low readout noise cases before and after

noise correction. SUPPLEMENTARY FIGURE 3 PIXEL FLUCTUATION COMPARISON BEFORE AND AFTER NOISE CORRECTION AT LOW PHOTON LEVELS. End-binding protein 3 in COS-7 cells tagged with tdEos were

imaged on a conventional wide-field fluorescence microscope. (A) A single sCMOS frame pre-corrected for gain and offset for comparison purpose with an exposure time of 10 ms and at time

point t = 0 s. (B) Time series of selected regions in A from sCMOS frames and the corresponding NCS frames showing the significant reduction of sCMOS noise while maintaining the underlying

signal. SUPPLEMENTARY FIGURE 4 RESOLUTION COMPARISON USING BOTH EXPERIMENTAL DATA AND SIMULATED DATA. (A) 100 nm yellow-green fluorescent bead images from sCMOS camera and NCS corrected

images. To cancel readout noise in sCMOS frames for a fair comparison between the sCMOS frames and NCS frames, images were averaged over 200 frames for both cases. The intensity profiles

were generated by averaging over the vertical dimension of each bead image and fitted with a Gaussian function to extract their widths, ΣsCMOS and ΣNCS. (B) Simulated bead images based on

the parameters in Supplementary Note 14. The simulated bead images were averaged over 20 frames from sCMOS and NCS frames. From both experimental data and simulated data, the ΣNCS is

slightly larger than ΣsCMOS, resulting in 5.5 nm and 4 nm decrease in resolution, a small decrease is potentially negligible compared with the diffraction limit of approximately 250 nm.

SUPPLEMENTARY FIGURE 5 COMPARISON OF NCS RESULT USING OTF WEIGHTED AND NOISE ONLY MASKS. Peroxisome membrane proteins in COS-7 cells tagged with tdEos were imaged on a conventional

wide-field fluorescence microscope. (A) Temporal standard deviation map over 400 sCMOS frames, NCS frames with OTF weighted mask and noise only mask respectively. The colormap scale is from

min (STD of 2.3, dark red) to max (STD of 12.3, white) in units of effective photon count. (B) Average of a sequence of sCMOS and NCS frames as in A and their corresponding amplitudes in

Fourier space. Average images were used to cancel pixel dependent noise for fair comparison of sCMOS and NCS frames. (C) Radial average of the amplitude in Fourier space from the sCMOS and

NCS frames. SUPPLEMENTARY FIGURE 6 COMPARISON OF NCS ALGORITHM WITH LOW PASS FILTER. In order to illustrate the fundamental differences between low pass filters and the NCS algorithm, the

simulated bead data uses a simulated high variance map (3000∼6000 ADU2). (A) sCMOS frame. (B) NCS frame. (C) The sCMOS frame blurred by a 2D Gaussian kernel with a sigma equal to 1 pixel.

(D) The sCMOS frame after a low pass filter with a cutoff frequency equal to the OTF radius. The cutout region in each image is the 2× zoom of the region above the white box. It shows that

both the Gaussian blur and the OTF filter cannot effectively remove the sCMOS noise (yellow boxes and yellow arrows), while the NCS algorithm can significantly reduce the sCMOS noise

fluctuations. Furthermore, the Gaussian blur method also reduces the resolution of the original image (red circles) (Supplementary Note 12). SUPPLEMENTARY INFORMATION SUPPLEMENTARY TEXT AND

FIGURES Supplementary Figures 1–6, Supplementary Methods and Supplementary Notes 1–15 (PDF 8395 kb) SUPPLEMENTARY SOFTWARE NCS software package (ZIP 15641 kb) SUPPLEMENTARY VIDEO 1 Time

series and pixel fluctuation of sCMOS and NCS frames - EB3 in COS-7 cells End-binding protein 3 in COS-7 cells tagged with tdEos were imaged on a conventional wide-field fluorescence

microscope. The top panel shows the time series of the sCMOS and NCS frames of the experimental data. The bottom traces show the pixel value fluctuations of the circled regions in sCMOS and

NCS frames. (AVI 78450 kb) SUPPLEMENTARY VIDEO 2 Time series of raw sCMOS and NCS frames – SiR-actin in Aplysia bag neuron cell F-actin in peripheral domain and transition zone of Aplysia

bag cell neuronal growth cones tagged with SiR-actin were imaged on a conventional wide-field fluorescence microscope. The movie shows the time series of the raw sCMOS (without offset and

gain correction) and NCS frames of the experimental data. (AVI 65351 kb) LIFE SCIENCES REPORTING SUMMARY Reporting Summary (PDF 67 kb) RIGHTS AND PERMISSIONS Reprints and permissions ABOUT

THIS ARTICLE CITE THIS ARTICLE Liu, S., Mlodzianoski, M., Hu, Z. _et al._ sCMOS noise-correction algorithm for microscopy images. _Nat Methods_ 14, 760–761 (2017).

https://doi.org/10.1038/nmeth.4379 Download citation * Published: 01 August 2017 * Issue Date: 01 August 2017 * DOI: https://doi.org/10.1038/nmeth.4379 SHARE THIS ARTICLE Anyone you share

the following link with will be able to read this content: Get shareable link Sorry, a shareable link is not currently available for this article. Copy to clipboard Provided by the Springer

Nature SharedIt content-sharing initiative