Super-resolution localization microscopy: Toward high throughput, high quality, and low cost

In the past few decades, fluorescence microscopy has significantly expanded our ability to study biological processes at the cellular and subcellular level, on the strength of its molecular specificity and multiplex imaging capability. However, due to the diffraction-limited spatial resolution of optical microscopy, conventional fluorescence microscopy cannot visualize biological structures smaller than the diffraction-limited resolution (∼200 nm). In the last decade, the advances in super-resolution fluorescence microscopy have revolutionized biological imaging by overcoming the fundamental diffraction barrier, recognized by a Nobel Prize in 2014. Distinct from other super-resolution imaging techniques such as stimulated emission depletion (STED) or structured illumination microscopy (SIM), single-molecule localization microscopy (SMLM) {i.e., (fluorescence) photo-activated localization microscopy [(f)PALM],^1,2 (direct) stochastic optical reconstruction microscopy [(d)STORM],^3,4 and point accumulation for imaging in nanoscale topography (PAINT⁵ or DNA-PAINT⁶)} does not require complex and expensive optics for patterned illumination and, thus, is the most cost-effective approach to achieve super-resolved imaging capability. Meanwhile, in combination with high photon counts (tens of thousands of photons per emitter through bright emitters and long exposure time), a spatial resolution down to ∼5 nm⁷ can be obtained with standard illumination power density, much better than STED (∼50 nm) and SIM (∼100 nm). With these advantages, SMLM has quickly become an essential tool to understand the biological systems at the molecular scale.

The fundamental principle of SMLM is based on single-molecule localization, in which a small subset of labeled fluorophores is sequentially turned “on,” then the centers of individual emitters are determined by localization algorithms at a nanometer precision, and the final reconstructed image is obtained after accumulating localized positions from tens of thousands of image frames. Therefore, SMLM is largely a computational imaging technique, built upon a simple configuration of a wide-field fluorescence microscope. The resolution or quality of the reconstructed super-resolution image is dependent on the performance of localization algorithms and the photophysical switching properties of the fluorophores.

The initial form of SMLM systems was not only expensive but limited to mostly ultrathin and transparent model systems (such as bacteria and thin cultured cells). Furthermore, it was limited by a small field of view (FOV) (∼50 × 50 µm²) and slow data acquisition and image reconstruction speed. In the past decade, many technical advancements have been made to address the limitations of the SMLM system, and its applications in biological research are rapidly expanding. We will provide our perspective on the latest advances in the technical development of SMLM systems for super-resolution microscopy and their implications in the field of biomedical research.

The need for higher throughput is driven by the increasing use of SMLM in understanding complex and heterogeneous biological systems. High-throughput analysis on thousands to tens of thousands of cells is essential to gain an unbiased view of biological processes. The throughput of the SMLM system has been improved significantly in the past few years through hardware, image processing, and automation discussed below.

To achieve a large FOV and fast frame rate in SMLM, the camera is a major factor. The traditional gold-standard camera for single-molecule imaging and early-stage SMLM systems is Electron Multiplying Charged Coupled Device (EMCCD), which usually can provide a FOV of ∼50 × 50 µm² and a frame rate of 30 fps. However, this situation has been changed by modern scientific CMOS (sCMOS) camera. The latest sCMOS camera can now provide a FOV of ∼300 × 300 µm² and a frame rate up to 400 fps (e.g., a Photometrics Kinetix sensor with 3200 × 3200 pixels) and, importantly, a better performance in the signal-to-noise ratio (SNR). Therefore, sCMOS cameras are ideally suited for high-throughput SMLM.

However, the imaging throughput of SMLM is not solely dependent on the camera. The photophysical property (blinking rate) of fluorophores is another important factor. For (f)PALM and (d)STORM imaging, higher power density can increase the blinking rate of the emitters or decrease the time a fluorophore spends at the “on” state. For DNA-PAINT or transient binding based imaging, this rate can be adjusted by changing the oligo design or imager concentration. Furthermore, considering that most laser beams assume a Gaussian profile, the power density decreases quickly at the boundary of the illumination field, which means the image quality can vary across the field of view. To ensure a consistent high-quality super-resolution image across the entire FOV, a uniform illumination field is also required.

Several approaches have been implemented to achieve flat-field large FOV illumination. As one of the earliest efforts, Douglass et al. used the microlens array to achieve flat-field epi-illumination⁸ in the SMLM system with the extended FOV of ∼100 × 100 µm². Other approaches used different types of beam shaping devices to transform the Gaussian beam into the flat-top beam such as aspheric optics (πShaper,⁹ TopShape¹⁰), multi-mode optical fibers,¹¹ and powell lens. When combined with a fast-rotating flat-top diffuser, the flat-field illumination can also be readily obtained. Furthermore, chip-based SMLM used the total internal reflection configuration for fluorescence (TIRF) excitation with the high-index waveguide, which showed an ultra-large FOV of ∼500 × 500 μm².^12,13 However, the ultra-large FOV often requires low-magnification objectives with lower NA compared to the traditional SMLM objective with an NA of 1.4 or more, thus reducing the photon collection efficiency and reducing the spatial resolution. The waveguide-based SMLM is also limited to the TIRF configuration with a restricted imaging depth of less than 200 nm. For non-TIRF based setup, as the FOV is expanding, the required laser power to achieve sufficient power density for fast photo-switching of emitters is also higher. For dSTORM imaging, a laser power of ∼2 W to 4 W is needed for SMLM at an extended FOV of ∼200 × 200 µm², and a further increase in FOV and the subsequent laser power risks damaging the objective lens and other optics. More importantly, increased illumination power density may also damage the biological sample, which can distort the actual molecular structure.

To precisely localize the positions of individual emitters, fitting a Gaussian function or an experimentally measured spline function^14,15 are the commonly used methods. However, these iterative algorithms are computationally intensive. Given the intensive computation needed for single-molecule localization, a considerable early effort was devoted to improving the computational efficiency of localization algorithms¹⁶ without compromising the accuracy for high-speed SMLM image reconstruction. One general approach is through mathematically simple non-iterative localization algorithms such as radial symmetry,^17,18 gradient fitting,¹⁹ and phasor.²⁰ The second general approach is based on the parallel computing architecture to speed up the localization process by graphics processing units (GPU).^21,22 These developments largely addressed the limited computational speed for image reconstruction in the sparse emitter scenario. SMLM image reconstruction can now be routinely performed for over a million localizations per second on a personal computer equipped with a consumer-level graphics card.

High-density emitter localization is an effective strategy to improve the throughput by increasing the emitter density at each frame, resulting in a reduced number of frames required to reconstruct the image. However, more complex numerical algorithms are required to identify the overlapping emitters, such as multiple-emitter fitting,²³ compress sensing,²⁴ deconvolution, and Bayesian analysis.²⁵ As these algorithms are significantly more complex than those for sparse emitter localization, even a small image size of 100 × 100 pixels could take hours to reconstruct on a personal desktop computer. Although a dense emitter localization approach reduces the time for data acquisition, it dramatically increases the time for image reconstruction. To address this challenge, several hardware approaches have been implemented, such as the use of cloud computing or GPU for parallel processing.²⁶ Several algorithmic improvements have also been made. For example, Deep-STORM used deep convolutional neural networks, which achieved localizing 20 000 emitters per second in GPU.²⁷ However, the above-mentioned methods are still unable to realize real-time image reconstruction. One promising method is WindSTORM, a mathematically simple non-iterative approach. It achieved real-time image reconstruction (approaching 1 × 10⁶ emitters per second) at a high level of accuracy and robustness for dense emitter scenarios.²⁸ However, a smaller pixel size (≤100 nm) is preferable to achieve its best performance. Of note, regardless of high-density localization algorithm, the image resolution rapidly degrades with the increase in the emitter density. Therefore, the high-density localization strategy is recommended only when the fast imaging speed is crucial.

Simultaneous imaging of multiple targets in the spectral domain is another strategy to improve imaging throughput. Early development of spectrally resolved STORM used a prism²⁹ or spectrometer³⁰ to spatially disperse multiple wavelengths for simultaneous localization of multi-color emitters. However, this kind of approach inevitably reduced the collection efficiency of photon numbers from each emitter, thus resulting in a lower spatial resolution of reconstructed images. A simpler approach was reported to simultaneously identify two to three colors based on their ratiometric measurement of the transmitted and reflected fluorescence.³¹ For a larger number of targets at sub-20 nm resolution, SMLM based on Exchange-PAINT³² was developed for sequential super-resolution imaging to simultaneously visualize more than ten targets on the same cell,³³ but it is a slower process that can take several days to complete. Nevertheless, multiplexed imaging coupled with the super-resolved resolution is a powerful approach to visualize the spatial relationship of multiple targets at molecular-scale resolution and improve our understanding of the complex microenvironment of biological systems.

Besides the above-mentioned strategies to improve the speed of image acquisition and reconstruction, the development of a fully automated pipeline for image acquisition and analysis is another important strategy to improve the overall imaging content. A fully automated workflow often includes the real-time control of laser excitation and activation, focus control, drift correction, automatic identification of imaging target, area scanning, buffer exchange, and image reconstruction.^9,33–37 The automated workflow enables the maintenance of consistent imaging conditions throughout the entire imaging process, which is crucial for generating reproducible image data. Analysis of tens of thousands of cells through rigorous statistical testing is essential for the optimal selection of proper drug combinations and understanding the complex and heterogeneous biological processes.

Given the above-discussed key elements for a high-throughput SMLM system, we used the following experimental setup and workflow for data processing for an automated high-throughput multi-color SMLM imaging system, as shown in Fig. 1. A high-power excitation laser (2 W) at 640 nm is used as the light source, and it generates a flat-field illumination that is used as the laser source generated by a fast vibrating multi-mode fiber. A high-NA oil-immersion objective lens (60×, NA = 1.42) is used. The sample is mounted on three-axis high-precision piezo motor linear stages (Agilis^TM Piezo Motor series, Newport), which offer a decent travel distance of 12 mm at a precision of 50 nm for both lateral and axial movement. We selected three fluorophores, Alexa Fluor 647 (Thermo Fisher) and CF660 and CF680 (Biotium) for simultaneous dSTORM imaging. The fluorescence signals from three colors are simultaneously collected, and a dichroic mirror (T685lpxr-UF3, Chroma) is used to split the transmitted and reflected fluorescence signals into two channels, collected by using sCMOS cameras (Hamamatsu ORCA-Flash4.0), respectively. The pixel size corresponding to the sample plane is ∼100 nm, thus providing a FOV of ∼200 × 200 µm². The ratio of the fluorescence signals between these two channels can be used to distinguish the three colors in the fluorescent emitters and reconstruct three-color STORM images simultaneously (e.g., Alexa Fluor 647, CF660, and CF680).

For high-throughput SMLM, we recommend the drift correction as an independent device that is de-coupled from the SMLM image acquisition process for best throughput. The drift correction can be implemented with two different approaches, depending on the application. First, if the aqueous imaging buffer is used, the strong reflection between the coverslip (n = 1.515) and the mounting medium (n ∼ 1.34) of the sample can be used for drift correction. A low-power 850 nm diode laser can be used as the light source for drift correction, which is distinct from the excitation laser with minimal interference with the photophysical properties of fluorophores. When STORM imaging of a tissue section is needed, we recommend the addition of 2,2-thiodiethanol (TDE) into the imaging buffer as the optical clearing agent, which also reduces the background with a closely matched refractive index as that of an immersion oil.³⁸ However, the index-matching condition significantly reduces the reflection between the coverslip and the mounting medium of the sample, and the first approach for drift correction does not work. We propose a second approach of drift correction using fiducial markers of gold nanoparticles. The 850 nm LED is used as the light source, and the transmitted light from the bright-field image of the sample is collected on the drift correction camera. The axial position of the fiducial markers can be derived based on the phase value from the Fourier transform of the diffraction pattern of the bright-field images of gold nanoparticles, a holographic microscopy method for single-particle tracking.³⁹ In both cases, the axial positions are tracked and corrected in real-time (>1 Hz).

When two sCMOS cameras are simultaneously used, large image data are quickly generated at ∼1.6 GB/s, faster than the writing speed of hard drive disks. We used the RAID storage system (e.g., LaCie 12big Thunderbolt 3) for high-speed writing and storage of large image data, with a desktop computer equipped with a GPU device (Nvidia GTX2080) for high-speed image reconstruction.

The quality of super-resolution images is largely dependent on the ability of localization algorithms to precisely localize the central positions of individual emitters without artifacts. Most localization algorithms assume a uniform background, so the best accuracy is achieved when the raw image contains bright molecules on a uniform and low background. However, the non-uniform background is common in most biological samples under a wide-field fluorescence microscope. The background in the raw images of SMLM may vary in different regions of the sample or for different samples. One of the main contributors to the background fluorescence is those unbleached fluorophores continuously emitting fluorescence, resulting in a high and often heterogeneous background. In addition, autofluorescence and scattered light from thick samples or densely labeled targets can also contribute to the heterogeneous background. The background fluorescence can introduce significant localization inaccuracies up to tens of nanometers and result in image artifacts (i.e., misrepresentation of samples’ structure), localization bias (i.e., shift the true positions of localized emitters), or reduced resolution that greatly affects the quality of the final reconstructed image.⁴⁰ Therefore, the image quality of the reconstructed super-resolution image can vary between sample to sample due to the variation in the heterogeneous background, lacking robustness.

Physical suppression of the background fluorescence is the most effective strategy to improve the signal-to-background ratio in SMLM. In the early form of SMLM, the ultrathin and transparent model systems (such as bacteria and thin cultured cells) were often imaged on a total internal reflection fluorescence (TIRF) microscope. The evanescent waves coming from only ∼100 nm to 200 nm of thickness from the sample surface in contact with the coverslip effectively suppress the background. Other forms of optic-based background suppression were also developed, including confocal-based,³² temporal focusing,⁴¹ interference-based,^42,43 or light-sheet illumination.^44–48 These strategies reduced the depth of optical sectioning, thus effectively suppressing background fluorescence. However, these approaches often significantly increased the complexity and the cost of the instrument, making them difficult to serve as routine tools for large-scale biological studies. Another simple strategy is to use a simple optical clearing agent with a better-matched refractive index with the immersion oil, such as 2,2-thiodiethanol (TDE). The clearing and index-matching strategy did not change any hardware but reduced the spherical aberration and the depth of focus, which showed a significant reduction in the background fluorescence, especially in the formalin-fixed paraffin-embedded tissue section.³⁸ 

Various strategies based on the algorithm-based background correction have also been developed. Earlier efforts to remove the impact of background relied on conventional image processing methods as a pre-processing step to generate a noise-reduced image for identifying candidate emitters to localize. Spatial filtering is commonly used, such as a rolling ball filter,⁴⁹ Gaussian or averaging filter,⁵⁰ and wavelet filter.⁵¹ However, spatial filters often lack robustness to the non-uniform background and emitter size, leading to significant image artifacts. On the other hand, given that individual emitters used for single-molecule localization undergo multiple rapid blinking cycles whereas the background fluorescence undergoes slower variation over a longer time, temporal filtering is inherently superior to the commonly used spatial filter to separate the fast-changing emitters from the slowly varying background. For example, a temporal median filter was shown to outperform other spatial filtering methods in improving the quality of the reconstructed super-resolution image.⁵² However, the performance of the temporal median filter depends on the characteristics of the sample and imaging targets. It works well for ultra-sparse emitter scenarios where the perturbation from emitters to the background is minimal.⁵² Although, as the emitter density goes up, the temporal median value significantly over-estimates the background, which, in turn, suppresses the emitter intensity and size, resulting in a reduced emitter recall rate and lower localization accuracy.⁵³ 

To address the limitations of the existing methods, our group developed an accurate method to correct the heterogeneous background using an extreme value-based estimation model, referred to as extreme value-based emitter recovery (EVER),^28,53 as shown in Fig. 2. This approach took advantage of the extreme value (minimum) of a time series (a series of image stacks) that remains stable regardless of the probability of emitter occurrence and, thus, is inherently robust. To link the temporal minimum value to the actual background signals, we established a simple algebraic relationship based on rigorous statistical models of Poisson distribution. We showed that our method (EVER) accurately separates emitters from the noisy background, in which the recovered emitters showed the best match with the ground-truth image. In comparison, the recovered emitters by the temporal median filter (MED) over-estimated the background and resulted in an apparent reduction in the intensity and size; while the recovered emitters by the spatial filter of a rolling ball (RB) showed erroneous structures due to the artifacts introduced by the heterogeneous background. Furthermore, the quantitative comparison of the image similarity of the recovered emitters showed the best result of ∼98% similarity¹² compared to the ground truth, while other methods (MED and RB) only showed <60% similarity with the ground truth. Given that this approach is based on a simple analytical relationship, it does not cost many computational resources. An ImageJ plugin (https://pitt.box.com/v/EVER-BackgroundCorrection) is also available for widespread use in the scientific community.

Another recently reported approach used a deep neural network (BGnet) to estimate the arbitrary background in the image of SMLM.⁵⁴ This approach was built upon the widely used U-net architecture and used the simulated point spread function (PSF) with complex shapes (open aperture, double-helix, and tetrapod) and background images to train the neural network and then predict the background. The authors showed that the BGnet-based background correction significantly improved the localization precision for both simulated and experimental data for PSFs with various shapes. This framework has great potential to reduce image artifacts and improve the robustness of SMLM.

Quantitative evaluation of image quality is important to ensure the robustness and reproducibility of super-resolution images and identify image artifacts. Fourier Ring Correlation⁵⁵ and decorrelation⁵⁶ are commonly used methods to quantify the resolution of the reconstructed image. However, note that the Fourier spectrum-based method can be affected by the image structure and image artifact and cannot fully and accurately characterize the image quality. A heuristic approach to evaluate image artifacts in the reconstructed SMLM images is to compare the image features between diffraction-limited and super-resolved images. Quantitative evaluation using image similarity-based methods such as NanoJ-SQUIRREL can be used to confirm the presence of the artifacts.⁵⁷ However, this method cannot identify image artifacts below the diffraction-limited resolution. Some method was developed for a specific application such as identifying membrane-protein nanoclusters at the super-resolved scale.⁵⁸ By varying the labeling density, the clustered molecules can be distinguished from the random and diffusely distributed clusters due to image artifacts. We expect that more application-specific methods will be developed for quality control of artifact-free super-resolution images.

SMLM was initially developed and commercialized as a high-end microscopy instrument, available in the core imaging centers at major academic institutions and a small number of laboratories. Its high price tag of more than $200K limits its widespread use as a routine microscopy system as a conventional fluorescence microscope. However, in principle, the SMLM system is a simple wide-field fluorescence microscope. Only a few years after the successful launch of SMLM systems by major microscope manufacturers, researchers began questioning the need for all high-end components (i.e., lasers and camera) and evaluated whether SMLM can be made more cost-effectively.^59–61 We summarized the balance between the cost of each key component in traditional SMLM and their impact on the overall image quality in Fig. 3.

A few technical characteristics make SMLM the most cost-effective candidate among all super-resolution microscopy systems (SIM and STED). First, the laser is often one of the most expensive components. Its high cost often comes from the difficulty to achieve a highly coherent high-power single-mode Gaussian beam, which can be crucial to generate a donut shape used in STED or the Moiré fringes used in SIM. However, the SMLM system uses wide-field illumination, and the imaging quality of SMLM is not sensitive to the slight degradation on the stability of the laser output and spectral line width. The most important factor for SMLM illumination is a uniform spatial distribution of illumination intensity at a sufficiently high power density. It turns out the industry-grade high-power laser diodes that cost just a few hundred dollars often satisfy the requirement for SMLM. To create uniform illumination, low-cost solutions include the use of multi-mode fiber and a fiber-based beam shaper and rotating diffuser.^60,61 The high-end scientific-grade laser is often not necessary for SMLM. Second, the camera is responsible for image recording. The quantum efficiency (QE) and read noise are the critical parameters that determine the reconstructed image resolution. The early version of SMLM systems almost exclusively used EMCCD, which costs about $35K. However, the advances of CMOS camera technology significantly improved the sensitivity and lowered the noise, with the advantage of much larger image sensor size and lower cost. The scientific CMOS (sCMOS) cameras at half of the cost of EMCCD⁶⁰ are now being widely adopted in SMLM systems. A modern industry-grade camera (∼$400) can provide a QE of 72%, which is ∼20% worse than sCMOS cameras (QE ∼ 95%), and has shown the capability of detecting single molecules and super-resolution imaging.⁶¹ However, considering the balance between cost and the overall performance in image resolution, the industry-grade camera is a cost-effective choice.

Objective lens is the core component of the SMLM imaging setup, as it directly determines the collected photon number and, thus, the image resolution of SMLM. The High-NA objective is generally expensive (NA > 1.4, ∼$7K). As we previously demonstrated, a low-cost objective lens (NA = 1.3, oil immersion) in theory results in around 50% less collection efficiency compared to the high-end counterpart (NA = 1.49, oil immersion).⁶¹ Although the use of a low-cost objective lens (NA = 1.3, ∼$700) did show a reasonable performance for SMLM imaging, the dramatically decreased collection efficiency by ∼50% resulted in a compromised resolution. The successful implementation of a low-cost STORM system demonstrated the initial success of super-resolution imaging with just a small fraction of the cost in a conventional SMLM system, ranging from ∼$4K to ∼$25K. Therefore, among the above-mentioned key factors (laser, camera, and objective) for the low-cost SMLM system, the quality of the objective lens has the most profound impact on the resolution of reconstructed super-resolution images.

Another important component for SMLM is automatic drift correction. Although the long-term drift correction may not be needed in many proof-of-concept experiments (as shown in many early versions of the low-cost SMLM), it is crucial for robust biological experiments. The autofocus module often needs a real-time detection system to analyze the system drift and nano-positioning system for real-time correction to maintain nanometer stability in the entire imaging process that lasts from several minutes to hours. The detection system can be implemented with hardware based on the reflection of an infrared LED light between the coverslip and the sample⁶² or by tracking the fiducial markers⁶³ and image correlation.⁶⁴ However, drift correction usually requires nano-positioners for active compensation of axial positions, which is usually expensive (∼$5K). Therefore, the development of a low-cost nano-positioner is critical for a low-cost SMLM system.

There is no doubt that the past decade has seen incredible progress in both technological advancement and biomedical applications of super-resolution microscopy and its gradual emergence as an essential microscopic technique in biomedical research. However, the gap remains between the engineering advances and biomedical applications. Much of the engineering development focuses on the proof-of-concept demonstration of new technical capabilities using a small number of well-established biological samples with known structures, while biologists need a super-resolution microscope that is highly robust, reliable, easy-to-use, and works on a wide range of biological samples (bacteria, cultured cells, and frozen or paraffin-embedded tissue) with the ability to analyze a large number of cells to discover new unknown biological structures. Driven by the need of super-resolution microscopy to solve complex biological questions, as discussed in this perspective, much progress has been made and new advances will continue to be made to address this gap between technical development and biological applications.

On the technical end, the SMLM will likely expand toward simultaneous multiplexed imaging (four colors or more), even higher resolution (less than 10 nm), and the ability to probe deeper into the 3D cell models (e.g., organoid) and tissue. We envision that the state-of-the-art, compact, and low-cost multi-color SMLM system will soon be widely available in biological laboratories, becoming the most cost-effective super-resolution microscopy. More machine-learning-based automated image analysis tools will become available in the super-resolution microscopy community.

Its cost-effectiveness, single-molecule detection, and quantitative nature make SMLM an ideal candidate to make an impact in clinical care. New applications of SMLM in addressing clinical challenges remain in infancy at the current stage, but they will likely expand in the new decade. Several promising clinical applications of SMLM have emerged. SMLM has been used to detect the antigens at an ultra-low density that could be important in the selection of proper patients who may benefit from CAR-T immunotherapy.⁶⁵ Quantitative SMLM (qSMLM) combined with tissue touch preparation has been used to quantify the copy number from the freshly excised tissue from breast cancer patients.⁶⁶ The same method has also been used to quantify the size and biomarker content of individual extracellular vehicles from patients’ plasma.⁶⁷ Recently, our group optimized the SMLM for robust and high-quality super-resolution imaging of pathological tissue (referred to as PathSTORM) and demonstrated the potential of SMLM to improve cancer detection,³⁸ as illustrated in Fig. 4. We envision that, in the future, SMLM will not only become a powerful tool for basic biological research but the robust, reproducible, and quantitative super-resolution imaging systems will also likely be used to analyze patient tissue, blood samples, and other bodily fluids to make an impact in improving patient care.

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

This work was supported by National Institutes of Health Grant Nos. R33CA225494 and R01CA185363 and the Charles E. Kaufman Foundation.