Multi-images deconvolution improves signal-to-noise ratio on gated stimulated emission depletion microscopy

Stimulated emission depletion (STED) microscopy reveals with sub-diffraction spatial resolution,¹ the distributions and dynamics of tagged molecules inside living cells,² or in inorganic materials.³ The sub-diffraction resolution is achieved by reducing the detection volume of a conventional scanning microscope, i.e., the signaling of the excited fluorophores at the periphery of the excitation focal region is precluded. To this end, a second beam (the so called STED beam) is used. This beam, which is shaped like a doughnut with a “zero” intensity in the center and co-aligned with the regular excitation beam of a scanning microscope, quenches the fluorescence via stimulated emission (SE). Increasing the intensity of the STED beam saturates the SE transition of the fluorophores at the periphery of the excitation volume and confines the effective fluorescent volume, and thus the detection volume, to sub-diffraction size.

Although STED microscopy was invented in 1994,⁴ its applications have only gained substantial momentum over the last few years. This as a consequence of the alliance with many other research fields such as laser technology, fluorescent labeling, optics, and spectroscopy. Among these advances, the synergy with time-gated detection represented a fundamental step towards the wide dissemination of STED microscopy.^5,6 Certainly, it is possible to effectively silence fluorophores by using time-gated detection with moderate STED beam intensity.⁷ In addition to the ability to reduce the illumination intensity, required to obtain a certain resolution (fundamental for live cells imaging), time-gated detection allows the implementation of STED microscopy with the STED beam operating in a continuous wave (CW), which reduces both the complexity and the cost.^6,8 Time-gated STED implementations are based on the principle that the later the fluorescent photons of a fluorophore are registered, with respect to its excitation events, the more likely it is to be silenced when exposed to stimulating photons.⁷ In other words, the inhibition of the fluorescent signal depends on the number of stimulating photons to which the fluorophore is exposed while residing in the excited-state. Recording the fluorescence photons after a time T_g, from the excitation events, ensures that the signal originates from fluorophores which have resided in the excited-state and thereby in the presence of the STED beam, and have been exposed to stimulating photons for at least the same length of time. Practically, if the excitation events are triggered by a pulsed excitation beam and the fluorophores are continuously stimulated by the STED beam, the region of the contributing fluorophores shrinks with the time-delay T_g. Strictly speaking, the resolution improves because the effective point-spread-function (E-PSF)⁹ of this STED system, usually called a gated CW-STED (gCW-STED) microscope,^6,8 reduces in size with increased T_g (Figure 1(a)).

Unfortunately, the contraction of the gCW-STED microscope's E-PSF comes along with a reduction of its amplitude (Figure 1(b)). Indeed, the wanted signal from the center of the doughnut scales as exp(−T_g/τ), where τ represents the excited-state lifetime of the fluorophore in the absence of stimulating photons. As a consequence, a long time-delay T_g reduces the signal-to-noise ratio (SNR) of the gCW-STED image and can cancel-out the gain in resolution. Here, we propose a deconvolution-based method that compensates for this SNR reduction. This method takes advantage of the early fluorescent photons (before T_g), usually discarded by the time-gated detection. In particular, it recombines the image formed by the early photons with the conventional gCW-STED image (late photons) through a multi-image (MI) deconvolution algorithm.

The region from which the early fluorescent photons originated is larger with respect to the counterpart of the late fluorescent photons (after T_g), because early fluorescent photons (T_g < τ) stem from fluorophores, which have resided in the excited-state for a shorter time and thus exposed to fewer stimulating photons. Thus, the E-PSF associated with this early photons image⁹ (Figures 1(c) and 1(d)) (i) is always larger than the gCW-STED microscope's E-PSF; (ii) reduces with increased T_g from the PSF of the confocal microscope (no STED beam active) to the E-PSF of the CW-STED microscope (no time-gated detection); and (iii) increases in amplitude with increased T_g. In practice, for a long time-delay (T_g ≥ τ), the late-photons image (gCW-STED) has higher resolution but lower SNR, whilst the early photons image has poorer resolution but comparatively higher SNR (Figures 1(e)–1(h)). From a frequency-content view (Figure S1) the early photons image contains only the low frequencies (the cut-off frequency increases with T_g), whereas the late-photons image theoretically contains “infinite” frequencies, though they are mostly dominated by noise.⁷ Due to the ill-posed nature of the deconvolution problem, the application of conventional deconvolution algorithms on the late-photons image (gCW-STED image) may amplify noise and introduce artifacts in the restored images.¹⁰ A common method used to compensate for noise-amplification is to introduce, into the deconvolution problem, prior knowledge about the sample (object). These methods are usually referred to as constrained deconvolution.¹⁰ The basic idea is to impose some constraints on the solution of the deconvolution problem in order to remove the solutions dominated by artifacts. The most important constraint in fluorescence microscopy is the non-negativity of the solution. Further, typical regularized algorithms employ edge-preserving^11–13 or sparsity-promoting,¹⁴ which are usually obtained in a Bayesian framework.^10,15

To compensate for noise-amplification, we did not use a-priori information about the sample, but we used the information provided by the early photons image. In general, we used the information provided by the temporal dependencies of the E-PSF. We introduced the temporal information through the E-PSFs of a MI deconvolution approach, instead of using a Bayesian framework. In particular, we used a MI generalization^16,17 of the Richardson-Lucy (RL) algorithm^18,19

where:⁹ (i) x^k denotes the restored images at the k-th iteration; (ii) y_l denotes the l-th image associated to the l-th PSF, h_l; and (iii) H_l is the notation for the discretization of the convolution operator associated to the PSF h_l. This strategy was already outlined by Vicidomini et al.⁷ and Ingaramo et al.,²⁰ but lacks an extensive study.

In the context of this work, we have two images (i.e., L = 2), which represent, respectively, the early and late-photons images with the respective PSFs, h_early and h_late. We first tested the proposed method on synthetic images of a phantom mimicking the micro-tubulin cytoskeleton⁹ (Figure 1(e), upper-left corner). We simulated the images of the confocal (Figure 1(e), lower-right corner) and the CW-STED (Figure 1(f)) microscopes, together with the images obtained separating the early (Figure 1(g)) and the late photons (gCW-STED, (Figure 1(h))). To quantify the fidelity of the raw and the restored images, we chose the signal-to-error ratio (SER)

where x is the original object and $x̃$ is the raw or the restore image. In order to compare, the factor $γ=Φ(x)/Φ(x̃)$ normalizes the values of the phantom to have the same flux Φ as the image. In this context, it is important to remember that the RL algorithm and its generalization conserve the photons-flux of the raw image,⁹ which is important for performing a quantitative analysis on the restored image. We stopped the iterative algorithms at the iteration with the maximum SER.

It is worth noting that the SER of the gCW-STED raw image (gray line, Figure 2(a)) improves for increasing time-delay T_g, but after a certain value (T_g > 0.75 ns, in this example) it degrades faster. This is in complete agreement with the limitation of the gCW-STED microscope, namely, the reduction of the SNR associated to longer time-delays T_g can cancel-out the expected resolution improvement. By applying the conventional RL algorithm to the solely gCW-STED image (late-photons image), the SER of the restored image (gCW-STED⁺) improves, but it degrades fast with the increase of the time-delay. For long time-delays (T_g > 2 ns, in this example) the SER reduces even below the value of the CW-STED raw image (black line, Figure 2(a)). The fidelity of the restored image (gCW-STED⁺⁺) significantly improves when the deconvolution algorithm also includes the image formed by the early photons (red line, Figure 2). In particular, as with the gCW-STED raw images, the SER improves for increasing time-delay T_g until reaching a maximum value (T_g > 0.75 ns, in this example). Very importantly, the SER for this optimal time-delay is higher than the SER associated with the restoration of the CW-STED image (blue point, Figure 2(a)). Even though the CW-STED and the gCW-STED raw images have the same frequency bandwidth and the respective high-frequencies have similar strength (Figure S1),⁷ the information provided by the temporal dependency of the E-PSF, and included in the MI algorithm, allows for better recovery of the high-frequencies of the object. For long time-delay (T_g > 1.5 ns, in this example), the uncorrelated background⁹ dominates in the late-photons image, and the SER of the gCW-STED⁺⁺ restored image starts to degrade. This limitation, together with the need to choose the best time-delay T_g, could be overcome by splitting the fluorescence photons before T_end = 2 × τ (later photons are likely uncorrelated background) into more time gates and applying the MI algorithm (Eq. (1)) to the relative images (Figure S2). However, this solution is compatible only with the gCW-STED implementations, which are based on a time-correlated-single-photon-counting (TCSPC) card.^8,21 The inclusion of images in the MI algorithm has to be weighted against the increase in the computational-time, which scales linearly with the number of images L. Fortunately, the MI algorithm reduces the number of iterations needed to reach the optimal restoration, compensating for the increase of computational time necessary for each iteration (Figure 2(b)). To mimic near real-time deconvolution (Figure S3),^22,23 we implemented all algorithms on a graphics processing unit (GPU).⁹ 

Visual inspections of the restored images (Figures 2(c)–2(e)) confirm the quantitative SER analysis. It is evident that the gCW-STED⁺ restored image (Figure 2(c)) suffers severely from the typical artifacts of the RL algorithm. Indeed, it is well known that the RL algorithm converges to a sparse solution (known as the checkerboard effect). Early interruption of the algorithm can mitigate this problem, but in the case of a low SNR image this does not lead to significant benefits. In our example, the tubulin structures appear as a collection of spots and not as continuous structures, as depicted on the phantom (Figure 2(e)). These typical artifacts reduce in the gCW-STED⁺⁺ (Figure 2(d)) and CW-STED⁺ restored images, however, a closer look at the results shows an higher reduction of blurring from the multi-image RL restoration compared to the conventional RL restoration. The information provided by the different E-PSFs allows to the MI algorithm to reach a better solution even though the algorithm deals with images with the same SNR (the CW-STED image is the sum of the early and late-photons images). We obtained similar results also for phantoms mimicking mitochondria membranes (Figure S4).

Next, we applied the proposed MI deconvolution method to data acquired with a custom made gCW-STED microscope.^6,7 The images (Figure 3) show the β–tubulin proteins located in the cytoskeleton network of a PtK2 cell immuno-labeled using ATTO647N. A comparison between the CW-STED (T_g = 0 ns, upper-right corner Figure 3(a)) and the gCW-STED (T_g = 1.5 ns, lower-left corner Figure 3(a)) clearly shows the resolution improvement, but also the SNR reduction. Strong artifacts appear when we applied the RL algorithm to the gCW-STED raw image (gCW-STED⁺, lower-right corner Figure 3(b)). These artifacts reduced when we applied the RL algorithm on the CW-STED image (gCW-STED⁺, upper-left corner Figure 3(b)) and when we applied the proposed method (gCW-STED⁺⁺, Figure 3(c)), but the blurring was also reduced only in this last case (Figures 3(d) and S5).

It is interesting to compare this MI deconvolution approach with the subtractive method proposed by Hao et al.,²⁴ where a new image is obtained by the subtraction of the early photons image (properly weighted) from the late-photons image. Since the early photons image contains only low-frequencies this subtraction highlights the high-frequencies of the late-photons image but at the same time suppresses the low-frequencies, thereby, the resulting image reveals mainly the edge of the structures observed. Diversely, our method tries to recover the high-frequencies contained solely in the late-photons image, which unfortunately are hidden by noise. Further, subtractive methods reduce SNR and can introduce negative values, whilst image deconvolution algorithms are well known for their ability to improve the SNR and can easily constrain the solution to non-negative values.

As a conclusion, we proposed a computational method that improves the SNR of a gated CW-STED microscope and thereby its effective resolution. The method uses a MI deconvolution algorithm for combining the conventional gated CW-STED image with the image obtained from the early photons, usually discarded. We therefore expect our approach to greatly benefit gCW-STED microscopy users by providing a robust and easy-to-use method to obtain optimal restored images.

The authors thank Maria Ingaramo and Andrew G. York from the National Institutes of Health and Zeno Lavagnino from the Italian Institute of Technology for useful discussions, Eileen Sheppard for English proofreading.