A direct electron detector for time-resolved MeV electron microscopy

Ultrafast electron microscopy (UEM) is a set of techniques for studying dynamical processes on the femtosecond and picosecond time scales. A wide range of physical and chemical phenomena can be examined this way.^1,2 Examples from ultrafast electron diffraction (UED) include gas phase molecular motion,³ electron-phonon coupling in solids,⁴ and thin film lattice dynamics.^5,6 UED is complementary to time-resolved x-ray scattering, and in some cases, it is actually the preferred technique due to its larger cross section of interaction and less sample damage per scattering event. Additionally, the two techniques probe materials differently. Electrons mainly interact with nuclei whereas X-rays primarily interact with the surrounding electrons.

The temporal resolution of UEM is limited by space-charge effects within the electron beam. For relativistic electron beams, the transverse and longitudinal space-charge effects scale as $1/γ3$ and $1/γ5$⁠, respectively, where γ is the relativistic beam energy.⁷ In order to reduce these effects, a radio-frequency (RF) photocathode gun can be used to quickly accelerate the electrons to MeV energies before temporal broadening can occur.⁸ This generates the highest charge possible in as temporally short a pulse as possible. The benefits of this approach can be seen in the recent success of MeV UED studies of gas phase dynamics.^3,9 Another advantage of MeV UEM is that it eliminates the velocity mismatch between the pump laser and the electron beam that can limit temporal resolution.¹⁰ Improving the temporal resolution to better than 100 fs is a key objective for the time-resolved electron microscopy community.

At the MeV UED facility in SLAC National Accelerator Laboratory, an RF gun is used to launch a few fC of charge at up to 5 MeV with less than 100 fs RMS bunch length at a 180 Hz repetition rate. The beam is delivered to a sample using sets of solenoid and corrector magnets with a pinhole for spatial filtering between them. The beam drifts after the sample before being recorded by one of two different detectors. One is the existing phosphor-based imaging system and the other is the new direct electron detector, both of which are shown in Figure 1. More detailed information on this facility can be found in Ref. 11.

The existing phosphor-based imaging system consists of a 50 $μm$ thick P43 phosphor screen. The screen is imaged on a cooled Andor iXon Electron Multiplying Charge Coupled Device (EMCCD) camera using a 50 mm f/1.2 telephoto lens on an extension tube. The photon collection efficiency of the imaging system is a few percent and the camera is capable of single photon detection. If each incident MeV electron generates several hundred photons, then the system is almost capable of single electron detection. However, this imaging system has several undesirable features that negatively affect the UED program. First, the P43 phosphor suffers from image burn in. Even though the fluorescence 1/e lifetime is 0.7 ms, intense fluorescence can persist at a low level for minutes afterwards. This is disadvantageous when analyzing subtle differences in diffraction patterns. Faster phosphors are available but they have reduced quantum efficiency and fluoresce at different wavelengths. Second, in time-resolved pump-probe experiments, it is important to keep scattered pump light from reaching the detector. This can be done using optical filters but doing so adds the constraint that the pump laser wavelength cannot match the wavelength at which the phosphor fluoresces. Therefore, some experiments require different types of phosphors with lower quantum efficiency and worse temporal performance. Third, phosphor screens generally have a spatial resolution on the order of about 100 $μm$⁠. This comes from a combination of the minimum grain size and the film thickness. Ultimately this limits the reciprocal space resolution of the system. This can be detrimental to some biological experiments where very high reciprocal space resolution is needed. Readers who are interested in other imaging systems used under similar experimental conditions are referred to Refs. 12 and 13.

The introduction of direct electron detectors played a critical role in the structural biology revolution of cryogenic electron microscopy.¹⁴ This revolution occurred in part because of the development of advanced algorithms for 3D reconstruction and in part because single-shot direct electron detectors were developed that permitted images to be aligned using post processing, thereby removing the blurring associated with sample motion. This dramatically reduced the amount of time required to collect high quality data. The revolution in structural biology was the motivation to develop a direct electron detection system at the MeV UED facility in SLAC.

The major challenge encountered in developing the direct electron detection system was that there are no commercially available sensors designed to operate at MeV beam energies. As a result, a new system had to be engineered by colleagues at Lawrence Berkeley National Laboratory (LBNL). The system is based on a complementary metal–oxide–semiconductor (CMOS) active-pixel sensor (APS) array that was originally developed for the TEAM project.¹⁵ The sensor consists of n+ diodes implanted into moderately resistive p-type epitaxial silicon, grown on a low-resistivity p-type silicon substrate. The sensor includes back end of line (BEOL) interconnects consisting of metal and insulating SiO₂. The thickness of the active epitaxial region, t_epi, is 14 $μm$ and the pitch of the individual sensors in the array is 9.5 $μm$⁠, as illustrated in Figure 2. The sensor array is 1024 × 1024 in size with a 500-$μm$ diode in the middle that can be used to monitor the electron beam intensity (small black circle in Figs. 7 and 9). Commercial sensors¹⁶ have about three times thinner active epitaxial regions than this in order to minimize multiple scattering of incident keV electrons.¹⁷ MeV electrons have less multiple scattering and approximately five times lower energy deposited per unit length of active region. Therefore, the thicker epitaxial region of the TEAM sensor significantly increases the signal collected for MeV electrons.

The energy an incident electron deposits per unit length of absorbing material, dE/dx, is given by the Bethe-Bloch formula.¹⁸ Empirically parameterized values for this formula are available online from the National Institute of Standards and Technology (NIST) website.¹⁹ For 2.2 MeV electrons, traversing silicon dE/dx is 368 eV/$μm$⁠, almost entirely due to ionization. At 3.7 MeV, the value increases moderately to 391 eV/$μm$⁠. Because there is only 14% variation in dE/dx over the range from 1 to 5 MeV, the detector is expected to have an essentially flat response with respect to the beam energy. The energy of creation of an electron-hole pair in silicon, η, is 3.66 eV. According to Equation (1), a primary electron is expected to generate close to 1450 electron-hole pairs, N_e−h, in the active epitaxial region. The sensor response was measured to be approximately 0.25 analogue-to-digital converter (ADC) counts per electron-hole pair. Combining these two results, primary electrons at several MeV are expected to generate approximately 360 ADC counts

The direct detector system consists of three main components, each of which is shown in Figs. 1 and 5. There is an in-vacuum TEAM1k CMOS APS array, an external Sensor Interface Assembly (SIA), and an external MicroTCA® Data Acquisition (DAQ) computer. To minimize noise, all three components are placed in close proximity to each other, inside of the UED radiation bunker. The TEAM1k sensor is Ultra High Vacuum (UHV) compatible. It is mounted to a silicon carbide (SiC) substrate and attached to a Kapton Polyimide printed circuit board with Al bond wires. The SiC substrate is mounted to a Cu cooling block using glass-filled nylon screws and a vacuum rated thermal sinking compound. This allows the sensor to be cooled to −20 °C by a regulated water chiller and a TE Technology Peltier thermoelectric cooler. Additionally, a 25 $μm$ Kapton foil was placed over the sensor to shield it from ambient light. Because this foil blocks all visible light, the system is compatible with any UED pump laser configuration. Figure 3 shows images of the sensor as it is installed.

The TEAM1k sensor is expected to be tolerant to radiation damage at MeV beam energies. Anecdotal evidence is that the sensor has been exposed to MeV radiation for about one year and so far only small changes in leakage current have been observed. Nothing has been done to mitigate or reverse these changes (e.g., sensor annealing). This sensor was originally developed for the TEAM project, and its performance characteristics and radiation hardness were previously reported in Refs. 15 and 20. More detailed investigation of radiation hardness at MeV beam energies is planned in the future. Figure 4 shows the linearity of sensor response as a function of beam energy.

The dynamic range of the direct electron detector is 15 bits. If one conservatively assumes that each electron deposits enough energy into a single pixel to generate 400 ADC counts, then the maximum number of electrons that can be detected by a single pixel without saturating is about 80. With uniform beam coverage, the 1 cm² detector will completely saturate at 10 pC. Alternatively, a fC beam will saturate the detector if it is focused onto a diameter of less than 10 pixels (95 $μm$⁠). Under nominal UED operating conditions, the direct detector has never saturated.

The SIA includes 16 analog amplifier channels, 12 channels of configurable constant current and voltage sources, a timing and configuration interface signal path, an external trigger input, a connection to the diode at the center of the TEAM1k sensor, and switchable linear power sources for the sensor. The vacuum interface that connects the APS to the SIA uses two custom built cables that provide physical isolation between the sensor analog and digital signals. The SIA interfaces the DAQ computer Peripheral Component Interconnect Express (PCIe) bus via the VadaTech AMC52 that contains a Xilinx Virtex-7 Field-Programmable Gate Array (FPGA) and eight dual-channel 250 megasamples per second (MSPS) high speed analog-to-digital converters (ADCs). Pixel data from the 16 ADC channels are selected and arranged into a cpu-optimized (x,y) sequential fashion then transferred over a Direct Memory Access (DMA) to the CPU. Each ADC channel corresponds to 64 columns of pixel data. The DAQ computer includes an Advanced Mezzanine Card (AMC), a Solid-State Drive storage AMC, a 2 channel 10GbE Fabric AMC, and an AMC521 custom DAQ AMC that are all built by VadaTech Inc. The DAQ computer also includes an event receiver (EVR) AMC that is compatible with SLAC’s lab-wide timing infrastructure. The EVR was included for future time stamping of images in order to make correlated measurements with other machine parameters and to correct for temporal jitter.

Three software interfaces to the data are available: a standalone system where the data are stored locally on a hardware RAID-0 in the DAQ computer, network data streaming using ZeroMQ, and through a C++ driver that provides low level integration at the DMA level. Slow controls are either sent over human-readable transmission control protocol/internet protocol (TCP/IP) commands or directly through the low-level C++ driver. Integration into the UED facility has been achieved using an Experimental Physics and Industrial Control System (EPICS) interface that communicates with the detector using the low-level C++ driver.

Finally, a special double exposure acquisition mode is used to capture a dark image in the non-illuminated period just before electrons arrive at the detector. In this mode, the readout sequence for every trigger event is first to reset all pixels and then to wait 2 $μs$⁠. Next comes a readout of the non-illuminated dark image followed by another 2 $μs$ wait. Then the exposure integration (0-1.5 $μs$ variable) is triggered and the exposed image is read out. There is no pixel reset between the dark image and the exposed image so that they both have the same pixel reset levels. Thus, pixel reset noise is eliminated by subtracting the first dark image from the second exposed image. The 1024 × 1024 signed 16 bit integer images recorded by the direct detector system are actually the subtraction of these double exposures. The pixel noise that is left behind is a small amount of leakage current per pixel that must be dealt with using background subtraction during image analysis. Typically, this is on the order of 20 ADC counts RMS.

The integrated direct detector system captures single images at 180 Hz, producing an uncompressed data stream of 360 MB/s. The major technical challenge that had to be overcome in commissioning the new system was to efficiently handle such large amounts of data. The architecture designed for this is illustrated in Figure 5. The architecture converged over a series of iterations and is one of the multiple possible strategies that could have been implemented.

The system is configured such that when images are captured they are broadcast as a process variable (PV) by an EPICS input/output controller (IOC) on the DAQ computer. A dedicated 10 Gb fiber network brings the data out of the bunker to the analysis, control, and archiving computers located in the UED control room. Images from the direct detector are continuously broadcast but are not always saved. An archiving computer records the images when instructed to do so. The archiving computer was built by Supermicro and has 40 TB of disk space in a striped RAID 0 configuration optimized for fast I/O. The RAID capacity is sufficient to store 30 h of continuous uncompressed raw data. In practice this is adequate for an experimental run over the course of a week. Simultaneous reading and writing at high-speed is not possible using platter based hard drives so a PCIe flash disk is also included in the archiving subsystem. When images first arrive, they are temporarily stored on the flash disk and can be simultaneously read from and written to at high speed—facilitating online analysis. The images are then copied to the RAID array using a fast data transfer script before they are deleted from the flash disk. The analysis computer mounts the archiving computer’s flash disk using the Network File System (NFS) protocol. The advantage of this approach is that much of the underlying infrastructure is transparent to users if they wish to implement their own data analysis routines.

The Hierarchical Data Format (HDF5) is used for storing the images. This is the same format used at the Linac Coherent Light Source (LCLS) and allows the UED program to leverage existing computational resources at SLAC. Several hundred TB of longer-term storage space is available on LCLS servers and a slow data transfer script copies the data over as network resources are available. The space is accessible from the world-wide-web so visiting scientist can view their data externally for a limited time after having completed an experiment. Current plans are to store all data for up to six months.

The intention of the new detector is to replace the existing phosphor-based imaging system. In practice this is actually more complex than it might seem. The reason for this is that each exposure from the direct detector is read out individually. Alternatively stated, the direct detector is not able to accumulate data over multiple exposures the way that a CCD camera can. The problem is that readout noise occurs in every single image captured by the direct detector instead of occurring only once—as it does in the phosphor-based imaging system. Generally speaking, the sum of images acquired through single exposures will have significantly more noise than a single image acquired from an accumulation of exposures, all other parameters being equal. The best way to counteract this is to process each image one at a time to remove as much noise as possible prior to image summation.

The image processing algorithm developed for this relies on a process called “clustering”²¹ and a statistical interpretation of the data resulting from two physical observations. The first is that single electrons at several MeV passing through 14 $μm$ of absorbing material have a point-spread function that exceeds 9.5 $μm$⁠. This means that single electron events will be observed in no less then 4 pixels that are grouped together and that large single pixel fluctuations can be treated as noise. The second is that the loss of energy of a charged particle as it passes through an absorbing material can be statistically modeled by a Landau distribution.²² The formalism for the derivation of the Landau distribution is similar to that used to derive the Bethe-Bloch formula referred to in Section III. For MeV electrons incident on the direct detector, the sum of isolated clusters of pixels representing single electron events can be fit with

On the other hand, a Gaussian distribution more closely models the statistical distribution of clusters of pixels originating from noise

For convenience, the scale, width, and location parameters used in Equations (2) and (3) are being functionally defined by Equations (2) and (3). The advantage to this is that it avoids repeating a somewhat complex discussion of the Landau distribution in an attempt to give physical interpretations to the parameters. The fitting methods are commonly used so that readers who wish to have a deeper understanding of the parameters are encouraged to refer to Refs. 21, 23, and 24. The steps involved in individual image analysis are given in Table I and are partially illustrated in Figure 6.

In order to validate the performance of the direct detector, data with sparse electron hits per image were taken at different incident beam energies. 200 s of data (∼36 000 images) at beam energies of 2.2 MeV, 2.7 MeV, 3.2 MeV, and 3.7 MeV were acquired both with and without an electron beam.The images were analyzed according to the steps outlined in Table I. Furthermore, a separate analysis was performed by summing the statistical distributions of single image clustersextracted by terminating the analysis at step 6. Histograms of these distributions were taken and then summed to create cluster ensemble histograms for each incident beam energy. The range of parameters used to fit the resulting statistical distributions according to Equations (2) and (3) is given in Table II.

A detailed evaluation of the data recorded at 3.2 MeV using the steps identified in Table I is given in Figs. 7 and 8. Using a $5σ$ cluster identification threshold combined with $3σ$ single pixel filtering produces images that when summed both visually appear to have very low noise and statistically match distributions that are modeled almost entirely by the single electron events with very low noise. The conclusion is that this is most likely the optimum set of parameters for analyzing individual images prior to summing them and that no additional cluster acceptance threshold is needed. Figs. 9 and 10 show the result of repeating this analysis on data taken at other incident beam energies using the same $5σ$ cluster identification threshold and $3σ$ single pixel filtering. Generally 40% of all single electron events have been recorded and only a few percent of the identified clusters are likely to have originated from noise. The results make a compelling case that the detector is working as expected and is capable of producing low noise images after image processing.

One interesting thing to note is that the peak of the Landau distributions representing single electron clusters occurred at 345 ADU counts across all incident beam energies and using different forms of noise reduction. The peak of the distribution is not equivalent to a mean energy deposited but it is very close to the 360 counts predicted in Section II. This indicates that the analysis methods used give results that are consistent with expectations.

Separate analysis of the clusters at each beam energy produces point spread functions for single electron events that are characterized by RMS widths of σ = 0.58/0.70 pixels (x/y) without single pixel filtering and σ = 0.67/0.79 pixels with $3σ$ single pixel filtering. This result was independent of incident beam energy. The 0.12 pixel x-y RMS width asymmetry is explained by a small nearest neighbor pixel correlation in the direction of the channel readout. As expected there is a 0.09 RMS pixel width increase with $3σ$ single pixel filtering because the contributions from sharp pixel fluctuations attributed to noise have been removed. Because it is possible to resolve two single electron clusters with peaks separated by three pixels, the detector is stated to have better than 30 $μm$ spatial resolution. These measurements show that the majority of single electron clusters are 3 × 3 in size. If execution speed becomes a critical issue in the future, this could present an opportunity to optimize analysis routines by initially defining all clusters to be 3 × 3 in size. Doing so could simplify the cluster acceptance routine at the cost of introducing some statistical error. The tradeoff might make sense though if algorithm execution times become prohibitively long.

Finally, for single shot alignment of diffraction images, the position of the direct beam is used as the spatial reference for the alignment procedure. Due to the large dynamic range of the detector, there is no need for a beam stop. However, the clustering algorithm cannot be used on regions where charge pileup has occurred. The image analysis must first exclude these regions, such as in the vicinity of the direct beam. For the clustering algorithm to succeed, the direct detector should not be exposed to electron beam current densities greater than 7 fC/cm² (1 electron/25 pixels). In the case of some crystalline samples, such as the single shot single crystal gold diffraction pattern shown in Figure 11, when illuminated with a few fC direct beams, the Bragg peaks are still too strong for clustering. In this case, the charge in the main beam must be attenuated until the threshold condition for clustering in the Bragg peaks is reached.

Successful commissioning of the new direct electron detector is a major technological accomplishment. The system performs as expected, recording individual images at 180 Hz. A significant subset of these images can be analyzed in real time and current strategies for data management are working satisfactorily. The detector has a 15-bit dynamic range, better than 30-$μm$ spatial resolution and less than 20 counts RMS pixel noise. The direct detector system is also capable of identifying MeV single-electron events. When the direct detector is exposed to electron beam current densities of less than 7 fC/cm², then there is little chance of charge pileup and single electron events can be statistically identified with high probability. Although the majority of single electron clusters are 3 × 3 pixels in size, it is recommended to analyze clusters with a $5σ$ identification threshold and $3σ$ single pixel filtering as demonstrated in Section IV. No additional acceptance threshold is needed. With this analysis, approximately 40% of single electron events are recorded and only a few percent of the recorded events are likely to have originated from noise.

The introduction of direct electron detectors played a critical role in the structural biology revolution of cryogenic electron microscopy. There are several reasons why the direct electron detector is similarly expected to have a dramatic impact on time-resolved MeV electron microscopy. First, the direct electron detector possesses higher spatial resolution and faster temporal response than the phosphor-based EMCCD imaging system. Second, the direct electron detector is capable of recording single electron images at 180 Hz while the imaging system is only capable of 10s of Hz. Recording each individual image makes it possible to correct both temporal and spatial jitter through post processing, similar to the technique now routinely used for cryogenic electron microscopy. Correcting electron beam jitter is important in diffraction experiments where spatial resolution translates to q-space resolution. Likewise, correcting sample jitter is important in imaging where sample motion can blur the image and limit spatial resolution. Third, one unique feature of the direct electron detector is that it includes a custom built timing system that allows each image to be time-stamped. An online diagnostic will be developed in the future to measure the temporal jitter of each exposure. The image timestamp will then make it possible to correct for temporal jitter with post processing. This will be especially important for studying phenomena that occur on the fs time scale. The approach is similar to that being used at the LCLS to achieve less then 10 fs time resolution.²⁵ Fourth, the direct detector is compatible with all pump laser wavelengths and does not suffer from phosphor image burn in. Finally, the direct detector has a very large dynamic range, allowing the main beam to be recorded during diffraction experiments. Because the main beam is present, intensity fluctuations in single-shot images can be accurately normalized. For these reasons, it is expected that the direct electron detector will have a dramatic impact on time-resolved MeV electron microscopy.

Commissioning of the direct detector has completed and regular operations are beginning. However some future improvements will be needed in order to keep pace with aggressive scientific program planned at the UED facility. The following is a summary of future improvements that are being discussed.

Current image analysis routines are able to process 110 images/s relying on parallel CPU execution. Because the rate of analysis is slower than the rate of acquisition, not all of the data can be analyzed in real time. To provide users with better online image analysis, other paradigms need to be considered. This includes using high performance computing clusters or converting algorithms to run on graphics processing units (GPUs).

The direct detector has 16 independent ADC channels, each of which has slightly different signal amplification. As a result, if the channels are not independently analyzed then image artifacts can be created. One way to improve this is to adjust the individual ADC channel responses, producing more uniform images without unnecessarily complicating the analysis routines. Another way is to incorporate individual channel gains in the image analysis.

The detector, external amplifier, and DAQ computer are all subject to significant external noise in a high radiation RF environment. This can manifest itself in low-level correlated pixel noise. Increasing the shielding on all of the components in the system will reduce this effect.

Eventually the UED facility will run at 360 Hz and the next generation of sensor array is expected to have 4 times as many pixels (2048 × 2048). Together these changes will increase the required bandwidth of the system by a factor of 8. The 10 Gb fiber network currently used can handle the 760 MB of bandwidth needed to transfer the data from the DAQ computer to the archiving computer and again to the analysis computer but the network will not be able to keep up with an order of magnitude increase in the amount of data. At some point, a re-evaluation of the system architecture will be required, focused primarily on increasing network performance.

Finally, managing the amount of data generated is a challenge that will only continue to grow. It is reasonable to expect that both storage and computational resources will increase in time but relying on this is not a legitimate strategy for long-term data management. Rather, work needs to begin on reducing the actual amount of data saved. This involves both improvements to the analysis algorithms used as well as changes to the format the data are stored in. For example, currently each image is saved independently. The disk space required for this could be reduced significantly by compressing image sequences both spatially and temporally—analogous to lossless video compression. Alternatively, orders of magnitude reductions in the required disk space are possible by saving only the coordinates of single electron events—provided that it is acceptable to dispose of the original data. The topic of data reduction in particular will require a significant investment in the future.

The technical support from the SLAC Accelerator Directorate, Technology Innovation Directorate, and LCLS Laser Science and Technology division and Test Facilities Department is gratefully acknowledged. The authors would also like to personally thank Wilko Kroeger and Daniel S. Damiani for their assistance in developing software for the direct detector system. The work at SLAC was supported in part by the U.S. Department of Energy (DOE) Contract No. DE-AC02-76SF00515, DOE Office of Basic Energy Sciences Scientific User Facilities Division’s Accelerator & Detector R&D program. The work at LBNL was supported by the Office of Science, Office of Basic Energy Sciences, of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231.