Digital holography (DH) is a powerful imaging modality that is capable of capturing the object wavefront information, making it very valuable for diverse scientific research applications. Generally, it requires ample illumination to enable good fringe visibility and a sufficient signal-to-noise ratio. As such, in situations such as probing live cells with minimal light interaction and high-speed volumetric tracking in flow cytometry, the holograms generated with a limited photon budget suffer from poor pattern visibility. While it is possible to make use of photon-counting detectors to improve the hologram quality, the long recording procedure coupled with the need for mechanical scanning means that real-time extremely low-light holographic imaging remains a formidable challenge. Here, we develop a snapshot DH that can operate at an ultra-low photon level (less than one photon per pixel). This is achieved by leveraging a quanta image sensor to capture a stack of binary holographic frames and then computationally reconstructing the wavefront through integrating the mathematical imaging model and the data-driven processing, an approach that we termed PSHoloNet. The robustness and versatility of our DH system are demonstrated on both synthetic and experimental holograms with two common DH tasks, namely particle volumetric reconstruction and phase imaging. Our results demonstrate that it is possible to expand DH to the photon-starved regime, and our method will enable more advanced holography applications in various scientific imaging systems.

Digital holography (DH) is a versatile optical imaging technique that records an interference pattern, known as a hologram, on an electronic sensor, which can then be computationally processed to retrieve wavefront information.1 Because of this unique capability, it is used in a wide range of applications, such as biomedical imaging,2,3 environmental research,4,5 and particle field reconstruction.6,7 Typically, good illumination and a sufficient exposure time are needed to provide clear holographic patterns; such requirements unfortunately restrict the further use of this powerful modality under more challenging imaging conditions. For instance, for some biological samples or photo-materials vulnerable to light irradiation, there is strict control of the illumination power in consideration of the phototoxicity of the specimen.8 Similarly, for high-speed particle tracking, the exposure time has to be very short in order to capture fast dynamics.9 There is a clear need to detect such weak signals amidst a noise level that is possibly significantly higher.10 In DH, we can observe in Fig. 1 that the low signal-to-noise ratio (SNR) causes the hologram quality to deteriorate substantially, thus posing great challenges for conventional reconstruction algorithms to extract meaningful information from the holograms containing highly contaminated fringe patterns.

FIG. 1.

Reconstruction results from holograms under different SNRs: (a) ideal condition without noise, (b) SNR = 2.8 dB, and (c) SNR = 0.8 dB. Upper row: Holograms recorded under different noise levels. Bottom row: Reconstruction results using backward propagation.

FIG. 1.

Reconstruction results from holograms under different SNRs: (a) ideal condition without noise, (b) SNR = 2.8 dB, and (c) SNR = 0.8 dB. Upper row: Holograms recorded under different noise levels. Bottom row: Reconstruction results using backward propagation.

Close modal

These situations necessitate research in photon-starved holography since there are scarce photons to form conventional holographic patterns at the sensor plane. Various approaches have been developed to enable DH at a low signal power.11–13 Without introducing extra hardware, the purely computational methods are mostly based on noise removal and signal enhancement, such as angular matching12 and deep-learning transformation.13 However, conventional image sensors, such as CCD and CMOS image sensors used for registering holograms have a minimum photon flux requirement of around 104 photons per pixel (PPP).14 As one pushes the capability of DH to extremely low photon flux situations, i.e., one PPP or lower, a new hologram formation mechanism with single-photon detection has to be considered.

Yamamoto et al. first attempted to tackle this challenging problem by using a photon-counting detector (PCD).15 Benefiting from the continuous progress in sensor development, Demoli et al.16 then used a single-photon avalanche photodiode (SPAD) to record the holographic information of signals overshadowed by noise. More recently, Kang et al. proposed a deep neural network for retrieving phase information at low photon counts using random phase modulation.17 In these methods, the long recording time for the mechanical scanning process (as long as 30 h16) and the extra hardware for phase generation17 restrict their practical deployment. Therefore, we further aim to make the photon-starved holography a snapshot imaging system, which can enable the in vitro capture of live cells with minimal sample interaction and high-speed volumetric imaging.

To make photon-starved snapshot holography possible, an image detector with photon-level sensitivity and high resolution is necessary. Among various photon-counting image sensors reported over the past decade,18–22 the quanta image sensor (QIS) stands out with its small pixel size (e.g., ≤1 μm), high photon detection efficiency, and ultra-low readout noise. This leads to various applications in the low quanta flux regimes, including security, night vision, and quantum computing.23–25 

As reported in Ref. 26, a QIS can be realized using a high-gain jot based on existing CMOS techniques18–20 or using a SPAD.21,22 In this paper, we primarily focus on the CMOS-based QIS although our mathematical formulation can be extended to a SPAD-based QIS. Among the many properties of these sensors, we consider the one-bit mode, where the measured signal is either 1 or 0. This would allow the fastest sampling rate for moving scenes. Yet, integrating this photodetector in DH is not a trivial task, due to its binary signal detection mechanism and very different noise characteristics, which would require specifically designed reconstruction algorithms.

In this work, we overcome several difficulties to present a photon-starved snapshot holography system, which integrates a QIS to take advantage of its low-light detection, and a physics-informed wavefront reconstruction network called PSHoloNet that we have designed to process the captured signals. Through numerical simulations, we demonstrate the generalization capability of the proposed PSHoloNet from a simplified sensor model during training to a more advanced sensor model during inference. To our best knowledge, it is the first time that a QIS is integrated into a coherent imaging system. To demonstrate the feasibility through synthetic experiments, we show two DH applications, namely 3D particle volumetric reconstruction and phase imaging.

The Gabor holographic imaging setup together with a QIS as the recording device is designed and shown in Fig. 2. The object ϑ(x, y) = A(x, y)e(x,y) is placed at the axial position z = 0, with A and ϕ representing its transmittance and phase response, respectively. Suppose that we have an incident plane wave with wavelength λ. Based on the diffraction theory,27 the transmitted wave received at the sensor plane is
O(x,y;z)=hz(x,y)ϑ(x,y),
(1)
where ⊗ stands for the convolution and hz denotes the impulse response of free-space propagation over a distance z. It is represented using the Fresnel approximation as
hz(x,y)=1jλzej2πλejπλz(x2+y2).
(2)
FIG. 2.

Illustration of the proposed imaging system.

FIG. 2.

Illustration of the proposed imaging system.

Close modal
For a 3D volume that is discretized as multiple slices [ϑ1, ϑ2, …, ϑL], assuming that there is no inter-object interference and any nonlinear terms are treated as noise (e.g., twin image and the DC term), the intensity pattern (hologram) captured by the image sensor is
I(x,y)=1L=1LO(x,y;z)2=1L=1Lhzϑ2.
(3)
Supposing that the hologram created at the sensor plane is discretized by the M unit space area, where the light exposure remains constant over an exposure time (M degree of freedom). It can be represented by a 1D vector (i.e., I = [i1, i2, …, iM]) through raster scanning, and each element is expressed using the matrix notation for a compact form as
im=[Hv]m,m[1,M],
(4)
where vCQ is a vector corresponding to the stack of discretized object light response in the 3D volume with Q = LM, each column of HCM×Q is a discretized representation of the impulse response kernel hz defined in Eq. (2) at distance z, and [⋅]m denotes the m-th element of a vector.
The QIS, which we use to record the hologram, is composed of an array of specialized photodetectors called jots.20 Because of the small jot size and fast temporal response, a QIS is regarded as an oversampling device in both spatial and temporal dimensions with sampling ratios Ks and KT, respectively.28,29 We use the overall upsampling factor K = KsKT for simplicity of notation. Therefore, the output of a QIS can be considered as a spatial–temporal cube with N = KM units. For a set of units responding to the m-th discrete hologram area with light exposure im, the accumulated light flux at a single unit is
φn=imK=[Hv]mK,formKn<(m+1)K.
(5)
Depending on the local values of φn, each jot collects a number of photons impinging on its surface, which is represented by Yn. The relationship between φn and photon count Yn is stochastic and is described by a Poisson–Gaussian distribution,28,
Yn=Poissonφn+ηdark+N0,σ2,
(6)
where ηdark is the dark current and σ is the standard deviation of the read noise. Since the dark noise of a QIS is negligible,30 the probability distribution of Yn is
pYn(y)=k=0φnkeφnk!×12πσ2e(yk)22σ2.
(7)
In a photosensitive device, the accumulated photons are converted to electrical signals through a quantization process. In a traditional sensor, the digital signals are generated from an A/D converter and composed of 8–14 bits. On the contrary, a QIS follows a binary (i.e., 1-bit) quantization scheme. Let q be the threshold, and the quantized binary output Bn at each unit is, then,
Bn=1,Ynq,0,Yn<q.
(8)
It is a Bernoulli random variable that has the probability distribution characterized by the random variable Yn, which is drawn from the distribution described in Eq. (7). The goal of reconstruction is to recover the object light response v from the binary measurements Bn, which is linked by the discretized light flux φn at the hologram plane. Thus, the probability distribution of Bn is first expressed as
Pr[Bn=1]=qpYn(y)dy
(9)
=qk=0φnkeφnk!×12πσ2e(yk)22σ2dy
(10)
=k=0φnkeφnk!Φkqσ,
(11)
where Φ is the cumulative density function of the standard normal distribution.
In this study, we investigate the system with a uniform quantization threshold q = 1. While the optimal threshold design is an emerged topic, which could potentially improve the performance of QIS devices, especially in high dynamic range imaging,31 we choose to follow a standard configuration for the sake of consistency and comparability with the existing literature.18–20 While this research direction is not included in this study, it is certainly worthy of further investigation. Previous computational methods of QIS signal processing focus on the Poisson component by ignoring the read noise.28,31 We follow the same assumption in this study. However, it is worth noting that we test our algorithms on the realistic model considering the influence of different read noise levels. In our implementation (during training), we assume that σ = 0. This gives us
Pr[Bn=1]=k=1φnkeφnk!=1eφn,
(12)
Pr[Bn=0]=1Pr[Bn=1]=eφn.
(13)
Then, given the discretized object light response for the 3D light field vCQ, following the mapping between v and φn in Eq. (5), the probability distribution at each sensing unit is further expressed as
Pr[Bn=1]=1e[Hv]mK,Pr[Bn=0]=e[Hv]mK,formKn<(m+1)K
(14)
where K is the overall sampling ratio as stated previously. Therefore, due to the identical and independent measurements at each sensing unit, a QIS outputs a signal b = [b1, b2, …, bN] with the joint distribution as
f(b|v)=n=1NPr[Bn=bn|v]=m=1Mk=1K(1bmK+k)e[Hv]mK+bmK+k1e[Hv]mK.
(15)
Since bn is a binary value, introducing two functions
Km1=k=1KbmK+k,Km0=Kk=1KbmK+k,
(16)
we can write
f(b|v)=m=1Me[Hv]mKKm01e[Hv]mKKm1,
(17)
which is the forward model for the entire imaging system.
To reconstruct the object wavefront information from the QIS recordings, the inverse problem approach is used, which aims to find the optimal object that would produce the observed output while satisfying some physical constraints. Concretely, given the forward measurement model f(b|v) in Eq. (17), the optimal object is estimated by the maximum a posteriori (MAP) solution of a Bayesian framework with the prior constraint p(v),
v*=argmaxvf(b|v)+μp(v),
(18)
where μ is used to control the relative effect of the regularizer on the reconstruction. Substituting Eq. (17) into Eq. (18) and taking the negative logarithm operation to transform the product term into summation, the MAP estimate solution of the proposed imaging modality is derived as
v*=argminvm=1MKm0[Hv]mKKm1log1e[Hv]mKμlogp(v).
(19)
Inspired by the previous work for Poisson denoising,25 the forward and prior terms are decoupled by introducing two independent auxiliary variables φ and α, which reformulate the original minimization problem into the constrained format,
v*=argminvF(φ)+μs(α)s.t.φ=Hv,α=v.
(20)
For notation simplicity, we define F(φ)=m=1M[Km0φmKKm1log(1eφmK)] and use s(α) to represent the task-oriented prior in an implicit form, which will be introduced later. The augmented Lagrangian function for this problem is given by
v*,φ*,α*,u1*,u2*=arg minv,φ,α,u1,u2F(φ)+μs(α)+ρ12φHv+u12ρ12u12+ρ22αv+u22ρ22u22,
(21)
where u1, u2 are the scaled Lagrangian multipliers and ρ1, ρ2 are the penalty parameters associated with two constraints in Eq. (20). The minimizer of Eq. (20) is the saddle point of Eq. (21),32 which is found by repeatedly performing the following alternating direction method of multipliers (ADMM)32 steps until convergence:
vt+1=argminvρ12φt+u1tHv2+ρ22αt+u2tv2,
(22)
φt+1=argminφF(φ)+ρ12φ(Hvt+1u1t)2,
(23)
αt+1=argminαμs(α)+ρ22α(vt+1u2t)2,
(24)
u1t+1=φt+1Hvt+1+u1t,
(25)
u2t+1=αt+1vt+1+u2t.
(26)
However, the requirement of hundreds to thousands of iterations to converge makes it time-consuming and not suitable for real-time imaging deployment. Moreover, the sensitivity of hand-tuned parameters, calibration errors, and reconstruction artifacts resulting from model mismatch degrades its performance in real-world tasks. Therefore, the one-time training scheme with automatic parameters updating is preferred, as featured by the data-driven methods. However, such methods suffer from less interpretability, which is essential in identifying the potential failures of an imaging system. Thus, to integrate the merits of the two approaches, we adapt the algorithm unrolling strategy to combine the model-based and data-driven methods.33–35 

The conceptual illustration of such a method is shown in Fig. 3(a). The iterative process (left) is unfolded into an end-to-end trainable network (right). Specifically, each updating step in the iteration is transferred into one layer that is cascaded in the deep network. Passing through the network is equivalent to executing the iterative algorithm a finite number of times. In this way, the algorithm parameters θ (such as the model parameters and regularization coefficients) are transferred to the trainable network parameters θ1, θ2, …, which are automatically tuned via back-propagation under supervised learning.

FIG. 3.

Network design of PSHoloNet. (a) Conceptual schematic of unrolling an iterative algorithm (left) with handcrafted tuned parameters into its corresponding unrolled network (right) with trainable parameters. (b) PSHoloNet (top) takes the snapshot QIS recordings as input, while the output is determined by different inference applications. The task-oriented operation design is indicated by the blocks filled with the strip pattern, which are demonstrated on two common DH inference tasks (bottom), i.e., 3D particle volumetric reconstruction and phase imaging.

FIG. 3.

Network design of PSHoloNet. (a) Conceptual schematic of unrolling an iterative algorithm (left) with handcrafted tuned parameters into its corresponding unrolled network (right) with trainable parameters. (b) PSHoloNet (top) takes the snapshot QIS recordings as input, while the output is determined by different inference applications. The task-oriented operation design is indicated by the blocks filled with the strip pattern, which are demonstrated on two common DH inference tasks (bottom), i.e., 3D particle volumetric reconstruction and phase imaging.

Close modal

Following this scheme, we design an end-to-end trainable network for the photon-starved holographic imaging system (PSHoloNet), which is the unrolled version of the iterative process in Eqs. (22)(26). Its backbone structure is shown in Fig. 3(b), which consists of several blocks. Each block contains three layers corresponding to the subproblems in Eqs. (22)(24), respectively. The network input is the snapshot QIS recordings, while the output differs for different inference applications.

As shown in Fig. 3(b), we use the closed-form expression for v-update that is defined in Eq. (22). It is a least-square estimate problem, which has a closed-form solution. In order to avoid matrix inversions, the fast Fourier transform is used for an efficient implementation as
vt+1=F1Fṽ1t+ρ2/ρ1F(h)̄Fṽ2t1+ρ2/ρ1|F(h)|2,
(27)
where F/F1 represents the forward/inverse discrete Fourier transform and h=[hz1,hz2,,hzl] is the concatenated propagation kernels at different depths defined in Eq. (2). For notation simplicity, we define ṽ1t=φt+u1t and ṽ2t=αt+u2t.
For φ-update described in Eq. (23), it is a proximal mapping of φ = [φ1, φ2, …, φN] associated with the negative likelihood for a truncated Poisson distribution. Defining φ̃t=Hvt+1u1t transforms Eq. (23) into
φt+1=argminφF(φ)+ρ12φφ̃t2=argminφm=1MKm0φmKKm1log1eφmK+ρ12φφ̃t2.
(28)
It is a separable convex optimization problem that could be solved without iteration.36 For the m-th pixel, the optimal is acquired by finding the root of the equation where the gradient is set to zero,
KeφmK1+ρ1(φmφ̃mt)=Km0+ρ1K(φmφ̃mt).
(29)

The α-update layer at the t-th block is the solution of Eq. (24), which is the proximal point of α̃t=vt+1u2t with respect to the prior term s(α). Therefore, it is customized to make PSHoloNet adaptive for various inference tasks.

For 3D particle volumetric reconstruction, the network output is the 3D location of the particles. The particle field is generally assumed to be real-valued with physically realistic sparsity properties in a linear basis.6,37 Therefore, the α-update is designed to introduce sparsity regularization. We consider a transform T() that sparsifies the layer input α̃t. To avoid handcrafted parameters for constructing such a sparsity basis, the convolutional neural network (CNN) blocks are used with their generalized approximation property.38 The sparsity of the transformed signal T(α̃t) is encouraged by the soft-threshold operation on the transformed domain. The inverse of T(), denoted by T1(), is then introduced to recover the signal back to its original signal domain. Moreover, it takes the same structure with T() for a consideration of structure symmetry. Such a symmetric constraint is further emphasized by incorporating it into the loss function during the network training (see Sec. II C). Therefore, αt can be efficiently computed as
αt=T1(soft(T(α̃t),θt)),
(30)
where θt is the shrinkage threshold and is a trainable parameter at the t-th block. The detailed structures are presented in the  Appendix.
For phase imaging, the output αt from the α-update layer at the t-th block can be interpreted as a denoising process of the layer input α̃t. Such an image-to-image registration can be rephrased as
αt=D(α̃t).
(31)
Considering its hourglass structure and prominent performance in various image reconstruction problems, we adapt the widely used U-Net structure with the residual connection.39 It is composed of a set of downsampling and upsampling operations, of which each has a skip connection between the resulting feature blocks of the same size. This reduces the semantic gap between the encoder and decoder features. The residual blocks are included within each operator, which aims to boost the network’s ability to overcome vanishing gradients. The detailed structures are presented in the  Appendix. To increase the network flexibility, the penalty parameters ρ1 and ρ2 are varied across different phases that are determined in one-time training.
For different interference tasks, the loss function is designed to consider the data fidelity and prior constraints simultaneously. The data fidelity penalty for each block is considered by minimizing the mean-square-error (MSE) loss function,
Ldt=αtvgt22,
(32)
where αt stands for the output of the t-th phase and vgt stands for the ground truth.
As for the prior constraint, in 3D particle volumetric reconstruction, the symmetry between the transform T() and its inverse T1() in the sparsity-inducing prior is enforced by
Lpt=T1(T(α̃t))α̃t22.
(33)
In phase imaging, since it is rephrased as an image-to-image reconstruction, the correlative metric, such as the negative Pearson correlation coefficient (NPCC),40 is included for preserving the global similarity between the output and its target equivalents, which is defined as
Lpt=covαt,vgtσαtσvgt,
(34)
where covαt,vgt is the covariance between the prediction αt at the t-th stage and the ground truth phase map vgt, and σαt and σvgt are their standard deviation, respectively.
Therefore, the total task-oriented loss function is the weighted sum of the data-fidelity penalty and task-oriented prior constraint,
Ltotal=1Tt=1T(Ldt+γLpt),
(35)
where T stands for the total number of phases in the unrolled network and γ is the regularization parameter. In our experiments, γ is empirically set as 0.01 and T = 5 considering the convergence speed and computational cost. In both applications, the network is implemented using Python, version 3.7.0, with the PyTorch framework, version 1.11.1. We use a single NVIDIA GeForce RTX 3070 GPU to accelerate the training process. The Adam optimizer41 is used with an initial learning rate of 0.0001, which can handle the sparse gradients and realize efficient computation.

We demonstrate photon-starved snapshot holography using a simple Gabor holographic setup equipped with a QIS as the recording device. To enable 3D particle volumetric reconstruction and phase imaging, we implement two variants of a modular network, which we term PSHoloNet, as described in Sec. II B. We first mathematically model the imaging system and investigate its performance via simulation. Then, we verify the system using experimental holograms of particle volumes and analyze the influence of read noise, which is unavoidable in real sensors. The results are presented in Secs. III A–III C.

A photon-starved condition with less than one photon per pixel is created by setting the mean rate of Poisson statistics of photon arrival to vary between 0 and 1. The QIS, which consists of an array of photodetectors called jots, detects photon arrivals following the “uniform-binning” scheme. This device is considered an oversampling mechanism with an overall ratio K, which is comprised of spatial oversampling Ks and temporal oversampling KT (refer to Sec. II A for more details). In this study, we set Ks = 4 and KT = 100, in accordance with the established practice.24,31 This configuration is equivalent to an integration time of 0.1 s under a 1 kHz frame rate.21 By utilizing this setup, we are able to obtain accurate and reliable measurements of photon arrivals with a larger field-of-view that supports a snapshot capture of the whole hologram without the mechanical scanning process.

We now consider a volume of 3D particles as the target object. The particles are assumed to be pure amplitude disks with a diameter of 20 µm, which are randomly spread in a volume with a distance of 5–7 cm from the hologram plane. It is discretized to 256 × 256 × 7 voxels, with an xy resolution of 10 µm. Following the image formation model of the QIS-DH system, the holograms generated on the sensor plane are normalized and set as the mean rate of the Poisson arrival, in accordance with the low-flux photon condition (<1ppp). We incorporate this physical model into an end-to-end training framework to construct the PSHoloNet, in order to detect the volumetric distribution of the 3D particles. We incorporate this physical model into an end-to-end training framework to construct the PSHoloNet, in order to detect the volumetric distribution of the 3D particles. The physical model provides a priori knowledge of the underlying system, which can be used to constrain the network and further improve its learning effectiveness that supports a small training dataset. Considering the complexity of the network and the size and diversity of the datasets, we generate 500 holograms and we follow the same typical data-splitting arrangement with a ratio of 7:2:1 for training, validation, and testing datasets. We train the network for 300 epochs and cost 4 h in total, where the loss function converges to a stable value and further training does not improve the performance of the network significantly. We find that it is sufficient to effectively train the network and achieve high accuracy in our tests. After the network training phase, which only needs to be done once, the trained PSHoloNet is capable of processing a single snapshot QIS recording in just 0.04 s.

We conduct a comparison of our system with a state-of-the-art method called OSNet,7 which does not take into account the physical model. The visual results are given in Fig. 4, where the green circles are the ground truth and the red triangles represent the prediction results from the tested algorithms. As expected, under sufficient illumination where the interference patterns are recorded with high visibility [Fig. 4(b)], OSNet succeeds in restoring the 3D particles distribution in Fig. 4(e). However, under low light [Fig. 4(c)], the captured hologram is heavily corrupted by the intrinsic read noise in the sensor (i.e., σ = 2e), which results in the barely recognizable patterns and, in turn, a poor performance of OSNet [Fig. 4(f)]. In contrast, the binary recording nature of QIS leads to a negligible amount of read noise, and the dominant factor affecting fringes quality is the randomness of the photon arrivals [Fig. 4(d)]. By carefully taking this into account, PSHoloNet successfully reconstructs the particle volume as shown in Fig. 4(g). Numerically, it has a small position error of only 1.26 voxels, compared to the very significant errors shown in Fig. 4(f).

FIG. 4.

Reconstruction result comparison of 3D particle volume using different imaging systems. Green circles: The ground truth of particle distribution. Red triangles: The prediction results from the tested algorithms. (a) Ground truth of the particle distribution. The reconstruction results from OSNet7 with holograms under (b) sufficient illumination and (c) photon-starved condition are shown in (e) and (f), respectively; the reconstruction result from PSHoloNet with (d) QIS-recorded hologram under photon-starved condition is shown in (g).

FIG. 4.

Reconstruction result comparison of 3D particle volume using different imaging systems. Green circles: The ground truth of particle distribution. Red triangles: The prediction results from the tested algorithms. (a) Ground truth of the particle distribution. The reconstruction results from OSNet7 with holograms under (b) sufficient illumination and (c) photon-starved condition are shown in (e) and (f), respectively; the reconstruction result from PSHoloNet with (d) QIS-recorded hologram under photon-starved condition is shown in (g).

Close modal
In order to further investigate the robustness of our system against particle density, the trained network is tested under various particle concentrations ranging from 1×104 to 6 × 10−4 particles per volume (ppv). Since the target distribution (ground truth) is available in the simulation, the quantitative evaluation can be made by considering two common metrics: (1) recall, which is calculated as
R=NtNt+Nm,
(36)
where Nt is the number of correct predictions and Nm is the number of missed ground truths; (2) precision, which is calculated as
P=NtNt+Nf,
(37)
where Nf is the number of wrong inferences.

The reconstruction results presented in Fig. 5 and Table I provide compelling evidence for the robustness of PSHoloNet in the face of particle density variation. In the 3D plot shown in Fig. 5, because the number of wrong and missed predictions is relatively low compared to the number of correct predictions, they may be obscured by the large number of correct detections in the plot and hardly discernible. Therefore, we use the precision and recall metrics in Table I to take into account the number of both correct and incorrect predictions quantitatively made by our system. As indicated, our system is able to accurately identify particles with a high degree of confidence. In addition, despite a sixfold increase in the number of particles from 50 to 300, PSHoloNet continues to deliver accurate predictions of particle locations. Across different density levels, PSHoloNet achieves an average recall of 93.6% and a precision of 96.73%. In order to make a quantitative assessment of the particle location prediction, we follow the common practice in object detection tasks and calculate the deviation. First, a buffer area is pre-defined around each ground-truth particle. Then, the predicted particle from the network that falls within this area is considered a matched particle, and the difference between their coordinates is taken as the prediction error in spatial distance. In our study, the buffer area is defined as a sphere with a five-voxel radius, and the results are shown in Table I. Specifically, for the 300-particle case, which is the densest situation, the maximum prediction error in correctly matched particle pairs is 24.4 µm along the xy-plane and 0.137 mm along the z axis. These values account for only 0.697% and 0.685% of the total range, respectively.

FIG. 5.

Reconstruction of 3D particle volume at different densities. The green, red, and purple symbols represent the correct, wrong, and missed predictions, respectively.

FIG. 5.

Reconstruction of 3D particle volume at different densities. The green, red, and purple symbols represent the correct, wrong, and missed predictions, respectively.

Close modal
TABLE I.

Evaluation results for particle extraction metrics and prediction errors at different particle densities.

Particle density (ppv)Recall (%)Precision (%)XY error (μm)Z error (mm)
1 × 10−4 99.6 98.5 23.3 0.031 
2 × 10−4 95.3 98.2 23.4 0.039 
4 × 10−4 87.2 99.5 23.6 0.085 
6 × 10−4 92.3 90.7 24.4 0.137 
Particle density (ppv)Recall (%)Precision (%)XY error (μm)Z error (mm)
1 × 10−4 99.6 98.5 23.3 0.031 
2 × 10−4 95.3 98.2 23.4 0.039 
4 × 10−4 87.2 99.5 23.6 0.085 
6 × 10−4 92.3 90.7 24.4 0.137 

We investigate the application of the proposed network in photon-starved phase imaging by replacing the object field with pure phase objects. They are simulated by mapping the intensity patterns from some public natural images42 to a phase modulation ranging from −π to π, as shown in Fig. 6(a). The transmissive phase object is considered at a distance of 2 cm in front of the sensor plane with the same configuration for the illumination source and QIS. We simulate 300 holograms and follow the same training strategy as Sec. III A.

FIG. 6.

Phase imaging comparison between the traditional DH system and the proposed system under the photon-starved situation. (a) The ground-truth phase images from Ref. 42. (b) The holograms recorded by the traditional sensor. We assume that the photon level is <1ppp. The sensor has a read noise of σ = 2e and operates at 8 bits. Shown in this figure is one image frame. (c) The reconstruction results from DCOD.43 (c) The holograms recorded by QIS. We use K = 400 frames, where each frame is binary and the read noise is σ = 0.19e following Ref. 44. (d) The reconstruction result from PSHoloNet.

FIG. 6.

Phase imaging comparison between the traditional DH system and the proposed system under the photon-starved situation. (a) The ground-truth phase images from Ref. 42. (b) The holograms recorded by the traditional sensor. We assume that the photon level is <1ppp. The sensor has a read noise of σ = 2e and operates at 8 bits. Shown in this figure is one image frame. (c) The reconstruction results from DCOD.43 (c) The holograms recorded by QIS. We use K = 400 frames, where each frame is binary and the read noise is σ = 0.19e following Ref. 44. (d) The reconstruction result from PSHoloNet.

Close modal

Figures 6(b) and 6(c) show examples of phase imaging under photon-starved situations in a conventional DH system. Again, the randomness of photon arrivals and intrinsic read-noise result in highly contaminated interference patterns. Under this situation, even a state-of-the-art iterative reconstruction algorithm called deep compressed object decoder (DCOD)43 fails to recover the phase information. On the other hand, considerable phase information is successfully captured in the QIS-recorded holograms, and PSHoloNet is able to recover the phase images.

Such adaptability of this system is achieved by designing and incorporating task-oriented priors and loss functions. In previous 3D particle volumetric reconstruction tasks, since the particle field is inherently real-valued due to the relatively small particle size (micro-/nano-scale), the use of mean-square-error (MSE) loss is straightforward with fast convergence and efficiency. However, for phase imaging, since there are much richer spatial (or spectral) details of the object, the selection of loss function has a significant influence on the reconstruction performance. To investigate such an effect, we further train PSHoloNet separately with the same synthetic phase object dataset while using different loss functions for the back-propagation and parameter updating. From the visual inspection shown in Fig. 7, undesired checkerboard artifacts are observed in the output images from the network trained purely by MSE loss. It reveals the insufficiency of retrieving the phase maps only from the pixel-wise comparison. On the contrary, the negative Pearson correlation (NPCC) loss preserves the spatial correlations between the neighbor pixels, which increases the structural similarity between the network prediction and the target feature map. As shown in Fig. 7, it leads to better reconstructed phase maps with more informative and distinguishable features. To quantify this comparison, we plot the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) values between the reconstructed results and their ground-truth equivalents along the training process in Fig. 7(b). As indicated, only using MSE as the loss function results in poor structure recovery (an SSIM of 0.1). By introducing the NPCC loss, the more structural similarity is preserved in the predicted phase maps with a three times improvement (an SSIM of 0.32).

FIG. 7.

Impact of loss functions. The condition of NPCC and MSE offers the best performance. (a) Reconstruction results from PSHoloNet trained with mean-square-error loss (PSHoloNet-M), negative Pearson correlation coefficient loss (PSHoloNet-N), and their combination (PSHoloNet-NM). (b) The SSIM and PSNR curves for each case, respectively.

FIG. 7.

Impact of loss functions. The condition of NPCC and MSE offers the best performance. (a) Reconstruction results from PSHoloNet trained with mean-square-error loss (PSHoloNet-M), negative Pearson correlation coefficient loss (PSHoloNet-N), and their combination (PSHoloNet-NM). (b) The SSIM and PSNR curves for each case, respectively.

Close modal

We now proceed to verify the above experimentally. To prepare the particle volume sample for our experiment, we disperse 50 µm polystyrene particles in a physiological saline solution, which covers a distance ranging from 10 to 60 mm from the sensor plane. This involves thoroughly mixing the plastic particles with the saline solution until the particles are evenly distributed throughout the solution. Due to the similar density of the particles and the liquid, the particles diffuse in the solution and exhibit Brownian motion. This sample preparation method allows us to study the behavior and trajectories of the plastic particles from consecutive holograms. The illumination source is a 660 nm laser that is collimated to create the planar wave. The photon level is controlled by inserting a neutral density filter between the light source and the sample. A Sony IMX264 CMOS sensor with a pixel pitch of 3.45 µm is used to record the holograms, and computationally converted to QIS data, following the same mathematical modeling in Sec. III A that has been experimentally verified extensively in the literature.20,26,30

Since there is no ground truth, the inference using the state-of-the-art OSNet hologram reconstruction method with the ideal imaging condition is used as the reference for performance evaluation. The prediction result example is shown in Fig. 8, with an 82.6% recall and an 88.1% precision on average. The slight decrease in accuracy levels compared with the simulation is acceptable due to the inevitable misalignment errors in optical components in the real experimental setup. Based on the volumetric reconstruction on consecutive frames, the particle tracking is conducted to get the trajectories as shown in Fig. 8(e). The reported mean lateral and axial errors are 11.2 µm and 0.137 mm, respectively. The precision is competitive with that achieved by OSNet under the ideal lighting condition.

FIG. 8.

Particle volumetric reconstruction on the experimental holograms. The visual inspection is provided with (a) a 3D view, (b) a top view, and (c) a side view. Green circles: The inference results under the photon-starved condition using PSHoloNet. Red triangles: The inference results under the ideal light condition using the previous reconstruction method (OSNet).7 (d) The schematic illustration of the experimental setup that we used in our experiment. ND: neutral density filter. CL: collimating lens. (e) Trajectories of the selected particles. The color of the trajectories indicates the frame index.

FIG. 8.

Particle volumetric reconstruction on the experimental holograms. The visual inspection is provided with (a) a 3D view, (b) a top view, and (c) a side view. Green circles: The inference results under the photon-starved condition using PSHoloNet. Red triangles: The inference results under the ideal light condition using the previous reconstruction method (OSNet).7 (d) The schematic illustration of the experimental setup that we used in our experiment. ND: neutral density filter. CL: collimating lens. (e) Trajectories of the selected particles. The color of the trajectories indicates the frame index.

Close modal

In our results reported so far, we have verified the feasibility of the proposed QIS-DH system to recover wavefront information under photon-starved scenarios. We study the possibility of snapshot DH at an extremely low signal budget (<1ppp). Under such conditions, the signal is so small that the Poisson random variable dominates, which is highly contaminated by the intrinsic read-out noise in traditional sensors. In contrast, QIS has a prominent reduction in read noise and provides more faithful access for the Poisson randomness of the photon arrival. However, it still cannot be free of it since there is no perfect sensor.

To investigate the influence of read noise on our system, we simulate QIS-recorded holograms for particle volumes under various levels of read noise. The mean photon flux is kept below 1 ppp, and the read noise variance σ increases from 0.1e to 2e. The quantitative metrics (i.e., recall and precision) are shown in Fig. 9 in the way of a box-and-whisker plot. For metric at each noise level, the box extends from the Q1 to Q3 quartile values with a line at the median (Q2) and the whiskers show their range. The observation is that regardless of the read noise level (at least for σ ≤ 1e), our system demonstrates robust and competitive prediction performance both in recall and precision (over 95%). As the noise level goes up, the number of missed particles increases, which results in a decreased recall (average of 18% under σ = 2e). Its dispersion also shows the unsteady performance for particle volumetric reconstruction under the photon-starved situation when the read noise is too large as is the case in the traditional sensors. On the contrary, a QIS has a relatively small value of σ = 0.19e, which is sufficient to guarantee its insensitivity to read noise.44 

FIG. 9.

Boxplot for particle volumetric reconstruction metrics under different noise levels. At each noise level, recall and precision results are depicted in the way of blue and orange box-and-whisker plots, respectively. The box extends from the Q1 to Q3 quartile values with a line at the median (Q2). The whiskers show the range of the metric.

FIG. 9.

Boxplot for particle volumetric reconstruction metrics under different noise levels. At each noise level, recall and precision results are depicted in the way of blue and orange box-and-whisker plots, respectively. The box extends from the Q1 to Q3 quartile values with a line at the median (Q2). The whiskers show the range of the metric.

Close modal

In this study, we demonstrate the integration of QIS in developing a snapshot photon-starved DH system. Enabled by the physics-informed PSHoloNet, our approach enables 3D particle volumetric reconstruction and phase imaging under a limited photon budget by single-bit holograms. This facilitates more advanced holography applications in situations when the scarcity of photons is essential and unavoidable, which has numerous applications in fields such as microscopy, biomedical imaging, and materials science. In addition, we have shown that QIS can be used as an effective imaging modality in scientific and life science imaging applications. Our findings open up new possibilities for QIS in this area, and we anticipate that it will encourage new research initiatives and collaborations, leading to the development of more advanced and efficient imaging techniques that can benefit the scientific community.

This work was supported by the Research Grants Council of Hong Kong (Grant Nos. GRF 17200019, GRF 17201620, and GRF 17201822).

The authors would like to thank Dr. Ni Chen (University of Arizona, USA) for her comments on an early version of this project and suggestions on the experimental setup.

The authors have no conflicts to disclose.

Yunping Zhang: Conceptualization (lead); Data curation (lead); Formal analysis (lead); Methodology (lead); Software (lead); Validation (lead); Visualization (lead); Writing – original draft (lead); Writing – review & editing (equal). Stanley H. Chan: Methodology (supporting); Writing – review & editing (supporting). Edmund Y. Lam: Conceptualization (supporting); Funding acquisition (lead); Supervision (lead); Writing – review & editing (equal).

The data that support the findings of this study are available from the corresponding author upon reasonable request.

As introduced in Sec. II, α-update can be treated as a Gaussian denoiser for data with prior information, which are task-oriented and integrated by using the powerful representation of CNNs with different structures. Different structures used in this project are presented in the following.

1. α-update for the reconstruction of 3D particles distribution

As shown in Fig. 10, the sparsity regularization is performed by the soft-threshold operation on the transformed domain. The CNN blocks are used with their generalized approximation property to represent the non-linear transform T().38 The symmetric inverse transform T1() from the sparsity domain is designed as the same structure. Specifically, each CNN block contains consecutive convolutional layers separated by a batch normalization layer and leaky rectified linear (ReLU) activation layer, which is a widely used structure with promising performance in various vision tasks.6,38,42 Specifically, 3 × 3 convolutional layers are used with the same volume depth. A skip connection is made from the input directly to the final activation layer. Overall, the update process in Eq. (31) is rephrased as
αt=T1(soft(T(α̃t),θt)),
(A1)
where α̃t=vt+1u2t and θt is the shrinkage threshold, which is a trainable parameter at the t-th phase.
FIG. 10.

α-update block for particle volumetric reconstruction task.

FIG. 10.

α-update block for particle volumetric reconstruction task.

Close modal

2. α-update for the reconstruction of the phase map

Figure 11 shows the architecture details of the task-oriented prior in the phase map reconstruction. The input of the α-update block is the revised signal α̃t, and the output is the phase prediction αt.

FIG. 11.

α-update block for phase imaging task.

FIG. 11.

α-update block for phase imaging task.

Close modal

In alignment with the typical setting of U-Net, four sets of downsampling and upsampling operations are included, of which each has a skip connection between the resulting feature blocks of the same size. The number of channels for each scale is set as (64, 128, 256, 512), respectively. The residual blocks are included in the downsampling and upsampling operators, of which each is composed of a Conv–ReLU–Conv unit with an extra adding operation from its input to its output.

1.
U.
Schnars
,
C.
Falldorf
,
J.
Watson
, and
W.
Jüptner
, “
Digital holography
,”
Digital Holography and Wavefront Sensing
(
Springer
,
2015
), pp.
39
68
.
2.
Y.
Wu
and
A.
Ozcan
, “
Lensless digital holographic microscopy and its applications in biomedicine and environmental monitoring
,”
Methods
136
,
4
16
(
2018
).
3.
B.
Kemper
and
G.
Von Bally
, “
Digital holographic microscopy for live cell applications and technical inspection
,”
Appl. Opt.
47
,
A52
A61
(
2008
).
4.
V.
Bianco
,
P.
Memmolo
,
P.
Carcagnì
,
F.
Merola
,
M.
Paturzo
,
C.
Distante
, and
P.
Ferraro
, “
Microplastic identification via holographic imaging and machine learning
,”
Adv. Intell. Syst.
2
,
1900153
(
2020
).
5.
Y.
Zhu
,
C.
Hang Yeung
, and
E. Y.
Lam
, “
Microplastic pollution monitoring with holographic classification and deep learning
,”
J. Phys.: Photonics
3
,
024013
(
2021
).
6.
N.
Chen
,
C.
Wang
, and
W.
Heidrich
, “
Holographic 3D particle imaging with model-based deep network
,”
IEEE Trans. Comput. Imaging
7
,
288
296
(
2021
).
7.
Y.
Zhang
,
Y.
Zhu
, and
E. Y.
Lam
, “
Holographic 3D particle reconstruction using a one-stage network
,”
Appl. Opt.
61
,
B111
B120
(
2022
).
8.
S.
Wäldchen
,
J.
Lehmann
,
T.
Klein
,
S.
Van De Linde
, and
M.
Sauer
, “
Light-induced cell damage in live-cell super-resolution microscopy
,”
Sci. Rep.
5
,
15348
(
2015
).
9.
F.
Saglimbeni
,
S.
Bianchi
,
A.
Lepore
, and
R.
Di Leonardo
, “
Three-axis digital holographic microscopy for high speed volumetric imaging
,”
Opt. Express
22
,
13710
13718
(
2014
).
10.
P. A.
Cheremkhin
,
N. N.
Evtikhiev
,
V. V.
Krasnov
,
V. G.
Rodin
, and
R. S.
Starikov
, “
Shot noise and fixed-pattern noise effects on digital hologram reconstruction
,”
Opt. Lasers Eng.
139
,
106461
(
2021
).
11.
B.
Mirecki
,
M.
Rogalski
,
P.
Arcab
,
P.
Rogujski
,
L.
Stanaszek
,
M.
Józwik
, and
M.
Trusiak
, “
Low-intensity illumination for lensless digital holographic microscopy with minimized sample interaction
,”
Biomed. Opt. Express
13
,
5667
5682
(
2022
).
12.
K.
Inoue
,
A.
Anand
, and
M.
Cho
, “
Angular spectrum matching for digital holographic microscopy under extremely low light conditions
,”
Opt. Lett.
46
,
1470
1473
(
2021
).
13.
Z.
Zhang
,
Y.
Zheng
,
T.
Xu
,
A.
Upadhya
,
Y. J.
Lim
,
A.
Mathews
,
L.
Xie
, and
W. M.
Lee
, “
Holo-UNet: Hologram-to-hologram neural network restoration for high fidelity low light quantitative phase imaging of live cells
,”
Biomed. Opt. Express
11
,
5478
5487
(
2020
).
14.
P. A.
Morris
,
R. S.
Aspden
,
J. E.
Bell
,
R. W.
Boyd
, and
M. J.
Padgett
, “
Imaging with a small number of photons
,”
Nat. Commun.
6
,
5913
(
2015
).
15.
M.
Yamamoto
,
H.
Yamamoto
, and
Y.
Hayasaki
, “
Photon-counting digital holography under ultraweak illumination
,”
Opt. Lett.
34
,
1081
1083
(
2009
).
16.
N.
Demoli
,
H.
Skenderović
, and
M.
Stipčević
, “
Digital holography at light levels below noise using a photon-counting approach
,”
Opt. Lett.
39
,
5010
5013
(
2014
).
17.
I.
Kang
,
F.
Zhang
, and
G.
Barbastathis
, “
Phase extraction neural network (PhENN) with coherent modulation imaging (CMI) for phase retrieval at low photon counts
,”
Opt. Express
28
,
21578
21600
(
2020
).
18.
J.
Ma
and
E. R.
Fossum
, “
Quanta image sensor jot with sub 0.3e- r.m.s. read noise and photon counting capability
,”
IEEE Electron Device Lett.
36
,
926
928
(
2015
).
19.
Z.
Yin
,
J.
Ma
,
S.
Masoodian
, and
E. R.
Fossum
, “
Threshold uniformity improvement in 1b quanta image sensor readout circuit
,”
Sensors
22
,
2578
(
2022
).
20.
E. R.
Fossum
, “
The quanta image sensor (QIS): Concepts and challenges
,” in
Imaging Systems and Applications
(
Optical Society of America
,
2011
), p.
JTuE1
.
21.
K.
Morimoto
,
A.
Ardelean
,
M.-L.
Wu
,
A. C.
Ulku
,
I. M.
Antolovic
,
C.
Bruschini
, and
E.
Charbon
, “
Megapixel time-gated SPAD image sensor for 2D and 3D imaging applications
,”
Optica
7
,
346
354
(
2020
).
22.
T.
Al Abbas
,
N.
Dutton
,
O.
Almer
,
S.
Pellegrini
,
Y.
Henrion
, and
R.
Henderson
, “
Backside illuminated SPAD image sensor with 7.83 μm pitch in 3D-stacked CMOS technology
,” in
2016 IEEE International Electron Devices Meeting
(
IEEE
,
2016
), pp.
8
-
1
.
23.
P.
Caramazza
,
A.
Boccolini
,
D.
Buschek
,
M.
Hullin
,
C. F.
Higham
,
R.
Henderson
,
R.
Murray-Smith
, and
D.
Faccio
, “
Neural network identification of people hidden from view with a single-pixel, single-photon detector
,”
Sci. Rep.
8
,
11945
(
2018
).
24.
A.
Gnanasambandam
and
S. H.
Chan
, “
HDR imaging with quanta image sensors: Theoretical limits and optimal reconstruction
,”
IEEE Trans. Comput. Imaging
6
,
1571
1585
(
2020
).
25.
Y.
Sanghvi
,
A.
Gnanasambandam
, and
S. H.
Chan
, “
Photon limited non-blind deblurring using algorithm unrolling
,”
IEEE Trans. Comput. Imaging
8
,
851
864
(
2022
).
26.
J.
Ma
,
S.
Chan
, and
E. R.
Fossum
, “
Review of quanta image sensors for ultralow-light imaging
,”
IEEE Trans. Electron Devices
69
,
2824
2839
(
2022
).
27.
J. W.
Goodman
,
Introduction to Fourier Optics
,
4th ed.
(
Roberts and Company Publishers
,
2017
).
28.
F.
Yang
,
Y. M.
Lu
,
L.
Sbaiz
, and
M.
Vetterli
, “
Bits from photons: Oversampled image acquisition using binary Poisson statistics
,”
IEEE Trans. Image Process.
21
,
1421
1436
(
2011
).
29.
L.
Sbaiz
,
F.
Yang
,
E.
Charbon
,
S.
Susstrunk
, and
M.
Vetterli
, “
The gigavision camera
,” in
2009 IEEE International Conference on Acoustics, Speech and Signal Processing
(
IEEE
,
2009
), pp.
1093
1096
.
30.
E. R.
Fossum
,
J.
Ma
,
S.
Masoodian
,
L.
Anzagira
, and
R.
Zizza
, “
The quanta image sensor: Every photon counts
,”
Sensors
16
,
1260
(
2016
).
31.
O. A.
Elgendy
and
S. H.
Chan
, “
Optimal threshold design for quanta image sensor
,”
IEEE Trans. Comput. Imaging
4
,
99
111
(
2017
).
32.
S.
Boyd
,
N.
Parikh
,
E.
Chu
,
B.
Peleato
,
J.
Eckstein
et al, “
Distributed optimization and statistical learning via the alternating direction method of multipliers
,”
Foundations and Trends in Machine Learning
3
,
1
122
(
2011
).
33.
V.
Monga
,
Y.
Li
, and
Y. C.
Eldar
, “
Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing
,”
IEEE Signal Process. Mag.
38
,
18
44
(
2021
).
34.
K.
Monakhova
,
J.
Yurtsever
,
G.
Kuo
,
N.
Antipa
,
K.
Yanny
, and
L.
Waller
, “
Learned reconstructions for practical mask-based lensless imaging
,”
Opt. Express
27
,
28075
28090
(
2019
).
35.
J.
Sun
,
H.
Li
,
Z.
Xu
et al, “
Deep ADMM-Net for compressive sensing MRI
,”
Advances in Neural Information Processing Systems
29
,
1
9
(
2016
).
36.
S. H.
Chan
,
X.
Wang
, and
O. A.
Elgendy
, “
Plug-and-play ADMM for image restoration: Fixed-point convergence and applications
,”
IEEE Trans. Comput. Imaging
3
,
84
98
(
2016
).
37.
L.
Denis
,
D.
Lorenz
,
E.
Thiébaut
,
C.
Fournier
, and
D.
Trede
, “
Inline hologram reconstruction with sparsity constraints
,”
Opt. Lett.
34
,
3475
3477
(
2009
).
38.
J.
Zhang
and
B.
Ghanem
, “
ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing
,” in
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(
IEEE
,
2018
), pp.
1828
1837
.
39.
K.
Zhang
,
L. V.
Gool
, and
R.
Timofte
, “
Deep unfolding network for image super-resolution
,” in
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
(
IEEE
,
2020
), pp.
3217
3226
.
40.
S.
Li
,
M.
Deng
,
J.
Lee
,
A.
Sinha
, and
G.
Barbastathis
, “
Imaging through glass diffusers using densely connected convolutional networks
,”
Optica
5
,
803
813
(
2018
).
41.
D. P.
Kingma
and
J.
Ba
, “
Adam: A method for stochastic optimization
,” in
International Conference on Learning Representations
,
2015
.
42.
J.
Deng
,
W.
Dong
,
R.
Socher
,
L.-J.
Li
,
K.
Li
, and
L.
Fei-Fei
, “
ImageNet: A large-scale hierarchical image database
,” in
2009 IEEE Conference on Computer Vision and Pattern Recognition
(
IEEE
,
2009
), pp.
248
255
.
43.
F.
Niknam
,
H.
Qazvini
, and
H.
Latifi
, “
Holographic optical field recovery using a regularized untrained deep decoder network
,”
Sci. Rep.
11
(
10903
),
1
13
(
2021
).
44.
J.
Ma
,
D.
Zhang
,
O. A.
Elgendy
, and
S.
Masoodian
, “
A 0.19e rms read noise 16.7 Mpixel stacked quanta image sensor with 1.1 μm-pitch backside illuminated pixels
,”
IEEE Electron Device Lett.
42
,
891
894
(
2021
).