We report a dual-modality ghost diffraction (GD) system to simultaneously enable high-fidelity data transmission and high-resolution object reconstruction through complex disordered media using an untrained neural network (UNN) with only one set of realizations. The pixels of a 2D image to be transmitted are sequentially encoded into a series of random amplitude-only patterns using a UNN without labels and datasets. The series of random patterns generated is sequentially displayed to interact with an object placed in a designed optical system through complex disordered media. The realizations recorded at the receiving end are used to retrieve the transmitted data and reconstruct the object at the same time. The experimental results demonstrate that the proposed dual-modality GD system can robustly enable high-fidelity data transmission and high-resolution object reconstruction in a complex disordered environment. This could be a promising step toward the development of AI-driven compact optical systems with multiple modalities through complex disordered media.
I. INTRODUCTION
Optical modulation through disordered media has become an active research topic1–5 and has various applications in biomedical and astronomy fields.6,7 The main challenge is that the disordered media are inhomogeneous and variable and corrupt effective information in the wave propagation path. Some approaches to addressing this challenge have been developed,1,4,8–10 e.g., phase conjugation,8 memory effect,9 and shower-curtain effect.10 In previous studies, pixelated arrays were usually used for intensity detection, but it could be difficult (or even not available) to implement in some applications, e.g., non-visible bandwidths or low light levels.
Recently, ghost diffraction (GD) with structured illumination and a single-pixel detector11–19 has emerged as an easy-to-implement approach. The GD was initially realized with entangled photons generated by spontaneous parametric downconversion in quantum domain.11,12 Subsequently, experiments of GD with pseudo-thermal light were realized, which promoted its development in classical domain.13,14 Optical information can be retrieved based on the second-order correlation between a series of illumination patterns and the realizations collected by using a single-pixel detector.15 Advanced algorithms, e.g., differential,20 normalized,21 and compressed sensing,22,23 have been developed to enhance the signal-to-noise ratio of ghost images. Furthermore, deep learning24–26 has been applied at low samplings and could perform properly. However, training-based deep learning requires a large dataset for optimization and could lack the generalization capability. To get rid of dataset constraints, an untrained neural network (UNN)27 was introduced28,29 and the GD could have a comparable performance by incorporating a physical model into neural networks. Although the GD is promising in real-world scenarios30–36 (e.g., microscopy33 and communication34), the current studies focus on developing one modality, and it is difficult to integrate dual or multiple modalities into one optical system, especially in complex disordered environments. It is desirable to explore an integrated GD system to enable multiple modalities in complex disordered environments.
Here, we report a dual-modality GD system to simultaneously enable high-fidelity optical data transmission and high-resolution ghost reconstruction through complex disordered media. The UNN is first designed to sequentially encode the pixels of a 2D image (to be optically transmitted) into a series of random amplitude-only patterns, and the zero-frequency component of the spectrum is designed to be proportional to each pixel of the 2D image. The series of generated random patterns is sequentially embedded into a spatial light modulator (SLM) in a designed optical system. Optical wave modulated by the generated random patterns illuminates an object, and a single-pixel detector is used to record a series of light intensities. High-fidelity optical data retrieval can be directly realized by using the realizations, and a high-resolution object is also recovered, which is enhanced by using block-matching and 3D filtering (BM3D) and UNN, regularized by an explicit denoiser (UNN-RED). A series of optical experiments are conducted in complex disordered environments, and the experimental results verify effectiveness and robustness of the designed dual-modality GD system.
II. METHODS
After the optimization, the zero-frequency component of the Fourier spectrum of pattern Pi can be proportional to the pixel Gi. The above-mentioned process is repeated until all pixels of the 2D image are encoded into random amplitude-only patterns.
Since the probability distributions of random patterns generated by the UNN are inconsistent with the light source,38 the transmission quality could be significantly affected. Furthermore, noise could be inevitably induced by complex disordered media, preventing a direct application of the generated patterns. To overcome this challenge, a strategy is further designed, as shown in Fig. 1(b). To align probability distributions of the generated patterns with Gaussian, a zero-mean Gaussian image is used to be superimposed with each generated pattern. Therefore, probability distribution of the pattern can be modified without affecting its original zero-frequency component in a Fourier domain. A differential approach is also employed to suppress noise. Each pattern is further divided, i.e., (1 + P)/2 and (1 − P)/2. Finally, a shuffle operation is applied to produce randomized illumination patterns.
A schematic experimental setup for the proposed dual-modality GD system is shown in Fig. 2(a). A green laser (MGI-III-532-50 mW) with a wavelength of 532.0 nm and a peak output power of 50.0 mW is used. The laser beam is expanded by using an objective lens with a 40× magnification and then collimated. The collimated beam is reflected by a mirror and then illuminates the generated patterns embedded into SLM (Holoeye, LC-R720) with a pixel pitch of 20.0 μm. A 4f system is designed to project the patterns onto an object, e.g., USAF 1951 resolution target. The lenses L1 and L2 in the 4f system have a focal length of 50.0 mm. A water tank with a dimension of 100.0 mm (length), 200.0 mm (width), and 300.0 mm (height) is placed in the optical path and is filled with 4000 ml of clean water. To create a dynamic disordered environment, 15 ml skimmed milk diluted with 1000 ml clean water is kept dropping into water tank. A rotator is used to keep operating at 600.0 revolutions per minute (rpm) to create dynamic scattering. Only one set of realizations is recorded by using a single-pixel silicon photodiode (Thorlabs, PDA100A2).
III. RESULTS AND DISCUSSION
A. Different 2D images to be optically transmitted
To verify the developed dual-modality GD system, different 2D grayscale images are optically transmitted in Fig. 2(a) and are individually encoded into a series of random amplitude-only patterns using the designed UNN. The generated patterns are sequentially displayed by SLM to illuminate an object (i.e., USAF 1951 resolution target) through disordered media. Figure 3 shows the experimental results in the proposed dual-modality GD system through static and dynamic disordered media, respectively. In static disordered environments, the experimentally retrieved data are shown in Figs. 3(a)–3(d), and the reconstructed ghost images are shown in Figs. 3(e)–3(h). Quality of the experimentally retrieved images is quantitatively evaluated by using peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM).46 As shown in Figs. 3(a)–3(d), the retrieved data have high PSNR and high SSIM, and it is demonstrated that the proposed dual-modality GD system enables high-fidelity image transmission.
To illustrate the quality of the retrieved images, the pixels along the 30th row of the retrieved images shown in Figs. 3(a) and 3(b) are shown in Figs. 4(a) and 4(b), respectively. It is demonstrated that the experimentally retrieved data overlap with the original data. PSNR values of Figs. 4(a) and 4(b) are 39.75 and 39.39 dB, respectively. MSE values are 1.06 × 10−4 and 1.15 × 10−4, respectively. The high PSNR and low MSE demonstrate that the proposed optical system is feasible and robust for optically transmitting 2D grayscale images. Contrast-to-noise ratio (CNR)47–49 is calculated to evaluate quality of the reconstructed ghost images. In Figs. 3(e)–3(h), the reconstructed ghost images have high CNR. It is demonstrated that the proposed dual-modality GD system can reconstruct a high-quality object at the same time. Here, element 5 in Group 3 is the finest resolvable feature, and high spatial resolution of 78.74 μm is achieved. The experimental results shown in Figs. 3(a)–3(h) demonstrate that the proposed dual-modality GD system can simultaneously realize high-fidelity optical data transmission and high-resolution object reconstruction in static disordered environments.
When optical experiments in dynamic disordered environments are conducted, optical transmission and imaging results are shown in Figs. 3(i)–3(p). In Figs. 3(i)–3(l), the experimentally retrieved 2D images are of high fidelity. Typical comparisons using the retrieved data at the 30th row of Figs. 3(k) and 3(l) are shown in Figs. 4(c) and 4(d). PSNR of the data in Figs. 4(c) and 4(d) is 40.87 and 40.31 dB, respectively. MSE values are 8.19 × 10−5 and 9.31 × 10−5, respectively. It is experimentally demonstrated that the retrieved data are in accordance with the original data. The reconstructed ghost images are shown in Figs. 3(m)–3(p), and element 5 in Group 3 is well-resolved. It is demonstrated that high spatial resolution of 78.74 μm is achieved in the proposed dual-modality GD system through dynamic disordered media. Dual modalities, i.e., high-fidelity optical data transmission and high-resolution object reconstruction, are realized in the proposed optical system through complex disordered media.
B. Different objects to be reconstructed
A 2D image (“butterfly”) is encoded into a series of random amplitude-only patterns, which are sequentially used to illuminate an object, and different objects (i.e., “1X,” “95,” “AF,” and “Triple-bar”) are individually tested in the optical path. Static (clean) water and dynamic (turbid) water are applied in Fig. 2(a). In the optical setup with static water, the experimental results are shown in Figs. 5(a)–5(h). It is shown in Figs. 5(a)–5(d) that the retrieved images are of high fidelity. PSNR values are higher than 40.0 dB, and SSIM values are close to 1. The comparisons in the 30th row of Figs. 5(a) and 5(b) are shown in Figs. 6(a) and 6(b). PSNR values are 41.00 and 43.45 dB, respectively. MSE values are 7.94 × 10−5 and 4.52 × 10−5, respectively. It is experimentally demonstrated that high-fidelity data transmission can always be realized, when different objects are placed in Fig. 2(a). The reconstructed objects are shown in Figs. 5(e)–5(h). It can be seen that the recovered objects are of high quality with CNR values higher than 33.0.
In dynamic disordered environments, the experimental results are shown in Figs. 5(i)–5(p). In Figs. 5(i)–5(l), high PSNR and high SSIM are achieved. The pixels along the 30th row of the retrieved 2D images in Fig. 5(k) and 5(l) are shown in Figs. 6(c) and 6(d), respectively. PSNR values of the experimentally retrieved data in Figs. 6(c) and 6(d) are 38.70 and 40.23 dB, respectively. MSE values are 1.35 × 10−4 and 9.49 × 10−5, respectively. It is demonstrated that the retrieved data are of high fidelity in dynamic disordered environments. The reconstructed objects are shown in Figs. 5(m)–5(p). The reconstructed images render detailed object information with high visibility. It is experimentally verified that the proposed dual-modality GD system has high robustness to simultaneously reconstruct a high-quality object and retrieve high-fidelity data using only one set of realizations in dynamic and complex disordered media.
C. Different sampling ratios
The proposed optical system is further verified through disordered media at different sampling ratios. Imaging through static (clean) water and dynamic (turbid) water is conducted, and the experimental results are shown in Fig. 7 when sampling ratios of 12.2%, 24.4%, 36.6%, 48.8%, 61.0%, 73.2%, 85.4%, and 97.6% are used, respectively. The comparisons in Fig. 7 show effectiveness of the proposed two-step enhancement approach. It is shown in Figs. 7(a) and 7(b) that the reconstruction quality is dramatically enhanced with the higher sampling ratio. With the developed two-step enhancement, object images with the higher visibility can always be obtained. When the sampling ratio is not smaller than 24.4%, the reconstructed ghost images can contain clear information. When dynamic and turbid water is considered, the experimental results are shown in Figs. 7(c) and 7(d). With the developed two-step enhancement, the visibility is significantly enhanced and noise is highly suppressed, as shown in Fig. 7(d). It is observed that the proposed optical system can recover high-quality objects at low sampling rates (e.g., 24.4%) in dynamic disordered environments. In the proposed dual-modality GD system, the sampling ratio is constrained by the length of the transmitted data. The experimental results in Fig. 7 demonstrate that high-quality objects can still be reconstructed in complex disordered environments even when the length of the transmitted data is small.
In Figs. 8(a)–8(d), CNR is calculated to quantitatively illustrate the quality of the reconstructed objects at different sampling ratios. In Figs. 8(a) and 8(b), CNR values of the reconstructed object images have similar variation trends, steadily increasing with the higher sampling ratio. CNR values of the reconstructed objects with BM3D and UNN-RED, as shown in Fig. 8(b), are much higher than those without enhancement as shown in Fig. 8(a). When optical experiments in dynamic turbid water are conducted, the trends of CNR variations are similar, as shown in Figs. 8(c) and 8(d). The average CNR values of the reconstructed object images without enhancement are in a range of 0.34–1.56, and the average CNR values with the two-step enhancement increase from 5.27 to 24.17.
IV. CONCLUSIONS
We have reported a dual-modality GD system using a UNN, simultaneously enabling high-fidelity data transmission and high-resolution object reconstruction through complex disordered media using only one set of realizations. A series of random amplitude-only patterns are generated to carry information of a 2D grayscale image (to be optically transmitted) using a UNN. The generated random patterns are embedded into SLM to modulate optical wave, and a series of single-pixel light intensities are recorded at the receiving end. With only one set of realizations, high-fidelity data information can be retrieved, and a high-resolution and high-visibility object can be recovered. A series of optical experiments have been conducted to verify the proposed dual-modality GD system. It is demonstrated that a dual-modality GD system can be realized in a complex disordered environment. The proposed approach could open an avenue for the development of AI-driven multi-modality optical systems in complex disordered environments.
SUPPLEMENTARY MATERIAL
Additional information supporting the findings of this work is provided as a separate file. The supplementary material includes information about the contrast-to-noise ratio (CNR).
ACKNOWLEDGMENTS
This work was supported by the Hong Kong Research Grants Council (Grant Nos. 15224921, 15223522), the Basic and Applied Basic Research Foundation of GuangDong Province (Grant No. 2022A1515011858), and the Hong Kong Polytechnic University (Grant No. 1-WZ4M).
AUTHOR DECLARATIONS
Conflict of Interest
The authors have no conflicts to disclose.
Author Contributions
Yang Peng: Data curation (equal); Formal analysis (equal); Investigation (equal); Methodology (equal); Software (equal); Visualization (equal); Writing – original draft (equal); Writing – review & editing (equal). Wen Chen: Conceptualization (lead); Formal analysis (equal); Funding acquisition (lead); Investigation (equal); Methodology (equal); Project administration (lead); Resources (equal); Supervision (lead); Writing – review & editing (equal).
DATA AVAILABILITY
The data and source codes that support the findings of this study are openly available in GitHub at https://github.com/YangPeng2021/Dual-modality-ghost-diffraction.