Optimizing Anti-Perturbation Capability in Single-shot wide-field Multimode Fiber Imaging Systems

In recent years, multimode fiber (MMF) has emerged as a focal point in ultrathin endoscopy owing to its high-capacity information transmission. Nevertheless, the technology’s susceptibility to external perturbances limits its practical applications. In this study, we employ a single MMF as both the illumination unit and imaging probe, and utilize this single-shot wide-field MMF imaging system to investigate the impact of LED and laser sources on anti-perturbation capabilities. Experimental results demonstrate that, in the absence of deformations in the MMF, both LED and laser-based systems achieve an average Structural Similarity (SSIM) index of around 0.8 for the reconstructed image, utilizing advanced deep learning (DNN) techniques, with the laser-based system performing slightly better. However, under unknown MMF configurations post-deformation, the SSIM remains robust at 0.67 for the LED-based system, while the laser-based system drops the average SSIM to 0.45. The results reveal that LED has anti-perturbation capability in single-shot wide-field MMF imaging systems


nBSTRnCT
In recent years, multimode fiber (MMF) has emerged as a focal point in ultrathin endoscopy owing to its highcapacity information transmission.Nevertheless, the technology's susceptibility to external perturbances limits its practical applications.In this study, we employ a single MMF as both the illumination unit and imaging probe, and utilize this single-shot wide-field MMF imaging system to investigate the impact of LED and laser sources on antiperturbation capabilities.Experimental results demonstrate that, in the absence of deformations in the MMF, both LED and laser-based systems achieve an average Structural Similarity (SSIM) index of around 0.8 for the reconstructed image, utilizing advanced deep learning (DNN) techniques, with the laser-based system performing slightly better.However, under unknown MMF configurations post-deformation, the SSIM remains robust at 0.67 for the LED-based system, while the laser-based system drops the average SSIM to 0.45.The results reveal that LED has anti-perturbation capability in single-shot wide-field MMF imaging systems.These findings indicate significant potential for future anti-perturbation studies in endoscopy employing MMF imaging.Medical endoscopy technology is advancing towards minimally invasive approaches, which is a crucial trend in the field of medical diagnosis and treatment. 16][7] Most fiber-optic imaging systems use single-mode fibers (SMFs).Still, SMFs can only transmit intensity information at specific points in an image. 8,9They are typically arranged into fiber bundles for complete image transmission.1][12] However, challenges like mode dispersion and coupling issues lead to the distortion of transmitted image information and the generation of random speckle patterns at the MMF's output facet. 132][23][24][25] With the continuous development of these technologies, the high-throughput image transmission capabilities of MMFs have been demonstrated, 26 and these technologies have also shown their potential in ultra-thin endoscopic imaging.
Despite the significant potential of using MMFs for image transmission, the sensitivity of this system to external perturbance during operation poses a considerable challenge. 25Some innovative approaches to enhance the resistance of MMF-based systems to deformation interference have been proposed.For instance, methods such as metasurface reflector stacks or guide stars positioned at the distal facet of the MMF, 27,28 or MMF surrounded by three SMFs containing fiber Bragg grating arrays 14 .However, these methods typically require complex optical paths with MMF recalibration and imaging speed limitations.Therefore, a simple method is needed to enhance the system's anti-perturbation capability.
In a recent study, Xiao's group 18 compared five different laser sources in MMF imaging systems, assessing the impact of light source coherence and linewidth on image reconstruction quality.They observed that the inverse transmission matrix (ITM) method has limited generalization capability, making it challenging to recover speckle patterns generated by low-coherence and wide-linewidth light sources.Although deep learning 29 has been proved to be an effective method to reconstruct speckle patterns from low-coherence, the utilization of low-coherence light sources for enhancing anti-perturbation capabilities using deep learning has not been explored.Additionally, most MMF imaging optical systems introduce extra illumination pathways, 30,31 leading to an expanded system footprint and increased design costs, making them impractical for endoscopic applications.
In this study, we validate the feasibility of using an incoherent light source, LED, in single-shot wide-field MMF imaging systems, and a laser-based system is also studied for comparison.Two image reconstruction methods, namely Principal Component Analysis-based Inverse Transmission Matrix (PCA-ITM) and Deep Neural Networks (DNN), are used to process the two acquired signal patterns (laser-speckles and LED-patterns).The experimental results show that the DNN method exhibits excellent image recovery for the LED-based single-shot wide-field MMF imaging system.The system's resistance to deformations in the MMF improves significantly.This work demonstrates that LED as a light source for single MMF endoscopy systems holds great potential for practical biomedical applications.
Figure .1(a) illustrates our MMF imaging system.It incorporates the critical components of the optical system, the signal patterns acquisition process, and the fiber deformation procedure.The endoscopic probe employs a MMF (NA: 0.22, core diameter: 200 μm), responsible for illuminating the Digital Micromirror Device (DMD) and collecting signals to a CMOS camera.The laser light source used is a semiconductor laser (λ=488nm, Δλ=4nm), and the LED is a blue light source with a similar central wavelength placed behind a filter (λ=488nm, Δλ=20nm).Their spectra are shown in Fig. 1(b).Both light sources undergo beam shaping, and in the experiment, only one light source is applied at a time.Moreover, the experimental conditions were maintained to stabilize temperature, humidity, and ambient light.
The ground truth datasets (10,000 images each from MNIST 32 and Fashion-MNIST 33 ) are projected to the DMD and coupled into the MMF.The MMF is fixed on a manual stage for adjusting fiber deformation, and the CMOS camera captures signal patterns under each illumination condition.In this study, we employ two image reconstruction methods.The first one is the PCA-ITM method, the detailed process of which can be found in S2 of the supplementary materials or previous related work. 34The PCA-ITM method exhibits superior imaging quality and has lower computational costs.However, its generalization capability may be insufficient, making it challenging to achieve image reconstruction for unknown speckle patterns.The other is the DNN method, as illustrated in Fig. 2. We adopt a cascaded U-Net architecture commonly applied in image segmentation and reconstruction tasks. 35The role of the second shallow U-net is to perform secondary encoding, enabling the extraction of additional feature information from the images.The key parameter settings are as follows: the CMOS-captured color images are converted to grayscale and fed into the input layer with an image size of 224224 pixels.The encoder consists of multiple convolutional layers with a 33 pixels kernel size for extracting features from the input image.The final convolutional layer outputs a single-channel image with a 2828 pixels size.We choose the Adam optimizer and continuously adjust the learning rate during training.The loss function used is mean square error (MSE).The batch size for each training iteration is 8, and the maximum training epoch is formed to 60.This study evaluates image reconstruction performance using structural similarity (SSIM) and peak signal-tonoise ratio (PSNR) parameters, quantifying the structural resemblance between the reconstructed image and the ground truth.SSIM comprehensively considers the structural features of an image, while PSNR focuses more on numerical variations, making it more effective for precision evaluation.Detailed information on these parameters is presented in S2.4 of the supplementary materials.We select 800 images as the test set, and the reconstructed results are presented in Fig. 3.The training loss curves are shown in Fig. S2 of the supplementary materials.We compare eight reconstructed images and display the average SSIM and PSNR of 800 test images under laser or LED illumination conditions.The PCA-ITM method exhibits more noise in the reconstructed images, while the DNN method produces smoother results.PCA-ITM recovers better image details for the laser illumination condition than DNN.This effect is more pronounced in the Fashion-MNIST dataset.Conversely, the DNN method excels in recovering less detailed images like "skirt" and "boot."For LED-patterns, PCA-ITM yields poor reconstruction results, while the DNN method shows a significant advantage, with its SSIM value being very close to that using lasers as the light source.It reflects the potential of LED in a single MMF illumination and imaging system.
To explore the impact of the light source on the system's anti-perturbation ability, we collected 800 laserspeckles and LED-patterns from test datasets across 20 MMF configurations.These images are then subjected to recovery using pre-trained PCA-ITM and DNN models before MMF deformation.Selected recovery results are presented in Fig. 4. It is apparent that PCA-ITM struggles with recovery for unknown MMF configurations, hindering meaningful information discernment in all test sets.In contrast, the DNN method adeptly recovers laserspeckles or LED-patterns for unknown MMF configurations.As shown in Fig. 4, for the LED light source, the DNN method effectively restores unknown MMF configurations, maintaining stable and distinguishable imaging quality throughout the deformation process, whether digit "2" or "high-heeled shoes."Both the MNIST dataset and Fashion-MNIST dataset can be recovered with a fiber deformation of up to 20mm (If the range of the manual stage allows, image reconstruction can be performed under larger MMF deformation.).However, for the laser light source, during the MNIST dataset recovery, laser-speckles are challenging to discern for deformation of 5mm.However, at the deformation of 10mm, digit "2" contours reappear, suggesting occasional feature within a specific range, possibly due to increased randomness from fiber deformation in laser-speckles, involving speckle boiling phenomena. 34For the Fashion-MNIST dataset, despite close-range MMF deformation, some outline information of "high-heeled shoes" from laser-speckles is recovered.The above results demonstrate that LED-patterns perform superiorly compared to laser-speckles.Analyzing image reconstruction under unknown MMF configurations, Fig. 5 shows the average SSIM for the different deformation distances.The average PSNR (see Fig. S3 of the supplementary materials) follows a similar trend.The average SSIM of the 800 test set images can reliably reflect the impact of fiber deformation on the image reconstruction capability, reducing experimental variability caused by chance.Comparing recovery results for MNIST and Fashion-MNIST datasets in (a) and (b), respectively, Fashion-MNIST exhibits slightly worse recovery, likely due to its richer details being more affected by MMF deformation.As MMF deformation increases, overall recovery degrades.DNN consistently outperforms PCA-ITM in average SSIM.Under the laser-based system, the decrease in SSIM due to the initial 1 mm MMF deformation is very pronounced, while the subsequent fiber deformation causes a less pronounced change in SSIM.It is also possibly due to higher randomness in laser-speckles changes.Consequently, within a specific range, it cannot be assumed that more significant deformations result in poorer quality of the recovered images.However, in the LED system, DNN shows a gentler change in average SSIM, with the highest overall recovery quality.The average SSIM for the LED system using the DNN method is around 0.67, significantly higher than that in the laser-based system, which is about 0.45.The results indicate that using LED light sources with the DNN method can resist disturbances and significantly improve the imaging system's resistance to fiber deformation interference.
When using the DNN method to reconstruct images in the LED-based single-shot wide-field MMF imaging system, the system's resistance to MMF deformation interference is significantly enhanced compared to the laserbased system.This improvement may stem from various factors.Through the analysis using the transmission matrix theory, the illumination light generated by the LED, after transmission through the MMF, can be considered as the superposition of a series of output fields with different frequencies: 18 1 where vi I is intensity data of a certain frequency of light in a broad-spectrum source.Different frequencies of light correspond to various transmission matrices, where X is detected image amplitude.
T represents the transmission matrix of the system.R Y is the vector of recovered image.Finding a suitable transmission matrix T that fits all images and light frequencies is challenging for the PCA-ITM method.Therefore, this method exhibits better recovery quality for the laser, which has a narrower linewidth but yields less favorable results for LED. 13 On the other hand, the DNN method can be more effective in extracting the invariant properties of the signal in MMF imaging systems. 36This approach can extract features and recover low-contrast LED-patterns.It also demonstrates feature extraction capabilities for speckle signals in unknown MMF configurations, yielding significantly better results than the PCA-ITM method.Regarding the signal patterns generated by laser and LED under unknown MMF configurations (see Fig. S4 of the supplementary materials).With fiber deformation, the laser undergoes a series of unpredictable mode coupling and phase changes in the fiber, resulting in a more random variation of the laser-speckles. 37In contrast, LED is almost unaffected by phase changes, 38 making LED-patterns more stable and continuous with fiber deformation.Ultimately, when inputting the pre-trained recovery model, the test set of LED-patterns has a higher correlation with the training set in the pretrained model, resulting in better recovery results.However, a challenge in LED-based MMF imaging technology lies in the relatively lower contrast of its output images.As the transmitted information volume increases, the recovery from the intensity map collected at the fiber output becomes more challenging.Nonetheless, current advancements in deep learning, particularly in super-resolution techniques, hold the potential to address this issue.
In summary, the impact of light sources on image reconstruction in single-shot wide-field MMF imaging systems has been comprehensively studied.PCA-ITM and DNN image reconstruction methods are employed to process the images obtained in the system using laser or LED as the light source.Without fiber deformation perturbation and using the PCA-ITM method, the performance of the laser-based system is much better than that of the LED-based system.However, when the DNN method is used for image reconstruction, the performance of the LED-based system is very close to that of the laser-based system.After considering external perturbations by introducing fiber deformation, the experimental results demonstrate the LED-based MMF imaging system is much better than the laser-based MMF imaging system for image reconstruction using the DNN method.The SSIM remains robust at 0.67 for the LED-based system, while the laser-based system drops the average SSIM to 0.45.This is due to the incoherent nature of LED, which is almost unaffected by phase changes with fiber deformation.These results highlight that the LED-based MMF imaging system can significantly enhance anti-perturbation capability, with potential application in endoscopy employing MMF imaging.From an algorithmic standpoint, integrating the super-resolution generative adversarial network algorithm may enhance the LED's imaging capability for complex tissue images.On a methodological level, combining it with compressed sensing imaging techniques may enable the imaging of natural objects.Furthermore, our work can be easily integrated with other MMF imaging technologies, providing valuable insights into the anti-perturbation research in MMF endoscopy imaging.

AUTHOR DECLARATIONS
Figure.1(c) illustrates the captured signal patterns, where laser-speckles exhibit higher contrast, while LED-patterns are more uniform.It is attributed to the incoherent nature of LED light, where the field formed at the MMF output is the intensity summation of individual light modes without generating speckle patterns due to interference.To analyze the anti-perturbation capability of the two illumination systems, we move the manual stage at 1 mm intervals and collect 20 sets of signal patterns with different MMF configurations as a test dataset.

Fig. 1 .
Fig. 1.Experimental setup schematic.(a) Schematic diagram of the optical system, with the top left corner illustrating the process of fiber deformation; (b) Relative spectral diagrams of laser and LED; (c) Signal patterns captured by the CMOS camera.Detailed parameters of the experiments can be found in S1 of the supplementary material.In this study, we employ two image reconstruction methods.The first one is the PCA-ITM method, the detailed process of which can be found in S2 of the supplementary materials or previous related work.34The PCA-ITM method exhibits superior imaging quality and has lower computational costs.However, its generalization capability may be insufficient, making it challenging to achieve image reconstruction for unknown speckle patterns.The other is the DNN method, as illustrated in Fig.2.We adopt a cascaded U-Net architecture commonly applied in image

Fig. 3 .
Fig. 3. Schematic diagram of image reconstruction results.The right side of the image shows the average SSIM and PSNR of 800 test image results under the laser and LED illumination conditions.We select 800 images as the test set, and the reconstructed results are presented in Fig.3.The training loss curves are shown in Fig.S2of the supplementary materials.We compare eight reconstructed images and display the average SSIM and PSNR of 800 test images under laser or LED illumination conditions.The PCA-ITM method exhibits more noise in the reconstructed images, while the DNN method produces smoother results.PCA-ITM recovers better image details for the laser illumination condition than DNN.This effect is more pronounced in the Fashion-MNIST dataset.Conversely, the DNN method excels in recovering less detailed images like "skirt" and "boot."For LED-patterns, PCA-ITM yields poor reconstruction results, while the DNN method shows a significant advantage, with its SSIM value being very close to that using lasers as the light source.It reflects the potential of LED in a single MMF illumination and imaging system.To explore the impact of the light source on the system's anti-perturbation ability, we collected 800 laserspeckles and LED-patterns from test datasets across 20 MMF configurations.These images are then subjected to recovery using pre-trained PCA-ITM and DNN models before MMF deformation.Selected recovery results are

Fig. 4 .
Fig. 4. Recovery of laser-speckles and LED-patterns under different MMF configurations using PCA-ITM and DNN Methods.

Fig. 5 .
Fig. 5. Average SSIM Trends of Reconstructed Images (laser-speckles and LED-patterns) under Various MMF States using PCA-ITM and DNN Methods.(a) MNIST Dataset, (b) Fashion-MNIST Dataset.Shadowed areas in the figure represent the standard deviation of the data.Analyzing image reconstruction under unknown MMF configurations, Fig.5shows the average SSIM for the different deformation distances.The average PSNR (see Fig.S3of the supplementary materials) follows a similar trend.The average SSIM of the 800 test set images can reliably reflect the impact of fiber deformation on the image reconstruction capability, reducing experimental variability caused by chance.Comparing recovery results for MNIST and Fashion-MNIST datasets in (a) and (b), respectively, Fashion-MNIST exhibits slightly worse recovery, likely due to its richer details being more affected by MMF deformation.As MMF deformation increases, overall recovery degrades.DNN consistently outperforms PCA-ITM in average SSIM.Under the laser-based system, the decrease in SSIM due to the initial 1 mm MMF deformation is very pronounced, while the subsequent fiber deformation causes a less pronounced change in SSIM.It is also possibly due to higher randomness in laser-speckles changes.Consequently, within a specific range, it cannot be assumed that more significant deformations result in See the S7 and S8 of the supplementary materials on experimental details and results.This work was supported by the National Key Research and Development Program of China (2021YFC2400100, 2022YFC2404400, zchj2022006L), National Natural Science Foundation of China (62375280); Natural Science Foundation of Shandong Province (ZR2023MF087), Youth Innovation Promotion Association of ACS (2023340), and Basic Research Program of Suzhou (szs2020308, SJC2022009, SSD2023010, SSD2023020).