Holographic wavefront manipulation enables converting hair-thin multimode optical fibers into minimally invasive lensless imaging instruments conveying much higher information densities than conventional endoscopes. Their most prominent applications focus on accessing delicate environments, including deep brain compartments, and recording micrometer-scale resolution images of structures in close proximity to the distal end of the instrument. Here, we introduce an alternative “far-field” endoscope capable of imaging macroscopic objects across a large depth of field. The endoscope shaft with dimensions of 0.2 × 0.4 mm2 consists of two parallel optical fibers: one for illumination and the other for signal collection. The system is optimized for speed, power efficiency, and signal quality, taking into account specific features of light transport through step-index multimode fibers. The characteristics of imaging quality are studied at distances between 20 mm and 400 mm. As a proof-of-concept, we provide imaging inside the cavities of a sweet pepper commonly used as a phantom for biomedically relevant conditions. Furthermore, we test the performance on a functioning mechanical clock, thus verifying its applicability in dynamically changing environments. With the performance reaching the standard definition of video endoscopes, this work paves the way toward the exploitation of minimally invasive holographic micro-endoscopes in clinical and diagnostics applications.

Holographic endoscopes harnessing controlled light transport through hair-thin multimode optical fibers have recently emerged as powerful technological candidates for minimally invasive observations in biomedical applications.1–5 The concept spun-out from groundbreaking research on photonics of complex media,6–10 utilizing empirical quantification of optical propagation through a random medium,11 thus heralding a new era of applications that cannot be met by conventional endoscopes. Their recognized potential to obtain detailed imagery from large depths of sensitive tissue structures12 has recently been exploited in in vivo neuroscience studies,13–15 where neurones and processes of neuronal circuits have been acquired in living animal models through fibers having footprints of ∼0.01 mm2.

The aspiration of this work is to extend the applicability of holographic endoscopes into clinical environments, where reduction of instrument’s footprint is equally desired, yet the demands on imaging performance differ considerably from those of in vivo neuroscience. This relates not only to spatial resolution and frame rate but importantly also to the working distance and field of view, which have to be significantly enhanced before the concept can be accepted as a credible strategy for new minimally invasive diagnostics and surgery-assisting instrumentation. In this work, we therefore focus on imaging objects placed at macroscopic distances away from the instrument’s distal end and address the limitations that arose from this new regime. Reaching a high imaging speed dictates short pixel dwell times, nowadays only possible with power-inefficient devices.16 Moreover, imaging distant objects combined with further minimization of the instrument’s footprint dramatically reduces the amount of photons returning from the object through the endoscope shaft. Therefore, the overall power efficiency and wavefront-control fidelity represent a major challenge. While initial studies in this field have treated multimode fibers as entirely random media, it has been shown recently that light transport at the distances relevant for endoscopy is almost perfectly predictable in straight as well as significantly deformed fibers.17 Even utilizing a fragmental knowledge about the optical transmission through various input–output correlations may lead to startling benefits in practical operation of fiber-based endoscopes.18 Among other technological advancements described in the following, the potential of light-transport predictability is exploited in this work by designing the system in a manner benefiting from radial k-space conservation observed in step-index multimode fibers. In the far-field imaging regime developed here, this alone greatly enhances the power efficiency and associated signal-to-noise ratio (SNR) of the resulting imagery. Additionally, in contrast to gradient-index optical fibers, step-index multimode fibers have nearly uniform (and twice higher) mode densities, thus yielding images with unchanging resolution across the field of view. To provide a qualitative appreciation of the endoscope system, we showcase the imaging performance inside a sweet pepper and a functioning clockwork mechanism. Furthermore, we provide a detailed quantitative analysis of the imaging fidelity for various distances and compare the radial k-space conservation benefiting system with the commonly used geometry. Although compressive sensing or machine-learning algorithms could accelerate imaging or eliminate inherently speckled nature of the images,19–24 all our results are presented as raw measurements with their original contrast in order to avoid any biases.

The endoscope exploits the principle of raster-scan imaging, whereby images are reconstructed from the local response of an object to a scanning beam preshaped by the holographic modulator and delivered by a multimode optical fiber. In its simplest implementation, the endoscope collects photons back-scattered by the object, with their amount depending on the object’s local reflectivity, roughness, orientation, and axial depth. Because only a small portion of the back-scattered light falls within the small collection area defined by the fiber core, at large imaging distances, the detected signals become feeble in comparison to the constant background formed by reflections at the air/glass interfaces of the fiber and other optical components in the system. For this reason, we collect the reflected light signals by a separate identical fiber, which guides them toward a highly sensitive bucket detector, as illustrated in Fig. 1(a). A fast digital micro-mirror device (DMD) is employed as the wavefront shaping element in the system. Light propagation through the illumination fiber is empirically characterized by a transmission matrix describing the linear relationship between conveniently chosen sets of input and output fields. In our case, we use the representation of orthogonal plane waves truncated by the DMD chip as the basis of input fields and diffraction-limited foci across a square grid in the far-field plane of the distal fiber facet as the basis of output fields. Once acquired, the transmission matrix contains the information for designing the binary DMD patterns for pre-shaping the proximal wavefront, which result in far-field foci on the distal end of the endoscope. A detailed description of the setup and holographic methods is given in the supplementary material.

FIG. 1.

Multimode fiber based far-field endoscope. (a) A sequence of holograms displayed by a digital micro-mirror device (DMD) spatially shapes the wavefronts coupled to a multimode optical fiber in such a way that a far-field focus scans the distal field of view. The light signals back-scattered by an object are partially captured by a collection fiber, allowing real-time image reconstruction at the proximal side. (b) Imaging a mechanical clock as an example of a dynamic object. (c) Demonstration of endoscopic imaging of a sweet pepper via a small opening. Multimedia views: https://doi.org/10.1063/5.0038367.1 and https://doi.org/10.1063/5.0038367.2

FIG. 1.

Multimode fiber based far-field endoscope. (a) A sequence of holograms displayed by a digital micro-mirror device (DMD) spatially shapes the wavefronts coupled to a multimode optical fiber in such a way that a far-field focus scans the distal field of view. The light signals back-scattered by an object are partially captured by a collection fiber, allowing real-time image reconstruction at the proximal side. (b) Imaging a mechanical clock as an example of a dynamic object. (c) Demonstration of endoscopic imaging of a sweet pepper via a small opening. Multimedia views: https://doi.org/10.1063/5.0038367.1 and https://doi.org/10.1063/5.0038367.2

Close modal

Imaging of three-dimensional natural scenes is shown in Figs. 1(b) and 1(c). The field of view is scanned by ∼100  000 far-field foci, yielding greyscale images with 0.1 megapixel definition, a value generally accepted as the standard definition of video endoscopes.25Figure 1(c) shows the selected frames from a video recording of a yellow sweet pepper, an object commonly used for testing endoscope performance since its cavities resemble orifices of higher-order organisms. A small opening carved on the lateral surface facilitated the access of the endoscope to observe its interior. The full video recording is available as multimedia view 1. Being a far-field imaging system, the field of view scales linearly with the imaging distance, provided that the far-field condition is fulfilled. This can be understood in terms of the Fresnel number, which, for the illumination fiber used (200 μm-diameter core, 0.22 numerical aperture, and 532 nm wavelength), translates into imaging distances greater than 2 cm. Naturally, a trade-off exists between the accessible field of view and the attainable spatial resolution, resulting from the fixed angular resolution of the endoscope. This is addressed in greater depth in Sec. II-A of the supplementary material.

As in any coherent imaging technique, the reconstructed images are affected by speckle artifacts. A spatially coherent far-field focus, back-scattered by a diffusive object, gives rise to speckle with the grain size exceeding that of the collection area of the endoscope. As the sample is scanned by identical far-field foci, the backscattered speckle field changes accordingly and overlaps differently with the collection core, manifesting itself as fluctuations in the detected intensity.

The imaging pixel rate is limited by the 23 kHz refresh rate of the state-of-the-art spatial light modulator employed, resulting in a 4.4 s acquisition time for each 100 kilopixel image shown here. When imaging a dynamic scene such as the rotating clockwork mechanism shown in Fig. 1(b), this can lead to motion artifacts. To mitigate this effect, the spatial sampling was decreased on-the-fly to speed up the acquisition frame rate to 0.91 fps, 3.6 fps, and 15 fps—resulting in 25-, 6.3-, and 1.6-kilopixel images, respectively—as shown in multimedia view 2. Random access to the sequence of holograms loaded to the DMD allows also restricting raster-scanning to a particular region of interest, which can be imaged at a frame rate scaling inversely with its number of pixels. This is discussed in greater detail in Sec. 2-B of the supplementary material.

Despite its apparently randomizing nature, light transport through multimode optical fibers has properties differing from other complex media. All multimode fibers support a number of propagation invariant modes (PIMs), which do not change their field distributions as they propagate through the fiber, each of which being characterized by a particular propagation constant dictating its phase velocity.26 Within distances relevant for endoscopic applications, energy exchange between PIMs is typically negligible.17 Specifically, for step-index fibers, which confine most of the optical power in their optically homogeneous core, PIMs entering or leaving the fiber from and to other homogeneous media are quasi-non-diffracting beams also featuring extremely narrow distributions of propagation constants. Their far-field intensities therefore take the shape of an annulus with the increase in diameter as the propagation constant of the PIM decreases. This leads to an input–output correlation between the proximal and distal far fields of a step-index multimode fiber, where power from a certain annular zone in the proximal far field is delivered to the corresponding annular region in the distal far field without mixing with others. This has been efficiently utilized for refocusing diffraction-limited foci,3,27 synthesizing Bessel beams,3,28 and spatially focusing short pulses29 in the close proximity of the fiber facet. In relevance to far-field endoscopy, creating a focus at the distal far field is equivalent to producing at the distal fiber facet a plane wave truncated by the fiber core, i.e., an optical field also featuring an extremely narrow distribution of propagation constants. The corresponding proximal far field is therefore a superposition of PIMs with very similar annular far-field intensity distributions. They feature diverse topological charges (azimuthal phase dependencies) and mutually intricate phase relationships due to the macroscopic lengths of fiber, and thus, their coherent sum leads to an apparently random character of speckles. Yet, the optical power remains concentrated in the commonly shared annular zone, as illustrated in Fig. 2(a). This simulation represents the ideal case where both the phase and amplitude of the incident wavefront are modulated with arbitrary resolution so that all optical power contributes to form a desired far-field focus. However, imprinting complex modulations onto a wavefront generally incurs prohibitive power losses, as it involves discarding most of the available power; therefore, in practice, phase-only modulation is usually employed. Since holographic modulators have gradually replaced beam-steering and wavefront-correcting elements, traditional wavefront-shaping systems adopt a far-field phase modulation arrangement, as depicted in Fig. 2(b), where the incident fields are preshaped by a spatial light modulator at the back-focal plane of the focusing optics—i.e., the Fourier plane of the proximal fiber facet—as illustrated in Fig. 2(g) (designs frequently use relay optics omitted here). In this case, light signals originating outside the aforementioned annular region in the proximal far field cannot contribute to the target far-field focus, instead giving rise to a speckle background spanning the acceptance cone of the fiber, and thus decrease the fraction of controlled power.

FIG. 2.

Far-field focusing through a step-index multimode fiber. (a) Due to conservation of the axial component of incident wavevectors in step-index fibers, only light signals originating from a particular annular region in the proximal far field can contribute to synthesizing a distal far-field focus at a given distance from the optical axis. In the ideal case, when complex modulation with arbitrary resolution is applied, all incident power is found in the target far-field focus. (b) In the case of far-field phase modulation, the input spectrum contains all spatial frequencies supported by the fiber, and a background signal arising from uncontrolled light is found on the distal far field spanning its entire acceptance cone. (c) When phase modulation takes place in a plane conjugate to the proximal fiber facet, the optical power can be efficiently redistributed in the proximal far field such that it closely matches the ideal target field, thus increasing the fraction of controlled power. (d)–(f) Intensity and phase profiles of the proximal fields coupled to the fiber for the modulation approaches depicted in (a)–(c). [(g) and (h)] Practical implementation of phase-only modulation in the far field (g) and in a plane conjugate to the proximal fiber facet (h).

FIG. 2.

Far-field focusing through a step-index multimode fiber. (a) Due to conservation of the axial component of incident wavevectors in step-index fibers, only light signals originating from a particular annular region in the proximal far field can contribute to synthesizing a distal far-field focus at a given distance from the optical axis. In the ideal case, when complex modulation with arbitrary resolution is applied, all incident power is found in the target far-field focus. (b) In the case of far-field phase modulation, the input spectrum contains all spatial frequencies supported by the fiber, and a background signal arising from uncontrolled light is found on the distal far field spanning its entire acceptance cone. (c) When phase modulation takes place in a plane conjugate to the proximal fiber facet, the optical power can be efficiently redistributed in the proximal far field such that it closely matches the ideal target field, thus increasing the fraction of controlled power. (d)–(f) Intensity and phase profiles of the proximal fields coupled to the fiber for the modulation approaches depicted in (a)–(c). [(g) and (h)] Practical implementation of phase-only modulation in the far field (g) and in a plane conjugate to the proximal fiber facet (h).

Close modal

The alternative arrangement exploited here performs the phase modulation in the facet plane, allowing an efficient redistribution of the available power in the proximal far field, as shown by the simulation in Fig. 2(c). This implementation involves placing the wavefront shaping element in a plane conjugate to the proximal fiber facet,21,27,30 as illustrated in Fig. 2(h), and allows generating a modulated field closely resembling the target ring-shaped field in the proximal far field from phase-only modulations. As a consequence, only the required spatial frequencies are coupled to the fiber and just a small fraction of stray power (∼20%) reaches the sample plane in this regime.31 

In the following, we compare the performance of the endoscope system in the two competing configurations discussed. For simplicity, these geometries are referred to hereinafter as “Fourier plane” and “field plane,” indicating the mutual placement of the DMD and the proximal fiber facet. Phase-only modulation is achieved from binary-amplitude DMD patterns using the Lee hologram approach.16,32,33

Figure 3 shows a comparison of the foci quality generated in the competing geometries introduced above. Panels (a) and (b) show the sum projection of ∼1500 far-field foci generated sequentially across a sparse square grid limited by the numerical aperture of the illumination fiber. The power uniformity is visibly better in the “field plane” configuration (b) and does not vanish with increasing angles (expressed in terms of normalized spatial frequency) until the edge of the field of view is reached. A much smaller background level can also be seen in this geometry, indicating that a smaller fraction of the output power falls outside the foci. The fraction of transmitted optical power contributing to the desired light output, termed power ratio, is used here as a metric to quantify the fidelity of the generated foci—as discussed in greater detail in Sec. 2-C of the supplementary material—since it directly impacts the contrast of the resulting imagery.

FIG. 3.

Focusing quality in the far field. [(a) and (b)] Sum projection of several far-field foci generated along a square grid in the “Fourier plane” (a) and in the “field plane” (b) configurations, shown in both linear (a1) and (b1) and logarithmic (a2) and (b2) scales. [(c) and (d)] Comparison of the focusing quality of the two designs in terms of the fraction of power contained in the desired focus (c) and its angular size (d) as a function of the normalized spatial frequency (i.e., distance to the optical axis in the far field) expressed as kt/k, where kt2=kx2+ky2 and k is the wavenumber in free space. The horizontal dashed line in (d) indicates the theoretical limit of 3.25 mrad. In (c) and (d), the vertical dotted line indicates the nominal NA of the fiber, 0.22, and the shadowed areas indicate standard deviations around the mean values.

FIG. 3.

Focusing quality in the far field. [(a) and (b)] Sum projection of several far-field foci generated along a square grid in the “Fourier plane” (a) and in the “field plane” (b) configurations, shown in both linear (a1) and (b1) and logarithmic (a2) and (b2) scales. [(c) and (d)] Comparison of the focusing quality of the two designs in terms of the fraction of power contained in the desired focus (c) and its angular size (d) as a function of the normalized spatial frequency (i.e., distance to the optical axis in the far field) expressed as kt/k, where kt2=kx2+ky2 and k is the wavenumber in free space. The horizontal dashed line in (d) indicates the theoretical limit of 3.25 mrad. In (c) and (d), the vertical dotted line indicates the nominal NA of the fiber, 0.22, and the shadowed areas indicate standard deviations around the mean values.

Close modal

The intensity distribution of each such far-field focus generated in the “Fourier plane” geometry was fitted to an Airy pattern. For foci synthesized using the “field plane” configuration, the Airy model function was modified to account for the Gaussian envelope (truncated by the circular fiber core) of the signal illuminating the DMD chip. In the presence of a radially decreasing intensity profile at the proximal fiber facet, coupling favors the fiber’s PIMs with the lowest radial order—i.e., which carry power closer to the fiber axis—and therefore, the same radial azimuthally invariant power distribution is also found at the distal facet. This fitting procedure of the point spread functions (PSFs) allowed estimating both the power ratio, computed as the fraction of power under the fitted surfaces with respect to the total power emerging from the fiber, and the resolution limit. The latter was retrieved from their modulation transfer functions (MTFs, the moduli of the Fourier transform of the PSFs) as the spatial frequency at which the contrast reaches 9% of their peak value, which in the case of the Airy model function corresponds to half the angular aperture of the first-zero ring in the Airy pattern (i.e., Rayleigh’s criterion). Figures 3(c) and 3(d) show the power ratio and resolution limit of the generated foci as a function of their radial coordinate in the far field for both geometries under test. The fraction of controlled power is shown to be improved when the proximal fiber facet is placed in the field plane of the DMD, confirming the hypothesis that a larger portion of the input power gives rise to the background signal when it is placed in the Fourier plane of the spatial light modulator. The “Fourier plane” configuration is able to generate far-field foci closely matching the theoretical value of 3.25 mrad expected for an Airy disk resulting from the diffraction of a plane wave truncated by the 200 μm-diameter fiber core at the used wavelength. The “field plane” geometry produced foci around 7% larger than the theoretical limit, which we believe is caused by non-uniform illumination of the DMD chip, which, due to the nature of light transport described above, affects the power distribution at the distal facet. As shown in the following, this leads to a relatively small reduction of the imaging resolution, which can be improved by employing a Gauss-to-top-hat beam-shaping element in front of the DMD.

An image-based assessment of performance is given in Fig. 4, where each image in Figs. 4(a) and 4(b) is normalized to five times its median intensity. An imaging comparison between the two configurations is shown in Fig. 4(a) for a two-dimensional quasi-Lambertian object placed at ∼23 cm from the endoscope. Higher image contrast is clearly obtained in the “field plane” configuration, especially when approaching the periphery of the field of view where the power ratio becomes very small for the “Fourier plane” geometry. A negative 1951 USAF resolution test chart (comprising groups −4 to 7) was imaged at a varying distance from the fiber endoscope, with the output optical power limited to ∼0.1 mW. Figure 4(b) shows selected examples for four imaging distances in each design configuration. In addition to their aforementioned speckled nature, the images are also affected by detection noise, particularly for larger imaging distances when very few back-scattered photons are effectively captured by the small collection area of the endoscope. This is quantified in Fig. 4(c), which shows the signal-to-noise ratio of the reconstructed images as a function of the imaging distance. We should note that the gain of the photomultiplier tube (PMT) detector used and the output optical power were re-adjusted at each imaging distance, as described in Subsection 2 of  Appendix, and that the sparsity of the images depends on the particular region of the object being examined. The “field plane” geometry yields higher signal fidelity, again due to the increased power ratio achieved in this configuration.

FIG. 4.

Assessment of the imaging performance. (a) Imaging a two-dimensional Lambertian object under each design geometry. (b) Imaging a negative 1951 USAF resolution test chart placed at varying distances d from the endoscope termination. (c) Signal-to-noise ratio retrieved from the test chart images as a function of imaging distance. (d) Contrast of several test chart elements as a function of their spatial frequency [expressed in line pairs per millimeter (lp/mm)] for an imaging distance of 26 cm fitted to the contrast transfer function (CTF). (e) Imaging resolution retrieved from the CTFs as a function of the imaging distance. The slopes in (e) correspond to the angular resolution of the system in each configuration. The error bars in (c)–(e) indicate standard deviations.

FIG. 4.

Assessment of the imaging performance. (a) Imaging a two-dimensional Lambertian object under each design geometry. (b) Imaging a negative 1951 USAF resolution test chart placed at varying distances d from the endoscope termination. (c) Signal-to-noise ratio retrieved from the test chart images as a function of imaging distance. (d) Contrast of several test chart elements as a function of their spatial frequency [expressed in line pairs per millimeter (lp/mm)] for an imaging distance of 26 cm fitted to the contrast transfer function (CTF). (e) Imaging resolution retrieved from the CTFs as a function of the imaging distance. The slopes in (e) correspond to the angular resolution of the system in each configuration. The error bars in (c)–(e) indicate standard deviations.

Close modal

The obtained images of the 1951 USAF test chart were used for further assessing the imaging performance. At any given imaging distance, the contrast of the target elements as a function of their spatial frequency follows the contrast transfer function (CTF), which describes the response of the imaging system to a square wave (a binary black and white stripe pattern). This is exemplified in Fig. 4(d), where the fitted CTF was computed using Coltman’s formula,34 from the modulation transfer function of a diffraction-limited imaging system with a circular aperture. The fitting procedure searched for two free parameters, corresponding to the amplitude (i.e., zero-frequency contrast) and cut-off spatial frequency. The latter allows retrieving the spatial resolution (Rayleigh’s definition), which is plotted against the imaging distance in Fig. 4(e). Here, the slopes of the fitted lines correspond to the angular resolution of the system, which is found to be (3.59 ± 0.07) mrad and (3.23 ± 0.07) mrad for the “field plane” and “Fourier plane” configurations, respectively. The imaging resolution is slightly higher when the input fiber facet is placed on the Fourier plane of the DMD, rather than on the field plane, which is consistent with the observation that the far-field foci are smaller in size in this configuration, shown above in Fig. 3(d).

While both the traditional “Fourier plane” geometry and the alternative “field plane” configuration exploited here lead to far-field images with comparable spatial resolution, the latter is shown to yield images with far improved signal-to-noise ratio owing to the increased degree of control over the output light fields forming diffraction-limited foci in the distal far field.

To the best of our knowledge, in this work, we show the first far-field endoscope based on multimode fiber. Due to its extended depth of field, the holographic endoscope is capable of imaging macroscopic objects in a wide range of distances. Here, we reach the current technological limits, employing the fastest available spatial light modulator and having an unprecedented control over 17 000 fiber modes. This allowed transmitting 0.1 megapixel images, thus reaching the standard definition of modern video endoscopes.25 The versatility of the instrument was demonstrated by imaging complex three-dimensional scenes, particularly the interior of a sweet pepper serving as a phantom for biomedically relevant environments, as well as a functioning clockwork mechanism as an example of an object with dynamic complexity. Because of radial k-space conservation of propagating fields in step-index fibers, a larger fraction of the optical power can be directed toward a far-field focus when phase-only modulation is performed in the field plane of the input fiber facet, rather than in its Fourier plane. This allows efficiently manipulating the spatial frequency spectra of the coupled fields and results in far-field imaging with a greatly improved signal-to-noise ratio.

The field of view, resolution, and minimum imaging distance of the endoscope depend solely on intrinsic parameters of the illumination fiber, namely, its numerical aperture and core size, whereas the number of resolvable image features (which affect the definition of the resulting imagery) depends on the number of guided modes it supports. As such, the imaging properties of the system can be tailored to the particular needs of the envisioned application by selecting the suitable fiber, as discussed to a greater extent in Sec. 2-B of the supplementary material. With the numerical aperture of currently available fibers now reaching up to 0.9 (Ref. 27), identical 0.1 megapixel images could be obtained using a 50 μm-diameter core illumination fiber, with the instrument footprint decreased by 16 fold. Naturally, the probe size could be further halved by adopting a single fiber design. A single core fiber can serve the dual purpose of illuminating the sample and collecting back-scattered photons, provided that the reflections at the fiber end faces are eliminated—using anti-reflective coatings or angled terminations or by exploiting alternative strategies for discriminating between the light signals, such as separating them in the time domain. The endoscope can also be extended to color imaging by calibrating the illumination fiber at multiple wavelengths. The different illumination colors can be either scanned sequentially or even simultaneously by combining them into optimized holograms, with the back-scattered signals being separated spectrally on the proximal side. Finally, the speckled nature of the reconstructed images can, in principle, be suppressed computationally using a multitude of available strategies.35 

This work sets the scene for the introduction of semi-rigid minimally invasive multimode fiber probes as a perspective alternative to the rigid endoscopes routinely used in clinical diagnostics and key-hole surgery. Relying in its heart on a reconfigurable device, this endoscope is compatible with a multi-modal operation, whereby dynamically switching from far-field imaging to detailed microscopic observations near the fiber is possible simply by updating the holograms displayed by the spatial light modulator. Both hologram sequences can be obtained from two transmission matrices acquired simultaneously or numerically by multiplying either one with a free-space propagation operator.17 The far-field modality can thus be used to guide the insertion of the probe while providing organ- or tissue-scale imagery, which can then be promptly converted into an instrument imaging at the subcellular scale. Combined with spectroscopic imaging methods,36–38 the technology has potential for in situ diagnostics at the cellular level.

See the supplementary material for further details, including the detailed description of the system and holographic methods as well as additional results.

The authors acknowledge the financial support from the European Research Council (724530), the European Regional Development Fund (CZ.02.1.01/0.0/15_003/0000476), the Thüringer Ministerium für Wirtschaft, Wissenschaft und Digitale Gesellschaft, the Thüringer Aufbaubank, and the Federal Ministry of Education and Research, Germany (BMBF). The project on which these results are based was funded by the Free State of Thuringia under the number 2018 FGI 0022 and cofinanced by the European Union in the framework of the European Regional Development Fund (ERDF).

Martin Šiler is also gratefully acknowledged for a C library allowing efficient parallel computation of the DMD patterns.

The data that support the findings of this study are available from the corresponding author upon reasonable request.

1. Foci analysis

Because the intensity distributions of the far-field foci (including the speckled background) shown in Fig. 3 span over the 8-bit depth of the CMOS camera used, several image frames were acquired for each focus with increasing exposure times until the speckled background was visible. This allowed reconstructing high dynamic range images Ixi,yj, where xi,yj indicate discrete pixel positions, of the intensity distributions for every far-field focus.

In the case of the “Fourier plane” configuration, where the distal facet field has uniform envelope, each focus was fitted to an Airy pattern,

f1x,y=A2J1ρ/wρ/w2+c,
(A1)

where J1 is the Bessel function of the first kind of order one, A is the amplitude, c is the offset, ρ=xx02+yy02 is the radial polar coordinate centered around x0,y0, and w measures the width of the distribution. The power ratio, η, of each focus was estimated as the fraction of power under each fitted surface f1x,y with respect to the total transmitted power,

η=f1x,ycdxdyi,jIxi,yj=4πAw2i,jIxi,yj.
(A2)

The resolution limit for each focus was determined as 2π/kl, where kl is the spatial frequency at which its modulation transfer function, obtained from the fitted point spread function as MTFkx,ky=Ff1x,y (here, F denotes the Fourier transform), reaches 9% of its maximum value.

For foci generated in the “field plane” geometry, the power distribution at the distal fiber facet (i.e., in their far field) has the form of a uniform circular aperture multiplied by a Gaussian envelope, which can be understood as a convolution product at the sample plane,

f2x,y=AA02J1ρ/wρ/w*eρσ22+c,
(A3)

where the symbol * denotes the convolution operation, σ measures the width of the Gaussian kernel, and the factor A0 normalizes the result from the convolution product. The value of σ is fixed and is determined by the illumination truncation ratio of the experimental settings. The resolution limit and power ratio were determined analogously to the previous case, the only difference consisting in the integral expression in Eq. (A2) being computed numerically rather than analytically.

2. Imaging methods

Upon the calibration procedure, where the transmission matrix of the fiber is measured and the sequence of holograms for generating each far-focus is calculated, the calibration module was removed and replaced by the object to be imaged. The sequence of binary holograms was arranged in such a way to perform 2D interlaced scanning of the object, where for each image frame acquired, the field of view is scanned by successive foci generated along sparse square grids of increasing spatial frequency (i.e., decreasing spacing between foci positions) without repeating the same foci positions. In other words, the hologram sequence is stored on the DMD memory in such a way that the field of view can be scanned with decimation (i.e., only illuminating every other n calibrated locations) by displaying the first N/n2 DMD holograms, where N is the total number of holograms in the sequence. This allowed modifying the spatial sampling on-the-fly by displaying the hologram sequence only partially, thus providing a convenient means of adjusting the image acquisition frame rate.

The imaging performance is typically well maintained over an entire imaging session spanning over a few hours, eventually degrading over time due to mechanical drift in the opto-mechanical constituents of the system. For this reason, a new calibration—completed in less than 6 min—took place at the beginning of each imaging session to ensure the best imaging quality.

When imaging objects at the smaller distances, the gain of the photomultiplier tube (PMT) detector was set to its minimum value, and the optical power is adjusted to obtain the maximum possible output voltage signals from the detector. As imaging distances increased, the optical power was also increased to the maximum values permitted by the PMT. Once the maximum available output optical power was reached (∼0.1 mW), the PMT gain was also adjusted to maximize the output voltage signals from the PMT detector.

3. Image analysis

a. Signal-to-noise ratio

Each recorded image Ixi,yj of the 1951 USAF test chart was compared to a binary image Bxi,yj, shifted and re-scaled to best fit Ixi,yj, serving as a “ground truth.” The signal-to-noise ratio (SNR) of each image I was then retrieved as

SNR=1N1i,jI.B1N0i,jI1B1N0i,jI1B,
(A4)

where N1 = ∑i,jB and N0=i,j1B are the number of “ones” and “zeros” in each binary image B, respectively.

b. Spatial resolution

For every image of the 1951 USAF test chart, each individual target element was fitted to a 2D surface representing three parallel bars to retrieve their width. For elements with vertical bars (i.e., oriented in the y direction), the fitted surface had the form

fx,y=m=051ma2erfbxx0mw+c,
(A5)

where erf is the error function, w is the width (also separation) between bars, a is an amplitude, b is a measure of the “steepness” of the transition, c is an offset, and x0 is the horizontal position of the first bar. The contrast of each target element was estimated as fmaxfmin/fmax+fmin, where fmax and fmin are the maximum and minimum of the fitted surface f, respectively.

The contrast transfer function (CTF) was computed from the modulation transfer function (MTF) using Coltman’s formula,

CTFν=4πn=01n2n+1MTF2n+1ν,
(A6)

where MTF is the modulation transfer function and ν is the spatial frequency. The infinite summation was truncated to the first 100 terms, which describe accurately the CTF for spatial frequencies larger than 0.5% of the cut-off frequency, which encompasses the range of experimental values obtained. We assume the case of an ideal imaging system with a circular aperture, with MTF given by

MTFν=A2πarccosνν1ν2,
(A7)

where ν′ = ν/2ν0 is the spatial frequency normalized to the cut-off value and A is an amplitude.

1.
R.
Di Leonardo
and
S.
Bianchi
, “
Hologram transmission through multi-mode optical fibers
,”
Opt. Express
19
,
247
254
(
2011
).
2.
T.
Čižmár
and
K.
Dholakia
, “
Shaping the light transmission through a multimode optical fibre: Complex transformation analysis and applications in biophotonics
,”
Opt. Express
19
,
18871
18884
(
2011
).
3.
T.
Čižmár
and
K.
Dholakia
, “
Exploiting multimode waveguides for pure fibre-based imaging
,”
Nat. Commun.
3
,
1027
(
2012
).
4.
I. N.
Papadopoulos
,
S.
Farahi
,
C.
Moser
, and
D.
Psaltis
, “
Focusing and scanning light through a multimode optical fiber using digital phase conjugation
,”
Opt. Express
20
,
10583
10590
(
2012
).
5.
Y.
Choi
,
C.
Yoon
,
M.
Kim
,
T. D.
Yang
,
C.
Fang-Yen
,
R. R.
Dasari
,
K. J.
Lee
, and
W.
Choi
, “
Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber
,”
Phys. Rev. Lett.
109
,
203901
(
2012
).
6.
I. M.
Vellekoop
and
A. P.
Mosk
, “
Focusing coherent light through opaque strongly scattering media
,”
Opt. Lett.
32
,
2309
2311
(
2007
).
7.
S. M.
Popoff
,
G.
Lerosey
,
M.
Fink
,
A. C.
Boccara
, and
S.
Gigan
, “
Image transmission through an opaque material
,”
Nat. Commun.
1
,
81
(
2010
).
8.
A. P.
Mosk
,
A.
Lagendijk
,
G.
Lerosey
, and
M.
Fink
, “
Controlling waves in space and time for imaging and focusing in complex media
,”
Nat. Photonics
6
,
283
292
(
2012
).
9.
S.
Rotter
and
S.
Gigan
, “
Light fields in complex media: Mesoscopic scattering meets wave control
,”
Rev. Mod. Phys.
89
,
015005
(
2017
).
10.
J.-H.
Park
,
Z.
Yu
,
K.
Lee
,
P.
Lai
, and
Y.
Park
, “
Perspective: Wavefront shaping techniques for controlling multiple light scattering in biological tissues: Toward in vivo applications
,”
APL Photonics
3
,
100901
(
2018
).
11.
S. M.
Popoff
,
G.
Lerosey
,
R.
Carminati
,
M.
Fink
,
A. C.
Boccara
, and
S.
Gigan
, “
Measuring the transmission matrix in optics: An approach to the study and control of light propagation in disordered media
,”
Phys. Rev. Lett.
104
,
100601
(
2010
).
12.
J. A.
Frank
,
M.-J.
Antonini
, and
P.
Anikeeva
, “
Next-generation interfaces for studying neural function
,”
Nat. Biotechnol.
37
,
1013
1023
(
2019
).
13.
S.
Ohayon
,
A.
Caravaca-Aguirre
,
R.
Piestun
, and
J. J.
DiCarlo
, “
Minimally invasive multimode optical fiber microendoscope for deep brain fluorescence imaging
,”
Biomed. Opt. Express
9
,
1492
1509
(
2018
).
14.
S.
Turtaev
,
I. T.
Leite
,
T.
Altwegg-Boussac
,
J. M. P.
Pakan
,
N. L.
Rochefort
, and
T.
Čižmár
, “
High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging
,”
Light Sci. Appl.
7
,
92
(
2018
).
15.
S. A.
Vasquez-Lopez
,
R.
Turcotte
,
V.
Koren
,
M.
Plöschner
,
Z.
Padamsey
,
M. J.
Booth
,
T.
Čižmár
, and
N. J.
Emptage
, “
Subcellular spatial resolution achieved for deep-brain imaging in vivo using a minimally invasive multimode fiber
,”
Light Sci. Appl.
7
,
110
(
2018
).
16.
S.
Turtaev
,
I. T.
Leite
,
K. J.
Mitchell
,
M. J.
Padgett
,
D. B.
Phillips
, and
T.
Čižmár
, “
Comparison of nematic liquid-crystal and DMD based spatial light modulation in complex photonics
,”
Opt. Express
25
,
29874
29884
(
2017
).
17.
M.
Plöschner
,
T.
Tyc
, and
T.
Čižmár
, “
Seeing through chaos in multimode fibres
,”
Nat. Photonics
9
,
529
535
(
2015
).
18.
S.
Li
,
C.
Saunders
,
D. J.
Lum
,
J.
Murray-Bruce
,
V. K.
Goyal
,
T.
Čižmár
, and
D. B.
Phillips
, “
Compressively sampling the optical transmission matrix of a multimode fibre
,” arXiv:2007.15891 (
2020
).
19.
L. V.
Amitonova
and
J. F.
de Boer
, “
Compressive imaging through a multimode fiber
,”
Opt. Lett.
43
,
5427
5430
(
2018
).
20.
B.
Rahmani
,
D.
Loterie
,
G.
Konstantinou
,
D.
Psaltis
, and
C.
Moser
, “
Multimode optical fiber transmission with a deep learning network
,”
Light Sci. Appl.
7
,
69
(
2018
).
21.
N.
Borhani
,
E.
Kakkava
,
C.
Moser
, and
D.
Psaltis
, “
Learning to see through multimode fibers
,”
Optica
5
,
960
965
(
2018
).
22.
P.
Fan
,
T.
Zhao
, and
L.
Su
, “
Deep learning the high variability and randomness inside multimode fibers
,”
Opt. Express
27
,
20241
20258
(
2019
).
23.
U.
Teǧin
,
B.
Rahmani
,
E.
Kakkava
,
N.
Borhani
,
C.
Moser
, and
D.
Psaltis
, “
Controlling spatiotemporal nonlinearities in multimode fibers with deep neural networks
,”
APL Photonics
5
,
030804
(
2020
).
24.
W.
Xiong
,
B.
Redding
,
S.
Gertler
,
Y.
Bromberg
,
H. D.
Tagare
, and
H.
Cao
, “
Deep learning of ultrafast pulses with a multimode fiber
,”
APL Photonics
5
,
096106
(
2020
).
25.
Y. M.
Bhat
,
B. K.
Abu Dayyeh
,
S. S.
Chauhan
,
K. T.
Gottlieb
,
J. H.
Hwang
,
S.
Komanduri
,
V.
Konda
,
S. K.
Lo
,
M. A.
Manfredi
,
J. T.
Maple
,
F. M.
Murad
,
U. D.
Siddiqui
,
S.
Banerjee
, and
M. B.
Wallace
, “
High-definition and high-magnification endoscopes
,”
Gastrointest. Endosc.
80
,
919
927
(
2014
).
26.
A. W.
Snyder
and
J. D.
Love
,
Optical Waveguide Theory
(
Springer US
,
Boston, MA
,
1983
).
27.
I. T.
Leite
,
S.
Turtaev
,
X.
Jiang
,
M.
Šiler
,
A.
Cuschieri
,
P. St. J.
Russell
, and
T.
Čižmár
, “
Three-dimensional holographic optical manipulation through a high-numerical-aperture soft-glass multimode fibre
,”
Nat. Photonics
12
,
33
39
(
2018
).
28.
M.
Plöschner
,
V.
Kollárová
,
Z.
Dostál
,
J.
Nylk
,
T.
Barton-Owen
,
D. E. K.
Ferrier
,
R.
Chmelík
,
K.
Dholakia
, and
T.
Čižmár
, “
Multimode fibre: Light-sheet microscopy at the tip of a needle
,”
Sci. Rep.
5
,
18050
(
2015
).
29.
E. E.
Morales-Delgado
,
S.
Farahi
,
I. N.
Papadopoulos
,
D.
Psaltis
, and
C.
Moser
, “
Delivery of focused short pulses through a multimode fiber
,”
Opt. Express
23
,
9109
9120
(
2015
).
30.
E.
Kakkava
,
B.
Rahmani
,
N.
Borhani
,
U.
Teğin
,
D.
Loterie
,
G.
Konstantinou
,
C.
Moser
, and
D.
Psaltis
, “
Imaging through multimode fibers using deep learning: The effects of intensity versus holographic recording of the speckle pattern
,”
Opt. Fiber Technol.
52
,
101985
(
2019
).
31.
S. N.
Chandrasekaran
,
H.
Ligtenberg
,
W.
Steenbergen
, and
I. M.
Vellekoop
, “
Using digital micromirror devices for focusing light through turbid media
,” in
Emerging Digital Micromirror Device Based Systems Applications VI
, edited by
M. R.
Douglass
,
P. S.
King
, and
B. L.
Lee
(
2014
), Vol. 8979, p.
897905
.
32.
W.-H.
Lee
, “
Binary computer-generated holograms
,”
Appl. Opt.
18
,
3661
3669
(
1979
).
33.
D. B.
Conkey
,
A. M.
Caravaca-Aguirre
, and
R.
Piestun
, “
High-speed scattering medium characterization with application to focusing light through turbid media
,”
Opt. Express
20
,
1733
1740
(
2012
).
34.
J. W.
Coltman
, “
The specification of imaging properties by response to a sine wave input
,”
J. Opt. Soc. Am.
44
,
468
471
(
1954
).
35.
V.
Bianco
,
P.
Memmolo
,
M.
Leo
,
S.
Montresor
,
C.
Distante
,
M.
Paturzo
,
P.
Picart
,
B.
Javidi
, and
P.
Ferraro
, “
Strategies for reducing speckle noise in digital holography
,”
Light Sci. Appl.
7
,
48
(
2018
).
36.
I.
Gusachenko
,
M.
Chen
, and
K.
Dholakia
, “
Raman imaging through a single multimode fibre
,”
Opt. Express
25
,
13782
13798
(
2017
).
37.
S.
Deng
,
D.
Loterie
,
G.
Konstantinou
,
D.
Psaltis
, and
C.
Moser
, “
Raman imaging through multimode sapphire fiber
,”
Opt. Express
27
,
1090
1098
(
2019
).
38.
J.
Trägårdh
,
T.
Pikálek
,
M.
Šerý
,
T.
Meyer
,
J.
Popp
, and
T.
Čižmár
, “
Label-free CARS microscopy through a multimode fiber endoscope
,”
Opt. Express
27
,
30055
30066
(
2019
).

Supplementary Material