Image sensors used in consumer cameras only detect intensity of light. Spectral, angular, and phase information of incident light is largely lost. For conventional applications like photography, the intensity information with three color bands (RGB) is sufficient. However, for advanced sensing applications, such as autonomous vehicles, biomedical imaging, and robotics, extracting more information from the incident light could help machines to make better decisions. It is expected that, in the future, more cameras will be built for machines than for people, which will be further accelerated by the rapid progress in machine learning and artificial intelligence. We envision that these applications will substantially benefit from multimodal measurement of light fields by advanced imaging sensors. In this Perspective, we discuss recent progress in image sensors with multimodal capabilities.
REFERENCES
1.
T. G.
Phillips
and
J.
Keene
, “
Submillimeter astronomy (heterodyne spectroscopy)
,” Proc. IEEE
80
, 1662
–1678
(1992
).2.
C. P.
Bacon
,
Y.
Mattley
, and
R.
DeFrece
, “
Miniature spectroscopic instrumentation: Applications to biology and chemistry
,” Rev. Sci. Instrum.
75
, 1
–16
(2004
).3.
L. P.
Schuler
,
J. S.
Milne
,
J. M.
Dell
, and
L.
Faraone
, “
MEMS-based microspectrometer technologies for NIR and MIR wavelengths
,” J. Phys. D
42
, 133001
(2009
).4.
J.
Malinen
,
A.
Rissanen
,
H.
Saari
,
P.
Karioja
,
M.
Karppinen
,
T.
Aalto
, and
K.
Tukkiniemi
, “
Advances in miniature spectrometer and sensor development
,” in Next-Generation Spectroscopic Technologies VII
(
SPIE
, 2014
), p. 91010C
.5.
R. A.
Crocombe
, “
Portable spectroscopy
,” Appl. Spectrosc.
72
, 1701
–1751
(2018
).6.
M.
Manley
, “
Near-infrared spectroscopy and hyperspectral imaging: Non-destructive analysis of biological materials
,” Chem. Soc. Rev.
43
, 8200
–8214
(2014
).7.
E.
Ryckeboer
,
R.
Bockstaele
,
M.
Vanslembrouck
, and
R.
Baets
, “
Glucose sensing by waveguide-based absorption spectroscopy on a silicon chip
,” Biomed. Opt. Express
5
, 1636
–1648
(2014
).8.
N. K.
Pervez
,
W.
Cheng
,
Z.
Jia
,
M. P.
Cox
,
H. M.
Edrees
, and
I.
Kymissis
, “
Photonic crystal spectrometer
,” Opt. Express
18
, 8277
–285
(2010
).9.
J. H.
Correia
,
G.
De Graaf
,
S. H.
Kong
,
M.
Bartek
, and
R. F.
Wolffenbuttel
, “
Single-chip CMOS optical microspectrometer
,” Sens. Actuators, A
82
, 191
–197
(2000
).10.
S. H.
Kong
,
J. H.
Correia
,
G.
De Graaf
,
M.
Bartek
, and
R. F.
Wolffenbuttel
, “
Integrated silicon microspectrometers
,” IEEE Instrum. Meas. Mag.
4
, 34
–38
(2001
).11.
S. W.
Wang
,
C.
Xia
,
X.
Chen
,
W.
Lu
,
M.
Li
,
H.
Wang
,
W.
Zheng
, and
T.
Zhang
, “
Concept of a high-resolution miniature spectrometer using an integrated filter array
,” Opt. Lett.
32
, 632
–634
(2007
).12.
A.
Tittl
,
A.
Leitis
,
M.
Liu
,
F.
Yesilkoy
,
D. Y.
Choi
,
D. N.
Neshev
,
Y. S.
Kivshar
, and
H.
Altug
, “
Imaging-based molecular barcoding with pixelated dielectric metasurfaces
,” Science
360
, 1105
–1109
(2018
).13.
A. M.
Mika
, “
Linear-wedge spectrometer
,” in Imaging Spectroscopy of the Terrestrial Environment
(
SPIE
, 1990
), Vol.
1298
, p. 127
.14.
M.
Grundmann
, “
Modeling of a waveguide‐based UV–VIS–IR spectrometer based on a lateral (In, Ga) N alloy gradient
,” Phys. Status Solidi A
216
, 1900170
(2019
).15.
S. R.
Mallinson
and
J. H.
Jerman
, “
Miniature micromachined Fabry-Perot interferometers in silicon
,” Electron. Lett.
23
, 1041
–1043
(1987
).16.
N.
Gat
, “
Imaging spectroscopy using tunable filters: A review
,” in Wavelet Applications VII
(
SPIE
, 2000
), Vol.
4056
.17.
H.
Zhang
,
X. L.
Wang
,
J. I.
Soos
, and
J. A.
Crisp
, “
Design of a miniature solid state NIR spectrometer
,” in Infrared Detectors and Instrumentation for Astronomy
(
SPIE
, 1995
).18.
T. A.
Kwa
and
R. F.
Wolffenbuttel
, “
Integrated grating/detector array fabricated in silicon using micromachining techniques
,” Sens. Actuators, A
31
, 259
–266
(1992
).19.
See www.hamamatsu.com/jp/en/product/type/C12666MA/index.html for “
Hamamatsu mini-spectrometer.
”20.
A. Y.
Zhu
,
W. T.
Chen
,
M.
Khorasaninejad
,
J.
Oh
,
A.
Zaidi
,
I.
Mishra
,
R. C.
Devlin
, and
F.
Capasso
, “
Ultra-compact visible chiral spectrometer with meta-lenses
,” APL Photonics
2
, 036103
(2017
).21.
C.
Yang
,
K.
Shi
,
P.
Edwards
, and
Z.
Liu
, “
Demonstration of a PDMS based hybrid grating and Fresnel lens (G-Fresnel) device
,” Opt. Express
18
, 23529
–23534
(2010
).22.
B.
Gao
,
Z.
Shi
, and
R. W.
Boyd
, “
Design of flat-band superprism structures for on-chip spectroscopy
,” Opt. Express
23
, 6491
–6496
(2015
).23.
D.
Sander
,
M.-O.
Duecker
,
O.
Blume
, and
J.
Mueller
, “
An optical microspectrometer in SiON-slab-waveguides
,” Proc. SPIE
2686
, 100
–107
(1996
).24.
G.
Calafiore
,
A.
Koshelev
,
S.
Dhuey
,
A.
Goltsov
,
P.
Sasorov
,
S.
Babin
,
V.
Yankov
,
S.
Cabrini
, and
C.
Peroz
, “
Holographic planar lightwave circuit for on-chip spectroscopy
,” Light: Sci. Appl.
3
, e203
(2014
).25.
S.
Nezhadbadeh
,
A.
Neumann
,
P.
Zarkesh-Ha
, and
S. R. J.
Brueck
, “
Chirped-grating spectrometer-on-a-chip
,” Opt. Express
28
, 24501
–24510
(2020
).26.
P.
Cheben
,
J. H.
Schmid
,
A.
Delâge
,
A.
Densmore
,
S.
Janz
,
B.
Lamontagne
,
J.
Lapointe
,
E.
Post
,
P.
Waldron
, and
D. X.
Xu
, “
A high-resolution silicon-on-insulator arrayed waveguide grating microspectrometer with sub-micrometer aperture waveguides
,” Opt. Express
15
, 2299
–2306
(2007
).27.
M.
Faraji-Dana
,
E.
Arbabi
,
A.
Arbabi
,
S. M.
Kamali
,
H.
Kwon
, and
A.
Faraon
, “
Compact folded metasurface spectrometer
,” Nat. Commun.
9
, 4196
(2018
).28.
P. R.
Griffiths
and
J. A.
De Haseth
, Fourier Transform Infrared Spectrometry
(
John Wiley & Sons
, 2007
).29.
O.
Manzardo
,
H. P.
Herzig
,
C. R.
Marxer
, and
N. F.
de Rooij
, “
Miniaturized time-scanning Fourier transform spectrometer based on silicon technology
,” Opt. Lett.
24
, 1705
–1707
(1999
).30.
U.
Wallrabe
,
C.
Solf
,
J.
Mohr
, and
J. G.
Korvink
, “
Miniaturized Fourier transform spectrometer for the near infrared wavelength regime incorporating an electromagnetic linear actuator
,” Sens. Actuators, A
123–124
, 459
–467
(2005
).31.
W.
Wang
,
S. R.
Samuelson
,
J.
Chen
, and
H.
Xie
, “
Miniaturizing Fourier transform spectrometer with an electrothermal micromirror
,” IEEE Photonics Technol. Lett.
27
, 1418
–1421
(2015
).32.
A. V.
Velasco
,
P.
Cheben
,
P. J.
Bock
,
A.
Delâge
,
J. H.
Schmid
,
J.
Lapointe
,
S.
Janz
,
M. L.
Calvo
,
D.
Xu
,
M.
Florjańczyk
, and
M.
Vachon
, “
High-resolution Fourier-transform spectrometer chip with microphotonic silicon spiral waveguides
,” Opt. Lett.
38
, 706
–708
(2013
).33.
J.
Li
,
D. F.
Lu
, and
Z. M.
Qi
, “
Miniature Fourier transform spectrometer based on wavelength dependence of half-wave voltage of a LiNbO3 waveguide interferometer
,” Opt. Lett.
39
, 3923
–3926
(2014
).34.
M. C.
Souza
,
A.
Grieco
,
N. C.
Frateschi
, and
Y.
Fainman
, “
Fourier transform spectrometer on silicon with thermo-optic non-linearity and dispersion correction
,” Nat. Commun.
9
, 665
(2018
).35.
S. N.
Zheng
,
J.
Zou
,
H.
Cai
,
J. F.
Song
,
L. K.
Chin
,
P. Y.
Liu
,
Z. P.
Lin
,
D. L.
Kwong
, and
A. Q.
Liu
, “
Microring resonator-assisted Fourier transform spectrometer with enhanced resolution and large bandwidth in single chip solution
,” Nat. Commun.
10
, 2349
(2019
).36.
Z.
Yang
,
T.
Albrow-Owen
,
W.
Cai
, and
T.
Hasan
, “
Miniaturization of optical spectrometers
,” Science
371
, 6528
(2021
).37.
W.
Zhang
,
H.
Song
,
X.
He
,
L.
Huang
,
X.
Zhang
,
J.
Zheng
,
W.
Shen
,
X.
Hao
, and
X.
Liu
, “
Deeply learned broadband encoding stochastic hyperspectral imaging
,” Light: Sci. Appl.
10
, 108
(2021
).38.
Z.
Wang
,
S.
Yi
,
A.
Chen
,
M.
Zhou
,
T. S.
Luk
,
A.
James
,
J.
Nogan
,
W.
Ross
,
G.
Joe
,
A.
Shahsafi
,
K.
Wang
,
M. A.
Kats
, and
Z.
Yu
, “
Single-shot on-chip spectral sensors based on photonic crystal slabs
,” Nat. Commun.
10
, 1020
(2019
).39.
Z.
Wang
and
Z.
Yu
, “
Spectral analysis based on compressive sensing in nanophotonic structures
,” Opt. Express
22
, 25608
–25614
(2014
).40.
J.
Meng
,
J. J.
Cadusch
, and
K. B.
Crozier
, “
Detector-only spectrometer based on structurally colored silicon nanowires and a reconstruction algorithm
,” Nano Lett.
20
, 320
–328
(2020
).41.
B.
Zheng
,
L.
Li
,
J.
Wang
,
M.
Zhuge
,
X.
Su
,
Y.
Xu
,
Q.
Yang
,
Y.
Shi
, and
X.
Wang
, “
On‐chip measurement of photoluminescence with high sensitivity monolithic spectrometer
,” Adv. Opt. Mater.
8
, 2000191
(2020
).42.
J.
Dong
, “
The smallest nanowire spectrometers
,” Front. Optoelectron.
12
, 341
–341
(2019
).43.
M. N.
Zhang
,
X.
Wu
,
A.
Riaud
,
X. L.
Wang
,
F.
Xie
,
W. J.
Liu
,
Y.
Mei
,
D. W.
Zhang
, and
S. J.
Ding
, “
Spectrum projection with a bandgap-gradient perovskite cell for colour perception
,” Light: Sci. Appl.
9
, 162
(2020
).44.
H.
Sun
,
W.
Tian
,
X.
Wang
,
K.
Deng
,
J.
Xiong
, and
L.
Li
, “
In situ formed gradient bandgap‐tunable perovskite for ultrahigh‐speed color/spectrum‐sensitive photodetectors via electron‐donor control
,” Adv. Mater.
32
, 1908108
(2020
).45.
Y.
August
and
A.
Stern
, “
Compressive sensing spectrometry based on liquid crystal devices
,” Opt. Lett.
38
, 4996
–4999
(2013
).46.
E.
Huang
,
Q.
Ma
, and
Z.
Liu
, “
Etalon array reconstructive spectrometry
,” Sci. Rep.
7
, 40693
(2017
).47.
Y.
Gao
,
H.
Cansizoglu
,
K. G.
Polat
,
S.
Ghandiparsi
,
A.
Kaya
,
H. H.
Mamtaz
,
A. S.
Mayet
,
Y.
Wang
,
X.
Zhang
,
T.
Yamada
,
E. P.
Devine
,
A. F.
Elrefaie
,
S.
Wang
, and
M. S.
Islam
, “
Photon-trapping microstructures enable high-speed high-efficiency silicon photodiodes
,” Nat. Photonics
11
, 301
–308
(2017
).48.
B.
Craig
,
V. R.
Shrestha
,
J.
Meng
,
J. J.
Cadusch
, and
K. B.
Crozier
, “
Experimental demonstration of infrared spectral reconstruction using plasmonic metasurfaces
,” Opt. Lett.
43
, 4481
–4484
(2018
).49.
J.
Bao
and
M. G.
Bawendi
, “
A colloidal quantum dot spectrometer
,” Nature
523
, 67
–70
(2015
).50.
X.
Zhu
,
L.
Bian
,
H.
Fu
,
L.
Wang
,
B.
Zou
,
Q.
Dai
,
J.
Zhang
, and
H.
Zhong
, “
Broadband perovskite quantum dot spectrometer beyond human visual resolution
,” Light: Sci. Appl.
9
, 73
(2020
).51.
B.
Redding
,
S. F.
Liew
,
Y.
Bromberg
,
R.
Sarma
, and
H.
Cao
, “
Evanescently coupled multimode spiral spectrometer
,” Optica
3
(9
), 956
–962
(2016
).52.
T.
Yang
,
C.
Xu
,
H. P.
Ho
,
Y. Y.
Zhu
,
X. H.
Hong
,
Q. J.
Wang
,
Y. C.
Chen
,
X. A.
Li
,
X. H.
Zhou
,
M. D.
Yi
, and
W.
Huang
, “
Miniature spectrometer based on diffraction in a dispersive hole array
,” Opt. Lett.
40
(13
), 3217
–3220
(2015
).53.
W.
Hartmann
,
P.
Varytis
,
H.
Gehring
,
N.
Walter
,
F.
Beutel
,
K.
Busch
, and
W.
Pernice
, “
Waveguide‐integrated broadband spectrometer based on tailored disorder
,” Adv. Opt. Mater.
8
(6
), 1901602
(2020
).54.
T.
Ermatov
,
R. E.
Noskov
,
A. A.
Machnev
,
I.
Gnusov
,
V.
Atkin
,
E. N.
Lazareva
,
S. V.
German
,
S. S.
Kosolobov
,
T. S.
Zatsepin
,
O. V.
Sergeeva
,
J. S.
Skibina
,
P.
Ginzburg
,
V. V.
Tuchin
,
P. G.
Lagoudakis
, and
D. A.
Gorin
, “
Multispectral sensing of biological liquids with hollow-core microstructured optical fibres
,” Light: Sci. Appl.
9
(1
), 173
(2020
).55.
S.
Perevoschikov
,
N.
Kaydanov
,
T.
Ermatov
,
O.
Bibikova
,
I.
Usenov
,
T.
Sakharova
,
A.
Bocharnikov
,
J.
Skibina
,
V.
Artyushenko
, and
D.
Gorin
, “
Light guidance up to 6.5 μm in borosilicate soft glass hollow-core microstructured optical waveguides
,” Opt. Express
28
(19
), 27940
–27950
(2020
).56.
O.
Avayu
,
E.
Almeida
,
Y.
Prior
, and
T.
Ellenbogen
, “
Composite functional metasurfaces for multispectral achromatic optics
,” Nat. Commun.
8
(1
), 14992
(2017
).57.
S.
Karepov
and
T.
Ellenbogen
, “
Metasurface-based contact lenses for color vision deficiency
,” Opt. Lett.
45
(6
), 1379
–1382
(2020
).58.
A.
Wang
,
P.
Gill
, and
A.
Molnar
, “
Light field image sensors based on the Talbot effect
,” Appl. Opt.
48
, 5897
–5905
(2009
).59.
S.
Sivaramakrishnan
,
A.
Wang
,
P. R.
Gill
, and
A.
Molnar
, “
Enhanced angle sensitive pixels for light field imaging
,” in International Electron Devices Meeting
(
IEEE
, 2011
).60.
S.
Yi
,
M.
Zhou
,
Z.
Yu
,
P.
Fan
,
N.
Behdad
,
D.
Lin
,
K. X.
Wang
,
S.
Fan
, and
M.
Brongersma
, “
Subwavelength angle-sensing photodetectors inspired by directional hearing in small animals
,” Nat. Nanotechnol.
13
, 1143
–1147
(2018
).61.
E. H.
Adelson
and
J. Y.
Wang
, “
Single lens stereo with a plenoptic camera
,” IEEE PAMI
14
, 99
–106
(1992
).62.
R.
Ng
,
M.
Levoy
,
M.
Brédif
,
G.
Duval
,
M.
Horowitz
, and
P.
Hanrahan
, “
Light field photography with a hand-held plenoptic camera
,” Doctoral dissertation (
Stanford University
, 2005
).63.
A.
Wang
,
P. R.
Gill
, and
A.
Molnar
, “
An angle-sensitive CMOS imager for single-sensor 3D photography
,” in IEEE International Solid-State Circuits Conference
(
IEEE
, 2011
), pp. 412
–414
.64.
P. L.
Wizinowich
,
D. L.
Mignant
,
A. H.
Bouchez
,
R. D.
Campbell
,
J. C.
Chin
,
A. R.
Contos
,
A.
Marcos
,
K. H.
Scott
,
M. J.
Erik
, and
D. M.
Pennington
, “
The W. M. Keck observatory laser guide star adaptive optics system: Overview
,” Publ. Astron. Soc. Pac.
118
, 297
–309
(2006
).65.
R.
Davies
and
M.
Kasper
, “
Adaptive optics for astronomy
,” Annu. Rev. Astron. Astrophys.
50
, 305
–351
(2012
).66.
J.
Liang
,
D. R.
Williams
, and
D. T.
Miller
, “
Supernormal vision and high-resolution retinal imaging through adaptive optics
,” J. Opt. Soc. Am. A
14
, 2884
–2892
(1997
).67.
K.
Wang
,
W.
Sun
,
C. T.
Richie
,
B. K.
Harvey
,
E.
Betzig
, and
N.
Ji
, “
Direct wavefront sensing for high-resolution in vivo imaging in scattering tissue
,” Nat. Commun.
6
, 7276
(2015
).68.
S.
Yi
,
J.
Xiang
,
M.
Zhou
,
Z.
Wu
,
L.
Yang
, and
Z.
Yu
, “
Angle-based wavefront sensing enabled by the near fields of flat optics
,” Nat. Commun.
12
, 1
–8
(2021
).69.
D. L.
Fried
, “
Least-square fitting a wave-front distortion estimate to an array of phase-difference measurements
,” J. Opt. Soc. Am.
67
, 370
–375
(1977
).70.
R.
Cubalchini
, “
Modal wave-front estimation from phase derivative measurements
,” J. Opt. Soc. Am.
69
, 972
–977
(1979
).71.
W. H.
Southwell
, “
Wave-front estimation from wave-front slope measurements
,” J. Opt. Soc. Am.
70
, 998
–1006
(1980
).72.
P.
Bon
,
G.
Maucort
,
B.
Wattellier
, and
S.
Monneret
, “
Quadriwave lateral shearing interferometry for quantitative phase microscopy of living cells
,” Opt. Express
17
, 13080
–13094
(2009
).73.
Y.
Park
,
C.
Depeursinge
, and
G.
Popescu
, “
Quantitative phase imaging in biomedicine
,” Nat. Photonics
12
, 578
–589
(2018
).74.
J. S.
Tyo
,
D. L.
Goldstein
,
D. B.
Chenault
, and
J. A.
Shaw
, “
Review of passive imaging polarimetry for remote sensing applications
,” Appl. Opt.
45
(22
), 5453
–5469
(2006
).75.
D. L.
Coffeen
, “
Polarization and scattering characteristics in the atmospheres of Earth, Venus, and Jupiter
,” J. Opt. Soc. Am.
69
, 1051
–1064
(1979
).76.
W. G.
Egan
,
W. R.
Johnson
, and
V. S.
Whitehead
, “
Terrestrial polarization imagery obtained from the Space Shuttle: Characterization and interpretation
,” Appl. Opt.
30
, 435
–442
(1991
).77.
M.
Garcia
,
C.
Edmiston
,
R.
Marinov
,
A.
Vail
, and
V.
Gruev
, “
Bio-inspired color-polarization imager for real-time in situ imaging
,” Optica
4
(10
), 1263
–1271
(2017
).78.
N. W.
Roberts
,
M. J.
How
,
M. L.
Porter
,
S. E.
Temple
,
R. L.
Caldwell
,
S. B.
Powell
,
V.
Gruev
,
N. J.
Marshall
, and
T. W.
Cronin
, “
Animal polarization imaging and implications for optical processing
,” Proc. IEEE
102
(10
), 1427
–1434
(2014
).79.
T.
Charanya
,
T.
York
,
S.
Bloch
,
G.
Sudlow
,
K.
Liang
,
M.
Garcia
,
W. J.
Akers
,
D.
Rubin
,
V.
Gruev
, and
S.
Achilefu
, “
Trimodal colorfluorescence-polarization endoscopy aided by a tumor selective molecular probe accurately detects flat lesions in colitis-associated cancer
,” J. Biomed. Opt.
19
, 126002
(2014
).80.
R.
Walraven
, “
Polarization imagery
,” Opt. Eng.
20
, 200114
(1981
).81.
R. M. A.
Azzam
, “
Arrangement of four photodetectors for measuring the state of polarization of light
,” Opt. Lett.
10
, 309
–311
(1985
).82.
J. D.
Barter
,
P. H. Y.
Lee
, and
H. R.
Thompson
, “
Stokes parameter imaging of scattering surfaces
,” in Polarization: Measurement, Analysis, and Remote Sensing
(
SPIE
, 1997
), Vol.
3121
, pp. 314
–320
.83.
J. L.
Pezzaniti
and
D. B.
Chenault
, “
A division of aperture MWIR imaging polarimeter
,” in Polarization Science and Remote Sensing II
, edited by
J. A.
Shaw
and
J. S.
Tyo
(
SPIE
, 2005
).84.
R. A.
Chipman
, “
Polarization analysis of optical systems
,” Opt. Eng.
28
, 280290
(1989
).85.
R.
Perkins
and
V.
Gruev
, “
Signal-to-noise analysis of Stokes parameters in division of focal plane polarimeters
,” Opt. Express
18
, 25815
–25824
(2010
).86.
A.
Arbabi
,
E.
Arbabi
,
S. M.
Kamali
,
Y.
Horie
,
S.
Han
, and
A.
Faraon
, “
Miniature optical planar camera based on a wide-angle metasurface doublet corrected for monochromatic aberrations
,” Nat. Commun.
7
, 13682
(2016
).87.
B.
Groever
,
W. T.
Chen
, and
F.
Capasso
, “
Meta-lens doublet in the visible region
,” Nano Lett.
17
, 4902
–4907
(2017
).88.
N. A.
Rubin
,
P.
Chevalier
,
M.
Juhl
,
M.
Tamagnone
,
R.
Chipman
, and
F.
Capasso
, “
Imaging polarimetry through metasurface polarization gratings
,” Opt. Express
30
(6
), 9389
–9412
(2022
).89.
N. A.
Rubin
,
G.
D'Aversa
,
P.
Chevalier
,
Z.
Shi
,
W. T.
Chen
, and
F.
Capasso
, “
Matrix Fourier optics enables a compact full-Stokes polarization camera
,” Science
365
(6448
), eaax1839
(2019
).90.
E.
Arbabi
,
S. M.
Kamali
,
A.
Arbabi
, and
A.
Faraon
, “
Full-Stokes imaging polarimetry using dielectric metasurfaces
,” ACS Photonics
5
(8
), 3132
–3140
(2018
).91.
C.
Brown
,
A.
Goncharov
,
Z. S.
Ballard
,
M.
Fordham
,
A.
Clemens
,
Y.
Qiu
,
Y.
Rivenson
, and
A.
Ozcan
, “
Neural network based on-chip spectroscopy using a scalable plasmonic encoder
,” ACS Nano
15
, 6305
–6315
(2021
).92.
J.
Zhang
,
X.
Zhu
, and
J.
Bao
, “
Solver-informed neural networks for spectrum reconstruction of colloidal quantum dot spectrometers
,” Opt. Express
28
, 33656
–33672
(2020
).93.
J.
Zhang
,
X.
Zhu
, and
J.
Bao
, “
Denoising autoencoder aided spectrum reconstruction for colloidal quantum dot spectrometers
,” IEEE Sens. J.
21
, 6450
–6458
(2021
).94.
C.
Kim
,
D.
Park
, and
H.-N.
Lee
, “
Convolutional neural networks for the reconstruction of spectra in compressive sensing spectrometers
,” in Conference on Optical Data Science II
,
San Francisco, CA
(
SPIE
2019
), p. 109370L
.95.
S.
Li
,
X.
Zhu
, and
J.
Bao
, “
Hierarchical multi-scale convolutional neural networks for hyperspectral image classification
,” Sensors
19
, 1714
(2019
).96.
C.
Kim
,
D.
Park
, and
H.-N.
Lee
, “
Compressive sensing spectroscopy using a residual convolutional neural network
,” Sensors
20
, 594
(2020
).97.
L.
Gao
,
Y.
Qu
,
L.
Wang
, and
Z.
Yu
, “
Computational spectrometers enabled by nanophotonics and deep learning
,” Nanophotonics
11
, 2507
(2022
).98.
Y.
Rivenson
,
Y.
Zhang
,
H.
Günaydın
,
D.
Teng
, and
A.
Ozcan
, “
Phase recovery and holographic image reconstruction using deep learning in neural networks
,” Light: Sci. Appl.
7
(2
), 17141
(2018
).99.
Y.
Rivenson
,
Z.
Göröcs
,
H.
Günaydin
,
Y.
Zhang
,
H.
Wang
, and
A.
Ozcan
, “
Deep learning microscopy
,” Optica
4
(11
), 1437
–1443
(2017
).© 2022 Author(s). Published under an exclusive license by AIP Publishing.
2022
Author(s)
You do not currently have access to this content.
