Taking advantage of optoelectronic hybrid neural networks, we propose a metasurface-single-pixel hybrid neural network for object recognition. It employs only eight illumination patterns trained by the digital neural network to convolve the object from two-dimensional images into only eight intensity values measured by a single-pixel detector, achieving a 93.8% accuracy rate in handwritten digit recognition. Our work therefore paves an image-free way for metasurface-based object recognition using only a single-pixel detector, which exhibits its powerful information compression and accurate extraction capabilities coupled with a compact structural design.
REFERENCES
1.
S.
Noy
and
W.
Zhang
, “
Experimental evidence on the productivity effects of generative artificial intelligence
,” Science
381
, 187
(2023
).2.
T.
Wang
,
S.-Y.
Ma
,
L. G.
Wright
et al, “
An optical neural network using less than 1 photon per multiplication
,” Nat. Commun.
13
, 123
(2022
).3.
Z.
Zhang
,
F.
Feng
,
J.
Gan
et al, “
Space-time projection enabled ultrafast all-optical diffractive neural network
,” Laser Photonics Rev.
18
, 2301367
(2024
).4.
G.
Qu
,
G.
Cai
,
X.
Sha
et al, “
All-dielectric metasurface empowered optical-electronic hybrid neural networks
,” Laser Photonics Rev.
16
, 2100732
(2022
).5.
J.
Sun
,
Y.
Choi
,
Y. J.
Choi
et al, “
2D-organic hybrid heterostructures for optoelectronic applications
,” Adv. Mater.
31
, 1803831
(2019
).6.
W.
Shi
,
Z.
Huang
,
H.
Huang
et al, “
LOEN: Lensless optoelectronic neural network empowered machine vision
,” Light
11
, 121
(2022
).7.
X.
Xu
,
M.
Tan
,
B.
Corcoran
et al, “
11 TOPS photonic convolutional accelerator for optical neural networks
,” Nature
589
, 44
(2021
).8.
G.
Barbastathis
,
A.
Ozcan
, and
G.
Situ
, “
On the use of deep learning for computational imaging
,” Optica
6
, 921
(2019
).9.
K.
Yadav
,
S.
Bidnyk
, and
A.
Balakrishnan
, “
Artificial intelligence and machine learning in optics: Tutorial
,” J. Opt. Soc. Am. B
41
, 1739
(2024
).10.
T. M.
Cover
, “
Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition
,” IEEE Trans. Electron. Comput.
EC-14
, 326
(1965
).11.
J.
Nickolls
and
W. J.
Dally
, “
The GPU computing era
,” IEEE Micro
30
, 56
(2010
).12.
S.
Zhang
, “
Intelligent metasurfaces: Digitalized, programmable, and intelligent platforms
,” Light
11
, 242
(2022
).13.
Z.
Wang
,
G.
Hu
,
X.
Wang
et al, “
Single-layer spatial analog meta-processor for imaging processing
,” Nat. Commun.
13
, 2188
(2022
).14.
C.
Qian
,
Z.
Wang
,
H.
Qian
et al, “
Dynamic recognition and mirage using neuro-metamaterials
,” Nat. Commun.
13
, 2694
(2022
).15.
P.
del Hougne
,
M. F.
Imani
,
A. V.
Diebold
et al, “
Learned integrated sensing pipeline: Reconfigurable metasurface transceivers as trainable physical layer in an artificial neural network
,” Adv. Sci.
7
, 1901913
(2020
).16.
C.
Wu
,
H.
Yu
,
S.
Lee
et al, “
Programmable phase-change metasurfaces on waveguides for multimode photonic convolutional neural network
,” Nat. Commun.
12
, 96
(2021
).17.
X.
Luo
,
Y.
Hu
,
X.
Ou
et al, “
Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible
,” Light
11
, 158
(2022
).18.
J.
Feldmann
,
N.
Youngblood
,
C. D.
Wright
et al, “
Parallel convolutional processing using an integrated photonic tensor core
,” Nature
589
, 52
(2021
).19.
R.
Hamerly
,
L.
Bernstein
,
A.
Sludds
et al, “
Large-scale optical neural networks based on photoelectric multiplication
,” Phys. Rev. X
9
, 021032
(2019
).20.
J.
Feldmann
,
N.
Youngblood
,
C. D.
Wright
et al, “
All-optical spiking neurosynaptic networks with self-learning capabilities
,” Nature
569
, 208
(2019
).21.
H.
Zhang
,
M.
Gu
,
X. D.
Jiang
et al, “
An optical neural chip for implementing complex-valued neural network
,” Nat. Commun.
12
, 457
(2021
).22.
J.
Fang
,
H.
Lin
,
X.
Chen
et al, “
A hybrid network of CNN and transformer for lightweight image super-resolution
,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
, 2022
.23.
L.
Li
,
Y.
Shuang
,
Q.
Ma
et al, “
Intelligent metasurface imager and recognizer
,” Light
8
, 97
(2019
).24.
M. K.
Chen
,
X.
Liu
,
Y.
Sun
et al, “
Artificial intelligence in meta-optics
,” Chem. Rev.
122
, 15356
(2022
).25.
J.
Jahns
and
A.
Huang
, “
Planar integration of free-space optical components
,” Appl. Opt.
28
, 1602
(1989
).26.
X.
Yuan
,
Y.
Wang
,
Z.
Xu
et al, “
Training large-scale optoelectronic neural networks with dual-neuron optical-artificial learning
,” Nat. Commun.
14
, 7110
(2023
).27.
T.
Yan
,
J.
Wu
,
T.
Zhou
et al, “
Fourier-space diffractive deep neural network
,” Phys. Rev. Lett.
123
, 023901
(2019
).28.
Y.
Luo
,
D.
Mengu
,
N. T.
Yardimci
et al, “
Design of task-specific optical systems using broadband diffractive neural networks
,” Light
8
, 112
(2019
).29.
X.
Lin
,
Y.
Rivenson
,
N. T.
Yardimci
et al, “
All-optical machine learning using diffractive deep neural networks
,” Science
361
, 1004
(2018
).30.
J. H.
Song
,
J.
van de Groep
,
S. J.
Kim
et al, “
Non-local metasurfaces for spectrally decoupled wavefront manipulation and eye tracking
,” Nat. Nanotechnol.
16
, 1224
(2021
).31.
J.
Zhou
,
H.
Qian
,
C.-F.
Chen
et al, “
Optical edge detection based on high-efficiency dielectric metasurface
,” Proc. Natl. Acad. Sci. U. S. A.
116
, 11137
(2019
).32.
J.
Xiong
,
Z.-H.
Zhang
,
Z.
Li
et al, “
Perovskite single-pixel detector for dual-color metasurface imaging recognition in complex environment
,” Light
12
, 286
(2023
).33.
Q.
Yang
,
H.
Xiong
,
J. H.
Deng
et al, “
Polarization-insensitive composite gradient-index metasurface array for microwave power reception
,” Appl. Phys. Lett.
122
, 253901
(2023
).34.
H.
Ren
,
X.
Fang
,
J.
Jang
et al, “
Complex-amplitude metasurface-based orbital angular momentum holography in momentum space
,” Nat. Nanotechnol.
15
, 948
(2020
).35.
Y. F.
Yu
,
A. Y.
Zhu
,
R.
Paniagua‐Domínguez
et al, “
High-transmission dielectric metasurface with 2π phase control at visible wavelengths
,” Laser Photonics Rev.
9
, 412
(2015
).36.
M. P.
Edgar
,
G. M.
Gibson
, and
M. J.
Padgett
, “
Principles and prospects for single-pixel imaging
,” Nat. Photonics
13
, 13
(2019
).37.
E.
Tajahuerce
,
V.
Durán
,
P.
Clemente
et al, “
Image transmission through dynamic scattering media by single-pixel photodetection
,” Opt. Express
22
, 16945
(2014
).38.
J.
Kim
,
T.
Jeong
,
S.-Y.
Lee
et al, “
Heralded single-pixel imaging with high loss-resistance and noise-robustness
,” Appl. Phys. Lett.
119
, 244002
(2021
).39.
Y.
LeCun
,
L.
Bottou
,
Y.
Bengio
et al, “
Gradient-based learning applied to document recognition
,” Proc. IEEE
86
, 2278
–2324
(1998
).40.
S.
Zhang
,
P.
Xiao
,
X.
Hong
et al, “
All-in-one hardware devices with event-based vision sensor arrays for image sensing, computing, and learning
,” Adv. Funct. Mater.
33
, 2306173
(2023
).41.
Y.
Wang
,
Y.
Zhu
,
Y.
Li
et al, “
Dual-modal optoelectronic synaptic devices with versatile synaptic plasticity
,” Adv. Funct. Mater.
32
, 2107973
(2022
).42.
W.
Pei
,
Y.
Li
,
S.
Siuly
et al, “
A hybrid deep learning scheme for multi-channel sleep stage classification
,” Comput. Mater. Continua
71
, 889
(2022
).43.
P.
Zheng
,
Q.
Dai
,
Z.
Li
et al, “
Metasurface-based key for computational imaging encryption
,” Sci. Adv.
7
, eabg0363
(2021
).44.
Y.
Peng
and
W.
Chen
, “
Deep learning-enhanced ghost imaging through dynamic and complex scattering media with supervised corrections of dynamic scaling factors
,” Appl. Phys. Lett.
124
, 181104
(2024
).45.
S.
Zhu
,
E.
Guo
,
J.
Gu
et al, “
Imaging through unknown scattering media based on physics-informed learning
,” Photonics Res.
9
, B210
(2021
).46.
P.
Wang
,
K.
Guo
, and
Z.
Guo
, “
Single-pixel imaging through scattering media employing a circular polarization multiplexing metasurface
,” Appl. Opt.
63
, 9029
(2024
).47.
Y.
Sun
,
J.
Shi
,
L.
Sun
et al, “
Image reconstruction through dynamic scattering media based on deep learning
,” Opt. Express
27
, 16032
(2019
).48.
Z.
Zhang
,
X.
Li
,
S.
Zheng
et al, “
Image-free classification of fast-moving objects using “learned” structured illumination and single-pixel detection
,” Opt. Express
28
, 13269
(2020
).49.
C. F.
Higham
,
R.
Murray-Smith
,
M. J.
Padgett
et al, “
Deep learning for real-time single-pixel video
,” Sci. Rep.
8
, 2369
(2018
).50.
V.-C.
Su
,
C. H.
Chu
,
G.
Sun
et al, “
Advances in optical metasurfaces: Fabrication and applications
,” Opt. Express
26
, 13148
(2018
).51.
K.
Yi
,
D.
Liu
,
X.
Chen
et al, “
Plasma-enhanced chemical vapor deposition of two-dimensional materials for applications
,” Acc. Chem. Res.
54
, 1011
(2021
).52.
L.
Bernstein
,
A.
Sludds
,
C.
Panuski
et al, “
Single-shot optical neural network
,” Sci. Adv.
9
, eadg7904
(2023
).53.
C.
Liu
,
Q.
Ma
,
Z. J.
Luo
et al, “
A programmable diffractive deep neural network based on a digital-coding metasurface array
,” Nat. Electron.
5
, 113
(2022
).54.
H.
Zheng
,
Q.
Liu
,
Y.
Zhou
et al, “
Meta-optic accelerators for object classifiers
,” Sci. Adv.
8
, eabo6410
(2022
).55.
H.
Zheng
,
Q.
Liu
,
I. I.
Kravchenko
et al, “
Multichannel meta-imagers for accelerating machine vision
,” Nat. Nanotechnol.
19
, 471
(2024
).© 2025 Author(s). Published under an exclusive license by AIP Publishing.
2025
Author(s)
You do not currently have access to this content.
