Photonic delay systems have revolutionized the hardware implementation of Recurrent Neural Networks and Reservoir Computing in particular. The fundamental principles of Reservoir Computing strongly facilitate a realization in such complex analog systems. Especially delay systems, which potentially provide large numbers of degrees of freedom even in simple architectures, can efficiently be exploited for information processing. The numerous demonstrations of their performance led to a revival of photonic Artificial Neural Network. Today, an astonishing variety of physical substrates, implementation techniques as well as network architectures based on this approach have been successfully employed. Important fundamental aspects of analog hardware Artificial Neural Networks have been investigated, and multiple high-performance applications have been demonstrated. Here, we introduce and explain the most relevant aspects of Artificial Neural Networks and delay systems, the seminal experimental demonstrations of Reservoir Computing in photonic delay systems, plus the most recent and advanced realizations.

1.
W. S.
McCulloch
and
W.
Pitts
,
Bull. Math. Biophys.
5
,
115
(
1943
).
2.
P. J.
Werbos
, “
Beyond regression: New tools for prediction and analysis in the behavioral sciences
,” Ph.D. thesis (
Harvard University
,
1974
).
3.
Y.
LeCun
,
Y.
Bengio
, and
G. E.
Hinton
,
Nature
521
,
436
(
2015
).
4.
B.
Amos
,
B.
Ludwiczuk
, and
M.
Satyanarayanan
, “OpenFace: A general-purpose face recognition library with mobile applications,” Tech. Rep. No. CMU-CS-16-118 (CMU School of Computer Science, 2016).
5.
A.
Graves
,
A.
Mohamed
, and
G.
Hinton
, in
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
(
IEEE
,
2013
), pp.
6645
6649
.
6.
H.
Jaeger
,
German National Research Center for Information Technology, GMD Technical Report 148
, 34 (
2001
).
7.
W.
Maass
,
T.
Natschläger
, and
H.
Markram
,
Neural Comput.
14
,
2531
(
2002
).
8.
H.
Jaeger
and
H.
Haas
,
Science
304
,
78
(
2004
).
9.
J. J.
Steil
, in
Proceedings of the International Joint Conference on Neural Networks (IJCNN)
(
2004
), Vol. 1, pp.
843
848
.
10.
L.
Appeltant
,
M. C.
Soriano
,
G. V. D.
Sande
,
J.
Danckaert
,
S.
Massar
,
J.
Dambre
,
B.
Schrauwen
,
C. R.
Mirasso
,
I.
Fischer
,
G.
Van der Sande
,
J.
Danckaert
,
S.
Massar
,
J.
Dambre
,
B.
Schrauwen
,
C. R.
Mirasso
, and
I.
Fischer
,
Nat. Commun.
2
,
468
(
2011
).
11.
L.
Larger
,
M. C.
Soriano
,
D.
Brunner
,
L.
Appeltant
,
J. M.
Gutierrez
,
L.
Pesquera
,
C. R.
Mirasso
, and
I.
Fischer
,
Opt. Express
20
,
3241
(
2012
).
12.
Y.
Paquot
,
F.
Duport
,
A.
Smerieri
,
J.
Dambre
,
B.
Schrauwen
,
M.
Haelterman
, and
S.
Massar
,
Sci. Rep.
2
,
287
(
2012
).
13.
F.
Duport
,
B.
Schneider
,
A.
Smerieri
,
M.
Haelterman
, and
S.
Massar
,
Opt. Express
20
,
22783
(
2012
).
14.
D.
Brunner
,
M. C.
Soriano
,
C. R.
Mirasso
, and
I.
Fischer
,
Nat. Commun.
4
,
1364
(
2013
).
15.
P.
Antonik
,
F.
Duport
,
M.
Hermans
,
A.
Smerieri
,
M.
Haelterman
, and
S.
Massar
,
IEEE Trans. Neural Netw. Learn. Syst.
PP
,
1
(
2016
).
16.
A.
Argyris
,
J.
Bueno
, and
I.
Fischer
,
Sci. Rep.
8
,
8487
(
2018
).
17.
R.
Martinenghi
,
S.
Rybalko
,
M.
Jacquot
,
Y. K.
Chembo
, and
L.
Larger
,
Phys. Rev. Lett.
108
,
244101
(
2012
).
18.
M.
Hermans
,
M. C.
Soriano
,
J.
Dambre
,
P.
Bienstman
, and
I.
Fischer
,
J. Mach. Learn. Res.
16
,
2081
(
2015
). arXiv:1501.02592.
19.
G.
Van der Sande
,
D.
Brunner
, and
M. C.
Soriano
,
Nanophotonics
6
,
561
(
2017
).
20.
I.
Goodfellow
,
Y.
Bengio
, and
A.
Courville
,
Deep Learning
(
MIT Press
,
2016
). http://www.deeplearningbook.org
21.
D. E.
Rumelhart
,
G. E.
Hinton
, and
R. J.
Williams
,
Institute for Cognitive Science Report No. 8506
(
1986
), p.
318
.
22.
D. E.
Rumelhart
,
G. E.
Hinton
, and
R. J.
Williams
,
Nature
323
,
533
(
1986
).
23.
K.
Funahashi
and
Y.
Nakamura
,
Neural Netw.
6
,
801
(
1993
).
24.
J.
Kilian
and
H. T.
Siegelmann
,
Inf. Comput.
128
,
48
(
1996
).
25.
L.
Grigoryeva
and
J.-P.
Ortega
,
Neural Netw.
arXiv:1806.00797.
26.
R. J.
Douglas
and
K. A.
Martin
,
Curr. Biol.
27
,
496
(
2004
).
27.
R.
Pascanu
,
T.
Mikolov
, and
Y.
Bengio
, in Proceedings of the 30th International Conference on Machine Learning (JMLR, 2012), p. 1310, arXiv:1211.5063v2.
28.
N. P.
Jouppi
,
C.
Young
,
N.
Patil
,
D.
Patterson
, et al., Google, Inc., Mountain View, CA USA 2017. In - Datacenter Performance Analysis of a Tensor Processing Unit, Proceedings of ISCA ’17, Toronto, ON, Canada, June 24–28, 2017.
29.
J. W.
Goodman
,
Introduction to Fourier Optics
(
Roberts and Co
,
2005
), p.
491
.
30.
C. S.
Weaver
and
J. W.
Goodman
,
Appl. Opt.
5
,
1248
(
1966
).
31.
S. D.
Smith
,
Nature
316
,
319
(
1985
).
32.
D. J.
Richardson
,
J. M.
Fini
, and
L. E.
Nelson
,
Nat. Photonics
7
,
354
(
2013
).
33.
A. N.
Tait
,
S.
Member
,
M. A.
Nahmias
,
B. J.
Shastri
, and
P. R.
Prucnal
,
J. Lightwave Technol.
32
,
3427
(
2014
).
34.
G. V.
Demidenko
,
V. A.
Likhoshvai
, and
A. V.
Mudrov
,
Differ. Equ.
45
,
33
(
2009
).
35.
J. C.
Sprott
,
Phys. Lett. A
366
,
397
(
2007
).
36.
M. C.
Soriano
,
D.
Brunner
,
M.
Escalona-Morán
,
C. R.
Mirasso
, and
I.
Fischer
,
Front. Comput. Neurosci.
9
,
68
(
2015
).
37.
A.
Neyer
and
E.
Voges
,
IEEE J. Quantum Electron.
QE-18
,
2009
(
1982
).
38.
L.
Larger
,
P.-A.
Lacourt
,
S.
Poinsot
, and
M.
Hanna
,
Phys. Rev. Lett.
95
,
043903
(
2005
).
39.
Y. C.
Kouomou
,
P.
Colet
,
L.
Larger
, and
N.
Gastaud
,
Phys. Rev. Lett.
95
,
203903
(
2005
).
40.
K. E.
Callan
,
L.
Illing
,
Z.
Gao
,
D. J.
Gauthier
, and
E.
Scholl
,
Phys. Rev. Lett.
104
,
113901
(
2010
).
41.
M.
Soriano
,
J.
García-Ojalvo
,
C.
Mirasso
, and
I.
Fischer
,
Rev. Mod. Phys.
85
,
421
(
2013
).
42.
A.
Argyris
,
D.
Syvridis
,
L.
Larger
,
V.
Annovazzi-Lodi
,
P.
Colet
,
I.
Fischer
,
J.
Garcia-Ojalvo
,
C. R.
Mirasso
,
L.
Pesquera
, and
K. A.
Shore
,
Nature
438
,
343
(
2005
).
43.
L.
Larger
,
Phil. Trans. R. Soc. A
371
,
20120464
(
2013
).
44.
M.
Peil
,
I.
Fischer
,
W.
Elsäßer
,
S.
Bakić
,
N.
Damaschke
,
C.
Tropea
,
S.
Stry
, and
J.
Sacher
,
Appl. Phys. Lett.
89
,
091106
(
2006
).
45.
X. S.
Yao
and
L.
Maleki
,
Electron. Lett.
30
,
1525
(
1994
).
46.
Y. K.
Chembo
,
K.
Volyanskiy
,
L.
Larger
,
E.
Rubiola
, and
P.
Colet
,
IEEE J. Quantum Electron.
45
,
178
(
2009
).
47.
L.
Maleki
,
Nat. Photonics
5
,
728
(
2011
).
48.
D.
Brunner
,
R.
Luna
,
A.
Delhom i Latorre
,
X.
Porte
, and
I.
Fischer
,
Opt. Lett.
42
,
163
(
2017
).
49.
A.
Uchida
,
K.
Amano
,
M.
Inoue
,
K.
Hirano
,
S.
Naito
,
H.
Someya
,
I.
Oowada
,
T.
Kurashige
,
M.
Shiki
,
S.
Yoshimori
,
K.
Yoshimura
, and
P.
Davis
,
Nat. Photonics
2
,
728
(
2008
).
50.
X.
Fang
,
B.
Wetzel
,
J.-M.
Merolla
,
J. M.
Dudley
,
L.
Larger
,
C.
Guyeux
, and
J. M.
Bahi
,
IEEE Trans. Circ. Syst. I
61
,
888
(
2014
).
51.
A. B.
Cohen
,
B.
Ravoori
,
T. E.
Murphy
, and
R.
Roy
,
Phys. Rev. Lett.
101
,
154102
(
2008
).
52.
T. E.
Murphy
et al.,
Phil. Trans. R. Soc. A
368
,
343
(
2010
).
53.
B.
Ravoori
,
A. B.
Cohen
,
J.
Sun
,
A. E.
Motter
,
T. E.
Murphy
, and
R.
Roy
,
Phys. Rev. Lett.
107
,
034102
(
2011
).
54.
L.
Illing
,
G.
Hoth
,
L.
Shareshian
, and
C.
May
,
Phys. Rev. E.
83
,
026107
(
2011
).
55.
L.
Illing
,
C. D.
Panda
, and
L.
Shareshian
,
Phys. Rev. E.
84
,
016213
(
2011
).
56.
D.
Brunner
,
M. C.
Soriano
,
X.
Porte
, and
I.
Fischer
,
Phys. Rev. Lett.
115
,
053901
(
2015
).
57.
B.
McNamara
,
K.
Wiesenfeld
, and
R.
Roy
,
Phys. Rev. Lett.
60
,
2626
(
1988
).
58.
A.
Uchida
,
R.
McAllister
, and
R.
Roy
,
Phys. Rev. Lett.
93
,
244102
(
2004
).
59.
B. A.
Marquez
,
L.
Larger
,
M.
Jacquot
,
Y. K.
Chembo
, and
D.
Brunner
,
Sci. Rep.
8
,
3319
(
2018
).
60.
N.
Oliver
,
L.
Larger
, and
I.
Fischer
,
Chaos Interdiscip. J. Nonlinear Sci.
26
,
103115
(
2016
).
61.
F. T.
Arecchi
,
G.
Giacomelli
,
A.
Lapucci
, and
R.
Meucci
,
Phys. Rev. A
45
,
4225
(
1992
).
62.
L.
Larger
,
B.
Penkovsky
, and
Y.
Maistrenko
,
Nat. Commun.
6
,
7752
(
2015
). arXiv:1411.4483v1.
63.
L.
Larger
,
A.
Baylón-Fuentes
,
R.
Martinenghi
,
V. S.
Udaltsov
,
Y. K.
Chembo
, and
M.
Jacquot
,
Phys. Rev. X
7
,
011015
(
2017
).
64.
J.
Bueno
,
S.
Maktoobi
,
L.
Froehly
,
I.
Fischer
,
M.
Jacquot
,
L.
Larger
, and
D.
Brunner
,
Optica
5
,
756
(
2018
). arXiv:1711.05133.
65.
A.
Rodan
and
P.
Tino
,
IEEE Trans. Neural Netw.
22
,
131
(
2011
).
66.
C.
Fernando
and
S.
Sojakka
, in
Advances in Artificial Life
, Lecture Notes in Computer Science, edited by
W.
Banzhaf
,
J.
Ziegler
,
T.
Christaller
,
P.
Dittrich
, and
J. T.
Kim
(
Springer
,
Berlin
,
2003
), pp.
588
597
.
67.
K.
Vandoorne
,
W.
Dierckx
,
B.
Schrauwen
,
D.
Verstraeten
,
R.
Baets
,
P.
Bienstman
,
J.
Van Campenhout
,
Opt. Express
16
,
11182
(
2008
).
68.
L.
Larger
,
J. P.
Goedgebuer
, and
V. S.
Udaltsov
,
C. R. Phys.
5
,
669
(
2004
).
69.
V.
Udaltsov
,
L.
Larger
,
J.-P.
Goedgebuer
,
M. W.
Lee
,
E.
Genin
, and
W.
Rhodes
,
IEEE Trans. Circuits Syst. I Fundam. Theory Appl.
49
,
1006
(
2002
).
70.
G.-B.
Huang
,
Q.-Y.
Zhu
, and
C.-K.
Siew
, in
2004 IEEE International Joint Conference on Neural Networks
(
IEEE
,
2004
), Vol. 2, pp.
985
990
.
71.
S.
Ortín
,
M. C.
Soriano
,
L.
Pesquera
,
D.
Brunner
,
D.
San-Martín
,
I.
Fischer
,
C. R.
Mirasso
, and
J. M.
Gutiérrez
,
Sci. Rep.
5
,
14945
(
2015
).
72.
K.
Vandoorne
,
P.
Mechet
,
T.
Van Vaerenbergh
,
M.
Fiers
,
G.
Morthier
,
D.
Verstraeten
,
B.
Schrauwen
,
J.
Dambre
, and
P.
Bienstman
,
Nat. Commun.
5
,
1
(
2014
).
73.
F. C.
Hoppensteadt
and
E. M.
Izhikevich
,
Phys. Rev. E
62
,
4010
(
2000
).
74.
S.
Wieczorek
,
B.
Krauskopf
,
T.
Simpson
, and
D.
Lenstra
,
Phys. Rep.
416
,
1
(
2005
).
75.
D.
Brunner
,
M. C.
Soriano
, and
I.
Fischer
,
IEEE Photonics Technol. Lett.
25
,
1680
(
2013
).
76.
J.
Bueno
,
D.
Brunner
,
M.
Soriano
, and
I.
Fischer
,
Opt. Express
25
,
2401
(
2017
).
77.
K.
Hicke
,
M. A.
Escalona-Moran
,
D.
Brunner
,
M. C.
Soriano
,
I.
Fischer
, and
C. R.
Mirasso
,
IEEE J. Sel. Top. Quantum Electron.
19
,
1501610
(
2013
).
78.
R. M.
Nguimdo
,
G.
Verschaffelt
,
J.
Danckaert
, and
G.
Van der Sande
,
IEEE Trans. Neural Netw. Learn. Syst.
26
,
3301
(
2015
).
79.
R. M.
Nguimdo
,
G.
Verschaffelt
,
J.
Danckaert
, and
G.
Van der Sande
,
Opt. Express
24
,
1238
(
2016
).
80.
R. M.
Nguimdo
,
E.
Lacot
,
O.
Jacquin
,
O.
Hugon
,
G.
Van der Sande
, and
H.
Guillet de Chatellus
,
Opt. Lett.
42
,
375
(
2017
).
81.
Q.
Vinckier
,
F.
Duport
,
A.
Smerieri
,
K.
Vandoorne
,
P.
Bienstmann
,
M.
Haeltermann
, and
S.
Massar
,
Optica
2
,
438
(
2015
).
82.
A.
Dejonckheere
,
F.
Duport
,
A.
Smerieri
,
L.
Fang
,
J.-L.
Oudar
,
M.
Haelterman
, and
S.
Massar
,
Opt. Express
22
,
10868
(
2014
).
83.
M. C.
Soriano
,
S.
Ortín
,
D.
Brunner
,
L.
Larger
,
C. R.
Mirasso
,
I.
Fischer
, and
L.
Pesquera
,
Opt. Express
21
,
12
(
2013
).
84.
J.
Nakayama
,
K.
Kanno
, and
A.
Uchida
,
Opt. Express
24
,
8679
(
2016
).
85.
L.
Appeltant
,
G. V. D.
Sande
,
J.
Danckaert
, and
I.
Fischer
,
Sci. Rep.
4
,
3629
(
2014
).
86.
Y.
Hou
,
G.
Xia
,
W.
Yang
,
D.
Wang
,
E.
Jayaprasath
,
Z.
Jiang
,
C.
Hu
, and
Z.
Wu
,
Opt. Express
26
,
10211
(
2018
).
87.
F.
Duport
,
A.
Smerieri
,
A.
Akrout
,
M.
Haelterman
, and
S.
Massar
,
Sci. Rep.
6
,
22381
(
2016
).
88.
F.
Duport
,
A.
Akrout
,
A.
Smerieri
,
M.
Haelterman
, and
S.
Massar
(2014); preprint arXiv:1406.3238.
89.
P.
Antonik
,
M.
Haelterman
, and
S.
Massar
,
Phys. Rev. Appl.
7
,
1
(
2017
).
90.
M.
Hermans
,
P.
Antonik
,
M.
Haelterman
, and
S.
Massar
,
Phys. Rev. Lett.
117
,
128301
(
2016
).
91.
M.
Hermans
,
J.
Dambre
, and
P.
Bienstman
,
IEEE Trans. Neural Netw. Learn. Syst.
26
,
1545
(
2015
).
92.
M.
Hermans
and
B.
Schrauwen
, in
2010 IEEE International Symposium on Circuits and Systems: Nano-Bio Circuit Fabrics and Systems, ISCAS 2010
(
2010
), p.
521
.
93.
M.
Hermans
,
B.
Schrauwen
,
P.
Bienstman
, and
J.
Dambre
,
PLoS ONE
9
,
e86696
(
2014
).
94.
M.
Hermans
,
J.
Dambre
, and
P.
Bienstman
,
IEEE Trans. Neural Netw. Learn. Syst.
26
,
1545
(
2015
).
95.
P.
Antonik
,
M.
Hermans
,
F.
Duport
,
M.
Haelterman
, and
S.
Massar
, “
Towards pattern generation and chaotic series prediction with photonic reservoir computers
,”
Proc. SPIE
9732
, 97320B1-12 (
2016
).
96.
B. A.
Marquez
,
J.
Suarez-Vargas
,
L.
Larger
,
M.
Jacquot
,
Y. K.
Chembo
, and
D.
Brunner
, in
2017 IEEE International Conference on Rebooting Computing (ICRC)
(
IEEE
,
2017
), pp.
1
4
.
97.
K.
Liu
,
C. R.
Ye
,
S.
Khan
, and
V. J.
Sorger
,
Laser Photon. Rev.
9
,
172
(
2015
).
98.
A.
Akrout
,
A.
Bouwens
,
F.
Duport
,
Q.
Vinckier
,
M.
Haelterman
, and
S.
Massar
(
2016
); e-print arXiv:1612.08606.
99.
Y.
Shen
,
N. C.
Harris
,
S.
Skirlo
,
M.
Prabhu
,
T.
Baehr-Jones
,
M.
Hochberg
,
X.
Sun
,
S.
Zhao
,
H.
Larochelle
,
D.
Englund
, and
M.
Soljacic
,
Nat. Photon.
11
,
441
(
2017
). arXiv:1610.02365.
100.
A.
Katumba
,
J.
Heyvaert
,
B.
Schneider
,
S.
Uvin
,
J.
Dambre
, and
P.
Bienstman
,
Sci. Rep.
8
,
2653
(
2018
).
101.
A. N.
Tait
,
T. F.
De Lima
,
E.
Zhou
,
A. X.
Wu
,
M. A.
Nahmias
,
B. J.
Shastri
, and
P. R.
Prucnal
,
Sci. Rep.
7
,
1
(
2017
). arXiv:1611.02272.
You do not currently have access to this content.