We investigate the predictive power of recurrent neural networks for oscillatory systems not only on the attractor but in its vicinity as well. For this, we consider systems perturbed by an external force. This allows us to not merely predict the time evolution of the system but also study its dynamical properties, such as bifurcations, dynamical response curves, characteristic exponents, etc. It is shown that they can be effectively estimated even in some regions of the state space where no input data were given. We consider several different oscillatory examples, including self-sustained, excitatory, time-delay, and chaotic systems. Furthermore, with a statistical analysis, we assess the amount of training data required for effective inference for two common recurrent neural network cells, the long short-term memory and the gated recurrent unit.

1.
M. E.
Marhic
,
Fiber Optical Parametric Amplifiers, Oscillators and Related Devices
(
Cambridge University Press
,
Cambridge
,
2008
), Chap. 8.
2.
M.
Tooley
,
Electronic Circuits: Fundamentals and Applications
(
Newnes
,
Oxford
,
2002
), Chap. 9.
3.
I. R.
Epstein
and
J. A.
Pojman
,
An Introduction to Nonlinear Chemical Dynamics, Oscillations, Waves, Patterns, and Chaos
(
Oxford University Press
,
Oxford
,
1998
), Chap. 8.
4.
A. T.
Winfree
,
The Geometry of Biological Time
(
Springer
,
Berlin
,
1980
).
5.
K. M.
Stiefel
and
G. B.
Ermentrout
,
J. Neurophysiol.
116
,
2950
2960
(
2016
).
6.
A. D. A.
Babloyantz
,
Biol. Cybern.
58
,
203
(
1988
).
7.
R.
Foster
and
L.
Kreitzman
,
Circadian Rhythms: A Very Short Introduction
(
Oxford University Press
,
Oxford
,
2017
).
8.
G.
Buzsáki
,
Rhythms of the Brain
(
Oxford University Press
,
Oxford
,
2006
).
9.
H. A.
Dijkstra
,
Nonlinear Climate Dynamics
(
Cambridge University Press
,
2013
).
10.
A.
Pikovsky
,
M.
Rosenblum
, and
J.
Kurths
,
Synchronization: A Universal Concept in Nonlinear Sciences
(
Cambridge University Press
,
2001
).
11.
H. U.
Voss
,
P.
Kolodner
,
M.
Abel
, and
J.
Kurths
,
Phys. Rev. Lett.
83
,
3422
(
1999
).
12.
H.
Voss
,
M. J.
Bünner
, and
M.
Abel
,
Phys. Rev. E
57
,
2820
(
1998
).
13.
M.
Abel
,
Int. J. Bifurcat. Chaos
14
,
2027
(
2004
).
14.
H.
Whitney
,
Ann. Math.
37
,
645
(
1936
).
15.
F.
Takens
, in
Dynamical Systems and Turbulence
,
Lecture Notes in Mathematics Vol. 898, edited by D. Rand and L. S. Young
(
Springer
,
Berlin
,
1981
).
16.
J.
Guckenheimer
and
P.
Holmes
, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, Applied Mathematical Sciences (Springer, New York, 2002).
17.
M.
Quade
,
M.
Abel
,
K.
Shafi
,
R. K.
Niven
, and
B. R.
Noack
,
Phys. Rev. E
94
,
012214
(
2016
).
18.
M.
Schmidt
and
H.
Lipson
,
Science
324
,
81
(
2009
).
19.
J.
Pathak
,
B.
Hunt
,
M.
Girvan
,
Z.
Lu
, and
E.
Ott
,
Phys. Rev. Lett.
120
,
024102
(
2018
).
20.
J.
Pathak
,
Z.
Lu
,
B. R.
Hunt
,
M.
Girvan
, and
E.
Ott
,
Chaos
27
,
121102
(
2017
).
21.
Z.
Lu
,
B. R.
Hunt
, and
E.
Ott
,
Chaos
29
,
061104
(
2018
).
22.
R. Z. U.
Parlitz
,
Chaos
28
,
043118
(
2018
).
23.
I.
Goodfellow
,
Y.
Bengio
, and
A.
Courville
,
Deep Learning
(
MIT Press
,
2016
), see http://www.deeplearningbook.org.
24.
Z. C.
Lipton
, CoRR abs/1506.00019 (2015).
25.
H.
Sak
,
A.
Senior
, and
F.
Beaufays
, in INTERSPEECH-2014 (International Speech Communication Association, 2014), Vol. 338.
26.
I.
Sutskever
,
J.
Martens
, and
G. E.
Hinton
, in ICML’11 Proceedings of the 28th International Conference on International Conference on Machine Learning (Omnipress, 2011), pp. 1017–1024.
27.
K.
Cho
,
B.
van Merrienboer
,
C.
Gulcehre
,
D.
Bahdanau
,
F.
Bougares
,
H.
Schwenk
, and
Y.
Bengio
, “
Learning phase representations using RNN encoder-decoder for statistical machine translation,
” preprint arXiv:1406.1078 (
2014
).
28.
C. C.
Canavier
,
Scholarpedia
1
,
1332
(
2006
).
30.
B. A.
Pearlmutter
,
Neural Comput.
1
,
263
(
1989
).
31.
K.
Funahashi
and
Y.
Nakamura
,
Neural Netw.
6
,
801
(
1993
).
32.
C. A.
Bailer-Jones
,
D. J.
MacKay
, and
P. J.
Withers
,
Network Comput. Neural Syst.
9
,
531
(
1998
).
33.
R.
Falahian
,
M. M.
Dastjerdi
,
M.
Molaie
,
S.
Jafari
, and
S.
Gharibyadeh
,
Nonlinear Dyn.
81
,
1951
(
2015
).
34.
D.
Svozil
,
V.
Kvasnička
, and
J.
Pospichal
,
Chemometr. Intell. Lab. Syst.
39
,
43
(
1997
).
35.
S.
Hochreiter
and
J.
Schmidhuber
,
Neural Comput.
9
,
1735
(
1997
).
36.
K.
Cho
,
B.
van Merrienboer
,
D.
Bahdanau
, and
Y.
Bengio
, e-print arXiv:1409.1259 (2014).
37.
C.
Olah
, see https://colah.github.io/posts/2015-08-Understanding-LSTMs/ for “Understanding LSTM Networks” (2015).
38.
M.
Abadi
,
A.
Agarwal
,
P.
Barham
,
E.
Brevdo
,
Z.
Chen
,
C.
Citro
,
G. S.
Corrado
,
A.
Davis
,
J.
Dean
,
M.
Devin
,
S.
Ghemawat
,
I.
Goodfellow
,
A.
Harp
,
G.
Irving
,
M.
Isard
,
Y.
Jia
,
R.
Jozefowicz
,
L.
Kaiser
,
M.
Kudlur
,
J.
Levenberg
,
D.
Mané
,
R.
Monga
,
S.
Moore
,
D.
Murray
,
C.
Olah
,
M.
Schuster
,
J.
Shlens
,
B.
Steiner
,
I.
Sutskever
,
K.
Talwar
,
P.
Tucker
,
V.
Vanhoucke
,
V.
Vasudevan
,
F.
Viégas
,
O.
Vinyals
,
P.
Warden
,
M.
Wattenberg
,
M.
Wicke
,
Y.
Yu
, and
X.
Zheng
, “TensorFlow: Large-scale Machine Learning on Heterogeneous Systems” (2015), software available from https://tensorflow.org.
39.
See https://github.com/keras-team/keras for “K. team: Keras” (2019).
40.
O. E.
Roessler
,
Phys. Lett.
57A
,
397
398
(
1976
).
41.
G. E.
Uhlenbeck
and
L. S.
Ornstein
,
Phys. Rev.
36
,
823
(
1930
).
43.
K.
Ota
,
M.
Nomura
, and
T.
Aoyagi
,
Phys. Rev. Lett.
103
,
024101
(
2009
).
44.
T.
Imai
,
K.
Ota
, and
T.
Aoyagi
,
J. Phys. Soc. Jpn.
86
,
024009
(
2017
).
45.
R.
Cestnik
and
M.
Rosenblum
,
Sci. Rep.
8
,
13606
(
2018
).
46.
A.
Wolf
,
J. B.
Swift
,
H. L.
Swinney
, and
J. A.
Vastano
,
Physica D
16
,
285
(
1985
).
47.
M. T.
Rosenstein
,
J. J.
Collins
, and
C. J. D.
Luca
,
Physica D
65
,
117
(
1993
).
48.
E. M.
Izhikevich
and
R.
FitzHugh
,
Scholarpedia
1
,
1349
(
2006
).
49.
L.
Glass
and
M. C.
Mackey
,
Ann. N.Y. Acad. Sci.
316
,
214
(
1979
).
50.
M.
Henon
,
Phys. D Nonlinear Phenom.
5
,
412
(
1982
).
51.
P.
Kuchment
,
Floquet Theory for Partial Differential Equations
, Operator Theory: Advances and Applications (
Birkhäuser
,
1993
), Vol. 60.
52.
K.
Josic
,
E. T.
Shea-Brown
, and
J.
Moehlis
,
Scholarpedia
1
,
1361
(
2006
).
53.
A.
Pikovsky
,
M.
Rosenblum
, and
J.
Kurths
,
Synchronization: A Universal Concept in Nonlinear Sciences.
(
Cambridge University Press
,
Cambridge
,
2001
).
54.
R.
Cestnik
, see https://github.com/rokcestnik/oscillator_snap for “Oscillator Snap” (2019).
You do not currently have access to this content.