We propose a novel type of neural networks known as “attention-based sequence-to-sequence architecture” for a model-free prediction of spatiotemporal systems. This architecture is composed of an encoder and a decoder in which the encoder acts upon a given input sequence and then the decoder yields another output sequence to make a multistep prediction at a time. In order to demonstrate the potential of this approach, we train the neural network using data numerically sampled from the Korteweg–de Vries equation—which describes the interaction between solitary waves—and then predict its future evolution. Furthermore, we validate the applicability of the approach on datasets sampled from the chaotic Lorenz system and three other partial differential equations. The results show that the proposed method can achieve good performance in predicting the evolutionary behavior of studied spatiotemporal dynamics. To the best of our knowledge, this work is the first attempt at applying attention-based sequence-to-sequence architecture to the prediction task of solitary waves.
REFERENCES
1.
R. S.
Cantrell
and C.
Cosner
, Spatial Ecology via Reaction-Diffusion Equations
(John Wiley & Sons
, 2004
).2.
J.
Smoller
, Shock Waves and Reaction-Diffusion Equations
(Springer Science & Business Media
, 2012
), Vol. 258.3.
S.
Patankar
, Numerical Heat Transfer and Fluid Flow
(CRC Press
, 2018
).4.
B. R.
Munson
, T. H.
Okiishi
, W. W.
Huebsch
, and A. P.
Rothmayer
, Fluid Mechanics
(Wiley
, Singapore
, 2013
).5.
U.
Parlitz
and C.
Merkwirth
, “Prediction of spatiotemporal time series based on reconstructed local states
,” Phys. Rev. Lett.
84
, 1890
(2000
). 6.
L.
Guo
and S. A.
Billings
, “State-space reconstruction and spatio-temporal prediction of lattice dynamical systems
,” IEEE Trans. Automat. Control
52
, 622
–632
(2007
). 7.
J.
Shawe-Taylor
, N.
Cristianini
et al., Kernel Methods for Pattern Analysis
(Cambridge University Press
, 2004
).8.
C. K.
Williams
and C. E.
Rasmussen
, “Gaussian processes for regression,” in Advances in Neural Information Processing Systems (MIT Press, 1996), pp. 514–520.9.
K.
Xu
et al., “Show, attend and tell: Neural image caption generation with visual attention,” arXiv:1502.03044 (2015).10.
J.
Pathak
, B.
Hunt
, M.
Girvan
, Z.
Lu
, and E.
Ott
, “Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach
,” Phys. Rev. Lett.
120
, 024102
(2018
). 11.
H.
Jaeger
and H.
Haas
, “Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication
,” Science
304
, 78
–80
(2004
). 12.
K.
Cho
, B.
Van Merriënboer
, C.
Gulcehre
, D.
Bahdanau
, F.
Bougares
, H.
Schwenk
, and Y.
Bengio
, “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” arXiv:1406.1078 (2014).13.
K.
Cho
, B.
Van Merriënboer
, D.
Bahdanau
, and Y.
Bengio
, “On the properties of neural machine translation: Encoder-decoder approaches,” arXiv:1409.1259 (2014).14.
I.
Sutskever
, O.
Vinyals
, and Q. V.
Le
, “Sequence to sequence learning with neural networks,” in Advances in Neural Information Processing Systems (Curran Associates, 2014), pp. 3104–3112.15.
A.
Karpathy
and L.
Fei-Fei
, “Deep visual-semantic alignments for generating image descriptions,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2015), pp. 3128–3137.16.
O.
Vinyals
, A.
Toshev
, S.
Bengio
, and D.
Erhan
, “Show and tell: A neural image caption generator,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2015), pp. 3156–3164.17.
D.
Tang
, B.
Qin
, and T.
Liu
, “Document modeling with gated recurrent neural network for sentiment classification,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (Association for Computational Linguistics, 2015), pp. 1422–1432.18.
D.
Bahdanau
, K.
Cho
, and Y.
Bengio
, “Neural machine translation by jointly learning to align and translate,” arXiv:1409.0473 (2014).19.
M.-T.
Luong
, H.
Pham
, and C. D.
Manning
, “Effective approaches to attention-based neural machine translation,” arXiv:1508.04025 (2015).20.
S.
Hochreiter
and J.
Schmidhuber
, “Long short-term memory
,” Neural Comput.
9
, 1735
–1780
(1997
). 21.
V.
Schwämmle
and H. J.
Herrmann
, “Geomorphology: Solitary wave behaviour of sand dunes
,” Nature
426
, 619
(2003
). 22.
S.
Amiranashvili
, U.
Bandelow
, and N.
Akhmediev
, “Few-cycle optical solitary waves in nonlinear dispersive media
,” Phys. Rev. A
87
, 013805
(2013
). 23.
A.
Choudhuri
and K.
Porsezian
, “Dark-in-the-bright solitary wave solution of higher-order nonlinear Schrödinger equation with non-Kerr terms
,” Opt. Commun.
285
, 364
–367
(2012
). 24.
A.
Hasegawa
and Y.
Kodama
, Solitons in Optical Communications
(Oxford University Press
, New York
, 1995
), Vol. 7.25.
C. A.
Cattell
, J.
Crumley
, J.
Dombeck
, J. R.
Wygant
, and F.
Mozer
, “Polar observations of solitary waves at the earth’s magnetopause
,” Geophys. Res. Lett.
29
, 9-1
, (2002
). 26.
S.
Russell
, in Fourteenth Meeting of the British Association of the Advancement of Science (John Murray, 1844).27.
D. J.
Korteweg
and G.
De Vries
, “XLI. On the change of form of long waves advancing in a rectangular canal, and on a new type of long stationary waves
,” Lond. Edinb. Dubl. Philos. Mag. J. Sci.
39
, 422
–443
(1895
). 28.
M.
Toda
, “Vibration of a chain with nonlinear interaction,” in Selected Papers of Morikazu Toda (World Scientific, 1993), pp. 97–102.29.
S.
Anco
and M. R.
Willoughby
, see http://lie.math.brocku.ca/sanco/solitons/kdv_solitons.php for “KdV solitons” (accessed 26 June 2019).30.
31.
J.
Pathak
, Z.
Lu
, B. R.
Hunt
, M.
Girvan
, and E.
Ott
, “Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data
,” Chaos
27
, 121102
(2017
). 32.
R. A.
Fisher
, “The wave of advance of advantageous genes
,” Ann. Eugen.
7
, 355
–369
(1937
). 33.
A.
Barone
, F.
Esposito
, C.
Magee
, and A.
Scott
, “Theory and applications of the sine-Gordon equation
,” La Rivista del Nuovo Cimento (1971–1977)
1
, 227
–267
(1971
). 34.
S. H.
Rudy
, S. L.
Brunton
, J. L.
Proctor
, and J. N.
Kutz
, “Data-driven discovery of partial differential equations
,” Sci. Adv.
3
, e1602614
(2017
). © 2020 Author(s).
2020
Author(s)
You do not currently have access to this content.