Concise, accurate descriptions of physical systems through their conserved quantities abound in the natural sciences. In data science, however, current research often focuses on regression problems, without routinely incorporating additional assumptions about the system that generated the data. Here, we propose to explore a particular type of underlying structure in the data: Hamiltonian systems, where an “energy” is conserved. Given a collection of observations of such a Hamiltonian system over time, we extract phase space coordinates and a Hamiltonian function of them that acts as the generator of the system dynamics. The approach employs an autoencoder neural network component to estimate the transformation from observations to the phase space of a Hamiltonian system. An additional neural network component is used to approximate the Hamiltonian function on this constructed space, and the two components are trained jointly. As an alternative approach, we also demonstrate the use of Gaussian processes for the estimation of such a Hamiltonian. After two illustrative examples, we extract an underlying phase space as well as the generating Hamiltonian from a collection of movies of a pendulum. The approach is fully data-driven and does not assume a particular form of the Hamiltonian function.
REFERENCES
1.
A. M.
Almeida
, Hamiltonian Systems: Chaos and Quantization
(Cambridge University Press
, 1992
).2.
A. L.
Caterini
, A.
Doucet
, and D.
Sejdinovic
, “Hamiltonian variational auto-encoder,” in Proceedings of the 32nd Conference on Neural Information Processing Systems (Curran Associates, Inc., 2018), p. 11.3.
B.
Chang
, L.
Meng
, E.
Haber
, F.
Tung
, and D.
Begert
, “Multi-level residual networks from dynamical systems view,” in International Conference on Learning Representations (2018).4.
R. T. Q.
Chen
, Y.
Rubanova
, J.
Bettencourt
, and D.
Duvenaud
, “Neural ordinary differential equations
,” in Advances in Neural Information Processing Systems, edited by S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Curran Associates, Inc., 2018), Vol. 31, pp. 6571
–6583
(2018
).5.
W. E
, “A proposal on machine learning via dynamical systems
,” Commun. Math. Stat.
5
(1
), 1
–11
(2017
).6.
R.
González-García
, R.
Rico-Martínez
, and I. G.
Kevrekidis
, “Identification of distributed parameter systems: A neural net based approach
,” Comput. Chem. Eng.
22
(98
), S965
–S968
(1998
). 7.
8.
K.
Greff
, R. K.
Srivastava
, and J.
Schmidhuber
, “Highway and residual networks learn unrolled iterative estimation,” in Proceedings of the International Conference on Learning Representations (2017).9.
S.
Greydanus
, M.
Dzamba
, and J.
Yosinski
, “Hamiltonian neural networks,” e-print arXiv:1906.01563 (2019).10.
W. R.
Hamilton
, “On a general method in dynamics
,” Philos. Trans. R. Soc. II
1834
, 247
–308
.11.
K.
He
, X.
Zhang
, S.
Ren
, and J.
Sun
, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2016), Vol. 7, pp. 171–180.12.
E.
Kaiser
, J.
Nathan Kutz
, and S. L.
Brunton
, “Discovering conservation laws from data for control,” in 2018 IEEE Conference on Decision and Control (CDC) (IEEE, 2018).13.
M.
Livio
, “Why symmetry matters
,” Nature
490
(7421
), 472
–473
(2012
). 14.
Y.
Lu
, A.
Zhong
, Q.
Li
, and B.
Dong
, “Beyond finite layer neural networks: Bridging deep architectures and numerical differential equations,” in Proceedings of the 35th International Conference on Machine Learning, edited by J. Dy and A. Krause (PMLR, Stockholm, Sweden, 2018), p. 10.15.
B.
Lusch
, J.
Nathan Kutz
, and S. L.
Brunton
, “Deep learning for universal linear embeddings of nonlinear dynamics
,” Nat. Commun.
9
(1
), 4950
(2018
). 16.
K. T.
McDonald
, “Hamiltonian with z as the independent variable,” Technical Report (2015
).17.
R.
Mottaghi
, H.
Bagherinezhad
, M.
Rastegari
, and A.
Farhadi
, “Newtonian image understanding: Unfolding the dynamics of objects in static images,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, Las Vegas, NV, 2016), pp. 3521–3529.18.
R.
Neal
, “MCMC using Hamiltonian dynamics,” in Handbook of Markov Chain Monte Carlo, edited by S. Brooks, A. Gelman, G. Jones, and X.-L. Meng (Chapman and Hall/CRC, 2011).19.
E.
Noether
, “Invariant variation problems
,” Transp. Theory Stat. Phys.
1
(3
), 186
–207
(1971
). 20.
R.
Kondor
, Z.
Lin
, and S.
Trivedi
, “Clebsch-Gordan nets: A fully Fourier space spherical convolutional neural network,” in Advances in Neural Information Processing Systems, edited by S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Curran Associates, Inc., 2018), Vol. 31, pp. 10117–10126.21.
M.
Raissi
and G. E.
Karniadakis
, “Hidden physics models: Machine learning of nonlinear partial differential equations
,” J. Comput. Phys.
357
, 125
–141
(2018
). 22.
M.
Raissi
, P.
Perdikaris
, and G. E.
Karniadakis
, “Inferring solutions of differential equations using noisy multi-fidelity data
,” J. Comput. Phys.
335
, 736
–746
(2017
). 23.
C. E.
Rasmussen
and C. K. I.
Williams
, Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning)
(The MIT Press
, 2005
).24.
D. J.
Rezende
and S.
Mohamed
, “Variational inference with normalizing flows,” in Proceedings of the 32nd International Conference on Machine Learning (PMLR, 2015), p. 9.25.
R.
Rico-Martinez
, R. A.
Adomaitis
, and I. G.
Kevrekidis
, “Noninvertibility in neural networks,” in Proceedings of the 1993 IEEE International Conference on Neural Networks (The Institute of Electrical and Electronics Engineers, 1993), pp. 382–386.26.
R.
Rico-Martínez
, R. A.
Adomaitis
, and I. G.
Kevrekidis
, “Noninvertibility in neural networks
,” Comput. Chem. Eng.
24
(11
), 2417
–2433
(2000
). 27.
R.
Rico-Martínez
, J. S.
Anderson
, and I. G
Kevrekidis
, “Continuous-time nonlinear signal processing: A neural network based approach for gray box identification,” in Proceedings of IEEE Workshop on Neural Networks for Signal Processing (The Institute of Electrical and Electronics Engineers, 1994).28.
R.
Rico-Martínez
and I. G
Kevrekidis
, “Nonlinear system identification using neural networks: Dynamics and instabilities,” in Neural Networks for Chemical Engineers, edited by A. B. Bulsari (Elsevier, 1995), pp. 409–442.29.
R.
Rico-Martínez
, I. G.
Kevrekidis
, and R. A.
Adomaitis
, “Noninvertible dynamics in neural network models,” in Proceedings of the Twenty-Eighth Annual Conference on Information Sciences and Systems (John Hopkins University, 1994), pp. 965–969.30.
R.
Rico-Martínez
, I. G.
Kevrekidis
, M. C.
Kube
, and J. L.
Hudson
, “Discrete- vs continuous-time nonlinear signal processing attractors, transitions and parallel implementation issues,” in American Control Conference (The Institute of Electrical and Electronic Engineers, 1993), pp. 1475–1479.31.
R.
Rico-Martínez
, K.
Krischer
, I. G.
Kevrekidis
, M. C.
Kube
, and J. L.
Hudson
, “Discrete- vs continuous-time nonlinear signal processing of Cu electrodissolution data
,” Chem. Eng. Commun.
, 118
(1
), 25
–48
(1992
), ISBN: 0098644920.32.
M.
Schmidt
and H.
Lipson
, “Distilling free-form natural laws from experimental data
,” Science
324
(5923
), 81
–85
(2009
). 33.
R. K.
Srivastava
, K.
Greff
, and J.
Schmidhuber
, “Highway networks,” in Proceedings of the International Conference on Machine Learning (PMLR, 2015).34.
P.
Toth
, D. J.
Rezende
, A.
Jaegle
, S.
Racanière
, A.
Botev
, and I.
Higgins
, “Hamiltonian generative networks,” e-print arXiv:1909.13789 (2019).35.
J. M.
Lee
, Introduction to Smooth Manifolds. Graduate Texts in Mathematics
(Springer New York
, 2012
).36.
G.
Darboux
, Sur le probléme de Pfaff. Bulletin des Sciences Mathématiques et Astronomiques, Gauthier-Villars, 1882, 2e serie, 6, 14–36
© 2019 Author(s).
2019
Author(s)
You do not currently have access to this content.
