In this work, we developed a nonlinear System Identification (SID) method that we called Entropic Regression. Our method adopts an information-theoretic measure for the data-driven discovery of the underlying dynamics. Our method shows robustness toward noise and outliers, and it outperforms many of the current state-of-the-art methods. Moreover, the method of Entropic Regression overcomes many of the major limitations of the current methods such as sloppy parameters, diverse scale, and SID in high-dimensional systems such as complex networks. The use of information-theoretic measures in entropic regression has unique advantages, due to the Asymptotic Equipartition Property of probability distributions, that outliers and other low-occurrence events are conveniently and intrinsically de-emphasized as not-typical, by definition. We provide a numerical comparison with the current state-of-the-art methods in sparse regression, and we apply the methods to different chaotic systems such as the Lorenz System, the Kuramoto-Sivashinsky equations, and the Double-Well Potential.
REFERENCES
1.
N. A.
Gershenfeld
and A. S.
Weigend
, The Future of Time Series. Time Series Prediction: Forecasting the Future and Understanding the Past
(Addison-Wesley
, 1993
).2.
S. L.
Brunton
, J. L.
Proctor
, and J. N.
Kutz
, “Discovering governing equations from data by sparse identification of nonlinear dynamical systems
,” Proc. Natl. Acad. Sci. U.S.A.
113
, 3932
–3937
(2016
). 3.
C.
Yao
and E. M.
Bollt
, “Modeling and nonlinear parameter estimation with Kronecker product representation for coupled oscillators and spatiotemporal systems
,” Physica D
1
, 78
–99
(2007
). 4.
S.
Chen
, S. A.
Billings
, and W.
Luo
, “Orthogonal least squares methods and their application to non-linear system identification
,” Int. J. Control
50
, 1873
–1896
(1989
). 5.
N.
Kalouptsidis
, G.
Mileounis
, B.
Babadi
, and V.
Tarokh
, “Adaptive algorithms for sparse system identification
,” Signal Process.
91
, 1910
–1919
(2011
). 6.
W.-X.
Wang
, R.
Yang
, Y.-C.
Lai
, V.
Kovanis
, and C.
Grebogi
, “Predicting catastrophes in nonlinear dynamical systems by compressive sensing
,” Phys. Rev. Lett.
106
, 154101
(2011
). 7.
W.-X.
Wang
, Y.-C.
Lai
, and C.
Grebogi
, “Data based identification and prediction of nonlinear and complex dynamical systems
,” Phys. Rep.
644
, 1
–76
(2016
). 8.
L.-Z.
Guo
, S. A.
Billings
, and D.
Zhu
, “An extended orthogonal forward regression algorithm for system identification using entropy
,” Int. J. Control
81
, 690
–699
(2008
). 9.
G.
Tran
and R.
Ward
, “Exact recovery of chaotic systems from highly corrupted data
,” Multiscale Model. Simul.
15
, 1108
–1129
(2017
). 10.
L.
Ljung
, “System identification
,” Wiley Encyclopedia Electrical Electron. Eng.
1
–19
(1999
).11.
H.
Akaike
, “A new look at the statistical model identification
,” IEEE. Trans. Automat. Contr.
19
, 716
–723
(1974
). 12.
G.
Prando
, A.
Chiuso
, and G.
Pillonetto
, “Maximum entropy vector kernels for MIMO system identification
,” Automatica
79
, 326
–339
(2017
). 13.
J.
Sun
, C.
Cafaro
, and E. M.
Bollt
, “Identifying the coupling structure in complex systems through the optimal causation entropy principle
,” Entropy
16
, 3416
–3433
(2014
). 14.
J.
Sun
, D.
Taylor
, and E.
Bollt
, “Causal network inference by optimal causation entropy
,” SIAM J. Appl. Dyn. Syst.
14
, 73
–106
(2015
). 15.
T. M.
Cover
and J. A.
Thomas
, Elements of Information Theory
(Wiley-Interscience
, Hoboken, NJ
, 2005
), pp. 1
–748
.16.
B.
Saltzman
, “Finite amplitude free convection as an initial value problem—I
,” J. Atmos. Sci.
19
, 329
–341
(1962
). 17.
E. N.
Lorenz
, “Deterministic nonperiodic flow
,” J. Atmos. Sci.
20
, 130
–141
(1963
). 18.
Y.
Kuramoto
and T.
Tsuzuki
, “Persistent propagation of concentration waves in dissipative media far from thermal equilibrium
,” Progress Theor. Phys.
55
, 356
–369
(1976
). 19.
Y.
Kuramoto
, “Diffusion-induced chaos in reaction systems
,” Progress Theor. Phys. Suppl.
64
, 346
–367
(1978
). 20.
G. I.
Sivashinsky
, “Nonlinear analysis of hydrodynamic instability in laminar flames—I. Derivation of basic equations
,” Acta Astronaut.
4
, 1177
–1206
(1977
). 21.
J. M.
Hyman
and B.
Nicolaenko
, “The Kuramoto-Sivashinsky equation: A bridge between PDE’s and dynamical systems
,” Physica D
18
, 113
–126
(1986
). 22.
Y.
Lan
and P.
Cvitanović
, “Unstable recurrent patterns in Kuramoto-Sivashinsky dynamics
,” Phys. Rev. E
78
, 026208
(2008
). 23.
F.
Christiansen
, P.
Cvitanovic
, and V.
Putkaradze
, “Spatiotemporal chaos in terms of unstable recurrent patterns
,” Nonlinearity
10
, 55
(1997
). 24.
P.
Hohenberg
and B. I.
Shraiman
, “Chaotic behavior of an extended system
,” Physica D
37
, 109
–115
(1989
). 25.
P.
Cvitanovic
, R.
Artuso
, R.
Mainieri
, G.
Tanner
, G.
Vattay
, N.
Whelan
, and A.
Wirzba
, Chaos: Classical and Quantum
(Niels Bohr Institute
, Copenhagen
, 2005
), Vol. 69.26.
J. C.
Robinson
, Infinite-Dimensional Dynamical Systems: An Introduction to Dissipative Parabolic PDEs and the Theory of Global Attractors, Cambridge Texts in Applied Mathematics (Cambridge University Press, 2001), ISBN: 9780521635646.27.
M. S.
Jolly
, I.
Kevrekidis
, and E. S.
Titi
, “Approximate inertial manifolds for the Kuramoto-Sivashinsky equation: Analysis and computations
,” Physica D
44
, 38
–60
(1990
). 28.
S.
Ramdani
, B.
Rossetto
, L. O.
Chua
, and R.
Lozi
, “Slow manifolds of some chaotic systems with applications to laser systems
,” Int. J. Bifurcat. Chaos
10
, 2729
–2744
(2000
). 29.
M. S.
Jolly
, R. M. S.
Rosa
, and R. M.
Temam
, “Accurate computations on inertial manifolds
,” J. Sci. Comp.
22
(6
), 2216
–2238
(2001
).30.
G. H.
Golub
and C. F.
Van Loan
, Matrix Computations
, 4th ed. (Johns Hopkins University Press
, Baltimore, MD
, 2013
), ISBN: 1-4214-0859-7.31.
L.
Wang
and R.
Langari
, “Building Sugeno-type models using fuzzy discretization and orthogonal parameter estimation techniques
,” IEEE Trans. Fuzzy Syst.
3
, 454
–458
(1995
). 32.
M.
Korenberg
, S. A.
Billings
, H.
Liu
, and P.
McIlroy
, “Orthogonal parameter estimation algorithm for non-linear stochastic systems
,” Int. J. Control
48
, 193
–210
(1988
). 33.
T.
Hastie
, R.
Tibshirani
, and M.
Wainwright
, Statistical Learning with Sparsity: The Lasso and Generalizations
(CRC Press
, 2015
).34.
M.
Grant
and S.
Boyd
, “CVX: Matlab software for disciplined convex programming,” 2008.35.
E. J.
Candès
, J. K.
Romberg
, and T.
Tao
, “Stable signal recovery from incomplete and inaccurate measurements
,” Commun. Pure Appl. Math.
59
, 1207
–1223
(2006
). 36.
E. J.
Candès
, J.
Romberg
, and T.
Tao
, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information
,” IEEE Trans. Inf. Theory
52
, 489
–509
(2006
). 37.
D. L.
Donoho
, “Compressed sensing
,” IEEE Trans. Inf. Theory
52
, 1289
–1306
(2006
). 38.
A.
Kraskov
, H.
Stögbauer
, and P.
Grassberger
, “Estimating mutual information
,” Phys. Rev. E
69
, 066
–138
(2004
). © 2020 Author(s).
2020
Author(s)
You do not currently have access to this content.
