Recently, a rigorous formalism has been established for information flow and causality within dynamical systems with respect to Shannon entropy. In this study, we re-establish the formalism with respect to relative entropy, or Kullback-Leiber divergence, a well-accepted measure of predictability because of its appealing properties such as invariance upon nonlinear transformation and consistency with the second law of thermodynamics. Different from previous studies (which yield consistent results only for 2D systems), the resulting information flow, say , is precisely the same as that with respect to Shannon entropy for systems of arbitrary dimensionality, except for a minus sign (reflecting the opposite notion of predictability vs. uncertainty). As before, possesses a property called principle of nil causality, a fact that classical formalisms fail to verify in many situation. Besides, it proves to be invariant upon nonlinear transformation, indicating that the so-obtained information flow should be an intrinsic physical property. This formalism has been validated with the stochastic gradient system, a nonlinear system that admits an analytical equilibrium solution of the Boltzmann type.
References
1.
A. P.
Dempster
, “Causality and statistics
,” J. Stat. Plan. Infer.
25
, 261
–278
(1990
).2.
X. S.
Liang
, “Information flow and causality as rigorous notions ab initio
,” Phys. Rev. E
94
, 052201
(2016
).3.
X. S.
Liang
, “Unraveling the cause-effect relation between time series
,” Phys. Rev. E
90
, 052150
(2014
).4.
A.
Stips
, D.
Macias
, C.
Coughlan
, E.
Garcia-Gorriz
, and X. S.
Liang
, “On the causal structure between CO2 and global temperature
,” Nat. Sci. Rep.
6
, 21691
(2016
).5.
R.
Kleeman
, “Measuring dynamical prediction utility using relative entropy
,” J. Atmos. Sci.
59
, 2057
–2072
(2002
).6.
C.
Granger
, “Investigating causal relations by econometric models and cross-spectral methods
,” Econometrica
37
, 424
(1969
).7.
K.
Kaneko
, “Lyapunov analysis and information flow in coupled map lattices
,” Physica D
23
, 436
–447
(1986
)..8.
J. A.
Vastano
and H. L.
Swinney
, “Information transport in spatiotemporal systems
,” Phys. Rev. Lett.
60
, 1773
(1988
).9.
T.
Schreiber
, “Measuring information transfer
,” Phys. Rev. Lett.
85
(2
), 461
–464
(2000
).10.
X. S.
Liang
and R.
Kleeman
, “Information transfer between dynamical system components
,” Phys. Rev. Lett.
95
(24
), 244101
(2005
).11.
B.
Pompe
and J.
Runge
, “Momentary information transfer as a coupling measure of time series
,” Phys. Rev. E
83
, 051122
(2011
).12.
J.
Sun
and E.
Bolt
, “Causation entropy identifies indirect influences, dominance of neighbors, and anticipatory couplings
,” Physica D
267
, 49
–57
(2014
).13.
L.
Barnett
, A. B.
Barrett
, and A. K.
Seth
, “Granger causality and transfer entropy are equivalent for Gaussian variables
,” Phys. Rev. Lett.
103
(23
), 238701
(2009
).14.
C. W. J.
Granger
, “Testing for causality: A personal viewpoint
,” J. Econ. Dyn. Control
2
, 329
–352
(1980
).15.
H.
Nalatore
, M.
Ding
, and G.
Rangarajan
, “Mitigating the effects of measurement noise on Granger causality
,” Phys. Rev. E
75
, 031123
(2007
).16.
D. A.
Smirnov
, “Spurious causalities with transfer entropy
,” Phys. Rev. E
87
, 042917
(2013
).17.
J. T.
Lizier
and M.
Prokopenko
, “Differentiating information transfer and causal effect
,” Eur. Phys. J. B
73
(4
), 605
–615
(2010
).18.
J.
Runge
, J.
Heitzig
, N.
Marwan
, and J.
Kurths
, “Quantifying causal coupling strength: A lag-specific measure for multivariate time series related to transfer entropy
,” Phys. Rev. E
86
, 061121
(2012
).19.
M.
Staniek
and K.
Lehnertz
, “Symbolic transfer entropy
,” Phys. Rev. Lett.
100
, 158101
(2008
).20.
G.
Sugihara
et al. “Detecting causality in complex ecosystems
,” Science
338
, 496
–500
(2012
).21.
X. S.
Liang
and R.
Kleeman
, “A rigorous formalism of information transfer between dynamical system components. I. Discrete mapping
,” Physica D
231
, 1
–9
(2007
).22.
X. S.
Liang
and R.
Kleeman
, “A rigorous formalism of information transfer between dynamical system components. I. Continuous flow
,” Physica D
227
, 173
–182
(2007
).23.
A. J.
Majda
and J.
Harlim
, “Information flow between subspaces of complex dynamical systems
,” Proc. Natl. Acad. Sci.
104
(23
), 9558
–9563
(2007
).24.
U.
Feldmann
and J.
Bhattacharya
, “Predictability improvement as an asymmetrical measure of interdependence in bivariate time series
,” Int. J. Bifurcat. Chaos Appl. Sci. Eng.
14
(2
), 505
–514
(2004
).25.
A.
Krakovská
and F.
Hanzely
, “Testing for causality in reconstructed state spaces by an optimized mixed prediction method
,” Phys. Rev. E
94
, 052203
(2016
).26.
R.
Kleeman
and A. J.
Majda
, “Predictability in a model of geostrophic turbulence
,” J. Atmos. Sci.
62
, 2864
–2879
(2005
).27.
R.
Kleeman
, “Limits, variability and general behavior of statistical predictability of the mid-latitude atmosphere
,” J. Atmos. Sci.
65
, 263
–275
(2008
).28.
X. S.
Liang
, “Local predictability and information flow in complex dynamical systems
,” Physica D
248
, 1
–15
(2013
).29.
X. S.
Liang
, “Information flow within stochastic dynamical systems
,” Phys. Rev. E
78
, 031113
(2008
).30.
D. W.
Hahs
and S. D.
Pethel
, “Distinguishing anticipation from causality: Anticipatory bias in the estimation of information flow
,” Phys. Rev. Lett.
107
, 128701
(2011
).31.
J. L.
Kaplan
and J. A.
Yorke
, Functional Differential Equations and Approximations of Fixed Points
, Lecture Notes in Mathematics Vol. 730 (Springer-Verlag
, 1979
).32.
O. E.
Rössler
, “An equation for continuous chaos
,” Phys. Lett.
57A
(5
), 397
–398
(1976
).33.
A. J.
Majda
and I.
Timofeyev
, “Remarkable statistical behavior for truncated Burgers-Hopf dynamics
,” Proc. Natl. Acad. Sci. U.S.A.
97
(23
), 12413
–12417
(2000
).34.
C.
Bai
, R.
Zhang
, S.
Bao
, X. S.
Liang
, and W.
Guo
, “Forecasting the tropical cyclone genesis over the Northwest Pacific through identifying the causal factors in the cyclone-climate interactions
,” J. Atmos. Ocean Tech.
35
, 247
–259
(2018
).35.
X. S.
Liang
and A.
Lozano-Durán
, “A preliminary study of the causal structure in fully developed near-wall turbulence,” in Proceedings of the Summer Program 2016
(Center for Turbulence Research
, 2016
), pp. 233
–242
.36.
X. S.
Liang
, “Normalizing the causality between time series
,” Phys. Rev. E
92
, 022126
(2015
).37.
B. H.
Vaid
and X. S.
Liang
, “The changing relationship between the convection over the western Tibetan Plateau and the sea surface temperature in the northern Bay of Bengal
,” Tellus A: Dyn. Meteorol. Oceanogr.
70
(1
), 1440869
(2018
).38.
I.
Hoyos
, J.
Ca nón-Barriga
, T.
Arenas-Suárez
, F.
Dominguez
, and B. A.
Rodríguez
, “Variability of regional atmospheric moisture over Northern South America
,” Clim. Dyn.
(2018
).39.
J.
Huang
, Y.
Li
, C.
Fu
, F.
Chen
, Q.
Fu
, A.
Dai
, M.
Shinoda
, Z.
Ma
, W.
Guo
, Z.
Li
, L.
Zhang
, Y.
Liu
, H.
Yu
, Y.
He
, Y.
Xie
, X.
Guan
, M.
Ji
, L.
Lin
, S.
Wang
, H.
Yan
, and G.
Wang
, “Dryland climate change: Recent progress and challenges
,” Rev. Geophys.
55
, 719
–778
(2017
).40.
T. M.
Cover
and J. A.
Thomas
, Elements of Information Theory
(John Wiley & Sons, Inc.
, 1991
).41.
A.
Lasota
and M. C.
Mackey
, Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics
(Springer
, New York
, 1994
).42.
X. S.
Liang
, “Entropy evolution and uncertainty estimation with dynamical system
,” Entropy
16
, 3605
–3634
(2014
).43.
T.
Schneider
, S. M.
Griffies
, “A conceptual framework for predictability studies
,” J. Clim.
12
, 3133
–3155
(1999
).44.
J. M.
Amigó
, R.
Monetti
, N.
Tort-Colet
, and M. V.
Sanchez-Vives
, “Infragranular layers lead information flow during slow oscillations according to information directionality indicators
,” J. Comput. Neurosci.
39
, 53
–62
(2015
).© 2018 Author(s).
2018
Author(s)
You do not currently have access to this content.