Estimating causal interactions in complex dynamical systems is an important problem encountered in many fields of current science. While a theoretical solution for detecting the causal interactions has been previously formulated in the framework of prediction improvement, it generally requires the computation of high-dimensional information functionals—a situation invoking the curse of dimensionality with increasing network size. Recently, several methods have been proposed to alleviate this problem, based on iterative procedures for the assessment of conditional (in)dependences. In the current work, we bring a comparison of several such prominent approaches. This is done both by theoretical comparison of the algorithms using a formulation in a common framework and by numerical simulations including realistic complex coupling patterns. The theoretical analysis highlights the key similarities and differences between the algorithms, hinting on their comparative strengths and weaknesses. The method assumptions and specific properties such as false positive control and order-dependence are discussed. Numerical simulations suggest that while the accuracy of most of the algorithms is almost indistinguishable, there are substantial differences in their computational demands, ranging theoretically from polynomial to exponential complexity and leading to substantial differences in computation time in realistic scenarios depending on the density and size of networks. Based on the analysis of the algorithms and numerical simulations, we propose a hybrid approach providing competitive accuracy with improved computational efficiency.

1
S.
Boccaletti
,
V.
Latora
,
Y.
Moreno
,
M.
Chavez
, and
D. U.
Hwang
, “
Complex networks: Structure and dynamics
,”
Phys. Rep.
424
,
175
308
(
2006
).
2
K. J.
Friston
, “
Functional and effective connectivity in neuroimaging: A synthesis
,”
Hum. Brain Mapp.
2
,
56
78
(
1994
).
3
K. G.
Schilling
,
A.
Daducci
,
K.
Maier-Hein
,
C.
Poupon
,
J.-C.
Houde
,
V.
Nath
,
A. W.
Anderson
,
B. A.
Landman
, and
M.
Descoteaux
, “
Challenges in diffusion MRI tractography—Lessons learned from international benchmark competitions
,”
Magn. Reson. Imaging
57
,
194
209
(
2018
).
4
J.
Hlinka
and
S.
Coombes
, “
Using computational models to relate structural and functional brain connectivity
,”
Eur. J. Neurosci.
36
,
2137
2145
(
2012
).
5
J.
Hlinka
and
M.
Hadrava
, “
On the danger of detecting network states in white noise
,”
Front. Comput. Neurosci.
9
,
11
(
2015
).
6
M. G.
Preti
,
T. A.
Bolton
, and
D. V. D.
Ville
, “
The dynamic functional connectome: State-of-the-art and perspectives
,”
NeuroImage
160
,
41
54
(
2017
). functional Architecture of the Brain.
7
N.
Wiener
, “The theory of prediction,” in Modern Mathematics for Engineers (McGraw-Hill, New York, 1956), pp. 165–190.
8
C. W.
Granger
, “
Investigating causal relations by econometric model and cross spectral methods
,”
Econometrica
37
,
424
438
(
1969
).
9
P.
Spirtes
,
C.
Glymour
, and
R.
Scheines
,
Causation, Prediction, and Search
, 2nd ed. (
MIT Press
,
2000
).
10
T.
Schreiber
, “
Measuring information transfer
,”
Phys. Rev. Lett.
85
,
461
464
(
2000
).
11
M.
Palus
,
V.
Komarek
,
T.
Prochazka
,
Z.
Hrncir
, and
K.
Sterbova
, “
Synchronization and information flow in EEG of epileptic patients
,”
IEEE Eng. Med. Biol. Mag.
20
(
5
),
65
71
(
2001
).
12
A.
Kraskov
,
H.
Stogbauer
, and
P.
Grassberger
, “
Estimating mutual information
,”
Phys. Rev. E
69
,
066138
(
2004
).
13
J.
Runge
,
J.
Heitzig
,
V.
Petoukhov
, and
J.
Kurths
, “
Escaping the curse of dimensionality in estimating multivariate transfer entropy
,”
Phys. Rev. Lett.
108
,
258701
(
2012
).
14
D.
Kugiumtzis
, “
Direct-coupling information measure from nonuniform embedding
,”
Phys. Rev. E
87
,
062918
(
2013
).
15
J.
Sun
,
D.
Taylor
, and
E. M.
Bollt
, “
Causal network inference by optimal causation entropy
,”
SIAM J. Appl. Dyn. Syst.
14
,
73
106
(
2015
).
16
J.
Runge
,
P.
Nowack
,
M.
Kretschmer
,
S.
Flaxman
, and
D.
Sejdinovic
, “Detecting causal associations in large nonlinear time series datasets,”
Sci. Adv.
5
,
11
(
2019
).
17
R. G.
James
,
N.
Barnett
, and
J. P.
Crutchfield
, “
Information flows? A critique of transfer entropies
,”
Phys. Rev. Lett.
116
,
238701
(
2016
).
18
R. G.
James
and
J. P.
Crutchfield
, “
Multivariate dependence beyond Shannon information
,”
Entropy
19
,
531
(
2017
).
19
E. A.
Martin
,
J.
Hlinka
,
A.
Meinke
,
F.
Děchtěrenko
,
J.
Tintěra
,
I.
Oliver
, and
J.
Davidsen
, “
Network inference and maximum entropy estimation on information diagrams
,”
Sci. Rep.
7
,
7062
(
2017
).
20
B.
Allen
,
B. C.
Stacey
, and
Y.
Bar-Yam
, “
Multiscale information theory and the marginal utility of information
,”
Entropy
19
,
273
(
2017
).
21
J.
Runge
, “
Causal network reconstruction from time series: From theoretical assumptions to practical estimation
,”
Chaos
28
,
075310
(
2018
).
22
J.
Runge
,
S.
Bathiany
,
E.
Bollt
,
G.
Camps-Valls
,
D.
Coumou
,
E.
Deyle
,
C.
Glymour
,
M.
Kretschmer
,
M. D.
Mahecha
,
J.
Muñoz-Marí
,
E. H.
van Nes
,
J.
Peters
,
R.
Quax
,
M.
Reichstein
,
M.
Scheffer
,
B.
Schlkopf
,
P.
Spirtes
,
G.
Sugihara
,
J.
Sun
,
K.
Zhang
, and
J.
Zscheischler
, “
Inferring causation from time series in earth system sciences
,”
Nat. Commun.
10
,
2553
(
2019
).
23
J.
Hlinka
,
D.
Hartman
,
M.
Vejmelka
,
J.
Runge
,
N.
Marwan
,
J.
Kurths
, and
M.
Paluš
, “
Reliability of inference of directed climate networks using conditional mutual information
,”
Entropy
15
,
2023
2045
(
2013
).
24
Y.
Benjamini
and
Y.
Hochberg
, “
Controlling the false discovery rate: A practical and powerful approach to multiple testing
,”
J. R. Stat. Soc. Series B Stat. Methodol.
57
,
289
300
(
1995
).
25
G.
Borboudakis
and
I.
Tsamardinos
, “Forward-backward selection with early dropping,”
J. Machine Learning Research
20
,
1
39
(
2019
); available at http://jmlr.org/papers/v20/17-334.htm.
26
J.
Hlinka
,
D.
Hartman
,
N.
Jajcay
,
D.
Tomeček
,
J.
Tintěra
, and
M.
Paluš
, “
Small-world bias of correlation networks: From brain to climate
,”
Chaos
27
,
035812
(
2017
).
27
J.
Hlinka
,
M.
Palus
,
M.
Vejmelka
,
D.
Mantini
, and
M.
Corbetta
, “
Functional connectivity in resting-state FMRI: Is linear correlation sufficient?
,”
NeuroImage
54
,
2218
2225
(
2011
).
28
D.
Hartman
,
J.
Hlinka
,
M.
Paluš
,
D.
Mantini
, and
M.
Corbetta
, “
The role of nonlinearity in computing graph-theoretical properties of resting-state functional magnetic resonance imaging brain networks
,”
Chaos
21
,
013119
(
2011
).
29
J.
Hlinka
,
D.
Hartman
,
M.
Vejmelka
,
D.
Novotna
, and
M.
Palus
, “
Non-linear dependence and teleconnections in climate data: Sources, relevance, nonstationarity
,”
Clim. Dyn.
42
,
1873
1886
(
2014
).
30
L.
Barnett
,
A. B.
Barrett
, and
A. K.
Seth
, “
Granger causality and transfer entropy are equivalent for Gaussian variables
,”
Phys. Rev. Lett.
103
,
238701
(
2009
).
31
J.
Hlinka
,
N.
Jajcay
,
D.
Hartman
, and
M.
Paluš
, “
Smooth information flow in temperature climate network reflects mass transport
,”
Chaos
27
,
035811
(
2017
).
32
M.
Vejmelka
,
L.
Pokorna
,
J.
Hlinka
,
D.
Hartman
,
N.
Jajcay
, and
M.
Palus
, “
Non-random correlation structures and dimensionality reduction in multivariate climate data
,”
Clim. Dyn.
44
,
2663
2682
(
2015
).
33
J.
Runge
,
V.
Petoukhov
,
J. F.
Donges
,
J.
Hlinka
,
N.
Jajcay
,
M.
Vejmelka
,
D.
Hartman
,
N.
Marwan
,
M.
Palus
, and
J.
Kurths
, “
Identifying causal gateways and mediators in complex spatio-temporal systems
,”
Nat. Commun.
6
,
8502
(
2015
).
34
D.
Marinazzo
,
M.
Pellicoro
, and
S.
Stramaglia
, “
Kernel method for nonlinear Granger causality
,”
Phys. Rev. Lett.
100
,
144103
(
2008
).
35
A.
Arnold
,
Y.
Liu
, and
N.
Abe
, “Temporal causal modeling with graphical Granger methods,” in Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’07) (ACM, New York, NY, 2007), pp. 66–75.
36
G.
Yang
,
L.
Wang
, and
X.
Wang
, “
Reconstruction of complex directional networks with group lasso nonlinear conditional Granger causality
,”
Sci. Rep.
7
,
2991
(
2017
).
37
E.
Pereda
,
R.
Quiroga
, and
J.
Bhattacharya
, “
Nonlinear multivariate analysis of neurophysiological signals
,”
Prog. Neurobiol.
77
,
1
37
(
2005
).
38
D.
Chicharro
and
R. G.
Andrzejak
, “
Reliable detection of directional couplings using rank statistics
,”
Phys. Rev. E
80
,
026217
(
2009
).
39
B.
Wahl
,
U.
Feudel
,
J.
Hlinka
,
M.
Wächter
,
J.
Peinke
, and
J. A.
Freund
, “
Granger-causality maps of diffusion processes
,”
Phys. Rev. E
93
,
022213
(
2016
).
40
B.
Wahl
,
U.
Feudel
,
J.
Hlinka
,
M.
Wächter
,
J.
Peinke
, and
J. A.
Freund
, “
Conditional Granger causality of diffusion processes
,”
Eur. Phys. J. B
90
,
197
(
2017
).

Supplementary Material

You do not currently have access to this content.