This work presents a comparison between different approaches for the model-free estimation of information-theoretic measures of the dynamic coupling between short realizations of random processes. The measures considered are the mutual information rate (MIR) between two random processes and and the terms of its decomposition evidencing either the individual entropy rates of and and their joint entropy rate, or the transfer entropies from to and from to and the instantaneous information shared by and . All measures are estimated through discretization of the random variables forming the processes, performed either via uniform quantization (binning approach) or rank ordering (permutation approach). The binning and permutation approaches are compared on simulations of two coupled non-identical Hènon systems and on three datasets, including short realizations of cardiorespiratory (CR, heart period and respiration flow), cardiovascular (CV, heart period and systolic arterial pressure), and cerebrovascular (CB, mean arterial pressure and cerebral blood flow velocity) measured in different physiological conditions, i.e., spontaneous vs paced breathing or supine vs upright positions. Our results show that, with careful selection of the estimation parameters (i.e., the embedding dimension and the number of quantization levels for the binning approach), meaningful patterns of the MIR and of its components can be achieved in the analyzed systems. On physiological time series, we found that paced breathing at slow breathing rates induces less complex and more coupled CR dynamics, while postural stress leads to unbalancing of CV interactions with prevalent baroreflex coupling and to less complex pressure dynamics with preserved CB interactions. These results are better highlighted by the permutation approach, thanks to its more parsimonious representation of the discretized dynamic patterns, which allows one to explore interactions with longer memory while limiting the curse of dimensionality.
REFERENCES
1.
N. F.
Rulkov
, M. M.
Sushchik
, L. S.
Tsimring
, and H. D.
Abarbanel
, “Generalized synchronization of chaos in directionally coupled chaotic systems
,” Phys. Rev. E
51
, 980
(1995
). 2.
J.
Arnhold
, P.
Grassberger
, K.
Lehnertz
, and C. E.
Elger
, “A robust method for detecting interdependences: Application to intracranially recorded EEG
,” Phys. D: Nonlinear Phenom.
134
, 419
–430
(1999
). 3.
L.
Faes
, A.
Porta
, and G.
Nollo
, “Mutual nonlinear prediction as a tool to evaluate coupling strength and directionality in bivariate time series: Comparison among different strategies based on k nearest neighbors
,” Phys. Rev. E
78
, 026201
(2008
). 4.
K.
Li
, L.
Guo
, J.
Nie
, G.
Li
, and T.
Liu
, “Review of methods for functional brain connectivity detection using fMRI
,” Comput. Med. Imaging Graph.
33
, 131
–139
(2009
). 5.
A.
Bashan
, R. P.
Bartsch
, J.
Kantelhardt
, S.
Havlin
, and P. C.
Ivanov
, “Network physiology reveals relations between network topology and physiological function
,” Nat. Commun.
3
, 702
(2012
). 6.
L.
Faes
, G.
Nollo
, and K. H.
Chon
, “Assessment of Granger causality by nonlinear model identification: Application to short-term cardiovascular variability
,” Ann. Biomed. Eng.
36
, 381
–395
(2008
). 7.
S. L.
Bressler
and A. K.
Seth
, “Wiener–Granger causality: A well established methodology
,” Neuroimage
58
, 323
–329
(2011
). 8.
J. T.
Lizier
, The Local Information Dynamics of Distributed Computation in Complex Systems
(Springer Science & Business Media
, 2012
).9.
M.
Wibral
, J. T.
Lizier
, and V.
Priesemann
, “Bits from brains for biologically inspired computing
,” Front. Robot. AI
2
, 5
(2015
). 10.
D.
Widjaja
, A.
Montalto
, E.
Vlemincx
, D.
Marinazzo
, S.
Van Huffel
, and L.
Faes
, “Cardiorespiratory information dynamics during mental arithmetic and sustained attention
,” PLoS One
10
, e0129112
(2015
). 11.
M.
Javorka
, J.
Krohova
, B.
Czippelova
, Z.
Turianikova
, Z.
Lazarova
, R.
Wiszt
, and L.
Faes
, “Towards understanding the complexity of cardiovascular oscillations: Insights from information theory
,” Comput. Biol. Med.
98
, 48
–57
(2018
). 12.
M.
Wibral
, J. T.
Lizier
, S.
Vögler
, V.
Priesemann
, and R.
Galuske
, “Local active information storage as a tool to understand distributed neural information processing
,” Front. Neuroinform.
8
, 1
(2014
). 13.
L.
Faes
, M. A.
Pereira
, M. E.
Silva
, R.
Pernice
, A.
Busacca
, M.
Javorka
, and A. P.
Rocha
, “Multiscale information storage of linear long-range correlated stochastic processes
,” Phys. Rev. E
99
, 032115
(2019
). 14.
R.
Vicente
, M.
Wibral
, M.
Lindner
, and G.
Pipa
, “Transfer entropy—A model-free measure of effective connectivity for the neurosciences
,” J. Comput. Neurosci.
30
, 45
–67
(2011
). 15.
L.
Faes
, D.
Marinazzo
, A.
Montalto
, and G.
Nollo
, “Lag-specific transfer entropy as a tool to assess cardiovascular and cardiorespiratory information transfer
,” IEEE Trans. Biomed. Eng.
61
, 2556
–2568
(2014
). 16.
L.
Faes
, M.
Gómez-Extremera
, R.
Pernice
, P.
Carpena
, G.
Nollo
, A.
Porta
, and P.
Bernaola-Galván
, “Comparison of methods for the assessment of nonlinearity in short-term heart rate variability under different physiopathological states
,” Chaos
29
, 123114
(2019
). 17.
V.
Bari
, B.
De Maria
, C. E.
Mazzucco
, G.
Rossato
, D.
Tonon
, G.
Nollo
, L.
Faes
, and A.
Porta
, “Cerebrovascular and cardiovascular variability interactions investigated through conditional joint transfer entropy in subjects prone to postural syncope
,” Physiol. Meas.
38
, 976
(2017
). 18.
T. E.
Duncan
, “On the calculation of mutual information
,” SIAM J. Appl. Math.
19
, 215
–220
(1970
). 19.
E.
Bianco-Martinez
, N.
Rubido
, C. G.
Antonopoulos
, and M.
Baptista
, “Successful network inference from time-series data using mutual information rate
,” Chaos
26
, 043102
(2016
). 20.
D.
Chicharro
, “On the spectral formulation of Granger causality
,” Biol. Cybern.
105
, 331
–347
(2011
). 21.
M.
Baptista
and J.
Kurths
, “Transmission of information in active networks
,” Phys. Rev. E
77
, 026205
(2008
). 22.
G.
Mijatovic
, Y.
Antonacci
, T.
Loncar-Turukalo
, L.
Minati
, and L.
Faes
, “An information-theoretic framework to measure the dynamic interaction between neural spike trains
,” IEEE Trans. Biomed. Eng.
68
, 3471
–3481
(2021
). 23.
G.
Mijatovic
, R.
Pernice
, A.
Perinelli
, Y.
Antonacci
, A.
Busacca
, M.
Javorka
, L.
Ricci
, and L.
Faes
, “Measuring the rate of information exchange in point-process data with application to cardiovascular variability
,” Front. Netw. Physiol.
1
, 765332
(2022
). 24.
L.
Faes
, G.
Mijatovic
, Y.
Antonacci
, R.
Pernice
, C.
Barà
, L.
Sparacino
, M.
Sammartino
, A.
Porta
, D.
Marinazzo
, and S.
Stramaglia
, “A new framework for the time-and frequency-domain assessment of high-order interactions in networks of random processes
,” IEEE Trans. Signal Process.
70
, 5766
–5777
(2022
). 25.
A.
Porta
, G.
Baselli
, D.
Liberati
, N.
Montano
, C.
Cogliati
, T.
Gnecchi-Ruscone
, A.
Malliani
, and S.
Cerutti
, “Measuring regularity by means of a corrected conditional entropy in sympathetic outflow
,” Biol. Cybern.
78
, 71
–78
(1998
). 26.
R.
Pernice
, L.
Sparacino
, V.
Bari
, F.
Gelpi
, B.
Cairo
, G.
Mijatovic
, Y.
Antonacci
, D.
Tonon
, G.
Rossato
, M.
Javorka
, and A.
Porta
, “Spectral decomposition of cerebrovascular and cardiovascular interactions in patients prone to postural syncope and healthy controls
,” Auton. Neurosci.
242
, 103021
(2022
). 27.
A. B.
Barrett
, L.
Barnett
, and A. K.
Seth
, “Multivariate Granger causality and generalized variance
,” Phys. Rev. E
81
, 041907
(2010
). 28.
H.
Azami
, L.
Faes
, J.
Escudero
, A.
Humeau-Heurtier
, and L. E.
Silva
, “Entropy analysis of univariate biomedical signals: Review and comparison of methods,” in Frontiers in Entropy Across the Disciplines: Panorama of Entropy: Theory, Computation, and Applications (World Scientific Publishing, 2020), pp. 233–286.29.
L.
Faes
and A.
Porta
, “Conditional entropy-based evaluation of information dynamics in physiological systems,” in Directed Information Measures in Neuroscience (Springer, 2014), pp. 61–86.30.
C.
Bandt
and B.
Pompe
, “Permutation entropy: A natural complexity measure for time series
,” Phys. Rev. Lett.
88
, 174102
(2002
). 31.
32.
W.
Xiong
, L.
Faes
, and P. C.
Ivanov
, “Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations
,” Phys. Rev. E
95
, 062114
(2017
). 33.
M.
Staniek
and K.
Lehnertz
, “Symbolic transfer entropy
,” Phys. Rev. Lett.
100
, 158101
(2008
). 34.
A. M.
Unakafov
and K.
Keller
, “Conditional entropy of ordinal patterns
,” Phys. D: Nonlinear Phenom.
269
, 94
–102
(2014
). 35.
D.
Kugiumtzis
, “Transfer entropy on rank vectors
,” J. Nonlinear Syst. Appl.
73
, 81
(2012
).36.
D.
Kugiumtzis
, “Partial transfer entropy on rank vectors
,” Eur. Phys. J. Spec. Top.
222
, 401
–420
(2013
). 37.
J.
Runge
, J.
Heitzig
, V.
Petoukhov
, and J.
Kurths
, “Escaping the curse of dimensionality in estimating multivariate transfer entropy
,” Phys. Rev. Lett.
108
, 258701
(2012
). 38.
J.
Theiler
, S.
Eubank
, A.
Longtin
, B.
Galdrikian
, and J. D.
Farmer
, “Testing for nonlinearity in time series: The method of surrogate data
,” Phys. D: Nonlinear Phenom.
58
, 77
–94
(1992
). 39.
R. Q.
Quiroga
, A.
Kraskov
, T.
Kreuz
, and P.
Grassberger
, “Performance of different synchronization measures in real data: A case study on electroencephalographic signals
,” Phys. Rev. E
65
, 041903
(2002
). 40.
M.
Wiesenfeldt
, U.
Parlitz
, and W.
Lauterborn
, “Mixed state analysis of multivariate time series
,” Int. J. Bifurc. Chaos
11
, 2217
–2226
(2001
). 41.
A.
Porta
, T.
Bassani
, V.
Bari
, and S.
Guzzetti
, “Granger causality in cardiovascular variability series: Comparison between model-based and model-free approaches,” in 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, 2012), pp. 3684–3687.42.
G.
Nollo
, L.
Faes
, A.
Porta
, R.
Antolini
, and F.
Ravelli
, “Exploring directionality in spontaneous heart period and systolic pressure variability interactions in humans: Implications in the evaluation of baroreflex gain
,” Am. J. Physiol.-Heart Circ. Physiol.
288
, H1777
–H1785
(2005
). 43.
J.
Cohen
, Statistical Power Analysis for the Behavioral Sciences
(Routledge
, 2013
).44.
J. P.
Saul
, R. D.
Berger
, M.
Chen
, and R. J.
Cohen
, “Transfer function analysis of autonomic regulation. II. Respiratory sinus arrhythmia
,” Am. J. Physiol.-Heart Circ. Physiol.
256
, H153
–H161
(1989
). 45.
H.-S.
Song
and P. M.
Lehrer
, “The effects of specific respiratory rates on heart rate and heart rate variability
,” Appl. Psychophysiol. Biofeedback
28
, 13
–23
(2003
). 46.
M.
Elstad
, E. L.
O’Callaghan
, A. J.
Smith
, A.
Ben-Tal
, and R.
Ramchandra
, “Cardiorespiratory interactions in humans and animals: Rhythms for life
,” Am. J. Physiol.-Heart Circ. Physiol.
315
, H6
–H17
(2018
). 47.
L.
Faes
, A.
Porta
, and G.
Nollo
, “Information decomposition in bivariate systems: Theory and application to cardiorespiratory dynamics
,” Entropy
17
, 277
–303
(2015
). 48.
W. H.
Cooke
, J. B.
Hoag
, A. A.
Crossman
, T. A.
Kuusela
, K. U.
Tahvanainen
, and D. L.
Eckberg
, “Human responses to upright tilt: A window on central autonomic integration
,” J. Physiol.
517
, 617
–628
(1999
). 49.
T. J.
Mullen
, M. L.
Appel
, R.
Mukkamala
, J. M.
Mathias
, and R. J.
Cohen
, “System identification of closed-loop cardiovascular control: Effects of posture and autonomic blockade
,” Am. J. Physiol.-Heart Circ. Physiol.
272
, H448
–H461
(1997
). 50.
R.
Zhang
, J. H.
Zuckerman
, C. A.
Giller
, and B. D.
Levine
, “Transfer function analysis of dynamic cerebral autoregulation in humans
,” Am. J. Physiol.-Heart Circ. Physiol.
274
, H233
–H241
(1998
). 51.
O.
Paulson
, S.
Strandgaard
, and L.
Edvinsson
, “Cerebral autoregulation
,” Cerebrovasc. Brain Metab. Rev.
2
, 161
–192
(1990
).52.
L.
Barnett
, A. B.
Barrett
, and A. K.
Seth
, “Granger causality and transfer entropy are equivalent for Gaussian variables
,” Phys. Rev. Lett.
103
, 238701
(2009
). 53.
L.
Barnett
and A. K.
Seth
, “The MVGC multivariate Granger causality toolbox: A new approach to Granger-causal inference
,” J. Neurosci. Methods
223
, 50
–68
(2014
). 54.
L.
Barnett
and A. K.
Seth
, “Granger causality for state-space models
,” Phys. Rev. E
91
, 040101
(2015
). 55.
Y.
Antonacci
, L.
Minati
, D.
Nuzzi
, G.
Mijatovic
, R.
Pernice
, D.
Marinazzo
, S.
Stramaglia
, and L.
Faes
, “Measuring high-order interactions in rhythmic processes through multivariate spectral information decomposition
,” IEEE Access
9
, 149486
–149505
(2021
). 56.
A.
Kraskov
, H.
Stögbauer
, and P.
Grassberger
, “Estimating mutual information
,” Phys. Rev. E
69
, 066138
(2004
). 57.
I.
Vlachos
and D.
Kugiumtzis
, “Nonuniform state-space reconstruction and coupling detection
,” Phys. Rev. E
82
, 016207
(2010
). 58.
L.
Faes
, G.
Nollo
, and A.
Porta
, “Non-uniform multivariate embedding to assess the information transfer in cardiovascular and cardiorespiratory variability series
,” Comput. Biol. Med.
42
, 290
–297
(2012
). 59.
M.
Papapetrou
, E.
Siggiridou
, and D.
Kugiumtzis
, “Adaptation of partial mutual information from mixed embedding to discrete-valued time series
,” Entropy
24
, 1505
(2022
). 60.
A.
Porta
, V.
Bari
, A.
Marchi
, B.
De Maria
, P.
Castiglioni
, M.
Di Rienzo
, S.
Guzzetti
, A.
Cividjian
, and L.
Quintin
, “Limits of permutation-based entropies in assessing complexity of short heart period variability
,” Physiol. Meas.
36
, 755
(2015
). 61.
D.
Cuesta-Frau
, M.
Varela-Entrecanales
, A.
Molina-Picó
, and B.
Vargas
, “Patterns with equal values in permutation entropy: Do they really matter for biosignal classification?
,” Complexity
2018
, 1324696
. 62.
C.
Bian
, C.
Qin
, Q. D.
Ma
, and Q.
Shen
, “Modified permutation-entropy analysis of heartbeat dynamics
,” Phys. Rev. E
85
, 021906
(2012
). 63.
D.
Cuesta Frau
, “Permutation entropy: Influence of amplitude information on time series classification performance
,” Math. Biosci. Eng.
16
, 6842
–6857
(2019
). 64.
A.
Myers
and F. A.
Khasawneh
, “On the automatic parameter selection for permutation entropy
,” Chaos
30
, 033130
(2020
). 65.
F.
Battiston
, G.
Cencetti
, I.
Iacopini
, V.
Latora
, M.
Lucas
, A.
Patania
, J.-G.
Young
, and G.
Petri
, “Networks beyond pairwise interactions: Structure and dynamics
,” Phys. Rep.
874
, 1
–92
(2020
). © 2023 Author(s). Published under an exclusive license by AIP Publishing.
2023
Author(s)
You do not currently have access to this content.