Biological neural systems encode and transmit information as patterns of activity tracing complex trajectories in high-dimensional state spaces, inspiring alternative paradigms of information processing. Heteroclinic networks, naturally emerging in artificial neural systems, are networks of saddles in state space that provide a transparent approach to generate complex trajectories via controlled switches among interconnected saddles. External signals induce specific switching sequences, thus dynamically encoding inputs as trajectories. Recent works have focused either on computational aspects of heteroclinic networks, i.e., Heteroclinic Computing, or their stochastic properties under noise. Yet, how well such systems may transmit information remains an open question. Here, we investigate the information transmission properties of heteroclinic networks, studying them as communication channels. Choosing a tractable but representative system exhibiting a heteroclinic network, we investigate the mutual information rate (MIR) between input signals and the resulting sequences of states as the level of noise varies. Intriguingly, MIR does not decrease monotonically with increasing noise. Intermediate noise levels indeed maximize the information transmission capacity by promoting an increased yet controlled exploration of the underlying network of states. Complementing standard stochastic resonance, these results highlight the constructive effect of stochastic facilitation (i.e., noise-enhanced information transfer) on heteroclinic communication channels and possibly on more general dynamical systems exhibiting complex trajectories in state space.
REFERENCES
1.
S. M.
Bohte
, “The evidence for neural information processing with precise spike-times: A survey
,” Nat. Comput.
3
, 195
–206
(2004
). 2.
S. J.
Thorpe
, “Spike arrival times: A highly efficient coding scheme for neural networks
,” Parallel Processing in Neural Systems
(North-Holland Elsevier
, 1990
), pp. 91
–94
.3.
G.
Laurent
and M.
Naraghi
, “Odorant-induced oscillations in the mushroom bodies of the locust
,” J. Neurosci.
14
, 2993
–3004
(1994
). 4.
S.
Steingrube
, M.
Timme
, F.
Wörgötter
, and P.
Manoonpong
, “Self-organized adaptation of a simple neural circuit enables complex robot behaviour
,” Nat. Phys.
6
, 224
–230
(2010
). 5.
A. J.
Ijspeert
, A.
Crespi
, and J.-M.
Cabelguen
, “Simulation and robotics studies of salamander locomotion
,” Neuroinformatics
3
, 171
–196
(2005
). 6.
W.
Maass
, T.
Natschläger
, and H.
Markram
, “Real-time computing without stable states: A new framework for neural computation based on perturbations
,” Neural Comput.
14
, 2531
–2560
(2002
). 7.
H.
Jaeger
and H.
Haas
, “Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication
,” Science
304
, 78
–80
(2004
). 8.
M.
Rabinovich
, R.
Huerta
, and G.
Laurent
, “Transient dynamics for neural processing
,” Science
321
, 48
–50
(2008
). 9.
M.
Lukoševičius
and H.
Jaeger
, “Reservoir computing approaches to recurrent neural network training
,” Comput. Sci. Rev.
3
, 127
–149
(2009
). 10.
L.
Larger
, M. C.
Soriano
, D.
Brunner
, L.
Appeltant
, J. M.
Gutierrez
, L.
Pesquera
, C. R.
Mirasso
, and I.
Fischer
, “Photonic information processing beyond turing: An optoelectronic implementation of reservoir computing
,” Opt. Express
20
, 3241
–3249
(2012
). 11.
C.
Kirst
, M.
Timme
, and D.
Battaglia
, “Dynamic information routing in complex networks
,” Nat. Commun.
7
, 011061
(2016
). 12.
P.
Ashwin
and J.
Borresen
, “Discrete computation using a perturbed heteroclinic network
,” Phys. Lett. A
347
, 208
–214
(2005
). 13.
O.
Mazor
and G.
Laurent
, “Transient dynamics versus fixed points in odor representations by locust antennal lobe projection neurons
,” Neuron
48
, 661
–673
(2005
). 14.
J.
Wordsworth
and P.
Ashwin
, “Spatiotemporal coding of inputs for a system of globally coupled phase oscillators
,” Phys. Rev. E
78
, 066203
(2008
). 15.
F. S.
Neves
and M.
Timme
, “Computation by switching in complex networks of states
,” Phys. Rev. Lett.
109
, 018701
(2012
). 16.
F. S.
Neves
and M.
Timme
, “Reconfigurable computation in spiking neural networks
,” IEEE Access
8
, 179648
–179655
(2020
). 17.
U.
Ernst
and K.
Pawelzik
, “Delay-induced multistable synchronization of biological oscillators
,” Phys. Rev. E
57
, 2150
(1998
). 18.
M.
Timme
, F.
Wolf
, and T.
Geisel
, “Prevalence of unstable attractors in networks of pulse-coupled oscillators
,” Phys. Rev. Lett.
89
, 154105
(2002
). 19.
M.
Timme
, F.
Wolf
, and T.
Geisel
, “Unstable attractors induce perpetual synchronization and desynchronization
,” Chaos
13
, 377
–387
(2003
). 20.
M.
Krupa
, “Robust heteroclinic cycles
,” J. Nonlinear Sci.
7
, 129
–176
(1997
). 21.
C.
Bick
and M. I.
Rabinovich
, “On the occurrence of stable heteroclinic channels in Lotka–Volterra models
,” Dyn. Syst.
25
, 97
–110
(2010
). 22.
Referring to these saddles as “states” stems from a parallel with finite state machines. In fact, when describing a perturbation-driven sequence of saddle-to-saddle switching in heteroclinic networks, it is rather natural to regard saddles in the network as states, and heteroclinic connections between saddles as transitions, with the resulting sequence of visited states being a function of a starting state and applied input. For this reason, we here refer to saddles in a heteroclinic network interchangeably as “saddle states” or, simply, “states.”
23.
U.
Ernst
, K.
Pawelzik
, and T.
Geisel
, “Synchronization induced by temporal delays in pulse-coupled oscillators
,” Phys. Rev. Lett.
74
, 1570
–1573
(1995
). 24.
P.
Ashwin
and M.
Timme
, “Unstable attractors: Existence and robustness in networks of oscillators with delayed pulse coupling
,” Nonlinearity
18
, 2035
–2060
(2005
). 25.
F. S.
Neves
and M.
Timme
, “Controlled perturbation-induced switching in pulse-coupled oscillator networks
,” J. Phys. A
42
, 345103
(2009
). 26.
M. J.
Field
, “Patterns of desynchronization and resynchronization in heteroclinic networks
,” Nonlinearity
30
, 516
–557
(2016
). 27.
P.
Ashwin
, S. B. S. D.
Castro
, and A.
Lohse
, “Almost complete and equable heteroclinic networks
,” J. Nonlinear Sci.
30
, 1
–22
(2020
). 28.
O.
Podvigina
, S. B. S. D.
Castro
, and I. S.
Labouriau
, “Asymptotic stability of robust heteroclinic networks
,” Nonlinearity
33
, 1757
–1788
(2020
). 29.
C.
Kirst
, T.
Geisel
, and M.
Timme
, “Sequential desynchronization in networks of spikin neurons with partial resets
,” Phys. Rev. Lett.
102
, 068101
(2009
). 30.
P.
Ashwin
and M.
Timme
, “When instability makes sense
,” Nature
436
, 36
–37
(2005
). 31.
E.
Stone
and D.
Armbruster
, “Noise and O(1) amplitude effects on heteroclinic cycles
,” Chaos
9
, 499
–506
(1999
). 32.
D.
Armbruster
, V.
Kirk
, and E.
Stone
, “Noisy heteroclinic networks
,” Chaos
13
, 71
–79
(2003
). 33.
Y.
Bakhtin
, “Small noise limit for diffusions near heteroclinic networks
,” Dyn. Syst.
25
, 413
–431
(2010
). 34.
P.
Ashwin
and C.
Postlethwaite
, “On designing heteroclinic networks from graphs
,” Physica D
265
, 26
–39
(2010
). 35.
Y.
Bakhtin
, “Noisy heteroclinic networks
,” Probab. Theory Relat. Fields
150
, 1
–42
(2011
). 36.
P.
Ashwin
and C.
Postlethwaite
, “Quantifying noisy attractors: From heteroclinic to excitable networks
,” SIAM J. Appl. Dyn. Syst.
15
, 1989
–2016
(2016
). 37.
M.
Voit
and H.
Meyer-Ortmanns
, “Dynamical inference of simple heteroclinic networks
,” Front. Appl. Math. Stat.
5
, 129
(2019
). 38.
F. S.
Neves
, M.
Voit
, and M.
Timme
, “Noise-constrained switching times for heteroclinic computing
,” Chaos
27
, 033107
(2017
). 39.
P.
Ashwin
and J.
Borresen
, “Encoding via conjugate symmetries of slow oscillations for globally coupled oscillators
,” Phys. Rev. E
70
, 026203
(2004
). 40.
G.
Laurent
, M.
Stopfer
, R. W.
Friedrich
, M. I.
Rabinovich
, A.
Volkovskii
, and H. D.
Abarbanel
, “Odor encoding as an active, dynamical process: Experiments, computation, and theory
,” Annu. Rev. Neurosci.
24
, 263
–297
(2001
). 41.
R.
Huerta
, T.
Nowotny
, M.
García-Sanchez
, H. D.
Abarbanel
, and M. I.
Rabinovich
, “Learning classification in the olfactory system of insects
,” Neural Comput.
16
, 1601
–1640
(2004
). 42.
V. S.
Afraimovich
, M. I.
Rabinovich
, and P.
Varona
, “Heteroclinic contours in neural ensembles and the winnerless competition principle
,” Int. J. Bifurcation Chaos
14
, 1195
–1208
(2004
). 43.
K.
Wiesenfeld
, F.
Moss
et al., “Stochastic resonance and the benefits of noise: From ice ages to crayfish and squids
,” Nature
373
, 33
–36
(1995
). 44.
F. H.
Magalhães
and A. F.
Kohn
, “Vibratory noise to the fingertip enhances balance improvement associated with light touch
,” Exp. Brain Res.
209
, 139
–151
(2011
). 45.
F.-G.
Zeng
, Q.-J.
Fu
, and R.
Morse
, “Human hearing enhanced by noise
,” Brain Res.
869
, 251
–255
(2000
). 46.
A.
Pikovsky
, A.
Zaikin
, and M. A.
de la Casa
, “System size resonance in coupled noisy systems and in the ising model
,” Phys. Rev. Lett.
88
, 050601
(2002
). 47.
A.
Pikovsky
and J.
Kurths
, “Coherence resonance in a noise-driven excitable system
,” Phys. Rev. Lett.
78
, 775
–778
(1997
). 48.
M. D.
McDonnell
and L. M.
Ward
, “The benefits of noise in neural systems: Bridging theory and experiment
,” Nat. Rev. Neurosci.
12
, 415
–425
(2011
). 49.
M.
Voit
and H.
Meyer-Ortmanns
, “A hierarchical heteroclinic network
,” Eur. Phys. J. Spec. Top.
227
, 1101
–1115
(2018
). 50.
V.
Afraimovich
, V.
Zhigulin
, and M.
Rabinovich
, “On the origin of reproducible sequential activity in neural circuits
,” Chaos
14
, 1123
–1129
(2004
). 51.
V. S.
Afraimovich
, M. A.
Zaks
, and M. I.
Rabinovich
, “Mind-to-mind heteroclinic coordination: Model of sequential episodic memory initiation
,” Chaos
28
, 053107
(2018
). 52.
M. I.
Rabinovich
and P.
Varona
, “Discrete sequential information coding: Heteroclinic cognitive dynamics
,” Front. Comput. Neurosci.
12
, 103118
(2018
). 53.
K. V.
Palem
, “Energy aware computing through probabilistic switching: A study of limits
,” Comput. IEEE Trans.
54
, 1123
–1137
(2005
). 54.
L. N.
Chakrapani
, P.
Korkmaz
, B. E.
Akgul
, and K. V.
Palem
, “Probabilistic system-on-a-chip architectures
,” ACM Trans. Des. Autom. Electron. Syst.
12
, 1
–28
(2007
). © 2021 Author(s). Published under an exclusive license by AIP Publishing.
2021
Author(s)
You do not currently have access to this content.
