We study the tipping point collective dynamics of an adaptive susceptible–infected–susceptible (SIS) epidemiological network in a data-driven, machine learning-assisted manner. We identify a parameter-dependent effective stochastic differential equation (eSDE) in terms of physically meaningful coarse mean-field variables through a deep-learning ResNet architecture inspired by numerical stochastic integrators. We construct an approximate effective bifurcation diagram based on the identified drift term of the eSDE and contrast it with the mean-field SIS model bifurcation diagram. We observe a subcritical Hopf bifurcation in the evolving network’s effective SIS dynamics that causes the tipping point behavior; this takes the form of large amplitude collective oscillations that spontaneously—yet rarely—arise from the neighborhood of a (noisy) stationary state. We study the statistics of these rare events both through repeated brute force simulations and by using established mathematical/computational tools exploiting the right-hand side of the identified SDE. We demonstrate that such a collective SDE can also be identified (and the rare event computations also performed) in terms of data-driven coarse observables, obtained here via manifold learning techniques, in particular, Diffusion Maps. The workflow of our study is straightforwardly applicable to other complex dynamic problems exhibiting tipping point dynamics.
REFERENCES
1.
S. H.
Strogatz
, Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering
(CRC Press
, 2018
).2.
J.
Guckenheimer
and P.
Holmes
, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields
(Springer Science & Business Media
, 2013
), Vol. 42.3.
A.
Uppal
, W. H.
Ray
, and A. B.
Poore
, “On the dynamic behavior of continuous stirred tank reactors
,” Chem. Eng. Sci.
29
(4
), 967
–985
(1974
). 4.
T.
Gross
, C. J.
Dommar D’Lima
, and B.
Blasius
, “Epidemic dynamics on an adaptive network
,” Phys. Rev. Lett.
96
(20
), 208701
(2006
). 5.
T.
Gross
and I. G.
Kevrekidis
, “Robust oscillations in SIS epidemics on adaptive networks: Coarse graining by automated moment closure
,” Europhys. Lett.
82
(3
), 38004
(2008
). 6.
M.
Scheffer
, Critical Transitions in Nature and Society
(Princeton University Press
, 2020
), Vol. 16.7.
A.
Gnanadesikan
, R.
Kelson
, and M.
Sten
, “Flux correction and overturning stability: Insights from a dynamical box model
,” J. Clim.
31
(22
), 9335
–9350
(2018
). 8.
S.
Gualdi
, M.
Tarzia
, F.
Zamponi
, and J.-P.
Bouchaud
, “Tipping points in macroeconomic agent-based models
,” J. Econ. Dyn. Control
50
, 29
–61
(2015
). 9.
P.
Liu
, C. I.
Siettos
, C.
William Gear
, and I. G.
Kevrekidis
, “Equation-free model reduction in agent-based computations: Coarse-grained bifurcation and variable-free rare event analysis
,” Math. Model. Nat. Phenom.
10
(3
), 71
–90
(2015
). 10.
A.
Omurtag
and L.
Sirovich
, “Modeling a large population of traders: Mimesis and stability
,” J. Econ. Behav. Organ.
61
(4
), 562
–576
(2006
). 11.
D.
Centola
, J.
Becker
, D.
Brackbill
, and A.
Baronchelli
, “Experimental evidence for tipping points in social convention
,” Science
360
(6393
), 1116
–1119
(2018
). 12.
M.
Milkoreit
, J.
Hodbod
, J.
Baggio
, K.
Benessaiah
, R.
Calderón-Contreras
, J. F.
Donges
, J.-D.
Mathias
, J.
Carlos Rocha
, M.
Schoon
, and S. E.
Werners
, “Defining tipping points for social-ecological systems scholarship—an interdisciplinary literature review
,” Environ. Res. Lett.
13
(3
), 033005
(2018
). 13.
I. M.
Otto
, J. F.
Donges
, R.
Cremades
, A.
Bhowmik
, R. J.
Hewitt
, W.
Lucht
, J.
Rockström
, F.
Allerberger
, M.
McCaffrey
, S. S. P.
Doe
, and A.
Lenferna
, “Social tipping dynamics for stabilizing Earth’s climate by 2050
,” Proc. Natl. Acad. Sci. U.S.A.
117
(5
), 2354
–2365
(2020
). 14.
M.
Gladwell
, The Tipping Point: How Little Things Can Make a Big Difference
(Little, Brown & Company
, 2006
).15.
F.
Dietrich
, A.
Makeev
, G.
Kevrekidis
, N.
Evangelou
, T.
Bertalan
, S.
Reich
, and I. G.
Kevrekidis
, “Learning effective stochastic differential equations from microscopic simulations: Linking stochastic numerics to deep learning
,” Chaos
33
(2
), 023121
(2023
). 16.
See https://github.com/scigma/scigma/tree/master/examples for more information about scigma. Last accessed June 18, 2023.
17.
R. R.
Coifman
and S.
Lafon
, “Diffusion maps
,” Appl. Comput. Harmon. Anal.
21
(1
), 5
–30
(2006
). 18.
C. C.
Aggarwal
and H.
Wang
, Managing and Mining Graph Data
(Springer
, 2010
), Vol. 40.19.
K. A.
Bold
, K.
Rajendran
, B.
Ráth
, and I. G.
Kevrekidis
, “An equation-free approach to coarse-graining the dynamics of networks,” arXiv:1202.5618 (2012).20.
A. A.
Kattis
, A.
Holiday
, A.-A.
Stoica
, and I. G.
Kevrekidis
, “Modeling epidemics on adaptively evolving networks: A data-mining perspective
,” Virulence
7
(2
), 153
–162
(2016
). 21.
K.
Rajendran
, A.
Kattis
, A.
Holiday
, R.
Kondor
, and I. G.
Kevrekidis
, “Data mining when each data point is a network,” in Patterns of Dynamics, edited by P. Gurevich, J. Hell, B. Sandstede, and A. Scheel (Springer International Publishing, Cham, 2017), pp. 289–317.22.
P.
Wills
and F. G.
Meyer
, “Metrics for graph comparison: A practitioner’s guide
,” PLoS One
15
(2
), e0228728
(2020
). 23.
A.
Athreya
, Z.
Lubberts
, Y.
Park
, and C. E.
Priebe
, “Discovering underlying dynamics in time series of networks,” arXiv:2205.06877 (2022).24.
B.
Oksendal
, Stochastic Differential Equations: An Introduction With Applications
(Springer Science & Business Media
, 2013
).25.
S. L.
Brunton
, J. L.
Proctor
, and J.
Nathan Kutz
, “Discovering governing equations from data by sparse identification of nonlinear dynamical systems
,” Proc. Natl. Acad. Sci. U.S.A.
113
(15
), 3932
–3937
(2016
). 26.
R. T.
Chen
, Y.
Rubanova
, J.
Bettencourt
, and D. K.
Duvenaud
, “Neural ordinary differential equations
,” in Advances in Neural Information Processing Systems
(2018), p. 31.27.
P.
Kidger
, J.
Morrill
, J.
Foster
, and T.
Lyons
, “Neural controlled differential equations for irregular time series,” in Advances in Neural Information Processing Systems, edited by H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Curran Associates, Inc., 2020), Vol. 33, pp. 6696–6707.28.
L.
Lu
, P.
Jin
, and G.
Em Karniadakis
, “Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators,” arXiv:1910.03193 (2019).29.
M.
Raissi
, P.
Perdikaris
, and G. E.
Karniadakis
, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
,” J. Comput. Phys.
378
, 686
–707
(2019
). 30.
J.
Brandstetter
, R.
van den Berg
, M.
Welling
, and J. K.
Gupta
, “Clifford neural layers for PDE modeling,” arXiv:2209.04934 (2022).31.
F. P.
Kemeth
, T.
Bertalan
, T.
Thiem
, F.
Dietrich
, S.
Joon Moon
, C. R.
Laing
, and I. G.
Kevrekidis
, “Learning emergent partial differential equations in a learned emergent space
,” Nat. Commun.
13
(1
), 3318
(2022
). 32.
S.
Lee
, Y. M.
Psarellis
, C. I.
Siettos
, and I. G.
Kevrekidis
, “Learning black-and gray-box chemotactic PDEs/closures from agent based monte carlo simulation data,” arXiv:2205.13545 (2022).33.
K.
Krischer
, R.
Rico-Martinez
, I. G.
Kevrekidis
, H. H.
Rotermund
, G.
Ertl
, and J. L.
Hudson
, “Model identification of a spatiotemporally varying catalytic reaction
,” AIChE J.
39
(1
), 89
–98
(1993
). 34.
R.
Rico-Martines
, I. G.
Kevrekidis
, M. C.
Kube
, and J. L.
Hudson
, “Discrete- vs. Continuous-time nonlinear signal processing: Attractors, transitions and parallel implementation issues,” in 1993 American Control Conference (IEEE, 1993), pp. 1475–1479.35.
R.
Rico-Martinez
, K.
Krischer
, I. G.
Kevrekidis
, M. C.
Kube
, and J. L.
Hudson
, “Discrete- vs. Continuous-time nonlinear signal processing of Cu electrodissolution data
,” Chem. Eng. Commun.
118
(1
), 25
–48
(1992
). 36.
R.
González-García
, R.
Rico-Martìnez
, and I. G.
Kevrekidis
, “Identification of distributed parameter systems: A neural net based approach
,” Comput. Chem. Eng.
22
, S965
–S968
(1998
). 37.
X.
Li
, T.-K.
Leonard Wong
, R. T. Q.
Chen
, and D. K.
Duvenaud
, “Scalable gradients and variational inference for stochastic differential equations,” in Symposium on Advances in Approximate Bayesian Inference (PMLR, 2020), pp. 1–28.38.
L.
Yang
, D.
Zhang
, and G.
Em Karniadakis
, “Physics-informed generative adversarial networks for stochastic differential equations
,” SIAM J. Sci. Comput.
42
(1
), A292
–A317
(2020
). 39.
L.
Yang
, C.
Daskalakis
, and G. E.
Karniadakis
, “Generative ensemble regression: Learning particle dynamics from observations of ensembles with physics-informed deep generative models
,” SIAM J. Sci. Comput.
44
(1
), B80
–B99
(2022
). 40.
P.
Kidger
, J.
Foster
, X.
Li
, and T. J.
Lyons
, “Neural SDEs as infinite-dimensional gans,” in International Conference on Machine Learning (PMLR, 2021), pp. 5453–5463.41.
Y.
Zhu
, Y.-H.
Tang
, and C.
Kim
, “Learning stochastic dynamics with statistics-informed neural network
,” J. Comput. Phys.
474
, 111819
(2023
). 42.
C.
Fang
, Y.
Lu
, T.
Gao
, and J.
Duan
, “An end-to-end deep learning approach for extracting stochastic dynamical systems with
-stable Lévy noise
,” Chaos
32
(6
), 063112
(2022
). 43.
A.
Hasan
, J. M.
Pereira
, S.
Farsiu
, and V.
Tarokh
, “Identifying latent stochastic differential equations
,” IEEE Trans. Signal Process.
70
, 89
–104
(2021
). 44.
K.
Pearson
, “LIII. On lines and planes of closest fit to systems of points in space
,” London, Edinburgh Dublin Philos. Mag. J. Sci.
2
(11
), 559
–572
(1901
). 45.
S. T.
Roweis
and L. K.
Saul
, “Nonlinear dimensionality reduction by locally linear embedding
,” Science
290
(5500
), 2323
–2326
(2000
). 46.
M.
Belkin
and P.
Niyogi
, “Laplacian eigenmaps for dimensionality reduction and data representation
,” Neural Comput.
15
(6
), 1373
–1396
(2003
). 47.
M.
Balasubramanian
and E. L.
Schwartz
, “The isomap algorithm and topological stability
,” Science
295
(5552
), 7
(2002
). 48.
L.
van der Maaten
and G.
Hinton
, “Visualizing data using t-SNE
,” J. Mach. Learn. Res.
9
(11
), 2579
–2605
(2008
).49.
L.
McInnes
, J.
Healy
, and J.
Melville
, “UMAP: Uniform manifold approximation and projection for dimension reduction,” arXiv:1802.03426 (2018).50.
M. A.
Kramer
, “Nonlinear principal component analysis using autoassociative neural networks
,” AIChE J.
37
(2
), 233
–243
(1991
). 51.
N.
Evangelou
, F.
Dietrich
, E.
Chiavazzo
, D.
Lehmberg
, M.
Meila
, and I. G.
Kevrekidis
, “Double diffusion maps and their latent harmonics for scientific computations in latent space,” arXiv:2204.12536 (2022).52.
N.
Evangelou
, N. J.
Wichrowski
, G. A.
Kevrekidis
, F.
Dietrich
, M.
Kooshkbaghi
, S.
McFann
, and I. G.
Kevrekidis
, “On the parameter combinations that matter and on those that do not: Data-driven studies of parameter (non)identifiability
,” PNAS Nexus
1
(4
), pgac154
(2022
). 53.
E. D.
Koronaki
, N.
Evangelou
, Y. M.
Psarellis
, A. G.
Boudouvis
, and I. G.
Kevrekidis
, “From partial data to out-of-sample parameter and observation estimation with diffusion maps and geometric harmonics,” arXiv:2301.11728 (2023).54.
R.
Talmon
, I.
Cohen
, S.
Gannot
, and R. R.
Coifman
, “Diffusion maps for signal processing: A deeper look at manifold-learning techniques based on kernels and graphs
,” IEEE Signal Process. Mag.
30
(4
), 75
–86
(2013
). 55.
B.
Nadler
, S.
Lafon
, R. R.
Coifman
, and I. G.
Kevrekidis
, “Diffusion maps, spectral clustering and reaction coordinates of dynamical systems
,” Appl. Comput. Harmon. Anal.
21
(1
), 113
–127
(2006
). 56.
A.
Holiday
, M.
Kooshkbaghi
, J. M.
Bello-Rivas
, C.
William Gear
, A.
Zagaris
, and I. G.
Kevrekidis
, “Manifold learning for parameter reduction
,” J. Comput. Phys.
392
, 419
–431
(2019
). 57.
D. W.
Sroczynski
, F. P.
Kemeth
, R. R.
Coifman
, and I. G.
Kevrekidis
, “Questionnaires to PDEs: From disorganized data to emergent generative dynamic models,” arXiv:2204.11961 (2022).58.
O.
Yair
, R.
Talmon
, R. R.
Coifman
, and I. G.
Kevrekidis
, “Reconstruction of normal forms by learning informed observation geometries from data
,” Proc. Natl. Acad. Sci. U.S.A.
114
(38
), E7865
–E7874
(2017
). 59.
E.
Chiavazzo
, R.
Covino
, R. R.
Coifman
, C.
William Gear
, A. S.
Georgiou
, G.
Hummer
, and I. G.
Kevrekidis
, “Intrinsic map dynamics exploration for uncharted effective free-energy landscapes
,” Proc. Natl. Acad. Sci. U.S.A.
114
(28
), E5494
–E5503
(2017
). 60.
A. L.
Ferguson
, A. Z.
Panagiotopoulos
, P. G.
Debenedetti
, and I. G.
Kevrekidis
, “Systematic determination of order parameters for chain dynamics using diffusion maps
,” Proc. Natl. Acad. Sci. U.S.A.
107
(31
), 13597
–13602
(2010
). 61.
N.
Evangelou
, F.
Dietrich
, J. M.
Bello-Rivas
, A. J.
Yeh
, R. S.
Hendley
, M. A.
Bevan
, and I. G.
Kevrekidis
, “Learning effective SDEs from Brownian dynamic simulations of colloidal particles
,” Mol. Syst. Des. Eng.
8
(7
), 887
–901
(2023
). 62.
F. P.
Kemeth
, T.
Bertalan
, N.
Evangelou
, T.
Cui
, S.
Malani
, and I. G.
Kevrekidis
, “Initializing LSTM internal states via manifold learning
,” Chaos
31
(9
), 093111
(2021
). 63.
I. G.
Kevrekidis
, C.
William Gear
, J. M.
Hyman
, P. G.
Kevrekidis
, O.
Runborg
, and C.
Theodoropoulos
, “Equation-free, coarse-grained multiscale computation: Enabling microscopic simulators to perform system-level analysis
,” Commun. Math. Sci.
1
(4
), 715
–762
(2003
). 64.
T. M.
Bury
, R. I.
Sujith
, I.
Pavithran
, M.
Scheffer
, T. M.
Lenton
, M.
Anand
, and C. T.
Bauch
, “Deep learning for early warning signals of tipping points
,” Proc. Natl. Acad. Sci. U.S.A.
118
(39
), e2106140118
(2021
). 65.
D.
Patel
and E.
Ott
, “Using machine learning to anticipate tipping points and extrapolate to post-tipping dynamics of non-stationary dynamical systems
,” Chaos
33
(2
), 023143
(2023
). 66.
S. H.
Lim
, L. T.
Giorgini
, W.
Moon
, and J. S.
Wettlaufer
, “Predicting critical transitions in multiscale dynamical systems using reservoir computing
,” Chaos
30
(12
), 123126
(2020
). 67.
L.-W.
Kong
, H.-W.
Fan
, C.
Grebogi
, and Y.-C.
Lai
, “Machine learning prediction of critical transition and system collapse
,” Phys. Rev. Res.
3
(1
), 013090
(2021
). 68.
R.
Xiao
, L.-W.
Kong
, Z.-K.
Sun
, and Y.-C.
Lai
, “Predicting amplitude death with machine learning
,” Phys. Rev. E
104
(1
), 014205
(2021
). 69.
E.
Galaris
, G.
Fabiani
, I.
Gallos
, I.
Kevrekidis
, and C.
Siettos
, “Numerical bifurcation analysis of PDEs from lattice Boltzmann model simulations: A parsimonious machine learning approach
,” J. Sci. Comput.
92
(2
), 34
(2022
). 70.
G.
Fabiani
, N.
Evangelou
, T.
Cui
, J. M.
Bello-Rivas
, C. P.
Martin-Linares
, C.
Siettos
, and I. G.
Kevrekidis
, “Tasks makyth models: Machine learning assisted surrogates for tipping points,” arXiv:2309.14334 (2023).71.
R. R.
Coifman
, S.
Lafon
, A. B.
Lee
, M.
Maggioni
, B.
Nadler
, F.
Warner
, and S. W.
Zucker
, “Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps
,” Proc. Natl. Acad. Sci. U.S.A.
102
(21
), 7426
–7431
(2005
). 72.
C. J.
Dsilva
, R.
Talmon
, R. R.
Coifman
, and I. G.
Kevrekidis
, “Parsimonious representation of nonlinear dynamical systems through manifold learning: A chemotaxis case study
,” Appl. Comput. Harmon. Anal.
44
(3
), 759
–773
(2018
). 73.
S.
Gillies
, see https://pypi.org/project/Shapely for “The shapely user manual” (2013).74.
A.
Logg
and G. N.
Wells
, “Dolfin: Automated finite element computing
,” ACM Trans. Math. Softw.
37
(2
), 1
–28
(2010
). 75.
A.
Logg
, K.-A.
Mardal
, and G.
Wells
, Automated Solution of Differential Equations by the Finite Element Method: The FEniCS Book
(Springer Science & Business Media
, 2012
), Vol. 84.76.
D.
Lehmberg
, F.
Dietrich
, G.
Köster
, and H.-J.
Bungartz
, “Datafold: Data-driven models for point clouds and time series on manifolds
,” J. Open Source Softw.
5
(51
), 2283
(2020
). 77.
A.
Bojchevski
, O.
Shchur
, D.
Zügner
, and S.
Günnemann
, “NetGAN: Generating graphs via random walks,” in International Conference on Machine Learning (PMLR, 2018), pp. 610–619.78.
S.
Pan
, R.
Hu
, G.
Long
, J.
Jiang
, L.
Yao
, and C.
Zhang
, “Adversarially regularized graph autoencoder for graph embedding,” arXiv:1802.04407 (2018).79.
E. R.
Crabtree
, J. M.
Bello-Rivas
, A. L.
Ferguson
, and I. G.
Kevrekidis
, “GANs and closures: Micro-macro consistency in multiscale modeling
,” Multiscale Model. Simul.
21
(3
), 1122
–1146
(2023
). 80.
E. R.
Crabtree
, J. M.
Bello-Rivas
, and I. G.
Kevrekidis
, “Micro-macro consistency in multiscale modeling: Score-based model assisted sampling of fast/slow dynamical systems
,” Chaos
34
(5
), 053110
(2024
). 81.
S. S.
Sharbayta
, B.
Buonomo
, A.
d’Onofrio
, and T.
Abdi
, “‘Period doubling’ induced by optimal control in a behavioral SIR epidemic model
,” Chaos, Solitons Fractals
161
, 112347
(2022
). 82.
A.
d’Onofrio
and P.
Manfredi
, “Information-related changes in contact patterns may trigger oscillations in the endemic prevalence of infectious diseases
,” J. Theor. Biol.
256
(3
), 473
–478
(2009
). 83.
A.
d’Onofrio
and P.
Manfredi
, “Behavioral SIR models with incidence-based social-distancing
,” Chaos, Solitons Fractals
159
, 112072
(2022
). 84.
Z.
Qiu
, B.
Espinoza
, V. V.
Vasconcelos
, C.
Chen
, S. M.
Constantino
, S. A.
Crabtree
, L.
Yang
, A.
Vullikanti
, J.
Chen
, J.
Weibull
, and K.
Basu
, “Understanding the coevolution of mask wearing and epidemics: A network perspective
,” Proc. Natl. Acad. Sci. U.S.A.
119
(26
), e2123355119
(2022
). 85.
N.
Ferguson
, “Capturing human behaviour
,” Nature
446
(7137
), 733
(2007
). 86.
C.
Prévôt
and M.
Röckner
, A Concise Course on Stochastic Partial Differential Equations
(Springer
, 2007
), Vol. 1905.87.
L.
Renson
, J.
Sieber
, D. A. W.
Barton
, A. D.
Shaw
, and S. A.
Neild
, “Numerical continuation in nonlinear experiments using local Gaussian process regression
,” Nonlinear Dyn.
98
, 2811
–2826
(2019
). 88.
R.
González-García
, R.
Rico-Martínez
, W.
Wolf
, M.
Lübke
, M.
Eiswirth
, J. S.
Anderson
, and I. G.
Kevrekidis
, “Characterization of a two-parameter mixed-mode electrochemical behavior regime using neural networks
,” Physica D
151
(1
), 27
–43
(2001
). 89.
M.
Scheffer
, J.
Bascompte
, W. A.
Brock
, V.
Brovkin
, S. R.
Carpenter
, V.
Dakos
, H.
Held
, E. H.
Van Nes
, M.
Rietkerk
, and G.
Sugihara
, “Early-warning signals for critical transitions
,” Nature
461
(7260
), 53
–59
(2009
). 90.
H.
Risken
, The Fokker–Planck Equation, Springer Series in Synergetics (Springer, 1989), Vol. 18.91.
B.
Kehlet
, A.
Logg
, J.
Ring
, and G. N.
Wells
, see https://bitbucket.org/fenics-project/mshr/ for “MSHR” (2020).© 2024 Author(s). Published under an exclusive license by AIP Publishing.
2024
Author(s)
You do not currently have access to this content.
