Controlling complex networks of nonlinear limit-cycle oscillators is an important problem pertinent to various applications in engineering and natural sciences. While in recent years the control of oscillator populations with comprehensive biophysical models or simplified models, e.g., phase models, has seen notable advances, learning appropriate controls directly from data without prior model assumptions or pre-existing data remains a challenging and less developed area of research. In this paper, we address this problem by leveraging the network’s current dynamics to iteratively learn an appropriate control online without constructing a global model of the system. We illustrate through a range of numerical simulations that the proposed technique can effectively regulate synchrony in various oscillator networks after a small number of trials using only one input and one noisy population-level output measurement. We provide a theoretical analysis of our approach, illustrate its robustness to system variations, and compare its performance with existing model-based and data-driven approaches.

1.
A. T.
Winfree
,
Circadian Rhythms in General
(
Springer New York
,
New York
,
2001
), pp. 545–591.
2.
G.
Buzsáki
,
Rhythms of the Brain
(
Oxford University Press
,
2006
).
3.
S.
Hanslmayr
and
T.
Staudigl
, “
How brain oscillations form memories—A processing based perspective on oscillatory subsequent memory effects
,”
NeuroImage
85
,
648
655
(
2014
).
4.
S.
Ahn
,
C.
Park
, and
L. L.
Rubchinsky
, “Neural synchronization in Parkinson’s disease on different time scales,” in Springer Series in Cognitive and Neural Systems (Springer International Publishing, 2019), pp. 57–65.
5.
J. W.
Simpson-Porco
,
F.
Dörfler
, and
F.
Bullo
, “
Synchronization and power sharing for droop-controlled inverters in islanded microgrids
,”
Automatica
49
(
9
),
2603
2611
(
2013
).
6.
P. S.
Skardal
and
A.
Arenas
, “
Control of coupled oscillator networks with application to microgrid technologies
,”
Sci. Adv.
1
(
7
),
e1500339
(
2015
).
7.
I. Z.
Kiss
,
C. G.
Rusin
,
H.
Kori
, and
J. L.
Hudson
, “
Engineering complex dynamical structures: Sequential patterns and desynchronization
,”
Science
316
(
5833
),
1886
1889
(
2007
).
8.
M.
Rosenblum
and
A.
Pikovsky
, “
Delayed feedback control of collective synchrony: An approach to suppression of pathological brain rhythms
,”
Phys. Rev. E
70
(
4
),
041904
(
2004
).
9.
P. A.
Tass
,
Phase Resetting in Medicine and Biology: Stochastic Modelling and Data Analysis
(
Springer Science & Business Media
,
2007
).
10.
P. A.
Tass
, “
Desynchronizing double-pulse phase resetting and application to deep brain stimulation
,”
Biol. Cybern.
85
(
5
),
343
354
(
2001
).
11.
P.
Tass
, “
Effective desynchronization by means of double-pulse phase resetting
,”
Europhys. Lett.
53
(
1
),
15
(
2001
).
12.
P. A.
Tass
, “
Effective desynchronization with bipolar double-pulse stimulation
,”
Phys. Rev. E
66
(
3
),
036226
(
2002
).
13.
P.
Tass
, “
Effective desynchronization with a resetting pulse train followed by a single pulse
,”
Europhys. Lett.
55
(
2
),
171
(
2001
).
14.
P.
Tass
, “
Effective desynchronization with a stimulation technique based on soft phase resetting
,”
Europhys. Lett.
57
(
2
),
164
(
2002
).
15.
C. J.
Wilson
,
B.
Beverlin
, and
T.
Netoff
, “
Chaotic desynchronization as the therapeutic mechanism of deep brain stimulation
,”
Front. Syst. Neurosci.
5
,
50
(
2011
).
16.
D.
Wilson
and
J.
Moehlis
, “
Locally optimal extracellular stimulation for chaotic desynchronization of neural populations
,”
J. Comput. Neurosci.
37
,
243
257
(
2014
).
17.
A.
Nabi
,
M.
Mirzadeh
,
F.
Gibou
, and
J.
Moehlis
, “
Minimum energy desynchronizing control for coupled neurons
,”
J. Comput. Neurosci.
34
(
2
),
259
271
(
2013
).
18.
B.
Singhal
,
I. Z.
Kiss
, and
J.-S.
Li
, “
Optimal phase-selective entrainment of heterogeneous oscillator ensembles
,”
SIAM J. Appl. Dyn. Syst.
22
(
3
),
2180
2205
(
2023
).
19.
D.
Wilson
and
J.
Moehlis
, “
Optimal chaotic desynchronization for neural populations
,”
SIAM J. Appl. Dyn. Syst.
13
(
1
),
276
(
2014
).
20.
B.
Monga
and
J.
Moehlis
, “
Phase distribution control of a population of oscillators
,”
Physica D
398
,
115
129
(
2019
).
21.
Y.
Kato
,
A.
Zlotnik
,
J.-S.
Li
, and
H.
Nakao
, “
Optimization of periodic input waveforms for global entrainment of weakly forced limit-cycle oscillators
,”
Nonlinear Dyn.
105
(
3
),
2247
2263
(
2021
).
22.
A. F.
Villaverde
,
N.
Tsiantis
, and
J. R.
Banga
, “
Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models
,”
J. R. Soc. Interface
16
(
156
),
20190043
(
2019
).
23.
H.
Nakao
, “
Phase reduction approach to synchronisation of nonlinear oscillators
,”
Contemp. Phys.
57
(
2
),
188
214
(
2016
).
24.
A.
Pikovsky
,
M.
Rosenblum
, and
J.
Kurths
, Synchronization—A Universal Concept in Nonlinear Sciences, Cambridge Nonlinear Science Series Vol. 12 (Cambridge University Press, 2001).
25.
M. J.
Panaggio
,
M.-V.
Ciocanel
,
L.
Lazarus
,
C. M.
Topaz
, and
B.
Xu
, “
Model reconstruction from temporal data for coupled oscillator networks
,”
Chaos
29
(
10
),
103116
(
2019
).
26.
B.
Monga
and
J.
Moehlis
, “
Supervised learning algorithms for controlling underactuated dynamical systems
,”
Physica D
412
,
132621
(
2020
).
27.
T. D.
Matchen
and
J.
Moehlis
, “
Leveraging deep learning to control neural oscillators
,”
Biol. Cybern.
115
(
3
),
219
235
(
2021
).
28.
G.
Baggio
,
D. S.
Bassett
, and
F.
Pasqualetti
, “
Data-driven control of complex networks
,”
Nat. Commun.
12
(
1
),
1429
(
2021
).
29.
C.
De Persis
and
P.
Tesi
, “
Formulas for data-driven control: Stabilization, optimality, and robustness
,”
IEEE Trans. Autom. Control
65
(
3
),
909
924
(
2019
).
30.
V.
Nagaraj
,
A.
Lamperski
, and
T. I.
Netoff
, “
Seizure control in a computational model using a reinforcement learning stimulation paradigm
,”
Int. J. Neural Syst.
27
(
07
),
1750012
(
2017
).
31.
B.
Mitchell
and
L. R.
Petzold
, “
Control of neural systems at multiple scales using model-free, deep reinforcement learning
,”
Sci. Rep.
8
(
1
),
10721
(
2018
).
32.
Y.-C.
Yu
,
V.
Narayanan
,
S.
Ching
, and
J.-S.
Li
, “Learning to control neurons using aggregated measurements,” in 2020 American Control Conference (ACC) (IEEE, 2020), pp. 4028–4033.
33.
D.
Krylov
,
D. V.
Dylov
, and
M.
Rosenblum
, “
Reinforcement learning for suppression of collective activity in oscillatory ensembles
,”
Chaos
30
(
3
),
033126
(
2020
).
34.
A. L.
Hodgkin
and
A. F.
Huxley
, “
A quantitative description of membrane current and its application to conduction and excitation in nerve
,”
J. Physiol.
117
(
4
),
500
544
(
1952
).
35.
W.
Bomela
,
B.
Singhal
, and
J.-S.
Li
, “
Engineering spatiotemporal patterns: Information encoding, processing, and controllability in oscillator ensembles
,”
Biomed. Phys. Eng. Express
9
(
4
),
045033
(
2023
).
36.
I. Z.
Kiss
,
Y.
Zhai
, and
J. L.
Hudson
, “
Emerging coherence in a population of chemical oscillators
,”
Science
296
(
5573
),
1676
1678
(
2002
).
37.
N.
Mitrou
,
A.
Laurin
,
T.
Dick
, and
J.
Inskip
, “
A peak detection method for identifying phase in physiological signals
,”
Biomed. Signal Process. Control
31
,
452
462
(
2017
).
38.
Y.
Kuramoto
, “Chemical turbulence,” in Chemical Oscillations, Waves, and Turbulence (Springer, 1984), pp. 111–140.
39.
S. H.
Strogatz
, “
From Kuramoto to Crawford: Exploring the onset of synchronization in populations of coupled oscillators
,”
Physica D
143
(
1–4
),
1
20
(
2000
).
40.
M. G.
Rosenblum
and
A. S.
Pikovsky
, “
Controlling synchronization in an ensemble of globally coupled oscillators
,”
Phys. Rev. Lett.
92
(
11
),
114102
(
2004
).
41.
A. V.
Oppenheim
,
A. S.
Willsky
, and
S. H.
Nawab
,
Signals & Systems
(
Prentice-Hall, Inc.
,
1996
).
42.
W.
Gerstner
,
W. M.
Kistler
,
R.
Naud
, and
L.
Paninski
,
Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition
(
Cambridge University Press
,
2014
).
43.
K.
Tsumoto
,
H.
Kitajima
,
T.
Yoshinaga
,
K.
Aihara
, and
H.
Kawakami
, “
Bifurcations in Morris–Lecar neuron model
,”
Neurocomputing
69
(
4–6
),
293
316
(
2006
).
44.
H. A.
Mallot
,
Computational Neuroscience
(
Springer
,
2013
), Vol. 486.
45.
E.
Nijholt
,
J. L.
Ocampo-Espindola
,
D.
Eroglu
,
I. Z.
Kiss
, and
T.
Pereira
, “
Emergent hypernetworks in weakly coupled oscillators
,”
Nat. Commun.
13
(
1
),
4849
(
2022
).
46.
L. B.
Wood
,
A.
Das
, and
H. H.
Asada
, “Broadcast feedback control of cell populations using stochastic Lyapunov functions with application to angiogenesis regulation,” in 2008 American Control Conference (IEEE, 2008), pp. 2105–2111.
47.
D.
Wilson
and
J.
Moehlis
, “
Determining individual phase response curves from aggregate population data
,”
Phys. Rev. E
92
(
2
),
022902
(
2015
).
48.
K.
Kuritz
,
S.
Zeng
, and
F.
Allgöwer
, “
Ensemble controllability of cellular oscillators
,”
IEEE Control Syst. Lett.
3
(
2
),
296
301
(
2018
).
49.
Y.
Kuramoto
, “Self-entrainment of a population of coupled non-linear oscillators,” in International Symposium on Mathematical Problems in Theoretical Physics, Kyoto University, Kyoto, Japan, 23–29 January 1975 (Springer, 1975), pp. 420–422.
50.
Y.
Wang
,
D. T.
Chik
, and
Z.
Wang
, “
Coherence resonance and noise-induced synchronization in globally coupled Hodgkin-Huxley neurons
,”
Phys. Rev. E
61
(
1
),
740
(
2000
).
51.
F.
Dörfler
and
F.
Bullo
, “
Synchronization in complex networks of phase oscillators: A survey
,”
Automatica
50
(
6
),
1539
1564
(
2014
).
52.
D. J.
Watts
and
S. H.
Strogatz
, “
Collective dynamics of “Small-World” networks
,”
Nature
393
(
6684
),
440
442
(
1998
).
53.
V.
Mnih
,
K.
Kavukcuoglu
,
D.
Silver
,
A. A.
Rusu
,
J.
Veness
,
M. G.
Bellemare
,
A.
Graves
,
M.
Riedmiller
,
A. K.
Fidjeland
,
G.
Ostrovski
, and
S.
Petersen
, “
Human-level control through deep reinforcement learning
,”
Nature
518
(
7540
),
529
533
(
2015
).
54.
J.
Schulman
,
F.
Wolski
,
P.
Dhariwal
,
A.
Radford
, and
O.
Klimov
, “Proximal policy optimization algorithms,” arXiv:1707.06347 (2017).
55.
T. M. Inc.
, “Reinforcement learning toolbox version: 23.2 (r2023b)” (2023).
56.
A.
Zlotnik
,
R.
Nagao
,
I. Z.
Kiss
, and
J.-S.
Li
, “
Phase-selective entrainment of nonlinear oscillator ensembles
,”
Nat. Commun.
7
(
1
),
10788
(
2016
).
57.
A. M.
Kuncel
and
W. M.
Grill
, “
Selection of stimulus parameters for deep brain stimulation
,”
Clin. Neurophysiol.
115
(
11
),
2431
2441
(
2004
).
58.
C. R.
Noback
,
D. A.
Ruggiero
,
N. L.
Strominger
, and
R. J.
Demarest
,
The Human Nervous System: Structure and Function
(
Springer Science & Business Media
,
2005
). No. 744.
59.
N. W.
Schultheiss
,
A. A.
Prinz
, and
R. J.
Butera
,
Phase Response Curves in Neuroscience: Theory, Experiment, and Analysis
(
Springer Science & Business Media
,
2011
).
60.
J. A. E.
Andersson
,
J.
Gillis
,
G.
Horn
,
J. B.
Rawlings
, and
M.
Diehl
, “
CasADi—A software framework for nonlinear optimization and optimal control
,”
Math. Program. Comput.
11
(
1
),
1
36
(
2019
).
61.
T. M.
Apostol
, Mathematical analysis,” in Addison-Wesley Series in Mathematics, 2nd ed. (Pearson, 1974).
62.
E.
Kamen
, “Fundamentals of linear time-varying systems,” in The Control Handbook (Second Edition): Control System Advanced Methods (CRC Press, 2010), pp. 3.1–3.33.

Supplementary Material

You do not currently have access to this content.