Empirically derived continuum models of collective behavior among large populations of dynamic agents are a subject of intense study in several fields, including biology, engineering, and finance. We formulate and study a mean-field game model whose behavior mimics an empirically derived nonlocal homogeneous flocking model for agents with gradient self-propulsion dynamics. The mean-field game framework provides a non-cooperative optimal control description of the behavior of a population of agents in a distributed setting. In this description, each agent's state is driven by optimally controlled dynamics that result in a Nash equilibrium between itself and the population. The optimal control is computed by minimizing a cost that depends only on its own state and a mean-field term. The agent distribution in phase space evolves under the optimal feedback control policy. We exploit the low-rank perturbative nature of the nonlocal term in the forward-backward system of equations governing the state and control distributions and provide a closed-loop linear stability analysis demonstrating that our model exhibits bifurcations similar to those found in the empirical model. The present work is a step towards developing a set of tools for systematic analysis, and eventually design, of collective behavior of non-cooperative dynamic agents via an inverse modeling approach.

1.
H.
Resat
,
L.
Petzold
, and
M. F.
Pettigrew
,
Kinetic Modeling of Biological Systems
(
Springer
,
2009
), pp.
311
335
.
2.
G.
Beni
, “
From swarm intelligence to swarm robotics
,” in
International Workshop on Swarm Robotics
(
Springer
,
2004
), pp.
1
9
.
3.
E.
Bonabeau
, “
Agent-based modeling: Methods and techniques for simulating human systems
,”
Proc. Natl. Acad. Sci. U. S. A.
99
,
7280
7287
(
2002
).
4.
G.
Whitham
, “
On kinematic waves ii. A theory of traffic flow on long crowded roads
,”
Proc. R. Soc. London, Ser. A
229
,
317
345
(
1955
).
5.
T.
Stankovski
,
T.
Pereira
,
P. V.
McClintock
, and
A.
Stefanovska
, “
Coupling functions: Universal insights into dynamical interaction mechanisms
,”
Rev. Mod. Phys.
89
,
045001
(
2017
).
6.
M.
Nourian
,
P. E.
Caines
,
R. P.
Malhame
, and
M.
Huang
, “
Nash, social and centralized solutions to consensus problems via mean field control theory
,”
IEEE Trans. Autom. Control
58
,
639
653
(
2013
).
7.
K.
Elamvazhuthi
and
P.
Grover
, “
Optimal transport over nonlinear systems via infinitesimal generators on graphs
,” preprint arXiv:1612.01193 (
2016
).
8.
P. E.
Caines
, “
Mean field games
,” in
Encyclopedia of Systems and Control
(
Springer
,
2015
), pp.
706
712
.
9.
J.-M.
Lasry
and
P.-L.
Lions
, “
Mean field games
,”
Jpn. J. Math.
2
,
229
260
(
2007
).
10.
M.
Huang
,
P. E.
Caines
, and
R. P.
Malhamé
, “
Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized ϵ-Nash equilibria
,”
IEEE Trans. Autom. Control
52
,
1560
1571
(
2007
).
11.
F.
Cucker
and
S.
Smale
, “
Emergent behavior in flocks
,”
IEEE Trans. Autom. Control
52
,
852
862
(
2007
).
12.
S.-Y.
Ha
,
J.-G.
Liu
 et al, “
A simple proof of the cucker-smale flocking dynamics and mean-field limit
,”
Commun. Math. Sci.
7
,
297
325
(
2009
).
13.
Y.-P.
Choi
,
S.-Y.
Ha
, and
Z.
Li
, “
Emergent dynamics of the Cucker–Smale flocking model and its variants
,” in
Active Particles
(
Springer
,
2017
), Vol.
1
, pp.
299
331
.
14.
A. B.
Barbaro
,
J. A.
Cañizo
,
J. A.
Carrillo
, and
P.
Degond
, “
Phase transitions in a kinetic flocking model of Cucker–Smale type
,”
Multiscale Model. Simul.
14
,
1063
1088
(
2016
).
15.
Y.
Kuramoto
, “
Self-entrainment of a population of coupled non-linear oscillators
,” in
International Symposium on Mathematical Problems in Theoretical Physics
(
Springer
,
1975
), pp.
420
422
.
16.
H.
Yin
,
P. G.
Mehta
,
S. P.
Meyn
, and
U. V.
Shanbhag
, “
Synchronization of coupled oscillators is a game
,”
IEEE Trans. Automat. Control
57
,
920
935
(
2012
).
17.
M.
Nourian
,
P. E.
Caines
, and
R. P.
Malhamé
, “
Mean field analysis of controlled cucker-smale type flocking: Linear analysis and perturbation equations
,”
IFAC Proc. Vol.
44
,
4471
4476
(
2011
).
18.
M.
Nourian
,
P. E.
Caines
, and
R. P.
Malhame
, “
A mean field game synthesis of initial mean consensus problems: A continuum approach for non-Gaussian behavior
,”
IEEE Trans. Automat. Control
59
,
449
455
(
2014
).
19.
P.
Degond
,
J.-G.
Liu
, and
C.
Ringhofer
, “
Large-scale dynamics of mean-field games driven by local nash equilibria
,”
J. Nonlinear Sci.
24
,
93
115
(
2014
).
20.
O.
Guéant
, “
A reference case for mean field games models
,”
J. Math. Pures Appl.
92
,
276
294
(
2009
).
21.
T. J.
Anastasio
,
A. K.
Barreiro
, and
J. C.
Bronski
, “
A geometric method for eigenvalue problems with low-rank perturbations
,”
R. Soc. Open Sci.
4
,
170390
(
2017
).
22.
S.
Wiggins
,
Introduction to Applied Nonlinear Dynamical Systems and Chaos
, Texts in Applied Mathematics Science, Vol.
2
(
Springer-Verlag
,
Berlin
,
1990
).
23.
J.
Tugaut
, “
Phase transitions of McKean–Vlasov processes in double-wells landscape
,”
Stochastics
86
,
257
284
(
2014
).
24.
G. A.
Pavliotis
,
Stochastic Processes and Applications: Diffusion Processes, the Fokker-Planck and Langevin Equations
(
Springer
,
2014
), Vol.
60
.
25.
P.
Freitas
, “
Stability of stationary solutions for a scalar non-local reaction-diffusion equation
,”
Q. J. Mech. Appl. Math.
48
,
557
582
(
1995
).
26.
T.
Kapitula
and
K.
Promislow
, “
Stability indices for constrained self-adjoint operators
,”
Proc. Am. Math. Soc.
140
,
865
880
(
2012
).
27.
T. A.
Driscoll
,
N.
Hale
, and
L. N.
Trefethen
, “
Chebfun guide
” (
2014
).
28.
V. S.
Borkar
, “
Ergodic control of diffusion processes
,” in
Proceedings ICM
(
2006
).
29.
E.
Todorov
, “
Eigenfunction approximation methods for linearly-solvable optimal control problems
,” in
IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning
(IEEE,
2009
), pp.
161
168
.
30.
J.
Alexander
,
R.
Gardner
, and
C.
Jones
, “
A topological invariant arising in the stability analysis of travelling waves
,”
J. Reine Angew. Math.
410
,
143
(
1990
).
31.
P.
Cardaliaguet
and
S.
Hadikhanloo
, “
Learning in mean field games: The fictitious play
,”
ESAIM: Control, Optim. Calculus Var.
23
,
569
591
(
2017
).
32.
N. E.
Leonard
, “
Multi-agent system dynamics: Bifurcation and behavior of animal groups
,”
Annu. Rev. Control
38
,
171
183
(
2014
).
33.
V.
Srivastava
and
N. E.
Leonard
, “
Bio-inspired decision-making and control: From honeybees and neurons to network design
,” in
American Control Conference (ACC)
(IEEE,
2017
), pp.
2026
2039
.
34.
R.
Gray
,
A.
Franci
,
V.
Srivastava
, and
N. E.
Leonard
, “
Multi-agent decision-making dynamics inspired by honeybees
,”
IEEE Transactions on Control of Network Systems
5
(
2
),
793
806
(
2018
).
You do not currently have access to this content.