We propose an adaptive importance sampling scheme for the simulation of rare events when the underlying dynamics is given by diffusion. The scheme is based on a Gibbs variational principle that is used to determine the optimal (i.e., zero-variance) change of measure and exploits the fact that the latter can be rephrased as a stochastic optimal control problem. The control problem can be solved by a stochastic approximation algorithm, using the Feynman–Kac representation of the associated dynamic programming equations, and we discuss numerical aspects for high-dimensional problems along with simple toy examples.
REFERENCES
1.
A.
Bachouch
, C.
Huré
, N.
Langrené
, and H.
Pham
, “Deep neural networks algorithms for stochastic control problems on finite horizon, part 2: Numerical applications,” e-print arXiv:1812.05916 (2018).2.
C.
Bender
and R.
Denk
, “A forward scheme for backward SDEs
,” Stoch. Proc. Appl.
117
(12
), 1793
–1812
(2007
). 3.
C.
Bender
and T.
Moseler
, “Importance sampling for backward SDEs
,” Stoch. Anal. Appl.
28
(2
), 226
–253
(2010
). 4.
C.
Bender
and J.
Steiner
, “Least-squares Monte Carlo for backward SDE,” in Numerical Methods in Finance, edited by R. A. Carmona, P. D. Moral, P. Hu, and N. Oudjane (Springer, Berlin, 2012), pp. 257–289.5.
M.
Boué
and P.
Dupuis
, “A variational representation for certain functionals of Brownian motion
,” Ann. Probab.
26
(4
), 1641
–1659
(1998
). 6.
F.
Cérou
, P.
Del Moral
, T.
Furon
, and A.
Guyader
, “Sequential Monte Carlo for rare event estimation
,” Stat. Comput.
22
(3
), 795
–808
(2012
). 7.
J.
Comer
, J. C.
Gumbart
, J.
Hénin
, T.
Lelièvre
, A.
Pohorille
, and C.
Chipot
, “The adaptive biasing force method: Everything you always wanted to know but were afraid to ask
,” J. Phys. Chem. B
119
(3
), 1129
–1151
(2015
). 8.
F.
Cérou
and A.
Guyader
, “Adaptive multilevel splitting for rare event analysis
,” Stoch. Anal. Appl.
25
(2
), 417
–443
(2007
). 9.
P.
Dai Pra
, L.
Meneghini
, and W.
Runggaldier
, “Connections between stochastic control and dynamic games
,” Math. Control Signals Syst.
9
, 303
–326
(1996
). 10.
F.
Delbaen
, Y.
Hu
, and A.
Richou
, “On the uniqueness of solutions to quadratic BSDEs with convex generators and unbounded terminal conditions
,” Ann. Inst. Henri Poincaré Probab. Stat.
47
(2
), 559
–574
(2011
). 11.
P.
Dupuis
and H.
Wang
, “Importance sampling, large deviations, and differential games
,” Stochastics
76
(6
), 481
–508
(2004
). 12.
P.
Dupuis
and H.
Wang
, “Subsolutions of an Isaacs equation and efficient schemes for importance sampling
,” Math. Oper. Res.
32
(3
), 723
–757
(2007
). 13.
W.
E
, J.
Han
, and A.
Jentzen
, “Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
,” Commun. Math. Stat.
5
(4
), 349
–380
(2017
). 14.
R. S.
Ellis
, Entropy, Large Deviations and Statistical Mechanics
(Springer
, Berlin
, 1985
).15.
I.
Exarchos
and E. A.
Theodorou
, “Stochastic optimal control via forward and backward stochastic differential equations and importance sampling
,” Automatica
87
, 159
–165
(2018
). 16.
W. H.
Fleming
, “Exit probabilities and optimal stochastic control
,” Appl. Math. Optim.
4
, 329
–346
(1977
). 17.
W. H.
Fleming
and W. M.
McEneaney
, “Risk-sensitive control on an infinite time horizon
,” SIAM J. Control Optim.
33
, 1881
–1915
(1995
). 18.
W. H.
Fleming
and S.-J.
Sheu
, “Asymptotics for the principal eigenvalue and eigenfunction of a nearly first-order operator with large potential
,” Ann. Probab.
25
, 1953
–1994
(1997
). 19.
W. H.
Fleming
and H. M.
Soner
, Controlled Markov Processes and Viscosity Solutions
(Springer
, 2006
).20.
P.
Glasserman
and Y.
Wang
, “Counterexamples in importance sampling for large deviations probabilities
,” Ann. Appl. Probab.
7
, 731
–746
(1997
). 21.
E.
Gobet
and P.
Turkedjiev
, “Linear regression MDP scheme for discrete backward stochastic differential equations under general conditions
,” Math. Comput.
85
(299
), 1359
–1391
(2016
). 22.
E.
Gobet
, J.-P.
Lemor
, and X.
Warin
, “A regression-based Monte Carlo method to solve backward stochastic differential equations
,” Ann. Appl. Probab.
15
(3
), 2172
–2202
(2005
). 23.
C.
Hartmann
and C.
Schütte
, “Efficient rare event simulation by optimal nonequilibrium forcing
,” J. Stat. Mech. Theor. Exp.
2012
, P11004
(2012
). 24.
C.
Hartmann
, L.
Richter
, C.
Schütte
, and W.
Zhang
, “Variational characterization of free energy: Theory and algorithms
,” Entropy
19
(11
), 626
(2017
). 25.
C.
Huré
, H.
Pham
, A.
Bachouch
, and N.
Langrené
, “Deep neural networks algorithms for stochastic control problems on finite horizon, part I: Convergence analysis,” e-print arXiv:1812.04300 (2018).26.
N.
Ikeda
and S.
Watanabe
, Stochastic Differential Equations and Diffusion Processes
(Kodansha/Elsevier
, North-Holland
, 1989
).27.
M.
James
, “Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games
,” Math. Control Signals Syst.
5
, 401
–417
(1992
). 28.
S.
Juneja
and P.
Shahabuddin
, “Rare-event simulation techniques: An introduction and recent advances,” in Handbooks in Operations Research and Management Science, edited by S. G. Henderson and B. L. Nelson (Elsevier, 2006), Vol. 13, pp. 291–350.29.
H. J.
Kappen
and H. C.
Ruiz
, “Adaptive importance sampling for control and inference
,” J. Stat. Phys.
162
(5
), 1244
–1266
(2016
). 30.
H. J.
Kappen
, V.
Gómez
, and M.
Opper
, “Optimal control as a graphical model inference problem
,” Mach. Learn.
87
(2
), 159
–182
(2012
). 31.
H. J.
Kappen
, “Path integrals and symmetry breaking for optimal control theory
,” J. Stat. Mech. Theor. Exp.
2005
(11
), P11011
(2005
). 32.
O.
Kebiri
, L.
Neureither
, and C.
Hartmann
, “Adaptive importance sampling with forward backward stochastic differential equations,” in Proceedings of the IHP Trimester Stochastic Dynamics Out of Equilibrium, Institut Henri Poincar, Paris, France, 2017 (Springer, Berlin, 2019).33.
D. P.
Kingma
and J. B.
Adam
, “A method for stochastic optimization,” in International Conference on Learning Representations (ICLR), May 7–9, 2015 (San Diego, 2015).34.
M.
Kobylanski
, “Backward stochastic differential equations and partial differential equations with quadratic growth
,” Ann. Probab.
28
(2
), 558
–602
(2000
). 35.
P.
L’Ecuyer
, M.
Mandjes
, and B.
Tuffin
, Importance Sampling in Rare Event Simulation (John Wiley & Sons, Ltd, 2009), Chap. 2, pp. 17–38.36.
B.
Øksendal
, Stochastic Differential Equations: An Introduction with Applications
(Springer
, Berlin
, 2003
).37.
O.
Papaspiliopoulos
and G.
Roberts
, “Importance sampling techniques for estimation of diffusion models,” in Statistical Methods for Stochastic Differential Equations (Chapman and Hall/CRC, 2012), pp. 329–357.38.
K.
Rawlik
, M.
Toussaint
, and S.
Vijayakumar
, “On stochastic optimal control and reinforcement learning by approximate inference,” in Proceedings of Robotics: Science and Systems Conference (R:SS ’12), July 9–13, 2012 (Sydney, Australia, 2012).39.
S.
Reich
, “Data assimilation—The Schrödinger perspective,” preprint arXiv:1807.08351 (2018).40.
C.
Schütte
, S.
Winkelmann
, and C.
Hartmann
, “Optimal control of molecular dynamics using Markov state models
,” Math. Program. Ser. B
134
, 259
–282
(2012
). 41.
E.
Todorov
, “Efficient computation of optimal actions
,” Proc. Natl. Acad. Sci. U.S.A.
106
(28
), 11478
–11483
(2009
). 42.
P.
Turkedjiev
, “Numerical methods for backward stochastic differential equations of quadratic and locally Lipschitz type,”
Ph.D. thesis (Humboldt-Universität zu Berlin, Mathematisch-Naturwissenschaftliche Fakultät II
, 2013
) .43.
E.
Vanden-Eijnden
and J.
Weare
, “Rare event simulation of small noise diffusions
,” Commun. Pure Appl. Math.
65
(12
), 1770
–1803
(2012
). 44.
M.
Villén-Altamirano
and J.
Villén-Altamirano
, “Restart: A straightforward method for fast simulation of rare events,” in Proceedings of the 26th Conference on Winter Simulation, WSC ’94 (Society for Computer Simulation International, San Diego, CA, 1994), pp. 282–289.45.
P.
Whittle
, “Risk-sensitivity large deviations and stochastic control
,” Eur. J. Oper. Res.
73
, 295
–303
(1994
). 46.
P.
Whittle
, “Risk-sensitivity a strangely pervasive concept
,” Macroecon. Dyn.
6
, 5
–18
(2002
). © 2019 Author(s).
2019
Author(s)
You do not currently have access to this content.
