We present a numerical algorithm that allows the approximation of optimal controls for stochastic reaction–diffusion equations with additive noise by first reducing the problem to controls of feedback form and then approximating the feedback function using finitely based approximations. Using structural assumptions on the finitely based approximations, rates for the approximation error of the cost can be obtained. Our algorithm significantly reduces the computational complexity of finding controls with asymptotically optimal cost. Numerical experiments using artificial neural networks as well as radial basis function networks illustrate the performance of our algorithm. Our approach can also be applied to stochastic control problems for high dimensional stochastic differential equations and more general stochastic partial differential equations.
REFERENCES
1.
R. E.
Showalter
, Monotone Operators in Banach Space and Nonlinear Partial Differential Equations
(American Mathematical Society
, Providence, RI
, 1997
).2.
W.
Liu
and M.
Röckner
, Stochastic Partial Differential Equations: An Introduction
(Springer
, 2015
).3.
M.
Hinze
, R.
Pinnau
, M.
Ulbrich
, and S.
Ulbrich
, Optimization with PDE Constraints
(Springer
, 2009
).4.
X.
Li
and J.
Yong
, Optimal Control Theory for Infinite Dimensional Systems
(Birkhäuser
, Boston, MA
, 1995
).5.
F.
Tröltzsch
, Optimal Control of Partial Differential Equations
(American Mathematical Society
, Providence, RI
, 2010
).6.
R.
Buchholz
, H.
Engel
, E.
Kammann
, and F.
Tröltzsch
, “On the optimal control of the Schlögl-model
,” Comput. Optim. Appl.
56
, 153
–185
(2013
). 7.
R.
Buchholz
, H.
Engel
, E.
Kammann
, and F.
Tröltzsch
, “Erratum to: On the optimal control of the Schlögl-model
,” Comput. Optim. Appl.
56
, 187
–188
(2013
). 8.
C.
Ryll
, “Optimal control of patterns in some reaction-diffusion-systems,” Doctoral thesis (Technische Universität Berlin, Berlin, 2017).9.
C.
Ryll
, J.
Löber
, S.
Martens
, H.
Engel
, and F.
Tröltzsch
, “Analytical, optimal, and sparse optimal control of traveling wave solutions to reaction-diffusion systems,” in Control of Self-Organizing Nonlinear Systems, edited by E. Schöll, S. H. L. Klapp, and P. Hövel (Springer, 2016), pp. 189–210.10.
F.
Cordoni
and L.
Di Persio
, “Optimal control for the stochastic FitzHugh-Nagumo model with recovery variable
,” Evol. Eq. Control Theory
7
, 571
–585
(2018
). 11.
K.
Du
and Q.
Meng
, “A maximum principle for optimal control of stochastic evolution equations
,” SIAM J. Control Optim.
51
, 4343
–4362
(2013
). 12.
G.
Fabbri
, F.
Gozzi
, and A.
Święch
, Stochastic Optimal Control in Infinite Dimension
(Springer
, 2017
).13.
H.
Frankowska
and X.
Zhang
, “Necessary conditions for stochastic optimal control problems in infinite dimensions
,” Stoch. Process. Appl.
130
, 4081
–4103
(2020
). 14.
M.
Fuhrman
, Y.
Hu
, and G.
Tessitore
, “Stochastic maximum principle for optimal control of SPDEs
,” Appl. Math. Optim.
68
, 181
–217
(2013
). 15.
M.
Fuhrman
, Y.
Hu
, and G.
Tessitore
, “Stochastic maximum principle for optimal control of partial differential equations driven by white noise
,” Stoch. Partial Differ. Equ. Anal. Comput.
6
, 255
–285
(2018
). 16.
M.
Fuhrman
and C.
Orrieri
, “Stochastic maximum principle for optimal control of a class of nonlinear SPDEs with dissipative drift
,” SIAM J. Control Optim.
54
, 341
–371
(2016
). 17.
Q.
Lü
and X.
Zhang
, General Pontryagin-Type Stochastic Maximum Principle and Backward Stochastic Evolution Equations in Infinite Dimensions
(Springer
, 2014
).18.
Q.
Lü
and X.
Zhang
, “Transposition method for backward stochastic evolution equations revisited, and its application
,” Math. Control Relat. Fields
5
, 529
–555
(2015
). 19.
Q.
Lü
and X.
Zhang
, “Operator-valued backward stochastic Lyapunov equations in infinite dimensions, and its application
,” Math. Control Relat. Fields
8
, 337
–381
(2018
). 20.
W.
Stannat
and L.
Wessels
, “Peng’s maximum principle for stochastic partial differential equations
,” SIAM J. Control Optim.
59
, 3552
–3573
(2021
). 21.
W.
Stannat
and L.
Wessels
, “Necessary and sufficient conditions for optimal control of semilinear stochastic partial differential equations,” arXiv:2112.09639 (2022).22.
L.
Wessels
, “Optimal control of stochastic reaction-diffusion equations,” Doctoral thesis (Technische Universität Berlin, Berlin, 2022).23.
C.
Beck
, S.
Becker
, P.
Cheridito
, A.
Jentzen
, and A.
Neufeld
, “Deep splitting method for parabolic PDEs
,” SIAM J. Sci. Comput.
43
, A3135
–A3154
(2021
). 24.
C.
Beck
and W. E.
A. Jentzen
, “Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
,” J. Nonlinear Sci.
29
, 1563
–1619
(2019
). 25.
S.
Dolgov
, D.
Kalise
, and K. K.
Kunisch
, “Tensor decomposition methods for high-dimensional Hamilton-Jacobi-Bellman equations
,” SIAM J. Sci. Comput.
43
, A1625
–A1650
(2021
). 26.
T.
Dunst
, A. K.
Majee
, A.
Prohl
, and G.
Vallet
, “On stochastic optimal control in ferromagnetism
,” Arch. Ration. Mech. Anal.
233
, 1383
–1440
(2019
). 27.
T.
Dunst
and A.
Prohl
, “The forward-backward stochastic heat equation: Numerical analysis and simulation
,” SIAM J. Sci. Comput.
38
, A2725
–A2755
(2016
). 28.
W. E. J.
Han
and A.
Jentzen
, “Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
,” Commun. Math. Stat.
5
, 349
–380
(2017
). 29.
A.
Gorodetsky
, S.
Karaman
, and Y.
Marzouk
, “High-dimensional stochastic optimal control using continuous tensor decompositions
,” Int. J. Robot. Res.
37
, 340
–377
(2018
). 30.
D.
Kalise
and K.
Kunisch
, “Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs
,” SIAM J. Sci. Comput.
40
, A629
–A652
(2018
). 31.
N.
Nüsken
and L.
Richter
, “Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: Perspectives from the theory of controlled diffusions and measures on path space
,” Partial Differ. Equ. Appl.
2
, 48
(2021
). 32.
M.
Oster
, L.
Sallandt
, and R.
Schneider
, “Approximating optimal feedback controllers of finite horizon control problems using hierarchical tensor formats
,” SIAM J. Sci. Comput.
44
, B746
–B770
(2022
). 33.
L.
Richter
, L.
Sallandt
, and N.
Nüsken
, “Solving high-dimensional parabolic PDEs using the tensor train format,” arXiv:2102.11830 (2021).34.
J.
Sirignano
and K.
Spiliopoulos
, “DGM: A deep learning algorithm for solving partial differential equations
,” J. Comput. Phys.
375
, 1339
–1364
(2018
). 35.
W.
Stannat
and L.
Wessels
, “Deterministic control of stochastic reaction-diffusion equations
,” Evol. Equ. Control Theory
10
, 701
–722
(2021
). 36.
W.
Stannat
and A.
Vogler
, “Approximation of optimal feedback controls for stochastic reaction-diffusion equations,” arXiv:submit/4710581 (2023).37.
C.
Tudor
, “Quadratic control for stochastic systems defined by evolution operators and square integrable martingales
,” Math. Nachr.
147
, 205
–218
(1990
). 38.
A.
Pinkus
, “Approximation theory of the MLP model in neural networks
,” Acta Numer.
8
, 143
–195
(1999
).39.
R.
Carmona
and M.
Laurière
, “Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games: II—The finite horizon case
,” Ann. Appl. Probab.
32
, 4065
–4105
(2022
). 40.
© 2023 Author(s). Published under an exclusive license by AIP Publishing.
2023
Author(s)
You do not currently have access to this content.