This study proposes a novel method for developing discretization-consistent closure schemes for implicitly filtered large eddy simulation (LES). Here, the induced filter kernel and, thus, the closure terms are determined by the properties of the grid and the discretization operator, leading to additional computational subgrid terms that are generally unknown in a priori analysis. In this work, the task of adapting the coefficients of LES closure models is thus framed as a Markov decision process and solved in an a posteriori manner with reinforcement learning (RL). This optimization framework is applied to both explicit and implicit closure models. The explicit model is based on an element-local eddy viscosity model. The optimized model is found to adapt its induced viscosity within discontinuous Galerkin (DG) methods to homogenize the dissipation within an element by adding more viscosity near its center. For the implicit modeling, RL is applied to identify an optimal blending strategy for a hybrid DG and finite volume (FV) scheme. The resulting optimized discretization yields more accurate results in LES than either the pure DG or FV method and renders itself as a viable modeling ansatz that could initiate a novel class of high-order schemes for compressible turbulence by combining turbulence modeling with shock capturing in a single framework. All newly derived models achieve accurate results that either match or outperform traditional models for different discretizations and resolutions. Overall, the results demonstrate that the proposed RL optimization can provide discretization-consistent closures that could reduce the uncertainty in implicitly filtered LES.
REFERENCES
1.
F. K.
Chow
and
P.
Moin
, “
A further study of numerical errors in large-eddy simulations
,” J. Comput. Phys.
184
, 366
(2003
).2.
B. J.
Geurts
and
F.
van der Bos
, “
Numerically induced high-pass dynamics in large-eddy simulation
,” Phys. Fluids
17
, 125103
(2005
).3.
C. D.
Pruett
, “
Toward the de-mystification of LES
,” in DNS/LES Progress and Challenges. Proceedings of the Third AFOSR International Conference on DNS/LES, edited by
C.
Liu
, L. Sakell, T. and Beutner (Greyden
, Columbus, OH
, 2001
), pp. 231
–238
4.
B. J.
Geurts
and
D. D.
Holm
, “
Commutator errors in large-eddy simulation
,” J. Phys. A: Math. Gen.
39
, 2213
(2006
).5.
L. C.
Berselli
,
C. R.
Grisanti
, and
V.
John
, “
Analysis of commutation errors for functions with low regularity
,” J. Comput. Appl. Math.
206
, 1027
(2007
).6.
C.
Fureby
and
G.
Tabor
, “
Mathematical and physical constraints on large-eddy simulations
,” Theor. Comput. Fluid Dyn.
9
, 85
(1997
).7.
S.
Ghosal
and
P.
Moin
, “
The basic equations for the large eddy simulation of turbulent flows in complex geometry
,” J. Comput. Phys.
118
, 24
(1995
).8.
T.
Lund
, “
The use of explicit filters in large eddy simulation
,” Comput. Math. Appl.
46
, 603
(2003
).9.
V.
John
, Large Eddy Simulation of Turbulent Incompressible Flows: Analytical and Numerical Results for a Class of LES Models
(
Springer Science & Business Media
, 2003
), Vol.
34
.10.
G. R.
Yalla
,
T. A.
Oliver
,
S. W.
Haering
,
B.
Engquist
, and
R. D.
Moser
, “
Effects of resolution inhomogeneity in large-eddy simulation
,” Phys. Rev. Fluids
6
, 074604
(2021
).11.
A.
Beck
,
D.
Flad
, and
C.-D.
Munz
, “
Deep neural networks for data-driven LES closure models
,” J. Comput. Phys.
398
, 108910
(2019
).12.
A similar argument can be arrived at when considering the integral form of the equations and a finite volume discretization. Here, the divergence operator is rewritten as an integral surface flux using Gauss's divergence theorem. While the numerical integration operator and thus projection onto the mean remain exact even in a discrete setting, the integrand can become a nonlinear approximation. A typical example would be (W)ENO-type schemes. Thus, the
errors also appear—not because the integration operator in finite volume is not a box-filter with known kernel, but because its arguments have been approximated.
13.
U.
Piomelli
,
P.
Moin
, and
J. H.
Ferziger
, “
Model consistency in large eddy simulation of turbulent channel flows
,” Phys. Fluids
31
, 1884
(1988
).14.
C.
Meneveau
and
J.
Katz
, “
Scale-invariance and turbulence models for large-eddy simulation
,” Annu. Rev. Fluid Mech.
32
, 1
(2000
).15.
O. V.
Vasilyev
and
D. E.
Goldstein
, “
Local spectrum of commutation error in large eddy simulations
,” Phys. Fluids
16
, 470
(2004
).16.
A. D.
Beck
,
D. G.
Flad
,
C.
Tonhäuser
,
G.
Gassner
, and
C.-D.
Munz
, “
On the influence of polynomial de-aliasing on subgrid scale models
,” Flow, Turbul. Combust.
97
, 475
(2016
).17.
M.
Kurz
and
A.
Beck
, “
Investigating model-data inconsistency in data-informed turbulence closure terms
,” in Proceedings of 14th WCCM-ECCOMAS Congress 2020
(2021
), Vol. 1700
.18.
M.
Kurz
,
P.
Offenhäuser
,
D.
Viola
,
M.
Resch
, and
A.
Beck
, “
Relexi—A scalable open source reinforcement learning framework for high-performance computing
,” Software Impacts
14
, 100422
(2022
).19.
M.
Kurz
,
P.
Offenhäuser
, and
A.
Beck
, “
Deep reinforcement learning for turbulence modeling in large eddy simulations
,” Int. J. Heat Fluid Flow
99
, 109094
(2023
).20.
J.
Meyers
,
P.
Sagaut
, and
B. J.
Geurts
, “
Optimal model parameters for multi-objective large-eddy simulations
,” Phys. Fluids
18
, 095103
(2006
).21.
G.
Matheou
, “
Numerical discretization and subgrid-scale model effects on large-eddy simulations of a stable boundary layer
,” Q. J. R. Meteorol. Soc.
142
, 3050
(2016
).22.
A.
Viré
,
D.
Krasnov
,
T.
Boeck
, and
B.
Knaepen
, “
Modeling and discretization errors in large eddy simulations of hydrodynamic and magnetohydrodynamic channel flows
,” J. Comput. Phys.
230
, 1903
(2011
).23.
N. A.
Adams
,
S.
Hickel
,
T.
Kempe
, and
J. A.
Domaradzki
, “
On the relation between subgrid-scale modeling and numerical discretization in large-eddy simulation
,” in Complex Effects in Large Eddy Simulations
, edited by
S. C.
Kassinos
,
C. A.
Langer
,
G.
Iaccarino
, and
P.
Moin
(
Springer Berlin Heidelberg
,
Berlin, Heidelberg
, 2007
), pp. 15
–27
.24.
D.
Flad
and
G.
Gassner
, “
On the use of kinetic energy preserving DG-schemes for large eddy simulation
,” J. Comput. Phys.
350
, 782
(2017
).25.
S.
Hickel
,
N. A.
Adams
, and
J. A.
Domaradzki
, “
An adaptive local deconvolution method for implicit LES
,” J. Comput. Phys.
213
, 413
(2006
).26.
F. S.
Schranner
,
V.
Rozov
, and
N.
Adams
, “
Optimization of an implicit LES method for underresolved simulations of incompressible flows
,” AIAA Paper No. 2016-0338, 2016
.27.
D.
Flad
,
A.
Beck
, and
P.
Guthke
, “
A large eddy simulation method for DGSEM using non-linearly optimized relaxation filters
,” J. Comput. Phys.
408
, 109303
(2020
).28.
R.
Maulik
,
O.
San
, and
J. D.
Jacob
, “
Spatiotemporally dynamic implicit large eddy simulation using machine learning classifiers
,” Phys. D
406
, 132409
(2020
).29.
J.
Rabault
,
M.
Kuchta
,
A.
Jensen
,
U.
Réglade
, and
N.
Cerardi
, “
Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control
,” J. Fluid Mech.
865
, 281
(2019
).30.
J.
Rabault
and
A.
Kuhnle
, “
Accelerating deep reinforcement learning strategies of flow control through a multi-environment approach
,” Phys. Fluids
31
, 094105
(2019
).31.
H.
Tang
,
J.
Rabault
,
A.
Kuhnle
,
Y.
Wang
, and
T.
Wang
, “
Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning
,” Phys. Fluids
32
, 053605
(2020
).32.
D.
Fan
,
L.
Yang
,
Z.
Wang
,
M. S.
Triantafyllou
, and
G. E.
Karniadakis
, “
Reinforcement learning for bluff body active flow control in experiments and simulations
,” Proc. Natl. Acad. Sci. U. S. A.
117
, 26091
(2020
).33.
P.
Varela
,
P.
Suárez
,
F.
Alcántara-Ávila
,
A.
Miró
,
J.
Rabault
,
B.
Font
,
L. M.
García-Cuevas
,
O.
Lehmkuhl
, and
R.
Vinuesa
, “
Deep reinforcement learning for flow control exploits different physics for increasing Reynolds number regimes
,” Actuators
11
, 359
(2022
).34.
C.
Vignon
,
J.
Rabault
, and
R.
Vinuesa
, “
Recent advances in applying deep reinforcement learning for flow control: Perspectives and future directions
,” Phys. Fluids
35
, 031301
(2023
).35.
R.
Vinuesa
,
O.
Lehmkuhl
,
A.
Lozano-Durán
, and
J.
Rabault
, “
Flow control in wings and discovery of novel approaches via deep reinforcement learning
,” Fluids
7
, 62
(2022
).36.
G.
Novati
,
H. L.
de Laroussilhe
, and
P.
Koumoutsakos
, “
Automating turbulence modelling by multi-agent reinforcement learning
,” Nat. Mach. Intell.
3
, 87
(2021
).37.
J.
Kim
,
H.
Kim
,
J.
Kim
, and
C.
Lee
, “
Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence
,” Phys. Fluids
34
, 105132
(2022
).38.
H. J.
Bae
and
P.
Koumoutsakos
, “
Scientific multi-agent reinforcement learning for wall-models of turbulent flows
,” Nat. Commun.
13
(1
), 1443
(2022
).39.
A.
Vadrot
,
X. I.
Yang
,
H. J.
Bae
, and
M.
Abkar
, “
Log-law recovery through reinforcement-learning wall model for large eddy simulation
,” Phys. Fluids
35
, 055122
(2023
).40.
Y.
Feng
,
F. S.
Schranner
,
J.
Winter
, and
N. A.
Adams
, “
A deep reinforcement learning framework for dynamic optimization of numerical schemes for compressible flow simulations
,” J. Comput. Phys.
493
, 112436
(2023
).41.
J.
Schulman
,
F.
Wolski
,
P.
Dhariwal
,
A.
Radford
, and
O.
Klimov
, “
Proximal policy optimization algorithms
,” arXiv:1707.06347 (2017
).42.
R. S.
Sutton
and
A. G.
Barto
, Reinforcement Learning: An Introduction
(
MIT Press
, 2018
).43.
S.
Notter
,
F.
Schimpf
, and
W.
Fichter
, “
Hierarchical reinforcement learning approach towards autonomous cross-country soaring
,” AIAA Paper No. 2021-2010, 2021
.44.
D. A.
Kopriva
, Implementing Spectral Methods for Partial Differential Equations: Algorithms for Scientists and Engineers
(
Springer Science & Business Media
, 2009
).45.
E.
Ferrer
,
G.
Rubio
,
G.
Ntoukas
,
W.
Laskowski
,
O.
Mariño
,
S.
Colombo
,
A.
Mateo-Gabín
,
H.
Marbona
,
F.
Manrique de Lara
,
D.
Huergo
,
J.
Manzanero
,
A.
Rueda-Ramírez
,
D.
Kopriva
,
E.
Valero
, and
1.
Image
, “
A high-order discontinuous Galerkin solver for flow simulations and multi-physics applications
,” Comput. Phys. Commun.
287
, 108700
(2023
).46.
F.
Witherden
,
A.
Farrington
, and
P.
Vincent
, “
PyFR: An open source framework for solving advection-diffusion type problems on streaming architectures using the flux reconstruction approach
,” Comput. Phys. Commun.
185
, 3028
(2014
).47.
N.
Krais
,
A.
Beck
,
T.
Bolemann
,
H.
Frank
,
D.
Flad
,
G.
Gassner
,
F.
Hindenlang
,
M.
Hoffmann
,
T.
Kuhn
,
M.
Sonntag
, and
C.-D.
Munz
, “
FLEXI: A high order discontinuous Galerkin framework for hyperbolic–parabolic conservation laws
,” Comput. Math. Appl.
81
, 186
(2021
).48.
G. J.
Gassner
,
A. R.
Winters
, and
D. A.
Kopriva
, “
Split form nodal discontinuous Galerkin schemes with summation-by-parts property for the compressible Euler equations
,” J. Comput. Phys.
327
, 39
(2016
).49.
S.
Pirozzoli
, “
Numerical methods for high-speed flows
,” Annu. Rev. Fluid Mech.
43
, 163
(2011
).50.
K.
Oßwald
,
A.
Siegmund
,
P.
Birken
,
V.
Hannemann
, and
A.
Meister
, “
L2Roe: A low dissipation version of Roe's approximate Riemann solver for low Mach numbers
,” Int. J. Numer. Methods Fluids
81
, 71
(2016
).51.
F.
Bassi
and
S.
Rebay
, “
A high-order accurate discontinuous finite element method for the numerical solution of the compressible Navier–Stokes equations
,” J. Comput. Phys.
131
, 267
(1997
).52.
G. J.
Gassner
and
A. D.
Beck
, “
On the accuracy of high-order discretizations for underresolved turbulence simulations
,” Theor. Comput. Fluid Dyn.
27
, 221
(2013
).53.
F.
Hindenlang
,
G. J.
Gassner
,
C.
Altmann
,
A.
Beck
,
M.
Staudenmaier
, and
C.-D.
Munz
, “
Explicit discontinuous Galerkin methods for unsteady problems
,” Comput. Fluids
61
, 86
(2012
).54.
K. S.
Klemmer
,
W.
Wu
, and
M. E.
Mueller
, “
Turbulence model form errors in separated flows
,” Phys. Rev. Fluids
8
, 024606
(2023
).55.
J. P.
Boris
,
F. F.
Grinstein
,
E. S.
Oran
, and
R. L.
Kolbe
, “
New insights into large eddy simulation
,” Fluid Dyn. Res.
10
, 199
(1992
).56.
A.
Uranga
,
P.-O.
Persson
,
M.
Drela
, and
J.
Peraire
, “
Implicit large eddy simulation of transition to turbulence at low Reynolds numbers using a discontinuous Galerkin method
,” Int. J. Numer. Methods Eng.
87
, 232
(2011
).57.
S.
Hennemann
,
A. M.
Rueda-Ramírez
,
F. J.
Hindenlang
, and
G. J.
Gassner
, “
A provably entropy stable subcell shock capturing approach for high order split form DG for the compressible Euler equations
,” J. Comput. Phys.
426
, 109935
(2021
).58.
A. M.
Rueda-Ramírez
and
G. J.
Gassner
, “
A subcell finite volume positivity-preserving limiter for DGSEM discretizations of the Euler equations
,” in Proceedings of 14th WCCM-ECCOMAS Congress 2020
, edited by
F.
Chinesta
,
R.
Abgrall
,
O.
Allix
, and
M.
Kaliske
(2021), pp. 1
–12
.59.
A. M.
Rueda-Ramírez
,
W.
Pazner
, and
G. J.
Gassner
, “
Subcell limiting strategies for discontinuous Galerkin spectral element methods
,” Comput. Fluids
247
, 105627
(2022
).60.
A. D.
Beck
,
J.
Zeifang
,
A.
Schwarz
, and
D. G.
Flad
, “
A neural network based shock detection and localization approach for discontinuous Galerkin methods
,” J. Comput. Phys.
423
, 109824
(2020
).61.
A.
Schwarz
,
J.
Keim
,
S.
Chiocchetti
, and
A.
Beck
, “
A reinforcement learning based slope limiter for second-order finite volume schemes
,” Proc. Appl. Math. Mech.
23
, e202200207
(2023
).62.
D.
Ray
and
J. S.
Hesthaven
, “
An artificial neural network as a troubled-cell indicator
,” J. Comput. Phys.
367
, 166
(2018
).63.
D.
Ray
and
J. S.
Hesthaven
, “
Detecting troubled-cells on two-dimensional unstructured grids using a neural network
,” J. Comput. Phys.
397
, 108845
(2019
).64.
R. S.
Rogallo
, Numerical Experiments in Homogeneous Turbulence
(
National Aeronautics and Space Administration
, 1981
), Vol.
81315
.65.
T. S.
Lundgren
, “
Linearly forced isotropic turbulence
,” in Proceedings of Annual Research Briefs 2OO3
(
Center for Turbulence Research
,
Standford
, 2003
), pp. 461
–473
.66.
B.
De Laage de Meux
,
B.
Audebert
,
R.
Manceau
, and
R.
Perrin
, “
Anisotropic linear forcing for synthetic turbulence generation in large eddy simulation and hybrid RANS/LES modeling
,” Phys. Fluids
27
, 035115
(2015
).67.
G.
Gassner
and
D. A.
Kopriva
, “
A comparison of the dispersion and dissipation errors of Gauss and Gauss–Lobatto discontinuous Galerkin spectral element methods
,” SIAM J. Sci. Comput.
33
, 2560
(2011
).68.
S. B.
Pope
, Turbulent Flows
, 7th ed. (
Cambridge University Press
,
Cambridge
, 2010
), p. 771
.69.
M.
Kurz
,
P.
Offenhäuser
,
D.
Viola
,
O.
Shcherbakov
,
M.
Resch
, and
A.
Beck
, “
Deep reinforcement learning for computational fluid dynamics on HPC systems
,” J. Comput. Sci.
65
, 101884
(2022
).70.
P.-W.
Chou
,
D.
Maturana
, and
S.
Scherer
, “
Improving stochastic policy gradients in continuous control with deep reinforcement learning using the beta distribution
,” in Proceedings of International Conference on Machine Learning
(
PMLR
, 2017
), pp. 834
–843
.71.
72.
J.
Smagorinsky
, “
General circulation experiments with the primitive equations: I. The basic experiment
,” Mon. Weather Rev.
91
, 99
(1963
).73.
P.
Moin
,
K.
Squires
,
W.
Cabot
, and
S.
Lee
, “
A dynamic subgrid-scale model for compressible turbulence and scalar transport
,” Phys. Fluids A
3
, 2746
(1991
).74.
A. D.
Beck
,
T.
Bolemann
,
D.
Flad
,
H.
Frank
,
G. J.
Gassner
,
F.
Hindenlang
, and
C.-D.
Munz
, “
High-order discontinuous Galerkin spectral element methods for transitional and turbulent flow simulations
,” Int. J. Numer. Methods Fluids
76
, 522
(2014
).75.
E.
Ferrer
, “
An interior penalty stabilised incompressible discontinuous Galerkin–Fourier solver for implicit large eddy simulations
,” J. Comput. Phys.
348
, 754
(2017
).76.
F.
Bassi
,
L.
Botti
,
A.
Colombo
,
A.
Crivellini
,
A.
Ghidoni
, and
F.
Massa
, “
On the development of an implicit high-order discontinuous Galerkin method for DNS and implicit LES of turbulent flows
,” Eur. J. Mech. - B/Fluids
55
, 367
(2016
).77.
R.
Moura
,
G.
Mengaldo
,
J.
Peiró
, and
S.
Sherwin
, “
On the eddy-resolving capability of high-order discontinuous Galerkin approaches to implicit LES/under-resolved DNS of Euler turbulence
,” J. Comput. Phys.
330
, 615
(2017
).78.
C. C.
de Wiart
,
K.
Hillewaert
,
L.
Bricteux
, and
G.
Winckelmans
, “
Implicit LES of free and wall-bounded turbulent flows based on the discontinuous Galerkin/symmetric interior penalty method
,” Int. J. Numer. Methods Fluids
78
, 335
(2015
).79.
A.
Vollant
,
G.
Balarac
, and
C.
Corre
, “
A dynamic regularized gradient model of the subgrid-scale stress tensor for large-eddy simulation
,” Phys. Fluids
28
, 025114
(2016
).80.
Z.
Zhu
,
K.
Lin
,
A. K.
Jain
, and
J.
Zhou
, “
Transfer learning in deep reinforcement learning: A survey
,” in Proceedings of IEEE Transaction on Pattern Analysis and Machine Intelligence
(
IEEE
, 2023
), Vol.
45
, p. 13344
.81.
M. E.
Taylor
and
P.
Stone
, “
Transfer learning for reinforcement learning domains: A survey
,” J. Mach. Learn. Res.
10
, 1633
(2009
), see https://dl.acm.org/doi/10.5555/1577069.1755839.82.
A.
Subel
,
A.
Chattopadhyay
,
Y.
Guan
, and
P.
Hassanzadeh
, “
Data-driven subgrid-scale modeling of forced Burgers turbulence using deep learning with generalization to higher Reynolds numbers via transfer learning
,” Phys. Fluids
33
, 031702
(2021
).83.
A.
Gupta
,
R.
Mendonca
,
Y.
Liu
,
P.
Abbeel
, and
S.
Levine
, “
Meta-reinforcement learning of structured exploration strategies
,” in Proceedings of the 32nd International Conference on Neural Information Processing Systems
Montreal, Canada (Curran Associates Inc., Red Hook, NY, 2018), pp. 5307–5316© 2023 Author(s). Published under an exclusive license by AIP Publishing.
2023
Author(s)
You do not currently have access to this content.
