In this paper, we present a deep learning technique for data-driven predictions of wave propagation in a fluid medium. The technique relies on an attention-based convolutional recurrent autoencoder network (AB-CRAN). To construct a low-dimensional representation of wave propagation data, we employ a denoising-based convolutional autoencoder. The AB-CRAN architecture with attention-based long short-term memory cells forms our deep neural network model for the time marching of the low-dimensional features. We assess the proposed AB-CRAN framework against the standard recurrent neural network for the low-dimensional learning of wave propagation. To demonstrate the effectiveness of the AB-CRAN model, we consider three benchmark problems, namely, one-dimensional linear convection, the nonlinear viscous Burgers equation, and the two-dimensional Saint-Venant shallow water system. Using the spatial-temporal datasets from the benchmark problems, our novel AB-CRAN architecture accurately captures the wave amplitude and preserves the wave characteristics of the solution for long time horizons. The attention-based sequence-to-sequence network increases the time-horizon of prediction compared to the standard recurrent neural network with long short-term memory cells. The denoising autoencoder further reduces the mean squared error of prediction and improves the generalization capability in the parameter space.

1.
M.
Abadi
 et al., “
TensorFlow: Large-scale machine learning on heterogeneous systems
,” arXiv:1603.04467 (2016).
2.
A. C.
Antoulas
,
Approximation of Large-Scale Dynamical Systems
(
SIAM
,
2005
).
3.
P.
Baldi
and
K.
Hornik
, “
Neural networks and principal component analysis: Learning from examples without local minima
,”
Neural Networks
2
(
1
),
53
58
(
1989
).
4.
Y.
Bengio
, “
Practical recommendations for gradient-based training of deep architectures
,” in
Neural Networks: Tricks of the Trade
(
Springer
,
2012
), pp.
437
478
.
5.
P.
Benner
,
W.
Schilders
,
S.
Grivet-Talocia
,
A.
Quarteroni
,
G.
Rozza
, and
L. M.
Silveira
,
Model Order Reduction: Volume 3 Applications
(
De Gruyter
,
2020
).
6.
P.
Benner
,
W.
Schilders
,
S.
Grivet-Talocia
,
A.
Quarteroni
,
G.
Rozza
, and
L. M.
Silveira
,
Model Order Reduction: Volume 2: Snapshot-Based Methods and Algorithms
(
De Gruyter
,
2020
).
7.
M. M.
Bronstein
,
J.
Bruna
,
T.
Cohen
, and
P.
Veličković
, “
Geometric deep learning: Grids, groups, graphs, geodesics, and gauges
,” arXiv:2104.13478 (
2021
).
8.
M. M.
Bronstein
,
J.
Bruna
,
Y.
LeCun
,
A.
Szlam
, and
P.
Vandergheynst
, “
Geometric deep learning: Going beyond Euclidean data
,”
IEEE Signal Process. Mag.
34
(
4
),
18
42
(
2017
).
9.
S. R.
Bukka
,
R.
Gupta
,
A. R.
Magee
, and
R. K.
Jaiman
, “
Assessment of unsteady flow predictions using hybrid deep learning based reduced-order models
,”
Phys. Fluids
33
(
1
),
013601
(
2021
).
10.
S. R.
Bukka
,
A. R.
Magee
, and
R. K.
Jaiman
, “
Deep convolutional recurrent autoencoders for flow field prediction
,” in
International Conference on Offshore Mechanics and Arctic Engineering
(
American Society of Mechanical Engineers
,
2020
), Vol.
84409
, p.
V008T08A005
.
11.
N.
Cagniart
,
Y.
Maday
, and
B.
Stamm
, “
Model order reduction for problems with large convection effects
,” in
Contributions to Partial Differential Equations and Applications
(
Springer
,
2019
), pp.
131
150
.
12.
K.
Carlberg
,
M. F.
Barone
, and
H.
Antil
, “
Galerkin v. discrete-optimal projection in nonlinear model reduction
,”
Report No. SAND-2015-2897J
(
2015
).
13.
N.
Cellier
, for locie/triflow: v0.4.3 (last accessed May
2017
).
14.
K.
Cho
,
B.
Van Merriënboer
,
C.
Gulcehre
,
D.
Bahdanau
,
F.
Bougares
,
H.
Schwenk
, and
Y.
Bengio
, “
Learning phrase representations using RNN encoder-decoder for statistical machine translation
,” arXiv:1406.1078 (
2014
).
15.
G.
Dong
,
G.
Liao
,
H.
Liu
, and
G.
Kuang
, “
A review of the autoencoder and its variants: A comparative perspective from target recognition in synthetic-aperture radar images
,”
IEEE Geosci. Remote Sens. Mag.
6
(
3
),
44
68
(
2018
).
16.
C. M.
Duarte
,
L.
Chapuis
,
S. P.
Collin
,
D. P.
Costa
,
R. P.
Devassy
,
V. M.
Eguiluz
,
C.
Erbe
,
T. A.
Gordon
,
B. S.
Halpern
,
H. R.
Harding
 et al., “
The soundscape of the Anthropocene ocean
,”
Science
371
(
6529
),
eaba4658
(
2021
).
17.
J. L.
Elman
, “
Finding structure in time
,”
Cognit. Sci.
14
(
2
),
179
211
(
1990
).
18.
C.
Erbe
,
S. A.
Marley
,
R. P.
Schoeman
,
J. N.
Smith
,
L. E.
Trigg
, and
C. B.
Embling
, “
The effects of ship noise on marine mammals: A review
,”
Front. Mar. Sci.
6
,
606
(
2019
).
19.
S.
Fresca
and
A.
Manzoni
 et al., “
A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs
,”
J. Sci. Comput.
87
(
2
),
1
36
(
2021
).
20.
N.
Geneva
and
N.
Zabaras
, “
Quantifying model form uncertainty in Reynolds-averaged turbulence models with Bayesian deep neural networks
,”
J. Comput. Phys.
383
,
125
147
(
2019
).
21.
F. J.
Gonzalez
and
M.
Balajewicz
, “
Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems
,” arXiv:1808.01346 (
2018
).
22.
I.
Goodfellow
,
Y.
Bengio
, and
A.
Courville
,
Deep Learning
(
MIT Press
,
2016
).
23.
A.
Goyal
and
Y.
Bengio
, “
Inductive biases for deep learning of higher-level cognition
,” arXiv:2011.15091 (
2020
).
24.
C.
Greif
and
K.
Urban
, “
Decay of the Kolmogorov N-width for wave problems
,”
Appl. Math. Lett.
96
,
216
222
(
2019
).
25.
R.
Gupta
and
R.
Jaiman
, “
Three-dimensional deep learning-based reduced order model for unsteady flow dynamics with variable Reynolds number
,”
Phys. Fluids
34
(
3
),
033612
(
2022
).
26.
R.
Gupta
and
R. K.
Jaiman
, “
A hybrid partitioned deep learning methodology for moving interface and fluid–structure interaction
,”
Comput. Fluids
233
,
105239
(
2022
).
27.
G. E.
Hinton
and
R. R.
Salakhutdinov
, “
Reducing the dimensionality of data with neural networks
,”
Science
313
(
5786
),
504
507
(
2006
).
28.
S.
Hochreiter
and
J.
Schmidhuber
, “
Long short-term memory
,”
Neural Comput.
9
(
8
),
1735
1780
(
1997
).
29.
D. P.
Kingma
and
J.
Ba
, “
Adam: A method for stochastic optimization
,” arXiv:1412.6980 (2014).
30.
Y.
LeCun
,
Y.
Bengio
, and
G.
Hinton
, “
Deep learning
,”
Nature
521
(
7553
),
436
444
(
2015
).
31.
K.
Lee
and
K. T.
Carlberg
, “
Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders
,”
J. Comput. Phys.
404
,
108973
(
2020
).
32.
R. J.
LeVeque
 et al.,
Finite Volume Methods for Hyperbolic Problems
(
Cambridge University Press
,
2002
), Vol. 31.
33.
M. J.
Lighthill
,
Waves in Fluids
(
Cambridge University Press
,
2001
).
34.
R.
Maulik
,
B.
Lusch
, and
P.
Balaprakash
, “
Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders
,”
Phys. Fluids
33
(
3
),
037106
(
2021
).
35.
S.
Mika
,
B.
Schölkopf
,
A. J.
Smola
,
K.-R.
Müller
,
M.
Scholz
, and
G.
Rätsch
, “
Kernel PCA and de-noising in feature spaces
,”
Adv. Neural Inf. Process. Syst.
11
,
536
542
(
1998
).
36.
T. P.
Miyanawala
and
R. K.
Jaiman
, “
An efficient deep learning technique for the Navier-Stokes equations: Application to unsteady wake flow dynamics
,” arXiv:1710.09099 (
2017
).
37.
T. P.
Miyanawala
and
R. K.
Jaiman
, “
A low-dimensional learning model via convolutional neural networks for unsteady wake-body interaction
,” arXiv:1807.09591 (
2018
).
38.
T. P.
Miyanawala
and
R. K.
Jaiman
, “
A novel deep learning method for the predictions of current forces on bluff bodies
,” in
International Conference on Offshore Mechanics and Arctic Engineering
(
American Society of Mechanical Engineers
,
2018
), Vol.
51210
, p.
V002T08A003
.
39.
R.
Mojgani
and
M.
Balajewicz
, “
Physics-aware registration based auto-encoder for convection dominated PDEs
,” arXiv:2006.15655 (
2020
).
40.
M.
Ohlberger
and
S.
Rave
, “
Reduced basis methods: Success, limitations and future challenges
,” arXiv:1511.02021 (
2015
).
41.
A.
Pinkus
,
N-Widths in Approximation Theory
(
Springer Science & Business Media
,
2012
), Vol.
7
.
42.
E.
Plaut
, “
From principal subspaces to principal components with linear autoencoders
,” arXiv:1804.10253 (
2018
).
43.
A.
Quarteroni
,
A.
Manzoni
, and
F.
Negri
,
Reduced Basis Methods for Partial Differential Equations: An Introduction
(
Springer
,
2015
), Vol.
92
.
44.
M.
Salvador
,
L.
Dede
, and
A.
Manzoni
, “
Non intrusive reduced order modeling of parametrized PDEs by kernel POD and neural networks
,” arXiv:2103.17152 (
2021
).
45.
W. H.
Schilders
,
H. A.
Van der Vorst
, and
J.
Rommes
,
Model Order Reduction: Theory, Research Aspects and Applications
(
Springer
,
2008
), Vol.
13
.
46.
M.
Seeger
,
H.
Nickisch
,
R.
Pohmann
, and
B.
Schölkopf
, in
Advances in Neural Information Processing Systems
, edited by
D.
Koller
,
D.
Schuurmans
,
Y.
Bengio
, and
L.
Bottou
(
MIT Press
,
2009
), Vol. 21.
47.
W. E.
Sorteberg
,
S.
Garasto
,
A. S.
Pouplin
,
C. D.
Cantwell
, and
A. A.
Bharath
, “
Approximating the solution to wave propagation using deep neural networks
,” arXiv:1812.01609 (
2018
).
48.
I.
Sutskever
,
O.
Vinyals
, and
Q. V.
Le
, “
Sequence to sequence learning with neural networks
,”
Adv. Neural Inf. Process. Syst.
27
,
3104
3112
(
2014
).
49.
J. B.
Tenenbaum
 et al., “
Mapping a manifold of perceptual observations
,”
Adv. Neural Inf. Proces. Syst.
10
,
682
688
(
1998
).
50.
B.
Unger
and
S.
Gugercin
, “
Kolmogorov N-widths for linear dynamical systems
,”
Adv. Comput. Math.
45
(
5
),
2273
2286
(
2019
).
51.
A.
Vaswani
,
N.
Shazeer
,
N.
Parmar
,
J.
Uszkoreit
,
L.
Jones
,
A. N.
Gomez
,
Ł.
Kaiser
, and
I.
Polosukhin
, “
Attention is all you need
,”
Adv. Neural Inf. Process. Syst.
30
,
1
–11 (
2017
).
52.
P.
Vincent
,
H.
Larochelle
,
Y.
Bengio
, and
P.-A.
Manzagol
, “
Extracting and composing robust features with denoising autoencoders
,” in
Proceedings of the 25th International Conference on Machine Learning
(Association for Computing Machinery,
2008
), pp.
1096
1103
.
53.
P.
Vincent
,
H.
Larochelle
,
I.
Lajoie
,
Y.
Bengio
,
P. A.
Manzagol
, and
L.
Bottou
, “
Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion
,”
J. Mach. Learn. Res.
11
(
12
),
3371
3408
(
2010
).
54.
J.
Willard
,
X.
Jia
,
S.
Xu
,
M.
Steinbach
, and
V.
Kumar
, “
Integrating scientific knowledge with machine learning for engineering and environmental systems
,” arXiv:2003.04919 (
2021
).
55.
P.
Wu
,
S.
Gong
,
K.
Pan
,
F.
Qiu
,
W.
Feng
, and
C.
Pain
, “
Reduced order model using convolutional auto-encoder with self-attention
,”
Phys. Fluids
33
(
7
),
077107
(
2021
).
56.
J.
Xu
and
K.
Duraisamy
, “
Multi-level convolutional autoencoder networks for parametric prediction of spatio-temporal dynamics
,”
Comput. Methods Appl. Mech. Eng.
372
,
113379
(
2020
).
57.
C.
Yang
,
X.
Yang
, and
X.
Xiao
, “
Data-driven projection method in fluid simulation
,”
Comput. Anim. Virtual Worlds
27
(
3–4
),
415
424
(
2016
).
58.
Y.
Yao
,
L.
Rosasco
, and
A.
Caponnetto
, “
On early stopping in gradient descent learning
,”
Constr. Approximation
26
(
2
),
289
315
(
2007
).
You do not currently have access to this content.