High-resolution (HR) information of fluid flows, although preferable, is usually less accessible due to limited computational or experimental resources. In many cases, fluid data are generally sparse, incomplete, and possibly noisy. How to enhance spatial resolution and decrease the noise level of flow data is essential and practically useful. Deep learning (DL) techniques have been demonstrated to be effective for super-resolution (SR) tasks, which, however, primarily rely on sufficient HR labels for training. In this work, we present a novel physics-informed DL-based SR solution using convolutional neural networks (CNNs), which is able to produce HR flow fields from low-resolution (LR) inputs in high-dimensional parameter space. By leveraging the conservation laws and boundary conditions of fluid flows, the CNN-SR model is trained without any HR labels. Moreover, the proposed CNN-SR solution unifies the forward SR and inverse data assimilation for the scenarios where the physics is partially known, e.g., unknown boundary conditions. A new network structure is designed to enable not only the parametric SR but also the parametric inference for the first time. Several flow SR problems relevant to cardiovascular applications have been studied to demonstrate the proposed method's effectiveness and merit. A series of different LR scenarios, including LR input with Gaussian noises, non-Gaussian magnetic resonance imaging noises, and downsampled measurements given either well-posed or ill-posed physics, are investigated to illustrate the SR, denoising, and inference capabilities of the proposed method.

1.
A.
Pollard
,
L.
Castillo
,
L.
Danaila
, and
M.
Glauser
,
Whither Turbulence and Big Data in the 21st Century?
(
Springer
,
2016
).
2.
C. M.
Lawley
,
K. M.
Broadhouse
,
F. M.
Callaghan
,
D. S.
Winlaw
,
G. A.
Figtree
, and
S. M.
Grieve
, “
4D flow magnetic resonance imaging: Role in pediatric congenital heart disease
,”
Asian Cardiovasc. Thorac. Ann.
26
(
1
),
28
37
(
2018
).
3.
Z.
Stankovic
,
B. D.
Allen
,
J.
Garcia
,
K. B.
Jarvis
, and
M.
Markl
, “
4D flow imaging with MRI
,”
Cardiovasc. Diagn. Ther.
4
(
2
),
173
(
2014
).
4.
F.
Ong
,
M.
Uecker
,
U.
Tariq
,
A.
Hsiao
,
M. T.
Alley
,
S. S.
Vasanawala
, and
M.
Lustig
, “
Robust 4D flow denoising using divergence-free wavelet transform
,”
Magn. Reson. Med.
73
(
2
),
828
842
(
2015
).
5.
M. F.
Fathi
,
A.
Bakhshinejad
,
A.
Baghaie
,
D.
Saloner
,
R. H.
Sacho
,
V. L.
Rayz
, and
R. M.
D'Souza
, “
Denoising and spatial resolution enhancement of 4D flow MRI using proper orthogonal decomposition and lasso regularization
,”
Comput. Med. Imaging Graph
70
,
165
120
(
2018
).
6.
F. M.
Callaghan
and
S. M.
Grieve
, “
Spatial resolution and velocity field improvement of 4D-flow MRI
,”
Magn. Reson. Med.
78
(
5
),
1959
1968
(
2017
).
7.
D.
Venturi
and
G. E.
Karniadakis
, “
Gappy data and reconstruction procedures for flow past a cylinder
,”
J. Fluid Mech.
519
,
315
336
(
2004
).
8.
T.
Bui-Thanh
,
M.
Damodaran
, and
K.
Willcox
, “
Proper orthogonal decomposition extensions for parametric applications in compressible aerodynamics
,” in
21st AIAA Applied Aerodynamics Conference
(
AIAA
,
2003
), p.
4213
.
9.
B.
Podvin
,
Y.
Fraigneau
,
F.
Lusseyran
, and
P.
Gougat
, “
A reconstruction method for the flow past an open cavity
,”
J. Fluids Eng.
128
(
3
),
531
540
(
2006
).
10.
A.
Yakhot
,
T.
Anor
, and
G. E.
Karniadakis
, “
A reconstruction method for gappy and noisy arterial flow data
,”
IEEE Trans. Med. Imaging
26
(
12
),
1681
1697
(
2007
).
11.
A. I.
Moreno
,
A. A.
Jarzabek
,
J. M.
Perales
, and
J. M.
Vega
, “
Aerodynamic database reconstruction via gappy high order singular value decomposition
,”
Aerosp. Sci. Technol.
52
,
115
128
(
2016
).
12.
M.
Mifsud
,
A.
Vendl
,
L.-U.
Hansen
, and
S.
Görtz
, “
Fusing wind-tunnel measurements and CFD data using constrained gappy proper orthogonal decomposition
,”
Aerosp. Sci. Technol.
86
,
312
326
(
2019
).
13.
P. J.
Schmid
, “
Dynamic mode decomposition of numerical and experimental data
,”
J. Fluid Mech.
656
,
5
28
(
2010
).
14.
J. H.
Tu
,
C. W.
Rowley
,
D. M.
Luchtenburg
,
S. L.
Brunton
, and
J. N.
Kutz
, “
On dynamicx mode decomposition: Theory and applications
,”
J. Comput. Dyn.
1
,
391
(
2014
).
15.
J. L.
Callaham
,
K.
Maeda
, and
S. L.
Brunton
, “
Robust flow reconstruction from limited measurements via sparse representation
,”
Phys. Rev. Fluids
4
(
10
),
103907
(
2019
).
16.
K.
Manohar
,
B. W.
Brunton
,
J. N.
Kutz
, and
S. L.
Brunton
, “
Data-driven sparse sensor placement for reconstruction: Demonstrating the benefits of exploiting known patterns
,”
IEEE Control Syst. Mag.
38
(
3
),
63
86
(
2018
).
17.
D. P.
Foures
,
N.
Dovetta
,
D.
Sipp
, and
P. J.
Schmid
, “
A data-assimilation method for Reynolds-averaged Navier–Stokes-driven mean flow reconstruction
,”
J. Fluid Mech.
759
,
404
431
(
2014
).
18.
B.
Combès
,
D.
Heitz
,
A.
Guibert
, and
E.
Mémin
, “
A particle filter to reconstruct a free-surface flow from a depth camera
,”
Fluid Dyn. Res.
47
(
5
),
051404
(
2015
).
19.
R.
Kikuchi
,
T.
Misaka
, and
S.
Obayashi
, “
Assessment of probability density function based on POD reduced-order model for ensemble-based data assimilation
,”
Fluid Dyn. Res.
47
(
5
),
051403
(
2015
).
20.
V.
Mons
,
J.-C.
Chassaing
,
T.
Gomez
, and
P.
Sagaut
, “
Reconstruction of unsteady viscous flows using data assimilation schemes
,”
J. Comput. Phys.
316
,
255
280
(
2016
).
21.
S.
Symon
,
N.
Dovetta
,
B. J.
McKeon
,
D.
Sipp
, and
P. J.
Schmid
, “
Data assimilation of mean velocity from 2D PIV measurements of flow over an idealized airfoil
,”
Exp. Fluids
58
(
5
),
61
(
2017
).
22.
J.-X.
Wang
and
H.
Xiao
, “
Data-driven CFD modeling of turbulent flows through complex structures
,”
Int. J. Heat Fluid Flow
62
,
138
149
(
2016
).
23.
H.
Xiao
,
J.-L.
Wu
,
J.-X.
Wang
,
R.
Sun
, and
C.
Roy
, “
Quantifying and reducing model-form uncertainties in Reynolds-averaged Navier–Stokes simulations: A data-driven, physics-informed Bayesian approach
,”
J. Comput. Phys.
324
,
115
136
(
2016
).
24.
S. L.
Brunton
,
B. R.
Noack
, and
P.
Koumoutsakos
, “
Machine learning for fluid mechanics
,”
Annu. Rev. Fluid Mech.
52
,
477
508
(
2020
).
25.
M.
Brenner
,
J.
Eldredge
, and
J.
Freund
, “
Perspective on machine learning for advancing fluid mechanics
,”
Phys. Rev. Fluids
4
(
10
),
100501
(
2019
).
26.
J.
Ling
,
A.
Kurzawski
, and
J.
Templeton
, “
Reynolds averaged turbulence modelling using deep neural networks with embedded invariance
,”
J. Fluid Mech.
807
,
155
166
(
2016
).
27.
K.
Duraisamy
,
G.
Iaccarino
, and
H.
Xiao
, “
Turbulence modeling in the age of data
,”
Annu. Rev. Fluid Mech.
51
,
357
377
(
2019
).
28.
J.-X.
Wang
,
J.-L.
Wu
, and
H.
Xiao
, “
Physics-informed machine learning approach for reconstructing Reynolds stress modeling discrepancies based on DNS data
,”
Phys. Rev. Fluids
2
(
3
),
034603
(
2017
).
29.
J.-X.
Wang
,
J.
Huang
,
L.
Duan
, and
H.
Xiao
, “
Prediction of Reynolds stresses in high-Mach-number turbulent boundary layers using physics-informed machine learning
,”
Theor. Comput. Fluid Dyn.
33
(
1
),
1
19
(
2019
).
30.
M. I.
Zafar
,
H.
Xiao
,
M. M.
Choudhari
,
F.
Li
,
C.-L.
Chang
,
P.
Paredes
, and
B.
Venkatachari
, “
Convolutional neural network for transition modeling based on linear stability theory
,” arXiv:2005.02599 (
2020
).
31.
K.
Fukami
,
Y.
Nabae
,
K.
Kawai
, and
K.
Fukagata
, “
Synthetic turbulent inflow generator using machine learning
,”
Phys. Rev. Fluids
4
(
6
),
064603
(
2019
).
32.
J.
Kim
and
C.
Lee
, “
Deep unsupervised learning of turbulence for inflow generation at various Reynolds numbers
,”
J. Comput. Phys.
406
,
109216
(
2020
).
33.
L.
Sun
,
H.
Gao
,
S.
Pan
, and
J.-X.
Wang
, “
Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data
,”
Comput. Methods Appl. Mech. Eng.
361
,
112732
(
2020
).
34.
R.
Maulik
,
K.
Fukami
,
N.
Ramachandra
,
K.
Fukagata
, and
K.
Taira
, “
Probabilistic neural networks for fluid flow surrogate modeling and data recovery
,”
Phys. Rev. Fluids
5
(
10
),
104401
(
2020
).
35.
H.
Gao
,
J.-X.
Wang
, and
M. J.
Zahr
, “
Non-intrusive model reduction of large-scale, nonlinear dynamical systems using deep learning
,”
Physica D
412
,
132614
(
2020
).
36.
C.
Ledig
,
L.
Theis
,
F.
Huszár
,
J.
Caballero
,
A.
Cunningham
,
A.
Acosta
,
A.
Aitken
,
A.
Tejani
,
J.
Totz
,
Z.
Wang
 et al., “
Photo-realistic single image super-resolution using a generative adversarial network
,”
in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(IEEE,
2017
), pp.
4681
4690
.
37.
K.
Fukami
,
K.
Fukagata
, and
K.
Taira
, “
Machine learning based spatio-temporal super resolution reconstruction of turbulent flows
,” arXiv:2004.11566 (
2020
).
38.
K.
Fukami
,
K.
Fukagata
, and
K.
Taira
, “
Super-resolution analysis with machine learning for low-resolution flow data
,” in
11th International Symposium on Turbulence and Shear Flow Phenomena (TSFP)
(
2019
).
39.
K.
Fukami
,
K.
Fukagata
, and
K.
Taira
, “
Super-resolution reconstruction of turbulent flows with machine learning
,” arXiv:1811.11328 (
2018
).
40.
Y.
Liu
,
C.
Ponce
,
S. L.
Brunton
, and
J. N.
Kutz
, “
Multiresolution convolutional autoencoders
,” arXiv:2004.04946 (
2020
).
41.
Z.
Deng
,
C.
He
,
Y.
Liu
, and
K. C.
Kim
, “
Super-resolution reconstruction of turbulent velocity fields using a generative adversarial network-based artificial intelligence framework
,”
Phys. Fluids
31
(
12
),
125111
(
2019
).
42.
M.
Bode
,
M.
Gauding
,
Z.
Lian
,
D.
Denker
,
M.
Davidovic
,
K.
Kleinheinz
,
J.
Jitsev
, and
H.
Pitsch
, “
Using physics-informed super-resolution generative adversarial networks for subgrid modeling in turbulent reactive flows
,” arXiv:1911.11380 (
2019
).
43.
K.
Bai
,
W.
Li
,
M.
Desbrun
, and
X.
Liu
, “
Dynamic upsampling of smoke through dictionary-based learning
,” arXiv:1910.09166 (
2019
).
44.
F. J.
Gonzalez
and
M.
Balajewicz
, “
Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems
,” arXiv:1808.01346 (
2018
).
45.
Y.
Xie
,
E.
Franz
,
M.
Chu
, and
N.
Thuerey
, “
tempoGAN: A temporally coherent, volumetric GAN for super-resolution fluid flow
,”
ACM Trans. Graph.
37
(
4
),
1
15
(
2018
).
46.
B.
Liu
,
J.
Tang
,
H.
Huang
, and
X.-Y.
Lu
, “
Deep learning methods for super-resolution reconstruction of turbulent flows
,”
Phys. Fluids
32
(
2
),
025105
(
2020
).
47.
M.
Werhahn
,
Y.
Xie
,
M.
Chu
, and
N.
Thuerey
, “
A multi-pass GAN for fluid flow super-resolution
,”
Proc. ACM Comput. Graph. Interact. Tech.
2
(
2
),
1
21
(
2019
).
48.
L.
Guo
,
S.
Ye
,
J.
Han
,
H.
Zheng
,
H.
Gao
,
D. Z.
Chen
,
J.-X.
Wang
, and
C.
Wang
, “
SSR-VFD: Spatial super-resolution for vector field data analysis and visualization
,” in
2020 IEEE Pacific Visualization Symposium (PacificVis)
(
IEEE
,
2020
), pp.
71
80
.
49.
N. B.
Erichson
,
L.
Mathelin
,
Z.
Yao
,
S. L.
Brunton
,
M. W.
Mahoney
, and
J. N.
Kutz
, “
Shallow learning for fluid flow reconstruction with limited sensors and limited data
,” arXiv:1902.07358 (
2019
).
50.
E.
Ferdian
,
A.
Suinesiaputra
,
D. J.
Dubowitz
,
D.
Zhao
,
A.
Wang
,
B.
Cowan
, and
A. A.
Young
, “
4DFlowNet: Super-resolution 4D flow MRI using deep learning and computational fluid dynamics
,”
Front. Phys.
8
,
138
(
2020
).
51.
M.
Raissi
,
P.
Perdikaris
, and
G.
Karniadakis
, “
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
,”
J. Comput. Phys.
378
,
686
707
(
2019
).
52.
V.
Dwivedi
,
N.
Parashar
, and
B.
Srinivasan
, “
Distributed learning machines for solving forward and inverse problems in partial differential equations
,”
Neurocomputing
420
,
299
316
(
2021
).
53.
Y.
Zhu
,
N.
Zabaras
,
P.-S.
Koutsourelakis
, and
P.
Perdikaris
, “
Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data
,”
J. Comput. Phys.
394
,
56
81
(
2019
).
54.
N.
Geneva
and
N.
Zabaras
, “
Modeling the dynamics of PDE systems with physics-constrained deep auto-regressive networks
,”
J. Comput. Phys.
403
,
109056
(
2020
).
55.
R.
Zhang
,
Y.
Liu
, and
H.
Sun
, “
Physics-informed multi-LSTM networks for metamodeling of nonlinear structures
,” arXiv:2002.10253 (
2020
).
56.
Z.
Long
,
Y.
Lu
,
X.
Ma
, and
B.
Dong
, “
PDE-Net: Learning PDEs from data
,” arXiv:1710.09668 (
2017
).
57.
Z.
Long
,
Y.
Lu
, and
B.
Dong
, “
PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network
,”
J. Comput. Phys.
399
,
108925
(
2019
).
58.
G.
Singh
,
S.
Gupta
,
M.
Lease
, and
C. N.
Dawson
, “
TIME: A transparent, interpretable, model-adaptive and explainable neural network for dynamic physical processes
,” arXiv:2003.02426 (
2020
).
59.
Z.
Chen
,
Y.
Liu
, and
H.
Sun
, “
Deep learning of physical laws from scarce data
,” arXiv:2005.03448 (
2020
).
60.
C. M.
Jiang
,
S.
Esmaeilzadeh
,
K.
Azizzadenesheli
,
K.
Kashinath
,
M.
Mustafa
,
H. A.
Tchelepi
,
P.
Marcus
,
A.
Anandkumar
 et al., “
Meshfreeflownet: A physics-constrained deep continuous space-time super-resolution framework
,” arXiv:2005.01463 (
2020
).
61.
A. T.
Mohan
,
N.
Lubbers
,
D.
Livescu
, and
M.
Chertkov
, “
Embedding hard physical constraints in neural network coarse-graining of 3D turbulence
,” arXiv:2002.00021 (
2020
).
62.
N.
Thuerey
,
Y.
Xie
,
M.
Chu
,
S.
Wiewel
, and
L.
Prantl
, “
Physics-based deep learning for fluid flow
,” see https://www.semanticscholar.org/paper/Physics-Based-Deep-Learning-for-Fluid-Flow-Thuerey-Xie/bf4623b356406133b7c9a0e8b28c67482da12df8.
63.
A.
Subramaniam
,
M. L.
Wong
,
R. D.
Borker
,
S.
Nimmagadda
, and
S. K.
Lele
, “
Turbulence enrichment using physics-informed generative adversarial networks
,” arXiv:2003.01907 (
2020
).
64.
L.
Sun
and
J.-X.
Wang
, “
Physics-constrained Bayesian neural network for fluid flow reconstruction with sparse and noisy data
,”
Theor. Appl. Mech. Lett.
10
(
3
),
161
169
(
2020
).
65.
J.
Töger
,
M. J.
Zahr
,
N.
Aristokleous
,
K.
Markenroth Bloch
,
M.
Carlsson
, and
P.-O.
Persson
, “
Blood flow imaging by optimal matching of computational fluid dynamics to 4D-flow data
,”
Magn. Reson. Med.
84
(
4
),
2231
2245
(
2020
).
66.
S. L.
Cotter
,
M.
Dashti
,
J. C.
Robinson
, and
A. M.
Stuart
, “
Bayesian inverse problems for functions and applications to fluid mechanics
,”
Inverse Probl.
25
(
11
),
115008
(
2009
).
67.
H.
Gao
and
J.-X.
Wang
, “
A bi-fidelity ensemble Kalman method for PDE-constrained inverse problems in computational mechanics
,”
Comput. Mech.
67
(
4
),
1115
1131
(
2021
).
68.
H.
Gao
,
L.
Sun
, and
J.-X.
Wang
, “
Phygeonet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain
,”
J. Comput. Phys.
428
,
110079
(
2021
).
69.
W.
Shi
,
J.
Caballero
,
F.
Huszár
,
J.
Totz
,
A. P.
Aitken
,
R.
Bishop
,
D.
Rueckert
, and
Z.
Wang
, “
Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network
,”
in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(IEEE,
2016
), pp.
1874
1883
.
70.
D. P.
Kingma
and
J.
Ba
, “
Adam: A method for stochastic optimization
,” arXiv:1412.6980 (
2014
).
71.
V.
Nair
and
G. E.
Hinton
, “
Rectified linear units improve restricted boltzmann machines
,” in
ICML'10: Proceedings of the 27th International Conference on International Conference on Machine Learning
(
2010
).
72.
K. M.
Johnson
and
M.
Markl
, “
Improved SNR in phase contrast velocimetry with five-point balanced flow encoding
,”
Magn. Reson. Med.
63
(
2
),
349
355
(
2010
).
73.
H.
Jasak
,
A.
Jemcov
,
Z.
Tukovic
 et al., “
Openfoam: A C++ library for complex physics simulations
,” in
International Workshop on Coupled Methods in Numerical Dynamics
(
IUC Dubrovnik
,
Croatia
,
2007
), Vol.
1000
, pp.
1
20
.
74.
R. H.
Pletcher
,
J. C.
Tannehill
, and
D.
Anderson
,
Computational Fluid Mechanics and Heat Transfer
(
CRC Press
,
2012
).
75.
C.
Rhie
and
W. L.
Chow
, “
Numerical study of the turbulent flow past an airfoil with trailing edge separation
,”
AIAA J.
21
(
11
),
1525
1532
(
1983
).
76.
A.
Paszke
,
S.
Gross
,
S.
Chintala
,
G.
Chanan
,
E.
Yang
,
Z.
DeVito
,
Z.
Lin
,
A.
Desmaison
,
L.
Antiga
, and
A.
Lerer
, “
Automatic differentiation in PyTorch
” in
31st Conference on Neural Information Processing Systems (NIPS 2017)
, Long Beach, CA,
2017
.
77.
C.
Rao
,
H.
Sun
, and
Y.
Liu
, “
Hard encoding of physics for learning spatiotemporal dynamics
,” arXiv:2105.00557 (
2021
).
78.
N.
Geneva
and
N.
Zabaras
, “
Multi-fidelity generative deep learning turbulent flows
,”
Found. Data Sci.
2
(
4
),
391
(
2020
).
79.
N.
Geneva
and
N.
Zabaras
, “
Transformers for modeling physical systems
,” arXiv:2010.03957 (
2020
).
80.
T.
Pfaff
,
M.
Fortunato
,
A.
Sanchez-Gonzalez
, and
P. W.
Battaglia
, “
Learning mesh-based simulation with graph networks
,” arXiv:2010.03409 (
2020
).
81.
S.
Patankar
,
Numerical Heat Transfer and Fluid Flow
(
Taylor & Francis
,
2018
).
You do not currently have access to this content.