High-resolution (HR) information of fluid flows, although preferable, is usually less accessible due to limited computational or experimental resources. In many cases, fluid data are generally sparse, incomplete, and possibly noisy. How to enhance spatial resolution and decrease the noise level of flow data is essential and practically useful. Deep learning (DL) techniques have been demonstrated to be effective for super-resolution (SR) tasks, which, however, primarily rely on sufficient HR labels for training. In this work, we present a novel physics-informed DL-based SR solution using convolutional neural networks (CNNs), which is able to produce HR flow fields from low-resolution (LR) inputs in high-dimensional parameter space. By leveraging the conservation laws and boundary conditions of fluid flows, the CNN-SR model is trained without any HR labels. Moreover, the proposed CNN-SR solution unifies the forward SR and inverse data assimilation for the scenarios where the physics is partially known, e.g., unknown boundary conditions. A new network structure is designed to enable not only the parametric SR but also the parametric inference for the first time. Several flow SR problems relevant to cardiovascular applications have been studied to demonstrate the proposed method's effectiveness and merit. A series of different LR scenarios, including LR input with Gaussian noises, non-Gaussian magnetic resonance imaging noises, and downsampled measurements given either well-posed or ill-posed physics, are investigated to illustrate the SR, denoising, and inference capabilities of the proposed method.

High-resolution (HR) information of fluid flow is critical for reliable qualitative and quantitative analyses for fluid systems in aerodynamics, mechanical, and biomedical engineering. Nonetheless, fluid flow data are often sparse, incomplete, and noisy in real-life scenarios due to the following reasons. First, flow data are typically spatiotemporal fields in large scales, which poses significant challenges to data analysis, sharing, and visualization due to limited storage space and large communication overhead. For example, the direct numerical simulation (DNS) of wall-bounded turbulent flows at Reynolds number of Reτ=104 can generate more than 20 TB files at each time step, and the file size will increase exponentially as Reτ grows.1 Hence, scientists could only afford to store a small fraction of data (e.g., temporally sparse sequences, spatially downscaled volumes, or selectively variable subsequences) for post hoc analysis. Second, the data resolution is often constrained by the ability of the measurement techniques. For example, flow magnetic resonance imaging (MRI) has been widely used to quantitatively study cardiovascular blood flow dynamics,2,3 but the spatial resolution and signal-to-noise ratio (SRN) of flow magnetic resonance (MR) data are far from sufficient, limiting their clinical applications.4–6 Therefore, it is significant and imperative to enhance the resolution and reduce the noise level, which is referred to as flow data super-resolution and denoising. Moreover, fluid flow data, even with high spatiotemporal resolutions, are usually sparse in the parameter space due to limited computational or experimental resources. For example, a single run of fully resolved DNS of turbulent flows often takes days or weeks on high-performance computing facilities.1 It becomes infeasible to perform massive queries in the parameter space to explore many different boundaries, geometries, and operational configurations for uncertainty quantification (UQ) and optimization. In such scenarios, data super-resolution in the parameter space can be treated as a cost-effective surrogate model that leverages the use of efficient low-resolution (LR) simulations or experiments. Scientists can opt to run their simulations or experiments at a low resolution and then upscale the results back to the target resolution, which will significantly save cost and speed up the process of scientific investigation and discovery.

Various efforts have been devoted to enhancing the spatial or/and temporal resolution of fluid flows. One type of approach focuses on extracting the coherent structures and correlation features from an existing HR database based on proper orthogonal decomposition (POD),7–12 dynamic mode decomposition (DMD),13,14 or other sparsity-promoting representation techniques.15,16 However, these approaches are limited by the linearity assumption made for the reduced basis. The other types of super-resolution methods take advantage of the computational fluid dynamics (CFD) model to provide full-field predictions instead of learning from the offline database. The sparse LR data are fused into the CFD predictions using data assimilation (DA) techniques, e.g., ensemble Kalman filter, particle filters, or variational DA algorithms.17–23 Nonetheless, physics-based CFD simulations are time-consuming in general, while the DA process usually involves numerous model evaluations, which could be computationally prohibitive.

The recent advances in machine learning (ML) and GPU computing open up a promising revenue to tackle this challenge. In the past few years, ML has been successfully applied in fluid dynamics,24,25 for example, for turbulence closure modeling,26–30 inflow turbulence generation,31,32 and fluid surrogate/reduced-order modeling,33–35 etc. In particular, the growing success of deep learning (DL)-based image super-resolution36 in computer vision inspires the application of deep neural networks (DNN) for the flow field super-resolution and reconstruction.37–45 Fukami et al.37–39 applied the convolutional neural network (CNN) and hybrid downsampled skip-connection multiscale (DSC/MC) models for super-resolving downsampled HR data of both laminar and turbulent flows. To achieve a similar goal, Deng et al.41 applied generative adversarial networks (GAN), while Liu et al.46 adopted multiple temporal paths convolutional neural network (MTPC). Thuerey and co-workers designed a more complicated GAN architecture by considering temporal coherence to up-sample 3D volumetric turbulent smoke data,45 and they further improved the scalability by decomposing the learning problem into multiple smaller sub-problems.47 Bai et al.43 used a dictionary learning strategy to super-resolve turbulent smoke flows in a variety of animation contexts. Guo et al.48 designed DL-based spatial upscaling solutions of vector fields for visualization purposes. Considering multi-scale features in fluid dynamics, Liu et al.40 proposed a multi-resolution convolutional autoencoder (MrCAE) super-resolution architecture to dynamically capture different scaled flow features at different depths of the network, where the multi-grid method and transfer learning techniques are leveraged. Instead of using deep networks, Erichson et al.49 proposed to directly capture an end-to-end mapping between the sparse measurements and the HR flow field using a shallow network. In the context of biomedical imaging, Ferdian et al.50 developed a DL model for 4D flow MRI super-resolution, where CFD simulation data are utilized as HR labels for training.

Despite the great promise, the success of these DL models mainly relies on a large amount of offline HR data as labels, which are inaccessible in many cases, e.g., super-resolution of 4D flow MR images. Moreover, these recent data-driven upsampling approaches add visual complexity to an LR input but cannot guarantee that the super-resolved fields are faithful to the physical laws and principles. A more promising strategy is to incorporate prior physics knowledge into deep learning models to alleviate data requirements and improve learning performance. This idea of physics-informed deep learning has been recently explored for solving forward and inverse PDEs,51,52 surrogate modeling,33,53–55 and equation discovery.56–59 For flow reconstruction and super-resolution, the divergence-free constraint for incompressible flow is the most straightforward one to be imposed on the learned solution, which can be done in a hard manner by either introducing stream functions or using spectral methods.42,60–62 Jiang et al.60 proposed a MeshfreeFlowNet for super-resolving of LR solution fields of Rayleigh–Benard convection equations, where the training is regularized by the governing PDEs. In a similar vein, Subramaniam et al.63 utilized the mass and momentum conservation law to constrain the training of a GAN for turbulence enrichment. Sun and Wang64 developed a Bayesian physics-informed neural network using Navier–Stokes constrained Stein variational gradient descent to reconstruct fluid flows from limited noisy measurements. These studies have demonstrated the merits of introducing physics constraints. However, technical challenges remain in developing effective physics-informed DL models for super-resolution, especially for irregular domain problems in label-scarce or label-free scenarios. Moreover, when the physics is partially given (e.g., unknown boundary conditions), how to efficiently assimilate data to recover unknown physics is also challenging. This inverse problem can be traditionally solved by CFD-based data assimilation techniques. For example, one can use adjoint-based variational optimization to fit the CFD solutions to measurement data,65 which, however, is highly code-intrusive and requires iterative forward simulations of expensive CFD models. Another way is the Bayesian data assimilation/optimization66,67 that searches for the posterior distributions of the unknown physics using Bayes' rule, which also requires a massive amount of CFD simulations and is, thus, infeasible in many practical scenarios. Physics-informed neural networks (PINNs)51 allows inference of unknowns, but it is usually non-parametric and needs to be re-trained given a new LF input or a new set of data, which is not effective for fast high-dimensional parametric SR and inference.

To fill these gaps, we developed a novel physics-informed deep learning framework for super-resolving and denoising LR noisy flow fields with irregular geometries without HR data (labels) and inferring unknown physics (e.g., boundary conditions) in a parametric setting. The novel contributions of this paper are summarized as follows: (a) we explored a deep learning solution for flow super-resolution without relying on HR data for training given well-posed physics; (b) the proposed method can simultaneously infer unknown conditions if the physics is partially known (e.g., boundary conditions is unknown); (c) a new hybrid CNN-MLP network structure is proposed to enable the inference in a parametric setting, which, to the best of our knowledge, is the first time to present a parametric inversion method; (d) the proposed work leverages discretized physics-informed deep learning for forward SR and inverse data assimilation without the need of penalty coefficient tunning; (e) we demonstrated the effectiveness of the proposed method on several fluid problems under a series of different LR scenarios, including LR input with Gaussian, non-Gaussian MRI noises, and downsampled measurements given either well-posed or ill-posed physics, in high-dimensional parameter space.

The rest of this paper is organized as follows: The methodology of the proposed method is introduced in Sec. II. Numerical results of several test cases, including vascular flow governed with known, unknown, and parametric boundary conditions, are presented in Sec. III. To illustrate the model's denoising capability, both Gaussian and non-Gaussian noises (e.g., MRI noise) are studied. Section IV discusses the success and limitations of the proposed method. Finally, Sec. V concludes this paper.

This work aims to reconstruct a high-resolution flow field from the corresponding low-resolution (possibly noisy) data obtained either by the low-fidelity simulations or measurements. Mathematically, this process can be described by the following mapping:

S:(Ψ̂l;μ)(Ψ̂h;μ),
(1)

where Ψ̂l denotes the low-resolution (LR) noisy velocity field on a coarse mesh (e.g., 4D flow MRI measurements), Ψ̂h denotes the high-resolution (HR) noise-free flow field, and μ represents the vector of physical parameters (e.g., geometry, inflow/outflow boundary conditions, and flow properties). In general, the dimension of Ψ̂h is much higher than that of Ψ̂l and, thus, could reveal more details of the flow field. We aim to develop a deep learning (DL) based SR solution, where a CNN model Sc is trained to approximate this LR-to-HR mapping as SSc. Once fully trained, the CNN model can be used to super-resolve any given LR data and generate the corresponding HR noise-reduced flow solutions. In contrast to the previous works, the proposed SR-CNN will be trained purely based on physical laws with strictly imposed boundary conditions and, thus, does not need any HR data (i.e., labels). Moreover, the proposed learning framework is able to assimilate sparse observation data, unifying the forward and inverse modeling processes. Namely, when the underlying physics are partially known (e.g., boundary condition or other physical parameters are unknown), extra observations can be parametrically assimilated to enable forward super-resolution and inverse determination of unknowns simultaneously. The overall schematic of the proposed physics-informed DL strategy for flow super-resolution is shown in Fig. 1, and each component of the framework will be detailed in Subsections II B–II F.

FIG. 1.

The schematic of physics-informed CNN for flow super-resolution (SR) and inverse data assimilation. The downsampled LR data can be leveraged and assimilated to enable parametric inference.

FIG. 1.

The schematic of physics-informed CNN for flow super-resolution (SR) and inverse data assimilation. The downsampled LR data can be leveraged and assimilated to enable parametric inference.

Close modal

A composite DL architecture is constructed (Fig. 1), which takes the (possibly noisy) LR velocity fields as the input channels and produces HR flow solutions. A separate sub-CNN is designed to capture each solution field individually, and thus the trainable network parameters are decoupled for different state variables with different magnitudes, facilitating multivariate learning. However, the network predictions of solution variables (u, v, p) are coupled by the Navier–Stokes equations during physics-informed network training. This composite learning structure with decoupled sub-nets has been demonstrated effective in enhancing the learning performance for multivariate regression problems.33,48,68 The composite DL model consists of several convolutional decoders. Each of them has an identical structure of three hidden convolution layers, which is a classic CNN structure for single image super-resolution (SISR).69 Specifically, the input layer is first up-sampled to the target resolution using the bicubic interpolation and then goes through three convolution layers with trainable filters of size 5 × 5, where 2D convolution operations with the padding of 2 and stride of 1 are applied. All the sub-nets are trained simultaneously with a unified physics-informed loss function as detailed in Sec. II C. The trainable parameters of the network are initialized from a uniform distribution of U(125Cin,125Cin), where Cin is the number of input channels. The network's hyper-parameters are summarized in Table I. The same hyper-parameter setting is used for all cases since we found that the model performance is insensitive to the hyper-parameters.

TABLE I.

Hyper-parameters of the sub-CNN.

Learning rateNo. of hidden layersNo. of hidden channelsOptimizer
103 [16,32,16] Adam70  
Padding size Strides Kernel size Non-linearity 
5 × 5 ReLU71  
Learning rateNo. of hidden layersNo. of hidden channelsOptimizer
103 [16,32,16] Adam70  
Padding size Strides Kernel size Non-linearity 
5 × 5 ReLU71  

The composite network Sc needs to be trained in order to approximate the LR-to-HR mapping S as

Ψ̂h=S(Ψ̂l)Sc(Ψ̂l;Wc),
(2)

which is an optimization problem traditionally solved based on a large amount of labeled data. Namely, when a set of ndh LR/HR data pairs {Ψ̂il,Ψ̂ih}i=1ndh are available, the network trainable parameters Wc can be optimized by iteratively minimizing the mismatch between the CNN predictions Sc(Ψ̂il) and the HR labels Ψ̂ih as follows:

W̃c=argminWci=1ndhSc(Ψ̂il;Wc)Ψ̂ihΩpdatabasedloss:Ld,
(3)

where ·Ω denotes the L2 norm over the entire domain Ωp. However, this data-driven training process requires enormous labeled data (i.e., HR samples), which are usually less accessible and way more expensive to acquire than the input data (i.e., LR samples). In many cases, the HR labels are not even available at all due to the resolution limit of the measurement techniques. In this work, we try to tackle this challenge and develop a CNN-SR solution without relying on HR data as training labels. A physics-informed learning strategy is adopted to enable label-free training in data-sparse/absent scenarios. The general idea is to leverage the (partially) known physics of the fluid flow (e.g., conservation laws and boundary conditions) to drive the CNN training such that the super-resolved flow information is learned from the flow governing equations instead of massive HR labels. Here, we consider fluid problems governed by the steady incompressible Navier–Stokes equations parameterized by μ,

R(u,p;μ)=0:={·u=0inΩp,(u·)u+1ρpν2u+bf=0inΩp,
(4)

where u is the velocity, p is the pressure, and ν and bf represents the viscosity and body force of the fluid flow, respectively. The flow solutions can be uniquely determined with given boundary condition (BC), B(u,p;μ)=0,onΩp. Since the CNN super-resolved flow fields should satisfy the governing equations, the training can be recast as a constrained optimization problem by minimizing the PDE residuals,

W̃c=argminWci=1ndpR(Sc(Ψ̂il;Wc))ΩpPDEbasedloss:Lp,s.t.B(Sc(Ψ̂il;Wc);μ)=0onΩp,
(5)

where Ψ̂il=u(Xl;μi) is the LR velocity field discretized on a coarse mesh Xl and Ψ̂ih=[u(Xh;μi),p(Xh;μi)]TSc(Ψ̂il;Wc) is the CNN super-resolved flow fields on a fine mesh Xh. To evaluate the PDE residuals on the discretized domain, we use convolution operations with the finite difference filters to compute the derivative terms in Eq. (4), and the details are given in  Appendix A. The boundary condition is strictly enforced into the CNN architecture, where the boundary operator B is discretized and imposed on the CNN-SR predictions in a hard manner using padding operations.68 In contrast to the traditional label-based data-driven approach, the number ndp of training samples is not constrained by the availability of HR data. Hence, the training space can be freely explored with a large number of LR data Ψ̂il, which are assumed to be very cheap to obtain. Moreover, the LR data can be further leveraged to facilitate the SR and inference. Namely, the CNN-SR output can assimilate the LR data to reversely determine unknown parameters (e.g., BCs) in a parametric manner (as shown in Fig. 1). However, the LR-data should only be assimilated strictly if the LR data are noise-free; otherwise, the network may tend to overfit the data noise. As an alternative, the CNN-SR output can be downsampled via the pooling or encoding process to build an LR data loss, which can assimilate LR data softly.

It is commonly known that the flow physics is governed by the Navier–Stokes equations R(u,p;μ)=0, but some of the physical parameters μ, such as inlet profiles or fluid properties, are unknown in many cases. On the other hand, it is possible to access additional observation data, which, however, is often spatially sparse and/or indirect to the quantity of interest. For example, the 3D full-field velocity information is obtained using 4D flow MRI techniques in cardiovascular applications, but the spatial resolution and SNR are unsatisfied and need enhancement.4 More accurate flow field data can be observed by 2D PC-MRI, which is only available on a limited number of 2D slices. The proposed physics-informed CNN-SR framework can naturally leverage these additional sparse observations to enable both forward super-resolution and inverse parameter determination in a unified manner. Here, we introduce a novel approach of assimilating additional sparse observation data to infer the under-determined physical conditions/parameters μsμ. First, the unknowns (e.g., Dirichlet boundary conditions) are parameterized as a trainable vector μs, which are incorporated into the SR learning architecture either through the equation-based loss function or strictly imposed boundary conditions. Second, the sparse observations Yobs are assimilated into the network in a hard manner, where the CNN-SR predictions are strictly enforced to be equal to the data at sparse locations by constructing the model output Ψc as

Fs2o(Ψc)=Fs2o(Ψ̂c)×0+Yobs,
(6)

where Ψ̂c is the CNN-SR raw output and Fs2o:ΨY indicates the state-to-observable map. Therefore, the constrained optimization for PDE-driven training is recast as

W̃c,μi=argminWc,μii=1ndpR(Sc(Ψ̂il;Wc);μi)ΩpPDEbasedloss:Lp,s.t.{B(Sc(Ψ̂il;Wc);μi)=0onΩp,Fs2o(Sc(Ψ̂il;Wc))Yobs=0,
(7)

where both the network parameter vector W̃c and physical parameter vector μi are inferred simultaneously via Adam optimizer. It worth mentioning that the proposed “hard” DA approach is based on the assumption that the observation data are relatively precise. When assimilating very noisy observations, the penalty-based “soft” approach used in the PINN51 should be employed to avoid overfitting the data noise.

Parametric deep learning solutions are attractive since the fast online inference speed of the DNN model can be leveraged for real-time predictions when the input LR and flow conditions/parameters are changed. However, it is impossible to conduct parametric SR when the physics is ill-posed (e.g., unknown boundary conditions). Namely, the inference should be formulated in a parametric setting as well along with the forward SR. To enable parametric data assimilation and inference, we propose a novel network structure that hybrids a multilayer perceptron (MLP) and CNN (Fig. 1). The unknown physical parameters μ introduced in Sec. II D become the output of the MLP model I:Ψ̂ilμi,

μi=I(Ψ̂il;WI),
(8)

where WI is the MLP network parameter vector. Substitution Eq. (8) into Eq. (7), we have the new constrained optimization as

W̃c,W̃I=argminWc,WIi=1ndpR(Sc(Ψ̂il;Wc),I(Ψ̂il;WI))ΩpPDEbasedloss:Lp,s.t.{B(Sc(Ψ̂il;Wc),I(Ψ̂il;WI))=0onΩp,Fs2o(Sc(Ψ̂il;Wc))Yobs=0,
(9)

the two networks are trained together by the unified physics-informed loss function. Once the model is sufficiently trained, it can parametrically infer the unknown physics and, meanwhile, super-resolve the flow field simultaneously.

A general limitation of CNNs is that they can only handle problems defined on rectangular domains with uniform grids since the convolution operations are originally designed for processing images described on uniform meshes. However, the geometries in most scientific applications are complex and irregular (e.g., subject-specific vessel geometries in cardiovascular applications). In order to perform physics-informed super-resolution on non-rectangular domains, we adopt the geometry-adaptive CNN formation proposed by Gao et al.,68 where the elliptic coordinate transformation is utilized to reformulate the PDE-constrained learning from the irregular physical domain (xΩp) to the regular reference domain (ξΩr). Particularly, the one-to-one coordinate transformation map G:ΩrΩp is numerically obtained by solving an elliptic problem, e.g., diffusion equations. The Jacobians of the map G are then computed to convert differential operators from the physical domain to the reference domain,

x=1J[(ξ)(yη)(η)(yξ)],
(10a)
y=1J[(η)(xξ)(ξ)(xη)],
(10b)

where coordinates of physical domain and reference domain are x=[x,y]T and ξ=[ξ,η]T, respectively; J=xξyηxηyξ is the determinant of the Jacobian matrix and metrics yη,yξ,xη; and xξ can be precomputed and remain constant for given G. Using elliptic coordinate transformation, the PDE-based loss function is reformulated on the reference domain, and thus the classic CNN backbone can be directly used for irregular geometries. For more details, see Ref. 68.

We demonstrate the physics-informed CNN-SR analysis on several internal flow cases relevant to cardiovascular applications. The LR input will be denoised and enhanced to the high-resolution field for both non-parametric (SR with a specific LR input) and parametric (SR with a set of different LR inputs that lie in a high-dimensional space) scenarios. We first study the flow field in a 2D vascular domain with a non-parametric setting, where the governing PDEs and boundary conditions are well defined. Moreover, we also investigate the scenario that the flow physics is partially known (e.g., the inlet boundary condition is unknown) to demonstrate the CNN-SR solution of unifying forward and inverse problems with additional observation data. Then, we present the parametric SR analysis for internal flows with a parameterized inlet velocity profile in a high-dimensional parameter space. Finally, we show the parametric SR and inference for internal flows in an idealized aneurysm geometry with parameterized inlet velocity profiles in a high-dimensional parameter space.

1. Synthesis of low-resolution, noisy data

Synthetic LR data are generated from finite volume (FV)-based CFD simulations on coarse meshes. The simulated LR velocity fields are corrupted by artificial measurement noises. The following two types of noise models are considered: (1) Gaussian noise and (2) non-Gaussian flow MRI noise.

  • Gaussian noise model: The LR velocity field Ψl is corrupted by an independent and identically distributed (i.i.d.) Gaussian noise factor ϵN(0,I) as
    Ψ̂l=Ψl·(I+cϵ),
    (11)

    where the parameter c[0,1] controls the noise level.

  • Non-Gaussian MRI noise model: To mimic the LR data obtained from the flow MRI, the five-point balanced phase-contrast method72 is employed to encode the CFD velocity field into the phase space by S=F(Ψl), where S is a complex matrix of five column vectors. Different levels of complex Gaussian noise are added to the complex data, and the synthetic noisy MR flow field can be then obtained via the inverse five-point map F1(·) following Ref. 4,
    Ŝ=S+c2|Ψl|·I(ϵ1+ϵ2i),Ψ̂l=F1(Ŝ),
    (12)

    where i is the imaginary unit, ϵ1,ϵ2i̇i̇dN(0,I), and c[0,1] controls the noise level. The Gaussian noise imposed in the phase space will become highly non-Gaussian once being mapped back to the physical velocity space, For more details of the forward and inverse five-point maps F(·) and F1(·), see Refs. 4, 72, and references therein.

2. Cases setup

To evaluate the CNN-SR performance, we generate high-resolution CFD data (Ψ̂h) as the reference. Moreover, the upsampled results from the bicubic interpolation will also be computed for comparison. Both the LR and HR CFD simulations are conducted using OpenFOAM,73 an open-source C++ library for FV simulations. The relative error metric e is defined as

e=Ψ̂cΨ̂hL2Ψ̂hL2.
(13)

Specifically, the FV solutions of the steady incompressible Navier–Stokes equations are solved using the semi-implicit method for pressure linked equation (SIMPLE) algorithms,74 where the Rhie and Chow interpolation with collocated grids is adopted to prevent the pressure–velocity decoupling.75 The nonlinear convection term is discretized based on the Gauss theorem with the second-order bounded linear upwind interpolation (i.e., Gauss linearUpwind Scheme in OpenFOAM), and the diffusion term is discretized using the central Gauss linear interpolation with the explicit non-orthogonal correction for surface normal gradients (i.e., Gauss linear corrected). The physics-informed CNN-SR model is implemented in PyTorch,76 and training is conducted on an NVIDIA GeForce RTX 2080 Graphics Processing Unit (GPU) card. The training histories for all the test cases are summarized in Fig. 20 of  Appendix C.

The proposed physics-informed CNN-SR model is constructed to super-resolve the LR flow field in a non-parametric setting. Namely, the DL model is trained to learn the LR-to-HR map, S:(Ψ̂l;μ)(Ψ̂h;μ), where the physical parameter μ is fixed. A 2D laminar flow with an irregular vascular geometry is investigated, as shown in Fig. 2. The flow starts at the bottom edge (i.e., inlet) and moves out at the upper edge (i.e., outlet), where the non-slip wall boundary condition is imposed on the left and right boundaries. The LR data are obtained from a coarse mesh of 126 cells, while the HR target mesh has 3773 cells. The CNN-SR will take the LR data as the input and spatially generates refined data by 30×.

FIG. 2.

(a) Coarse mesh and (b) fine mesh [the low-resolution mesh (126 cells) and high-resolution mesh (3773 cells)]. The LR input data are refined by 30×.

FIG. 2.

(a) Coarse mesh and (b) fine mesh [the low-resolution mesh (126 cells) and high-resolution mesh (3773 cells)]. The LR input data are refined by 30×.

Close modal

1. Known boundary condition

We first consider a non-parametric scenario with well-posed physics, where both the governing PDEs (i.e., incompressible Navier–Stokes equations) and boundary conditions are well defined. In particular, the inlet boundary condition is known as a constant profile u=[0,1] and the outlet is defined by u·n=0 and p =0, where n is the local wall-normal vector. The density ρ is 1, and the viscosity ν is 0.01. Figure 3 shows the CNN-SR velocity fields from the LR data with a 100% Gaussian noise (c =1.0). Due to the mesh coarseness, the LR field presents a mosaic pattern and provides very limited information. The SR solution directly upsampled by the bicubic interpolation is unsatisfactory since the large Gaussian noise makes the bicubic-SR solution highly unphysical. In contrast, the CNN-SR solution well agrees with the HR reference data (truth). The flow details of the boundary layer and velocity development can be accurately captured, where the large Gaussian noises are significantly reduced. The relative error of the CNN-SR field is 0.067, which is an order lower than that of the bicubic-SR result (0.520). The pointwise error maps are given in  Appendix D [Fig. 21(a)]. It shows that the CNN-SR solution has less error than the bicubic-SR one over the entire domain. Then, we also examine the velocity along the streamwise centerline and the wall shear stress (WSS) predicted by CNN-SR and bicubic-SR models, as shown in Fig. 4. The bicubic-SR results have a notably large discrepancy compared to the CFD reference, while the CNN-SR results agree with the CFD reference very well.

FIG. 3.

The super-resolved results of the LR input with the 100% Gaussian noise (c =1.0). The relative errors of the bicubic-SR and CNN-SR fields are 0.520 and 0.067, respectively.

FIG. 3.

The super-resolved results of the LR input with the 100% Gaussian noise (c =1.0). The relative errors of the bicubic-SR and CNN-SR fields are 0.520 and 0.067, respectively.

Close modal
FIG. 4.

The super-resolved centerline velocity and WSS results of the LR input with the 100% Gaussian noise (c =1.0).

FIG. 4.

The super-resolved centerline velocity and WSS results of the LR input with the 100% Gaussian noise (c =1.0).

Close modal

For the experiment with artificial MRI noises shown in Fig. 5, the LR velocity field becomes even more unphysical, and the flow information is overwhelmed by the large non-Gaussian MRI noises. Using bicubic interpolation does not show any improvement. Compared to the HR reference, the bicubic-SR velocity field barely captures any flow physics and has a large relative error of 0.617. However, the CNN-SR model, informed by the Navier–Stokes equations, can largely remove the MRI noises and generate an accurate super-resolved velocity field with a relative error of 0.066. By further investigating the pointwise error maps [Fig. 21(b)], we can see that the bicubic-SR solution becomes worse, while in the CNN-SR model, the performance remains the same as that with Gaussian noise. For the centerline velocity and WSS predictions (Fig. 6), the CNN-SR solution almost overlaps the CFD reference, which is much better than bicubic-SR results. The encouraging results show great promise of the proposed method for enhancing the spatial resolution of 4D flow MRI, for which the HR labels are often unavailable (see Table II).

TABLE II.

Summary of quantitative error analysis for non-parametric cases with the well-defined boundary conditions.

100% Gaussian noise100% MRI noise
SR modelVelocity errorWSS errorSR modelVelocity errorWSS error
CNN-SR 6.67×102 1.32×101 CNN-SR 6.66×102 1.33×101 
Bicubic-SR 5.2×101 1.67 Bicubic-SR 6.17×101 1.47 
100% Gaussian noise100% MRI noise
SR modelVelocity errorWSS errorSR modelVelocity errorWSS error
CNN-SR 6.67×102 1.32×101 CNN-SR 6.66×102 1.33×101 
Bicubic-SR 5.2×101 1.67 Bicubic-SR 6.17×101 1.47 
FIG. 5.

The super-resolved results of the LR input with the 100% MRI noise (c =1.0). The relative errors of bicubic-SR and CNN-SR fields are 0.617 and 0.066, respectively.

FIG. 5.

The super-resolved results of the LR input with the 100% MRI noise (c =1.0). The relative errors of bicubic-SR and CNN-SR fields are 0.617 and 0.066, respectively.

Close modal
FIG. 6.

The super-resolved centerline velocity and WSS results of the LR input with the 100% MRI noise (c =1.0), where BC is given.

FIG. 6.

The super-resolved centerline velocity and WSS results of the LR input with the 100% MRI noise (c =1.0), where BC is given.

Close modal

2. Unknown boundary condition

In this subsection, we demonstrate the capability of unifying the super-resolution and data assimilation for the situation where the physics is ill-posed (e.g., unknown boundary conditions), but additional sparse observation data are available. This scenario is quite common in cardiovascular applications. The 4D flow MRI techniques enable noninvasive and in vivo measurements of full-field blood flow information, whose spatial resolution, however, is too low to perform any quantitative analysis. Although the Navier–Stokes equations can be used to refine the LR data, the boundary conditions (e.g., inlet velocity field and outlet pressure distributions) are often not available in clinical practice. On the other hand, some sparse high-fidelity observations can be obtained by the 2D phase contrast MRI (PC-MRI) on a few 2D slices, which can be assimilated to infer the unknown boundaries. To mimic this scenario, we conduct a numerical experiment with the same setting as above, where the synthetic LR data corrupted by 100% MRI noises are super-resolved using the Navier–Stokes-informed CNN. In contrast to the previous example, we here assume that the true inlet velocity profile u=[0,100x(0.2x)] is unknown, but more accurate velocity observations are given only on four slices (2% of mesh grids), as an analog of the sparse 2-D PC-MRI data (see Fig. 7). Since the inlet boundary is discretized by 45 stencils and no prior correlation structure is imposed, we are solving a 45-dimensional inverse problem [i.e., the network directly searches the true inlet in the entire 45-dimensional space (45)]. It can be seen that not only the LR noisy flow field has been super-resolved to be in good agreement with the HR reference (e =0.029), but also the unknown inlet velocity profile (a 45-dimensional field) can be accurately recovered in a unified manner (see Table III). However, it also can be noticed that the inferred inlet profile slightly deviates from the ground truth near the wall, which might be due to following potential reasons: (1) the velocity value near the wall is relatively small, and thus the inference accuracy is accordingly low since the network optimization is conducted in an averaged manner (L2 norm); (2) the gradient is relatively large near the wall, where the inference performance typically downgrades; (3) the observation data are obtained from the finite volume (FV)-based simulation (OpenFOAM). However, the finite difference (FD) method is used to construct the physics-informed loss function. The inconsistency between FV and FD scheme is exaggerated near the boundary and then contributes to the inference discrepancy. We examined the pointwise error maps for bicubic-SR and CNN-SR solutions in Fig. 21(c), which indicates that the CNN-SR result is still very accurate even if the boundary condition is not given. For the predictions of centerline velocity and WSS (Fig. 8), the CNN-SR significantly outperforms the bicubic-SR in terms of accuracy. Note that we have tested many different realizations of the data noise, and the performance remains the same. Moreover, the minimum measurement data required to infer the inlet depends on the sensor placement. Based on our test, if sensors are randomly placed, observation data on at least 2%7% of total grid points can uncover the unknown boundary condition.

TABLE III.

Summary of quantitative error analysis for the non-parametric case with ill-posed physics.

100% MRI noise
SR modelVelocity errorWSS errorUnknown inlet
CNN-SR 2.9×102 1.32×101 6.21×102 
Bicubic-SR 7.9×101 3.09 Unavailable 
100% MRI noise
SR modelVelocity errorWSS errorUnknown inlet
CNN-SR 2.9×102 1.32×101 6.21×102 
Bicubic-SR 7.9×101 3.09 Unavailable 
FIG. 7.

The super-resolved results of the LR input with the 100% MRI noise (c =1.0), where inlet boundary is unknown. The sparse velocity observations (on 2% of mesh grids) are assimilated to infer the inlet velocity profile (a 45-dimensional field).

FIG. 7.

The super-resolved results of the LR input with the 100% MRI noise (c =1.0), where inlet boundary is unknown. The sparse velocity observations (on 2% of mesh grids) are assimilated to infer the inlet velocity profile (a 45-dimensional field).

Close modal
FIG. 8.

The super-resolved centerline velocity and WSS results of the LR input with the 100% MRI noise (c =1.0), where BC is unknown.

FIG. 8.

The super-resolved centerline velocity and WSS results of the LR input with the 100% MRI noise (c =1.0), where BC is unknown.

Close modal

The proposed CNN-SR solution can be applied for flow super-resolution in the parametric setting, leveraging the powerful interpolatory capability of DL models in high-dimensional parameter space. Namely, the DL model is trained to capture the operator, S:(Ψ̂l;μ)(Ψ̂h;μ), where the physical parameters μ could vary in a high-dimensional space. Even with physical parameter μ being unknown, the DL models can be trained as S:Ψ̂lΨ̂h and I:Ψ̂lμ. For both situations, once fully trained, the CNN-SR model can be treated as a cost-effective surrogate model that takes the LR data to produce HR solutions and much more reliable than the bicubic-SR model. Since the low-fidelity simulations or experiments are relatively cheap to conduct, the trained CNN-SR surrogate could significantly facilitate many query applications, e.g., uncertainty propagation, optimization, and sample-based Bayesian inference.

1. Parametric super-resolution with well-posed physics

To demonstrate parametric SR capability, we consider internal flows with spatially varying inflow boundary conditions (including non-zero secondary flow). The CNN-SR is trained to refine the LR flow fields by 400×. As shown in Fig. 9, the inlet velocity field u(x) is set on the left edge (x =0), while the zero-pressure outlet is prescribed on the right edge (x =1). Both the top and bottom edges (y=0,y=1) are set as non-slip walls. The density ρ is 1, and the viscosity ν is 0.01. Each component of the inlet velocity field (u=[u(x),v(x)]T) is modeled by a scalar stationary Gaussian process,

f(x)GP(0,K(x,x)),K(x,x)=σ2exp(|xx|2l2),inΩinlet
(14)

where K(x,x) is the exponential kernel function and l and σ represent the homogeneous length scale and standard deviation of the Gaussian random field. Here, we set the length scale l =0.1 and the standard deviation σ=0.33. To represent the Gaussian process in a compact form, we use Karhunen–Loeve (KL) expansion,

f(x)=i=1nkλiϕi(x)ωi,
(15)

where λ and ϕ(x) are eigenvalues and eigenfunctions of the kernel, respectively, and ωi are uncorrelated random variables with zero mean and unit variance. We further truncate the KL expansion with a finite number (nk = 10) of basis to capture 96% energy of the random field. The streamwise velocity field (u) and the transverse velocity field (v) are defined as

u(x)=1+f(x),v(x)=f(x).
(16)
FIG. 9.

(a) Coarse mesh and (b) fine mesh [the low-resolution input mesh (10 × 10) and the high-resolution output mesh (200 × 200)]. The LR data will be refined by 400×.

FIG. 9.

(a) Coarse mesh and (b) fine mesh [the low-resolution input mesh (10 × 10) and the high-resolution output mesh (200 × 200)]. The LR data will be refined by 400×.

Close modal

Hence, the inlet boundary condition is parameterized by a 20-dimensional parameter vector,

μ=[ω1,,ω20]T20,
(17)

where ω1,,ω10 parameterize the streamwise velocity and ω11,,ω20 are for the transverse velocity field. The first 10 KL modes are shown in Fig. 19.

The CNN-SR model is trained on 15 inlet samples randomly drawn from the Gaussian process defined by Eq. (14), where the PDE-based loss function is minimized with 103 iterations. Once the CNN-SR model is trained, it can super-resolve the LR flow fields of these 15 inlet samples and be used as a surrogate model to rapidly refine any LR data with unseen inlets in the 20-dimensional parameter space. To evaluate the model's generalizability, we generate 985 new testing inlets that are unseen during the training. Figure 10 shows the CNN-SR results of 16 samples randomly selected from the test set, where noise-free LR data are used as the input. Though without any noise, the LR data contain very limited flow information because of the mesh coarseness (10 × 10). Both bicubic interpolation and trained CNN-SR model can spatially refine the LR data by 400× and show improvements. However, the CNN-SR results reveal more flow details and have better agreement with the HR reference. Specifically, the boundary layer in the CNN-SR model can be captured more accurately than the bicubic-SR model does. Figure 11 shows the WSS along the bottom wall from both CNN-SR and bicubic-SR models. It is clear that the CNN-SR model outperforms the bicubic-SR model and agrees with the ground truth very well. This is also true for the upper wall. The error maps (Fig. 22) also indicate that, in the near-wall region, the error of the bicubic-SR solution is always significantly greater than that of the CNN-SR solution. However, for the centerline velocity, the SR performance is comparable for CNN-SR and bicubic-SR models when LR data do not contain any noise.

FIG. 10.

The CNN-SR results of noisy-free LR data on a subset of testing inlet samples randomly drawn from the Gaussian process.

FIG. 10.

The CNN-SR results of noisy-free LR data on a subset of testing inlet samples randomly drawn from the Gaussian process.

Close modal
FIG. 11.

The SR bottom WSS results of noise-free LR data on a subset of testing inlet samples randomly drawn from the Gaussian process.

FIG. 11.

The SR bottom WSS results of noise-free LR data on a subset of testing inlet samples randomly drawn from the Gaussian process.

Close modal

When the LR data contain noises, the superiority of the CNN-SR solution becomes more significant compared to the bicubic interpolation. To demonstrate this merit, we conduct another numerical experiment, where the LR data are corrupted with 20% Gaussian noise (noise level c =0.2). The CNN-SR model is trained on the LR data by resampling the noise at every iteration to recognize the noisy inputs. Figure 12 shows the super-resolution results of noisy LR data. It is apparent that the bicubic-SR velocity fields are visually unphysical by directly interpolating the data noises. The SR performance of the physics-informed CNN model still remains excellent as it accurately refined the spatial resolution of the LR data by 400×, and the SR results agree with the HR reference very well on all testing samples. The error maps (Fig. 23) show that, although the noise influences both models, the CNN-SR model has better accuracy, particularly for the near-wall region. This can also be observed in the WSS comparisons (Fig. 13), where CNN-SR notably outperforms the bicubic model, showing the robustness.

FIG. 12.

The CNN-SR results of noisy LR data on a subset of testing inlet samples randomly drawn from the Gaussian process.

FIG. 12.

The CNN-SR results of noisy LR data on a subset of testing inlet samples randomly drawn from the Gaussian process.

Close modal
FIG. 13.

The bottom WSS results of noisy LR data on a subset of testing inlet samples randomly drawn from the Gaussian process. It is also true for the upper wall.

FIG. 13.

The bottom WSS results of noisy LR data on a subset of testing inlet samples randomly drawn from the Gaussian process. It is also true for the upper wall.

Close modal

Figure 14 shows the mean relative errors vs LR data noise levels for both CNN-SR and bicubic-SR results over 985 testing samples. We can see the SR performance of bicubic interpolation remarkably deteriorates as the input noise level increases. When the LR data contain 100% noises, the naive bicubic interpolation completely fails as the relative error grows up to nearly 50%. In comparison, CNN-SR solutions are less sensitive to the growth of data noise. Although showing a similar trend, the CNN-SR solutions' relative error remains small (less than 20%), even if the data noise level reaches to 100%.

FIG. 14.

(a) Velocity and (b) wall shear stress. The relative errors of CNN and Bicubic SR results of LR inputs with different noise levels over 985 testing samples.

FIG. 14.

(a) Velocity and (b) wall shear stress. The relative errors of CNN and Bicubic SR results of LR inputs with different noise levels over 985 testing samples.

Close modal

The computational costs of a single model evaluation for the HR CFD and CNN-SR models are listed in Table IV. For a single run on each parameter point, the speedup of the CNN-SR model is more than 3000 times compared to the CFD simulation. It shows the potential of using the CNN-SR model as a surrogate for massive queries in the high-dimensional input space, which could enable or facilitate ensemble-based uncertainty quantification or inverse optimization.

TABLE IV.

Computational costs of the online evaluation of the HR CFD and CNN-SR models.

HR CFDCNN
Wall-clock time (s) 1.189×103 
Hardware Intel Xeon(R) Gold 6138 GeForce RTX 2080 
Speedup 3364  
HR CFDCNN
Wall-clock time (s) 1.189×103 
Hardware Intel Xeon(R) Gold 6138 GeForce RTX 2080 
Speedup 3364  

Finally, we demonstrate parametric SR solutions where boundary conditions are unknown and can be recovered simultaneously. Similar to the non-parametric case in Sec. III B 2, the model will leverage sparse or LR data to deal with ill-posed physics. However, compared to the non-parametric example, a more challenging task here is to parametrically infer unknown BCs given different LR inputs. It is noteworthy that HR labels are still not available, but the trained model is expected to super-resolve any new LR flow fields and rapidly recover unknown BCs accordingly. In this case, the flow in an idealized aneurysm geometry is studied. The LF input is obtained by downsampling the HF solution data (Fig. 15). Only 7% of the HR grid points are downsampled to form the LF data. The inlet velocity field u(x)=[0,v(x)] is prescribed on the bottom edge (y =0), while the zero-pressure outlet is prescribed on the top edge (y =1). The left and right edges are set as non-slip walls. The streamwise inlet profile v(x) is unknown, which will be modeled as a Gaussian process (GP) combined with a parabolic profile,

v(x)=GP(0,K)4(x0.5)(x+0.5),K(x,x)=σ2exp(|xx|2l2)inΩinlet,
(18)

where the hyperparameters of the exponential kernel function is set as l =0.2 and σ=0.3. Similarly, we use the KL expansion [Eq. (15)] to represent this stochastic field with 10 truncated modes (covers over 99% energy). Namely, the unknown inlet BC is parameterized by a 10-dimensional vector, μ=[ω1,,ω10]10. An MLP model is applied to parametrically infer the unknown vector for different LR inputs. The MLP contains three identical hidden layers, each of which has 100 neurons. The CNN-MLP SR model is trained on 250 inlet samples with 3500 epochs. Once being fully trained, the model not only can super-resolve the LR fields but also can infer the corresponding unknown inlet profiles given any randomly sampled LR data from the high-dimensional parameter space. To test the generalizability, we generate 2500 new testing samples. Figure 16 shows the SR results of 16 samples randomly selected from the test set. Although the contours generated by bicubic-SR model look reasonably similar to the reference, the pointwise error maps shown in Fig. 24 clearly indicate that the bicubic-SR model contains a relatively large error, particularly in the near-wall region. In contrast, the errors of CNN-SR results are very low over all test samples. Figure 17 shows that the CNN-SR model can accurately capture the WSS and significantly outperforms the bicubic-SR model. Finally, we examine the performance of recovering unknown boundary conditions in the parametric setting. Figure 18 shows that all the unknown inlet profiles are precisely inferred and agree with the truth very well (Table V).

FIG. 15.

(a) Coarse mesh and (b) is fine mesh. The downsampled coarse mesh as the input mesh has 169 points, and the high-resolution output mesh has 2401 points. The LR input data are refined by 15×.

FIG. 15.

(a) Coarse mesh and (b) is fine mesh. The downsampled coarse mesh as the input mesh has 169 points, and the high-resolution output mesh has 2401 points. The LR input data are refined by 15×.

Close modal
FIG. 16.

The CNN-SR and bicubic-SR results of downsampled LR data on a subset of testing inlet samples randomly drawn from the Gaussian process.

FIG. 16.

The CNN-SR and bicubic-SR results of downsampled LR data on a subset of testing inlet samples randomly drawn from the Gaussian process.

Close modal
FIG. 17.

The left wall shear stress of CNN-SR and bicubic-SR results on a subset of testing inlet samples randomly drawn from the Gaussian process. It is also true for the right wall shear stress.

FIG. 17.

The left wall shear stress of CNN-SR and bicubic-SR results on a subset of testing inlet samples randomly drawn from the Gaussian process. It is also true for the right wall shear stress.

Close modal
FIG. 18.

The inferred unknown inlet boundary conditions on a subset of testing samples randomly drawn from the Gaussian process. Note that the bicubic-SR model is not able to infer the unknowns.

FIG. 18.

The inferred unknown inlet boundary conditions on a subset of testing samples randomly drawn from the Gaussian process. Note that the bicubic-SR model is not able to infer the unknowns.

Close modal
FIG. 19.

The first 10 KL modes of the Gaussian random fields for the spatially varying inlet boundary conditions.

FIG. 19.

The first 10 KL modes of the Gaussian random fields for the spatially varying inlet boundary conditions.

Close modal
TABLE V.

Summary of quantitative error analysis for the parametric case with ill-posed physics.

Parametric SR and inference
SR modelVelocity errorWSS errorUnknown inlet
CNN-SR 1.39×102 1.65×102 3.82×102 
Bicubic-SR 1.01×101 7.2×101 Unavailable 
Parametric SR and inference
SR modelVelocity errorWSS errorUnknown inlet
CNN-SR 1.39×102 1.65×102 3.82×102 
Bicubic-SR 1.01×101 7.2×101 Unavailable 

This work proposed a general label-free physics-informed deep learning-based SR framework that can handle various SR applications. It is the first time to comprehensively study SR in many different scenarios, such as LR inputs with Gaussian noise, non-Gaussian noise, or downsampled measurements, given either well-posed physics or ill-posed physics. The key novelty of the proposed method is that it unifies forward SR and inverse data assimilation in a parametric setting. Namely, once fully trained, the model can rapidly super-resolve any newly provided LR field and infer the corresponding unknown boundary condition without the need to re-train the network, enabling real-time online applications. Although showing great promise, the current framework has several limitations and technical challenges. We list the current limitations and also provide our perspective on how to improve in future work. (1) This paper is focused on the steady-state problem. However, the cardiovascular flow is usually unsteady in practice. To handle dynamic flow SR problems, the standard forward/backward Euler or Runge–Kutta methods can be used to formulate temporal derivatives.77 Also, more advanced learning architectures should be designed based on sequential neural nets such as convolutional long short term memory (convLSTM) networks78 or transformer.79 (2) Although the elliptic coordinate transformation enables the CNN-SR model to work on simple irregular domains. It is still very challenging to tackle more complex geometries, e.g., 3D patient-specific geometries. This limitation potentially can be addressed by leveraging geometric deep learning techniques, e.g., graph convolutional neural networks.80 (3) The current framework cannot accurately recover the hidden pressure field because of the implicit coupling of pressure and velocity (i.e., the pressure appears as a source term in the momentum equations). The governing equation needs to be reformulated to strongly couple the pressure and velocity predictions based on, e.g., elliptic equation of pressure81 and Rich–Chow interpolation.75 

In this paper, we proposed a novel physics-informed deep learning solution for the spatial super-resolution of flow fields. Leveraging the physical laws and boundary conditions of fluid flows, the training of the CNN-SR model only needs LR samples instead of its HR counterparts as labels. Once sufficiently trained, the CNN-SR model can produce the spatially refined flow field, given a noisy LR input in the parameter space. When the flow boundary conditions are unknown, the proposed framework can naturally assimilate additional sparse observation data to simultaneously enable forward SR and inverse determination of unknown boundary conditions parametrically. The effectiveness and merit of the proposed CNN-SR model have been demonstrated on a number of non-parametric and parametric spatial flow SR problems relevant to cardiovascular applications, where both downsampled input or input with Gaussian and non-Gaussian MRI noises are investigated. In particular, we demonstrated that the CNN-SR model, by training on only 15 LR input samples, is able to accurately refine the spatial resolution by 400× for the flow fields with any new inlet BCs sampled in the 20-dimensional parameter space (μ20). Compared to the standard SR approach based on the bicubic interpolation, the CNN-SR model shows significantly higher accuracy and robustness. Compared to the standard FV simulation, the single sample speedup is more than 103 times, showing its potential for many-query applications. The current work is only focused on the spatial super-resolution for 2D flow fields. Future work will extend the framework for both spatial and temporal super-resolution of unsteady fluid flows in 3D complex domains.

The author gratefully acknowledge the funds from the National Science Foundation (NSF Contract No. CMMI-1934300) in supporting this project. The authors thank the anonymous reviewers for their insightful comments and suggestions to improve the quality of this paper.

The authors declare that they have no conflict of interest.

The data that support the findings of this study are openly available in GitHub at https://github.com/Jianxun-Wang/PICNNSR.

The convolution filters for gradient and Laplacian terms are stored as 4D tensors shown as below:

uxfilter=[[[[0000000000180810000000000]]]]×112δx,
(A1)
uyfilter=[[[[0010000800000000080000100]]]]×112δy,
(A2)

and

(2ux2+2uy2)filter=[[[[001000016001166016100160000100]]]]×112δxδy,δx=δy.
(A3)

The first 10 KL modes of the Gaussian random fields for the spatially varying inlet boundary conditions (see Fig. 19).

Figure 20 shows learning histories of all test cases.

FIG. 20.

Training histories for nonparametric and parametric super-resolution cases: (a) nonparametric (BC known, Gauss), (b) nonparametric (BC known, MRI), (c) nonparametric (BC unknown, MRI), (d) parametric (BC known, noise-free), (e) parametric (BC known, Gauss), and (f) parametric (BC unknown, downsampled LF).

FIG. 20.

Training histories for nonparametric and parametric super-resolution cases: (a) nonparametric (BC known, Gauss), (b) nonparametric (BC known, MRI), (c) nonparametric (BC unknown, MRI), (d) parametric (BC known, noise-free), (e) parametric (BC known, Gauss), and (f) parametric (BC unknown, downsampled LF).

Close modal

Pointwise error maps of non-parametric CNN-SR and bicubic-SR velocities, where (a) LR with the 100% Gaussian noise and known BC, (b) LR with the 100% Gaussian noise and known BC, and (c) LR with the 100% MRI noise and unknown BC. Pointwise error maps of parametric CNN-SR and bicubic-SR velocities with noise-free LR data on the test set of inlets randomly drawn from the Gaussian process (parametric square-duct case). Pointwise error maps of parametric CNN-SR and bicubic-SR velocities with noisy LR data on the test set of inlets randomly drawn from the Gaussian process (parametric square-duct case). Pointwise error map of CNN-SR and bicubic-SR results with downsampled LR data on a subset of testing inlet samples randomly drawn from the Gaussian process (parametric aneurysm case).

FIG. 21.

Pointwise error maps of non-parametric CNN-SR and bicubic-SR velocities, where (a) LR with the 100% Gaussian noise and known BC, (b) LR with the 100% Gaussian noise and known BC, and (c) LR with the 100% MRI noise and unknown BC.

FIG. 21.

Pointwise error maps of non-parametric CNN-SR and bicubic-SR velocities, where (a) LR with the 100% Gaussian noise and known BC, (b) LR with the 100% Gaussian noise and known BC, and (c) LR with the 100% MRI noise and unknown BC.

Close modal
FIG. 22.

Pointwise error maps of parametric CNN-SR and bicubic-SR velocities with noise-free LR data on the test set of inlets randomly drawn from the Gaussian process (parametric square-duct case).

FIG. 22.

Pointwise error maps of parametric CNN-SR and bicubic-SR velocities with noise-free LR data on the test set of inlets randomly drawn from the Gaussian process (parametric square-duct case).

Close modal
FIG. 23.

Pointwise error maps of parametric CNN-SR and bicubic-SR velocities with noisy LR data on the test set of inlets randomly drawn from the Gaussian process (parametric square-duct case).

FIG. 23.

Pointwise error maps of parametric CNN-SR and bicubic-SR velocities with noisy LR data on the test set of inlets randomly drawn from the Gaussian process (parametric square-duct case).

Close modal
FIG. 24.

Pointwise error map of CNN-SR and bicubic-SR results with downsampled LR data on a subset of testing inlet samples randomly drawn from the Gaussian process (parametric aneurysm case).

FIG. 24.

Pointwise error map of CNN-SR and bicubic-SR results with downsampled LR data on a subset of testing inlet samples randomly drawn from the Gaussian process (parametric aneurysm case).

Close modal
1.
A.
Pollard
,
L.
Castillo
,
L.
Danaila
, and
M.
Glauser
,
Whither Turbulence and Big Data in the 21st Century?
(
Springer
,
2016
).
2.
C. M.
Lawley
,
K. M.
Broadhouse
,
F. M.
Callaghan
,
D. S.
Winlaw
,
G. A.
Figtree
, and
S. M.
Grieve
, “
4D flow magnetic resonance imaging: Role in pediatric congenital heart disease
,”
Asian Cardiovasc. Thorac. Ann.
26
(
1
),
28
37
(
2018
).
3.
Z.
Stankovic
,
B. D.
Allen
,
J.
Garcia
,
K. B.
Jarvis
, and
M.
Markl
, “
4D flow imaging with MRI
,”
Cardiovasc. Diagn. Ther.
4
(
2
),
173
(
2014
).
4.
F.
Ong
,
M.
Uecker
,
U.
Tariq
,
A.
Hsiao
,
M. T.
Alley
,
S. S.
Vasanawala
, and
M.
Lustig
, “
Robust 4D flow denoising using divergence-free wavelet transform
,”
Magn. Reson. Med.
73
(
2
),
828
842
(
2015
).
5.
M. F.
Fathi
,
A.
Bakhshinejad
,
A.
Baghaie
,
D.
Saloner
,
R. H.
Sacho
,
V. L.
Rayz
, and
R. M.
D'Souza
, “
Denoising and spatial resolution enhancement of 4D flow MRI using proper orthogonal decomposition and lasso regularization
,”
Comput. Med. Imaging Graph
70
,
165
120
(
2018
).
6.
F. M.
Callaghan
and
S. M.
Grieve
, “
Spatial resolution and velocity field improvement of 4D-flow MRI
,”
Magn. Reson. Med.
78
(
5
),
1959
1968
(
2017
).
7.
D.
Venturi
and
G. E.
Karniadakis
, “
Gappy data and reconstruction procedures for flow past a cylinder
,”
J. Fluid Mech.
519
,
315
336
(
2004
).
8.
T.
Bui-Thanh
,
M.
Damodaran
, and
K.
Willcox
, “
Proper orthogonal decomposition extensions for parametric applications in compressible aerodynamics
,” in
21st AIAA Applied Aerodynamics Conference
(
AIAA
,
2003
), p.
4213
.
9.
B.
Podvin
,
Y.
Fraigneau
,
F.
Lusseyran
, and
P.
Gougat
, “
A reconstruction method for the flow past an open cavity
,”
J. Fluids Eng.
128
(
3
),
531
540
(
2006
).
10.
A.
Yakhot
,
T.
Anor
, and
G. E.
Karniadakis
, “
A reconstruction method for gappy and noisy arterial flow data
,”
IEEE Trans. Med. Imaging
26
(
12
),
1681
1697
(
2007
).
11.
A. I.
Moreno
,
A. A.
Jarzabek
,
J. M.
Perales
, and
J. M.
Vega
, “
Aerodynamic database reconstruction via gappy high order singular value decomposition
,”
Aerosp. Sci. Technol.
52
,
115
128
(
2016
).
12.
M.
Mifsud
,
A.
Vendl
,
L.-U.
Hansen
, and
S.
Görtz
, “
Fusing wind-tunnel measurements and CFD data using constrained gappy proper orthogonal decomposition
,”
Aerosp. Sci. Technol.
86
,
312
326
(
2019
).
13.
P. J.
Schmid
, “
Dynamic mode decomposition of numerical and experimental data
,”
J. Fluid Mech.
656
,
5
28
(
2010
).
14.
J. H.
Tu
,
C. W.
Rowley
,
D. M.
Luchtenburg
,
S. L.
Brunton
, and
J. N.
Kutz
, “
On dynamicx mode decomposition: Theory and applications
,”
J. Comput. Dyn.
1
,
391
(
2014
).
15.
J. L.
Callaham
,
K.
Maeda
, and
S. L.
Brunton
, “
Robust flow reconstruction from limited measurements via sparse representation
,”
Phys. Rev. Fluids
4
(
10
),
103907
(
2019
).
16.
K.
Manohar
,
B. W.
Brunton
,
J. N.
Kutz
, and
S. L.
Brunton
, “
Data-driven sparse sensor placement for reconstruction: Demonstrating the benefits of exploiting known patterns
,”
IEEE Control Syst. Mag.
38
(
3
),
63
86
(
2018
).
17.
D. P.
Foures
,
N.
Dovetta
,
D.
Sipp
, and
P. J.
Schmid
, “
A data-assimilation method for Reynolds-averaged Navier–Stokes-driven mean flow reconstruction
,”
J. Fluid Mech.
759
,
404
431
(
2014
).
18.
B.
Combès
,
D.
Heitz
,
A.
Guibert
, and
E.
Mémin
, “
A particle filter to reconstruct a free-surface flow from a depth camera
,”
Fluid Dyn. Res.
47
(
5
),
051404
(
2015
).
19.
R.
Kikuchi
,
T.
Misaka
, and
S.
Obayashi
, “
Assessment of probability density function based on POD reduced-order model for ensemble-based data assimilation
,”
Fluid Dyn. Res.
47
(
5
),
051403
(
2015
).
20.
V.
Mons
,
J.-C.
Chassaing
,
T.
Gomez
, and
P.
Sagaut
, “
Reconstruction of unsteady viscous flows using data assimilation schemes
,”
J. Comput. Phys.
316
,
255
280
(
2016
).
21.
S.
Symon
,
N.
Dovetta
,
B. J.
McKeon
,
D.
Sipp
, and
P. J.
Schmid
, “
Data assimilation of mean velocity from 2D PIV measurements of flow over an idealized airfoil
,”
Exp. Fluids
58
(
5
),
61
(
2017
).
22.
J.-X.
Wang
and
H.
Xiao
, “
Data-driven CFD modeling of turbulent flows through complex structures
,”
Int. J. Heat Fluid Flow
62
,
138
149
(
2016
).
23.
H.
Xiao
,
J.-L.
Wu
,
J.-X.
Wang
,
R.
Sun
, and
C.
Roy
, “
Quantifying and reducing model-form uncertainties in Reynolds-averaged Navier–Stokes simulations: A data-driven, physics-informed Bayesian approach
,”
J. Comput. Phys.
324
,
115
136
(
2016
).
24.
S. L.
Brunton
,
B. R.
Noack
, and
P.
Koumoutsakos
, “
Machine learning for fluid mechanics
,”
Annu. Rev. Fluid Mech.
52
,
477
508
(
2020
).
25.
M.
Brenner
,
J.
Eldredge
, and
J.
Freund
, “
Perspective on machine learning for advancing fluid mechanics
,”
Phys. Rev. Fluids
4
(
10
),
100501
(
2019
).
26.
J.
Ling
,
A.
Kurzawski
, and
J.
Templeton
, “
Reynolds averaged turbulence modelling using deep neural networks with embedded invariance
,”
J. Fluid Mech.
807
,
155
166
(
2016
).
27.
K.
Duraisamy
,
G.
Iaccarino
, and
H.
Xiao
, “
Turbulence modeling in the age of data
,”
Annu. Rev. Fluid Mech.
51
,
357
377
(
2019
).
28.
J.-X.
Wang
,
J.-L.
Wu
, and
H.
Xiao
, “
Physics-informed machine learning approach for reconstructing Reynolds stress modeling discrepancies based on DNS data
,”
Phys. Rev. Fluids
2
(
3
),
034603
(
2017
).
29.
J.-X.
Wang
,
J.
Huang
,
L.
Duan
, and
H.
Xiao
, “
Prediction of Reynolds stresses in high-Mach-number turbulent boundary layers using physics-informed machine learning
,”
Theor. Comput. Fluid Dyn.
33
(
1
),
1
19
(
2019
).
30.
M. I.
Zafar
,
H.
Xiao
,
M. M.
Choudhari
,
F.
Li
,
C.-L.
Chang
,
P.
Paredes
, and
B.
Venkatachari
, “
Convolutional neural network for transition modeling based on linear stability theory
,” arXiv:2005.02599 (
2020
).
31.
K.
Fukami
,
Y.
Nabae
,
K.
Kawai
, and
K.
Fukagata
, “
Synthetic turbulent inflow generator using machine learning
,”
Phys. Rev. Fluids
4
(
6
),
064603
(
2019
).
32.
J.
Kim
and
C.
Lee
, “
Deep unsupervised learning of turbulence for inflow generation at various Reynolds numbers
,”
J. Comput. Phys.
406
,
109216
(
2020
).
33.
L.
Sun
,
H.
Gao
,
S.
Pan
, and
J.-X.
Wang
, “
Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data
,”
Comput. Methods Appl. Mech. Eng.
361
,
112732
(
2020
).
34.
R.
Maulik
,
K.
Fukami
,
N.
Ramachandra
,
K.
Fukagata
, and
K.
Taira
, “
Probabilistic neural networks for fluid flow surrogate modeling and data recovery
,”
Phys. Rev. Fluids
5
(
10
),
104401
(
2020
).
35.
H.
Gao
,
J.-X.
Wang
, and
M. J.
Zahr
, “
Non-intrusive model reduction of large-scale, nonlinear dynamical systems using deep learning
,”
Physica D
412
,
132614
(
2020
).
36.
C.
Ledig
,
L.
Theis
,
F.
Huszár
,
J.
Caballero
,
A.
Cunningham
,
A.
Acosta
,
A.
Aitken
,
A.
Tejani
,
J.
Totz
,
Z.
Wang
 et al, “
Photo-realistic single image super-resolution using a generative adversarial network
,”
in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(IEEE,
2017
), pp.
4681
4690
.
37.
K.
Fukami
,
K.
Fukagata
, and
K.
Taira
, “
Machine learning based spatio-temporal super resolution reconstruction of turbulent flows
,” arXiv:2004.11566 (
2020
).
38.
K.
Fukami
,
K.
Fukagata
, and
K.
Taira
, “
Super-resolution analysis with machine learning for low-resolution flow data
,” in
11th International Symposium on Turbulence and Shear Flow Phenomena (TSFP)
(
2019
).
39.
K.
Fukami
,
K.
Fukagata
, and
K.
Taira
, “
Super-resolution reconstruction of turbulent flows with machine learning
,” arXiv:1811.11328 (
2018
).
40.
Y.
Liu
,
C.
Ponce
,
S. L.
Brunton
, and
J. N.
Kutz
, “
Multiresolution convolutional autoencoders
,” arXiv:2004.04946 (
2020
).
41.
Z.
Deng
,
C.
He
,
Y.
Liu
, and
K. C.
Kim
, “
Super-resolution reconstruction of turbulent velocity fields using a generative adversarial network-based artificial intelligence framework
,”
Phys. Fluids
31
(
12
),
125111
(
2019
).
42.
M.
Bode
,
M.
Gauding
,
Z.
Lian
,
D.
Denker
,
M.
Davidovic
,
K.
Kleinheinz
,
J.
Jitsev
, and
H.
Pitsch
, “
Using physics-informed super-resolution generative adversarial networks for subgrid modeling in turbulent reactive flows
,” arXiv:1911.11380 (
2019
).
43.
K.
Bai
,
W.
Li
,
M.
Desbrun
, and
X.
Liu
, “
Dynamic upsampling of smoke through dictionary-based learning
,” arXiv:1910.09166 (
2019
).
44.
F. J.
Gonzalez
and
M.
Balajewicz
, “
Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems
,” arXiv:1808.01346 (
2018
).
45.
Y.
Xie
,
E.
Franz
,
M.
Chu
, and
N.
Thuerey
, “
tempoGAN: A temporally coherent, volumetric GAN for super-resolution fluid flow
,”
ACM Trans. Graph.
37
(
4
),
1
15
(
2018
).
46.
B.
Liu
,
J.
Tang
,
H.
Huang
, and
X.-Y.
Lu
, “
Deep learning methods for super-resolution reconstruction of turbulent flows
,”
Phys. Fluids
32
(
2
),
025105
(
2020
).
47.
M.
Werhahn
,
Y.
Xie
,
M.
Chu
, and
N.
Thuerey
, “
A multi-pass GAN for fluid flow super-resolution
,”
Proc. ACM Comput. Graph. Interact. Tech.
2
(
2
),
1
21
(
2019
).
48.
L.
Guo
,
S.
Ye
,
J.
Han
,
H.
Zheng
,
H.
Gao
,
D. Z.
Chen
,
J.-X.
Wang
, and
C.
Wang
, “
SSR-VFD: Spatial super-resolution for vector field data analysis and visualization
,” in
2020 IEEE Pacific Visualization Symposium (PacificVis)
(
IEEE
,
2020
), pp.
71
80
.
49.
N. B.
Erichson
,
L.
Mathelin
,
Z.
Yao
,
S. L.
Brunton
,
M. W.
Mahoney
, and
J. N.
Kutz
, “
Shallow learning for fluid flow reconstruction with limited sensors and limited data
,” arXiv:1902.07358 (
2019
).
50.
E.
Ferdian
,
A.
Suinesiaputra
,
D. J.
Dubowitz
,
D.
Zhao
,
A.
Wang
,
B.
Cowan
, and
A. A.
Young
, “
4DFlowNet: Super-resolution 4D flow MRI using deep learning and computational fluid dynamics
,”
Front. Phys.
8
,
138
(
2020
).
51.
M.
Raissi
,
P.
Perdikaris
, and
G.
Karniadakis
, “
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
,”
J. Comput. Phys.
378
,
686
707
(
2019
).
52.
V.
Dwivedi
,
N.
Parashar
, and
B.
Srinivasan
, “
Distributed learning machines for solving forward and inverse problems in partial differential equations
,”
Neurocomputing
420
,
299
316
(
2021
).
53.
Y.
Zhu
,
N.
Zabaras
,
P.-S.
Koutsourelakis
, and
P.
Perdikaris
, “
Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data
,”
J. Comput. Phys.
394
,
56
81
(
2019
).
54.
N.
Geneva
and
N.
Zabaras
, “
Modeling the dynamics of PDE systems with physics-constrained deep auto-regressive networks
,”
J. Comput. Phys.
403
,
109056
(
2020
).
55.
R.
Zhang
,
Y.
Liu
, and
H.
Sun
, “
Physics-informed multi-LSTM networks for metamodeling of nonlinear structures
,” arXiv:2002.10253 (
2020
).
56.
Z.
Long
,
Y.
Lu
,
X.
Ma
, and
B.
Dong
, “
PDE-Net: Learning PDEs from data
,” arXiv:1710.09668 (
2017
).
57.
Z.
Long
,
Y.
Lu
, and
B.
Dong
, “
PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network
,”
J. Comput. Phys.
399
,
108925
(
2019
).
58.
G.
Singh
,
S.
Gupta
,
M.
Lease
, and
C. N.
Dawson
, “
TIME: A transparent, interpretable, model-adaptive and explainable neural network for dynamic physical processes
,” arXiv:2003.02426 (
2020
).
59.
Z.
Chen
,
Y.
Liu
, and
H.
Sun
, “
Deep learning of physical laws from scarce data
,” arXiv:2005.03448 (
2020
).
60.
C. M.
Jiang
,
S.
Esmaeilzadeh
,
K.
Azizzadenesheli
,
K.
Kashinath
,
M.
Mustafa
,
H. A.
Tchelepi
,
P.
Marcus
,
A.
Anandkumar
 et al, “
Meshfreeflownet: A physics-constrained deep continuous space-time super-resolution framework
,” arXiv:2005.01463 (
2020
).
61.
A. T.
Mohan
,
N.
Lubbers
,
D.
Livescu
, and
M.
Chertkov
, “
Embedding hard physical constraints in neural network coarse-graining of 3D turbulence
,” arXiv:2002.00021 (
2020
).
62.
N.
Thuerey
,
Y.
Xie
,
M.
Chu
,
S.
Wiewel
, and
L.
Prantl
, “
Physics-based deep learning for fluid flow
,” see https://www.semanticscholar.org/paper/Physics-Based-Deep-Learning-for-Fluid-Flow-Thuerey-Xie/bf4623b356406133b7c9a0e8b28c67482da12df8.
63.
A.
Subramaniam
,
M. L.
Wong
,
R. D.
Borker
,
S.
Nimmagadda
, and
S. K.
Lele
, “
Turbulence enrichment using physics-informed generative adversarial networks
,” arXiv:2003.01907 (
2020
).
64.
L.
Sun
and
J.-X.
Wang
, “
Physics-constrained Bayesian neural network for fluid flow reconstruction with sparse and noisy data
,”
Theor. Appl. Mech. Lett.
10
(
3
),
161
169
(
2020
).
65.
J.
Töger
,
M. J.
Zahr
,
N.
Aristokleous
,
K.
Markenroth Bloch
,
M.
Carlsson
, and
P.-O.
Persson
, “
Blood flow imaging by optimal matching of computational fluid dynamics to 4D-flow data
,”
Magn. Reson. Med.
84
(
4
),
2231
2245
(
2020
).
66.
S. L.
Cotter
,
M.
Dashti
,
J. C.
Robinson
, and
A. M.
Stuart
, “
Bayesian inverse problems for functions and applications to fluid mechanics
,”
Inverse Probl.
25
(
11
),
115008
(
2009
).
67.
H.
Gao
and
J.-X.
Wang
, “
A bi-fidelity ensemble Kalman method for PDE-constrained inverse problems in computational mechanics
,”
Comput. Mech.
67
(
4
),
1115
1131
(
2021
).
68.
H.
Gao
,
L.
Sun
, and
J.-X.
Wang
, “
Phygeonet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain
,”
J. Comput. Phys.
428
,
110079
(
2021
).
69.
W.
Shi
,
J.
Caballero
,
F.
Huszár
,
J.
Totz
,
A. P.
Aitken
,
R.
Bishop
,
D.
Rueckert
, and
Z.
Wang
, “
Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network
,”
in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(IEEE,
2016
), pp.
1874
1883
.
70.
D. P.
Kingma
and
J.
Ba
, “
Adam: A method for stochastic optimization
,” arXiv:1412.6980 (
2014
).
71.
V.
Nair
and
G. E.
Hinton
, “
Rectified linear units improve restricted boltzmann machines
,” in
ICML'10: Proceedings of the 27th International Conference on International Conference on Machine Learning
(
2010
).
72.
K. M.
Johnson
and
M.
Markl
, “
Improved SNR in phase contrast velocimetry with five-point balanced flow encoding
,”
Magn. Reson. Med.
63
(
2
),
349
355
(
2010
).
73.
H.
Jasak
,
A.
Jemcov
,
Z.
Tukovic
 et al, “
Openfoam: A C++ library for complex physics simulations
,” in
International Workshop on Coupled Methods in Numerical Dynamics
(
IUC Dubrovnik
,
Croatia
,
2007
), Vol.
1000
, pp.
1
20
.
74.
R. H.
Pletcher
,
J. C.
Tannehill
, and
D.
Anderson
,
Computational Fluid Mechanics and Heat Transfer
(
CRC Press
,
2012
).
75.
C.
Rhie
and
W. L.
Chow
, “
Numerical study of the turbulent flow past an airfoil with trailing edge separation
,”
AIAA J.
21
(
11
),
1525
1532
(
1983
).
76.
A.
Paszke
,
S.
Gross
,
S.
Chintala
,
G.
Chanan
,
E.
Yang
,
Z.
DeVito
,
Z.
Lin
,
A.
Desmaison
,
L.
Antiga
, and
A.
Lerer
, “
Automatic differentiation in PyTorch
” in
31st Conference on Neural Information Processing Systems (NIPS 2017)
, Long Beach, CA,
2017
.
77.
C.
Rao
,
H.
Sun
, and
Y.
Liu
, “
Hard encoding of physics for learning spatiotemporal dynamics
,” arXiv:2105.00557 (
2021
).
78.
N.
Geneva
and
N.
Zabaras
, “
Multi-fidelity generative deep learning turbulent flows
,”
Found. Data Sci.
2
(
4
),
391
(
2020
).
79.
N.
Geneva
and
N.
Zabaras
, “
Transformers for modeling physical systems
,” arXiv:2010.03957 (
2020
).
80.
T.
Pfaff
,
M.
Fortunato
,
A.
Sanchez-Gonzalez
, and
P. W.
Battaglia
, “
Learning mesh-based simulation with graph networks
,” arXiv:2010.03409 (
2020
).
81.
S.
Patankar
,
Numerical Heat Transfer and Fluid Flow
(
Taylor & Francis
,
2018
).