The Principle of Minimum Pressure Gradient: An Alternative Basis for Physics-Informed Learning of Incompressible Fluid Mechanics

Recent advances in the application of physics-informed learning into the field of fluid mechanics have been predominantly grounded in the Newtonian framework, primarly leveraging Navier-Stokes Equation or one of its various derivative to train a neural network. Here, we propose an alternative approach based on variational methods. The proposed approach uses the principle of minimum pressure gradient combined with the continuity constraint to train a neural network and predict the flow field in incompressible fluids. We describe the underlying principles of the proposed approach, then use a demonstrative example to illustrate its implementation and show that it reduces the computational time per training epoch when compared to the conventional approach.


I. INTRODUCTION
Before digital computing, the field of mechanics relied mostly on analytical tools, which placed limitations on our ability to solve many important problems that are described by complex multi-dimensional and/or nonlinear partial differential equations (PDEs).To tackle them, even the most astute mathematicians need to invoke simplifying assumptions often diluting the underlying physics.Motivated by the need to solve such problems, and enabled by the availability of fast computing resources, the field of computational mechanics grew rapidly over the past five decades becoming the method of choice for delineating the complex mechanics of solids, fluids, and their interactions [1][2][3][4] .To this end, various computational methods including finite element (FE) and finite volume (FV) methods, as well as spectral techniques were developed and employed 2,5,6 .
Despite such advancements, traditional computational techniques are continually challenged by several enduring complexities.Mesh generation remains a time-consuming process that is an art form for most complex domains involving interacting energy fields.Moreover, solving inverse computational problems, such as those involving unknown boundary conditions is often prohibitively expensive.Commercial software packages such as COMSOL, ANSYS, and OpenFOAM involve extensive amount of coding at the backend to handle solver algorithms, mesh generation, and to set up the boundary conditions.This makes the process of maintenance and generational updates an astounding task.Amid these enduring challenges, the field of computational mechanics has recently turned towards unconventional solutions hoping to overcome these hurdles.One such promising avenue is the application of machine learning techniques to solve complex PDEs 7 .However, largely due to the absence of advanced computational tools such as efficient deep machine learning algorithms and the advancements in GPU technology, such approaches did not gain significant traction until 2019, when Raissi et al. 8 introduced the concept of physicsinformed deep learning (PINN).In principle, the proposed method starts by using a neural network to initialize a random guess for the unknown dependent variables, then substitutes them into the governing PDEs, initial, and boundary conditions.A loss function is defined and used to measure the error between the predicted output based on the guessed variables and the target solution.The loss function is then minimized with respect to the parameters of the neural network.Iterations continue until the loss function converges to a predefined threshold around zero.
The development of PINN has inspired a number of applications in fluid mechanics involving the prediction of flows fields in various domains [9][10][11][12][13][14] .The main attractive advantage of PINNs is that a unified flexible and robust framework can be used for both forward and inverse problems 15,16 .Moreover, compared to traditional computational fluid dynamics (CFD) solvers, PINNs are superior at incorporating real-world observations of the flow quantities into the governing equations.A notable example is presented in Ref. 17 , where PINNs were proposed to infer the coefficient of the convection term in the Navier-Stokes equations (NSE) based on experimental velocity measurements for the 2D flow over a cylinder.
In the domain of fluid mechanics, the application of PINNs is predominantly grounded in the Newtonian framework, primarily leveraging NSE or one of its various derivatives 18 .Unlike in other domains where variational methods have effectively solved numerous problems 19 , the use of these principles in fluid mechanics has been somewhat limited.This limitation stems from the less mature theoretical foundation in fluid mechanics.For instance, extensive studies have attempted to extend the formulation of the NSE to align with Hamilton's principle of least action.However, these efforts have faced obstacles, largely because the Hamiltonian framework struggles to incorporate non-conservative forces like viscosity [20][21][22] .As a result, variational principles in fluid mechanics have faced constraints in effectively addressing the complexities of realworld scenarios.
In a departure from traditional approaches, Taha and Gonzales 23,24 introduced a transformative concept in fluid mechanics based on Gauss' principle of least constraint, leading to the development of the Principle of Minimum Pressure Gradient (PMPG).This principle asserts that, within any incompressible flow, the pressure gradient is minimized at all times.This framework diverges markedly from conventional methodologies, offering a unique variational approach to the problems of fluid dynamics.
Building upon this foundation, our research introduces a paradigm shift in addressing incompressible flow problems by combining PMPG with PINN, thus creating PMPG-PINN.This approach uniquely informs the neural network's loss function by focusing on minimizing a cost function derived from the PMPG that is independent of the pressure.The removal of the pressure dependence from the unknown variables reduces computational cost both in terms of reducing the size of the unknowns, and in terms of evaluating derivatives in the loss function.To the authors' knowledge, this is the first work that combines PMPG and PINN to solve incompressible flow problems.
To elucidate our proposed method, the paper is structured as follows.Section 2 provides a mathematical formulation of the principle of PMPG, shedding light into its theoretical foundation.Section 3 illustrates the conventional implementation of PINN using Navier-Stokes Equations.Section 4 demonstrates the newly proposed implementation of PINN using PMPG.Section 5 presents an illustrative example that showcases the effectiveness of the approach in solving incompressible flow problems.Finally, Section 6 presents the conclusions, and potential applications and extensions of the proposed method.

II. PRINCIPLE OF MINIMUM PRESSURE GRADIENT
To effectively present the PMPG, a foundational understanding of Gauss' principle of least constraint is essential.This principle, which forms the core of the derivation, is extensively detailed in references 25,26 .Consider the dynamics of N constrained particles, each of a fixed mass m i , whose motion can be described by the generalized coordinates, q.The dynamics of such particles is dictated by Newton's second law as: where a i is the inertial acceleration of the ith particle, F i are the external forces acting on the particle, and R i are the constraint forces, which do zero work as they preserve the constraints.
Gauss' principle asserts that the quantity is a minimum with respect to the generalized accelerations, q, at every instant of time.An equivalent S can be written in terms of the constraint forces as: where the sum of the squares of the constraint force must be a minimum.
In the absence of constraint forces, R i = 0, S assumes its absolute minimum, which is equal to zero.In other words, a particle follows the external force applied to it.However, in a constrained setting, nature acts like a mathematician.It picks, at each instant of time, an acceleration for the constrained system that minimizes, in a weighted least-squares sense, the difference between the acceleration of the free motion and the constrained system.This means that, the particle adjusts its path only to the extent necessary to meet the constraints, ensuring the least possible deviation from the unconstrained trajectory.
Gauss' principle of least constraint can be applied to any dynamical system including the two-dimensional motion of incompressible fluids described by the 2D incompressible NSE, which can be written in the following form: where u = (u, v) T , is the fluid velocity vector, p is the pressure, and ν is the kinematic viscosity.Equation ( 4) represents a balance of forces applied on the moving fluid in a domain Ω ∈ R 2 .The left-hand side of the Equation is the total acceleration of the fluid, incorporating both inertial and convective components, while the right-hand side comprises the forces acting on the fluid, which can be decomposed into either external or constraint forces.Finally, Equation ( 5) represents a mass balance often referred to as continuity.
Illustration of the evolution of the flow field in the spacetime configuration, highlighting the path traced by u with a stationary quantity ∇S = 0. .Several works 18,23 have demonstrated that in incompressible flow, the pressure gradient function acts as a constraint force, primarily enforcing the continuity.Hence, by applying Gauss' principle of least constraint to the dynamics of incompressible fluids governed by NSE, and classifying the pressure gradient as a constraint force, we can write the action, S, in Euler coordinates as: subject to the continuity and any boundary conditions defined on the boundary of the domain Γ = ∂ Ω.
As shown in Fig. 1, when the system evolves, multiple trajectories of the flow field u become possible, of which only few are shown.The trajectory followed by the system, represented by the solid line, is the one that renders ∇S = 0. Thus, the flow field will deviate from the motion dictated by the inertia and the viscous forces only by the amount necessary to satisfy the continuity.In other words, no larger pressure gradient will be generated than necessary to satisfy the continuity 24 .Thus, the name the principle of minimum pressure gradient (PMPG).

III. PHYSICS-INFORMED LEARNING USING NAVIER STOKES EQUATION
In general, PINN integrates information from the governing physical laws of a given system into the training process of a deep neural network (NN) so that the unknown variables can be approximated using a limited set of training samples.In the case of the NSE, Equations ( 4), ( 5), the unknown flow field variables, u, v, as well as the pressure, p, are attained by converting the process of finding the solution of the PDEs into an optimization problem in which a loss function (error function) is iteratively minimized by updating the parameters of the neural network.This process is denoted by NN training.
As shown in Fig. 2, the NN is a fully-connected feedforward network composed of L multiple-hidden layers.It takes a concatenation of time and the state-space z 0 = (t, x) as an input, and outputs a guess for the unknown variables (u, v, p).Each layer creates data for the next layer through a tensorial nested transformation of the form 27 : where the functions σ l are called activation functions.These can be chosen based on nature of the problem to be solved.The variables W l and b l denote, respectively, the weights and biases of each NN layer, l.These are updated after every iteration (epoch) by minimizing a loss function, L , with respect to those variables.The loss function, denoted here by L (θ ) where θ = [W l , b l ], measures the difference between the predicted solution and the target solution.When the the residual change between successive iterations is less than a predefined threshold, say ε, the training stops and the output of the last iteration is considered the solution of the PDE.Further details of this process are presented in Appendix A.
In the case of Equation ( 4), ( 5), the loss function, L (θ ), is defined as: and is formed by adding two losses subject to predefined weights (penalties).The first is the loss function, L PDE , weighted by µ p , which defines the loss in the calculation of Equation ( 4) and is given by: (10) where N B is the number of collocation points over the domain.The second is the loss function, L c , weighted by µ c , which defines the loss in the calculation of the continuity, Equation (5), and is defined as: Note that, the mean square error (MSE) is employed to compute the losses.

IV. PMPG-BASED PHYSICS-INFORMED LEARNING
Deviating from the conventional approach of including the loss function associated with NSE into the training process, the proposed PMPG-PINN searches for the velocity field u which minimizes the function S in Equation ( 6) subject to the continuity constraint (7) and the associated boundary conditions.Thus, the pressure and its gradients are completely eliminated from the training process.This can be simply achieved by converting the constrained optimization problem into an unconstrained optimization problem using the following loss function: where the integral S can be reformulated as a sum using the mean rule, leading to Here A is the area of the domain, and J is the discretized operator used to approximate the integrand in Equation ( 6), and can be written as: In comparing the definitions of the loss functions in Equation (9) and Equation ( 12), a key distinction emerges.The optimization problem in Equation ( 9) is easier to solve given that the losses associated with the PDE and the continuity, L PDE and L c , can be jointly driven to zero.This consistency enables simultaneous achievement of both constraints.In contrast, for the PMPG-PINN problem, the minimization of the loss function is achieved even when the objective function, S, is different from zero.Thus, a scheme that merely optimizes the sum of the objective and the constraint loss will usually end up in an optimum that is not a solution of the PDE 28 .
To overcome this issue, we utilize the Automated Lagrangian Method (ALM) 28 , which replaces the constrained .optimization problem with a sequence of unconstrained problems with an additional term designed to mimic Lagrangian multipliers.As such, the updated loss within this methodology at the k-th iteration can be written as: where µ c is the penalty weight, and λ k i are the Lagrange multipliers.Additional details specifying the algorithm governing the updates of λ k i following each iteration can be found in Appendix B.

V. EXAMPLE: LID-DRIVEN CAVITY
To illustrate the effectiveness of the proposed approach, we consider the widely-known lid-driven cavity problem.Among the many reasons which made this particular problem one of the most common benchmarks in CFD is the combination of its simple geometry and the presence of various corner singularities.Furthermore, only velocity Dirichlet boundary conditions are required to define the mathematical problem.We initially tackle the problem using conventional PINN.This initial step sets a baseline, enabling a comparison with the new approach.Following this, we proceed to illustrate the application of the PMPG-PINN method in two scenarios; one while employing Lagrange multipliers and the other without.Finally, we evaluate the computational efficiency of PMPG-PINN by comparing it to the conventional PINN.
As shown in Fig. 3, the computational domain is defined on the area [0, 1] x [0, 1].A no-slip condition is enforced on three walls, while the upper wall is assumed to move to the right with a velocity u = f 0 (x), where, in order to ensure continuous differentiability along the boundaries, f 0 (x) is assumed to be a parabolic function in x.Furthermore, zero-pressure is prescribed at the lower left cavity corner.The maximum axial velocity is chosen such that the flow remains laminar with a Reynolds number that is equal to Re = 100.
Since the problem considered is stationary, the time variable is eliminated and the traditional PINN takes x = (x, y) as inputs, and outputs the velocity and pressure fields as (u, v, p).To force the output solution to satisfy the boundary conditions, we modify the network and construct a new trial solu- . tion, ( û, v, p), as: During the training process utilizing the conventional PINN, the PDE and continuity were equally penalized by using equal loss weights (µ p = µ c = 2).A parametric study was then performed to determine the minimum number of neurons per layer, n, collocation points, N B , and network layers, L, that minimize computational time while leading to acceptable residual errors.This led to the following numerical values: n = 20, L = 8 and N B = 90000, which were used throughout the presented simulations.
Figure 4a presents convergence of the residual loss functions, L PDE , and L c , with the number of epochs as obtained using the conventional PINN.Both residuals demonstrate a consistent decrease until they reach an acceptable value in the order of 10 −6 after nearly 10000 epochs.The number of epochs differs based on the initialization of the PINN, yet the computational time per epoch remains nearly constant at 0.050 s.The velocity magnitude resulting from these computations is depicted in Fig. 4(b).
Next, PMPG-PINN is implemented on the same domain with the same network parameters.The first implementation does not use Lagrange multipliers in order to determine if the fixed weight scheme can result in accurate results.The value of S which minimizes the loss function is obtained for fixed values of µ c ranging between 2 and 2000.Figure 5 demonstrates that S varies considerably with µ c , which indicates that the fixed-weight training scheme results in erroneous predictions that are weight dependent.To see how far those values of S are from the reference value that the conventional PINN converges to, we substitute the velocity field obtained using the conventional PINN into Equation ( 6) and obtain a reference value of S = 0.002.It is apparent that, because the continuity constraint is not highly penalized when µ c is small, the Langrange-less PMPG-PINN yields a value of S that is much lower than the reference value.A higher penalty of µ c = 200 yields the best enforcement of the continuity constraint and a value of S ≈ 0.002.However, contrary to natural intuition, penalizing the continuity further causes the value of S to diverge from the reference value because the optimization problem becomes ill-conditioned 28 .
Next, we explore the benefits of using Lagrange multipliers in the loss function using two different penalties; namely µ c = 10 and µ c = 50.For the case µ c = 10, both the value of S and the residual of the loss constraint, L c , are plotted against the number of epochs in Fig. 6(a).The figure is sectioned into three areas, each representing one outer iteration in which the Lagrange multiplier is updated.It can be seen that adjustments to the Lagrange multipliers after every outer iteration following the procedure described in Appendix B correlates with significant reductions in the value of the residual yielding a value of S = 0.0018 and a residual of the order of 10 −6 after nearly 12000 epochs.
When PMPG-PINN is trained using µ c = 50, the value of S converges to 0.00195, while the residual of the constraint loss, L c , decreases to an acceptable value of the order of 10 −6 after nearly 11000 epochs.It can be clearly seen that the error is low, implying a good agreement between the two frameworks.In general, the accuracy of the PMPG-PINN framework can be improved further by considering higher-order integration methods of S, such as trapezoidal or Simpson's rule 29 .Beyond this, several other suggestions could be considered, such as, network architecture tuning, and learning rate optimization.When implementing PMPG-PINN, we observed a notable enhancement in the computational efficiency.Specifically, the time required per epoch in PMPG-PINN is measured at 0.044 s, in contrast to 0.050 s observed using the conventional PINN.This 12% reduction in training time per epoch was found to be consistent regardless of the value of µ c , the initial guesses of the NN parameters, or whether ALM is implemented or not.The reduction in time can be attributed to two factors: i) reduction in the number of automatic derivative calculations due to the elimination of the pressure gradients, and ii) reduction in the output variables, which means fewer parameters to adjust and consequently a smaller output space to learn.The reduction in time per epoch when implementing PMPG-PINN highlights its potential applicability in solving complex and computationally demanding problems.It is important to note, however, that this reduction in computational cost comes with a trade-off because direct calculation of the pressure field is not part of PMPG-PINN framework.Nevertheless, the pressure field can be recovered by applying the resultant velocity field, u and v, to Equation 4, followed by integrating the resulting linear, first-order, and time-independent PDE.

VI. CONCLUSIONS
In this paper, we developed a new approach to solve incompressible flow problems by integrating the Principle of Minimum Pressure Gradient (PMPG) with Physics-Informed Neural Networks (PINN) to create the PMPG-PINN framework.A critical component in the development of the new approach is the incorporation of the Augmented Lagrange Multipliers (ALM) in the loss function of the neural network.This implementation is key towards effectively balancing the objective and the constraint of the model.
Applying the PMPG-PINN framework to the stationary liddriven cavity problem yielded results that are in excellent agreement with those obtained using the conventional PINN approach.Moreover, PMPG-PINN results in nearly 12% reduction in computational time per epoch during neural network training independent of the network initialization and the penalty imposed on the loss function.The enhanced efficiency is primarily attributed to the elimination of pressure gradient calculations and the reduction of output parameter space in the neural network.
We believe that PMPG-PINN may offer a potentially important alternative to the conventional PINN based on NSE.Its apparent computational efficiency can pave the way toward solving more complex and non-stationary problems.For example, a natural progression would be to adapt the PMPG-PINN framework to transient and turbulent flow conditions.Venturing into the domain of fluid-structure interaction presents another frontier for exploration.Here, a key challenge lies in integrating the dynamics of solids into the formulation of the quantity S, which in turn can be minimized using PMPG-PINN framework.

DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available from the corresponding author upon reasonable request Appendix A: NN structure, activation and optimizer A fully-connected neural network with L hidden layers and n neurons per layer is employed.Because of its effectiveness, we used the tanh activation function 30 .The weights of the network are initialized with Xavier initialization 31 , while the biases are initialized as zero.No dropout methods were applied.To minimize the loss function, L , we combine ADAM 32 and L-BFGS-B 33 optimizers.We first apply the ADAM optimizer for gradient descent training and then employ the L-BFGS-B optimizer to fine tune the results.During the Adam-based training, the optimizer calculates the direction of the gradient at each iteration considering a full-batch size.The initial learning rate is 0.001.We set the stopping criterion of L-BFGS-B to the smallest positive normalized floating-point number represented in Python3.The Augmented Lagrangian Method (ALM) is an optimization technique designed to handle constraints more effectively in optimization problems.It is particularly useful in scenarios where conventional methods might suffer from illconditioning issues 28 .The ALM modifies the loss function defined in Equation ( 12) by incorporating explicit Lagrange multiplier estimates, λ k i , which transforms the original constrained optimization problem into a series of unconstrained problems, each characterized by varying coefficients of λ i .Thus, the new ALM loss function can be written as: This ALM process involves adjusting the Lagrange multipliers to ensure that the constraints are increasingly satisfied as the outer iterations proceed.During each outer iteration, the PINN algorithm seeks an approximate minimizer θ k , which will then become θ k+1 .As such, we can write the optimality condition as: In the k + 1 iteration, we anticipate that the constraint function g is nearly satisfied, and consequently rendering g 2 close to zero.Therefore, the Lagrange multipliers, λ k+1 i , must be updated to replicate the gradient in the previous iteration as: Comparison of (B3) and (B4) indicates the correct update formula for λ k+1 is: A general framework for algorithm based on the ALM can be specified as follows:

FIG. 3 .
FIG. 3. Schematic diagram of the lid-driven cavity problem showing the domain and boundary conditions..

FIG. 4 .
FIG. 4. A physics-informed solution of the Lid-driven cavity problem using conventional PINN.(a) Convergence of the residual loss functions, (dashed) L PDE , and (line) L c .(b) Heat map of the velocity magnitude:

FIG. 5 .
FIG. 5. Variation of the quantity S with the penalty weight µ c .

FIG. 6 .
FIG. 6.A physics-informed solution of the Lid-driven cavity problem using PMPG-PINN.(a) Convergence of the residual loss functions for µ c = 10, (dashed) S and (line) L c .(b) Point-wise error of the velocity magnitude between conventional PINN and PMPG-PINN with µ c = 50..