Nonlinear model reduction from equations and data

The increasing demand to provide reliable and real-time simulations of complex physical models has demonstrated the fundamental importance of developing advanced numerical methods. This goes in parallel with the availability of larger and larger amounts of data, which provide important information on the phenomena to be simulated but require sophisticated computational techniques to be handled effectively and efficiently.

In this context, data-driven model reduction has the potential to address these challenges by providing low-dimensional approximations of complex models to enable tasks such as rapid forecasting, state estimation, and feedback control. Apart from computational and numerical efficiency, special considerations are required to ensure robust and physically meaningful predictions from reduced-order models.

This Focus Issue presents a collection of works on the development, analysis, and implementation of reduction techniques and surrogate models aimed at replacing computationally expensive models with more efficient, yet accurate ones learned from data, as well as from governing equations. A particular emphasis is on how to properly deal with data and to keep physical interpretability of the approximate models. To this end, this issue presents a wide range of computational developments leveraging neural networks, automatic differentiation, operator inference, as well as theoretical developments. In several works, this emphasis also manifests in terms of identification of low-dimensional invariant manifolds that preserve robust features of the underlying dynamics.

This issue builds upon and complements several important contributions in literature. An overwhelming majority of model reduction techniques in the literature fall in the category of projection-based methods.^1,2 These methods project the governing equations onto linear subspaces and are widely adopted due to simplicity of implementations and developments that have improved their efficiency over the decades of research.^3,4 However, projection-based methods tacitly assume invariance of projection subspaces, which is generally not guaranteed for nonlinear systems. In recent years reduced-order models based on nonlinear approximations have emerged as a powerful tool to address this limitation. The topic has seen several important contributions; we refer to Ref. 5 for an overview.

From a purely data-driven perspective, the dynamic mode decomposition (DMD)⁶ and its extensions⁷ are particularly well-suited for an equation-free construction of ROMs for linearizable dynamics. However, DMD is generally incapable of predicting nonlinearizable phenomena such as multiple isolated steady-states.⁸ 

To address these issues, various techniques aim to identify nonlinear manifolds relevant for nonlinear model reduction. With limited theoretical understanding of the underlying system, such nonlinear manifolds may be discovered by training autoencoders on time-series data.^9,10 Other reduction approaches rooted in dynamical systems theory seek to compute low-dimensional, attracting invariant manifolds. Related methods include equation and data-driven constructions of spectral submanifolds^11–13 that have been demonstrated to be effective for reducing complex nonlinearizable phenomena.¹⁴ 

Numerical approximations based on nonlinear parameterizations have emerged as a powerful tool for the complexity reduction of high-dimensional complex problems. These nonlinear parameterizations may be used to model invariant manifolds and their reduced dynamics and for online simulation for unseen parameter changes. Numerical approximations include online adaptive model order reduction and neural networks.

In this Focus Issue, Ref. 15 explores the potential usage of graph neural networks (GNNs) for the simulation of time-dependent partial differential equations in parameter-dependent spatial domains. This is achieved by constructing surrogate models based on a data-driven time-stepping scheme where a graph neural network architecture is used to evolve the system.

Reference 16 addresses the challenges of dealing with the effects of fast dynamics and non-normal sensitivity mechanisms in transient dynamics near an underlying manifold. The authors introduce a parametric class of nonlinear projections described by constrained autoencoder neural networks in which both the manifold and the projection fibers are learned from data. Key aspects are invertible activation functions, biorthogonal weight matrices, and dynamics-aware cost functions that promote learning of oblique projection fibers.

In Ref. 17, the identification of nonlinear structure in the data through a general representation learning problem is used to learn a nonlinear manifold on which reduced-order models are constructed. The matrix operators of the reduced-order model are then inferred from the data using operator inference.

The focus of Ref. 18 is a judicious selection of data to nonlinearly adapt the approximate model of time-dependent problems. A lookahead data-gathering strategy is developed to predict the next state of the full model for adapting reduced spaces toward dynamics that are likely to be seen in the immediate future.

Some contributions in this issue focus on specific important problems arising in fluid dynamics and ecosystems behavior. For instance, in Ref. 19, model order reduction for the two-dimensional Rayleigh–Bénard problem is considered in different flow regimes, with the development of suitable data-driven techniques able to ensure long-time stability of the solution. The focus of Ref. 20 is on a dimension-reduction method for analyzing the resilience of hybrid herbivore–plant–pollinator networks. Prey-predator systems are considered in Ref. 21 where a fractional-order model and its discretization are studied to shed light on the role of group effects and ecosystem stability. The work in Ref. 22 focuses on computing relative equilibrium states of a superfluid in rotating cylinder as stationary vortices in the rotating frame. To this end, the authors develop an automatic differentiation-based technique to minimize the free energy via gradient-based optimization. They discover relative equilibria along with low-energy saddle-type solutions and the associated homoclinic orbits to examine the local nonlinear dynamics near a minimizing state.

Model order reduction relies on the assumption that the problems considered are characterized by low-dimensional structures, at least locally around operating points in the phase space. For some complex problems, however, the neglected components associated with such approximations might play a crucial role in the dynamics. Closure modeling and model correction are important numerical techniques developed to address this issue. In this Focus Issue, Ref. 23 pertains to closure models for efficient ensemble prediction of leading-order statistical moments and probability density functions in multiscale complex turbulent systems. This is achieved via a calibration of the high order feedback using ensemble solutions of the consistent stochastic equations.

Reference 24 proposes a model correction framework for decreasing the discrepancies between reduced model predictions and observations from the true system of interest. Focusing on the Lotka–Volterra equations, a stochastic enrichment operator is embedded into the reduced model: the enrichment operator is theory-informed, calibrated with observations from the complete model, and extended to extrapolative combinations of parameters and initial conditions.

Indeed, discovering the low-dimensional structures associated with complex nonlinear systems is a challenging task and techniques rooted in dynamical systems theory are particularly interesting for addressing this challenge. References 25–27 pertain to data-driven approaches for reducing the dimensionality of complex systems using spectral submanifolds (SSMs). More in details, Ref. 25 deals with low-dimensional inertial manifolds containing the chaotic attractor of the underlying high-dimensional system. The reduced dynamics on the SSMs is used to predict chaotic dynamics over a few Lyapunov times and to reproduce long-term statistical features of the chaotic attractor.

Reference 26 proposes an extended class of SSMs that also contains invariant manifolds with mixed internal stability types and of lower smoothness class arising from fractional powers in their parametrization. This has application in shear flows, dynamic buckling of beams, and periodically forced nonlinear oscillatory systems.

On the other hand, the focus of Ref. 27 discusses an equation-driven approach for SSM-based reduction of nonlinear mechanical systems subject to parametric excitations. The authors develop expressions for higher-order nonautonomous terms in the parameterization of SSMs and their reduced dynamics using a multi index-based approach able to optimize memory requirements and the computational procedure. An open-source implementation in the software package SSMTool is provided.

The real-time simulation of large-scale nonlinear systems is a challenging task and requires the development of sophisticated computational techniques. Indeed, traditional numerical approximation based on simplified low-resolution models are ineffective in reproducing such complex dynamics. Reduced-order models hold promise for a quick assessment of changes under parameters and uncertainties, as well as for effective prediction and control. This Focus Issue surveys some of the latest trends in nonlinear model reduction for equations and data sets across various fields of applications, ranging from computational to theoretical aspects from a dynamical systems perspective.

The editors of the “Nonlinear Model Reduction From Equations and Data” Focus Issue, Cecilia Pagliantini and Shobhit Jain, would like to thank all the authors and reviewers for their contributions. Also, they would like to thank Professor Jan Hesthaven and Professor George Haller and the Chaos editors and staff, Michael Small, Jürgen Kurths, Felisa Conrad-Burton, Deborah Doherty, and Brian H. Solis, for their kind support.

The authors have no conflicts to disclose.