Cascades of scales: Applications and mathematical methodologies

The present special collection presents reviews and genuine research articles addressing real-life applications involving many interacting scales and related methodological developments. Specifically, applications from the biomolecular, material, atmospheric, and geophysical sciences form the core of the dedicated collection, while this does not exclude selected ventures into other areas.

Of particular interest here are problems involving couplings across their scales from small to large and vice versa,^1–3 and mathematical concepts and methodologies that have proven to be transferable from specific applications to a generalized framework guided by mathematical abstraction.^4–9 In many cases, these abstractions result in related computational codes and software libraries for specific applications of interest to the different disciplines involved.^10–13 

Being, in this way, inherently interdisciplinary in nature, contributions to this volume give high priority to explaining relations between the application-oriented domain-specific languages used to frame their scientific problem in an application of mathematical physics on the one hand, and the complementary unifying abstract mathematical language that enables the generalization of methodologies to other fields on the other hand.

Thus, the mission of the special issue is two-pronged: It reports on exciting recent methodological developments for challenging problems in mathematical physics involving cascades of interacting scales, and it showcases the importance of precise language(s) and the power abstraction in fostering efficient interdisciplinary research. May it become a well-visited point of reference and point of departure for further conceptual developments. With solid foundations in concrete applications and phrased in the general language of mathematics, they may gain substantial momentum beyond their original discipline of application.

The contributions are published as regular articles of the Journal of Mathematical Physics, while they are linked at the same time in the present themed (online) collection.

Physical systems featuring multiple spatio-temporal scales have posed notoriously difficult challenges to theory and mathematical analysis in past decades, as witnessed, e.g., by a myriad of articles published in the present Journal of Mathematical Physics (JMP), by the establishment in 2003 of the Journal of Multiscale Modeling and Simulation of the Society of Industrial and Applied Mathematics (SIAM), or the references listed below. For a long time, the complexities of real-life physical problems were beyond the reach of mathematical analysis, both formal and rigorous. Thus, theoreticians developed and honed their tools and techniques on physically motivated but substantially simplified models, while researchers from the applied sciences (Physics, Chemistry, Biology, and the Geo-Sciences) pragmatically bridged gaps in theoretical knowledge by well-informed and intuitive closure schemes that would address the ubiquitous multiscale-effects. Over the past 1 1/2 decades, however, mathematical/theoretical multiscale techniques have matured to levels which increasingly allow their transfer to real-life applications, see, e.g., Refs. 4, 6, 7, 14, and 15. In addition, the wealth of observational data available today for physical processes from laboratory setups to the Earth system jointly with the exciting capabilities of recent machine learning techniques have lended tremendous thrust to data-based modeling activities.^16–18 The recent rapid rise of data-based modeling in the physical and chemical sciences offers a promising path to accessing a large class of problems across scales. This path comes with the danger, however, of artificial results due to insufficient and/or corrupted data—especially in the face of the curse of dimensionality. For this reason, current research calls strongly for physical models that must be as rigorous as possible to ensure scientific consistency and avoidance of any misuse of data (see, e.g., Refs. 17 and 18). One aim of the present collection is to highlight this aspect, among the others, and build a path to a sustainable research based also on data but with the control of rigor and consistency of physico-mathematical models.

These and similar developments have motivated JMP’s special collection on “Cascades of Scales: Applications and Mathematical Methodologies” of which we provide an overview in this paper.

A very important aspect in this context, besides impressive recent methodological advances and particularly successful real-life applications, concerns the challenges of interdisciplinary cooperation. To a large extent, these challenges are due to the fact that the scientific disciplines participating in a project rely on fine-tuned domain-specific languages for efficient communication. This is compounded by often similar vocabulary, which carries different meanings in different disciplines. A prominent example is the notion of “multiscale” itself, which, depending on the scientific community considered, can be associated with “broad and continuous Fourier spectra” (e.g., in turbulence theory), with the presence of large (asymptotic) scale separations [e.g., in (stiff) chemistry or geophysical fluid dynamics], or with non-asymptotic metastabilities (e.g., in molecular dynamics). Progress in interdisciplinary projects is often hampered by subtle misunderstandings arising from such discipline-dependent “concept overloading.” The authors who have contributed to the present special collection of papers have taken particular care to use precise and unambiguous language that will hopefully help make their contributions efficiently accessible to the readers of the Journal of Mathematical Physics.

Multiscale asymptotics, homogenization, WKB theory, and other formal tools of analysis of scale interactions have been developed successfully over the past seven-odd decades. The contributions to this special collection may be associated with four topical groups:

1. Mathematical model formulations,

2. Computational approaches,

3. Rigorous analysis,

4. Data-based stochastic modeling.

This thematic group features two papers, both of which address the challenge of formulating mathematical models for challenging multiscale physical systems. The first paper by del Razo et al. on “Chemical diffusion master equation: Formulations of reaction–diffusion processes on the molecular level”¹⁹ addresses situations in molecular systems that feature multiple scales in terms of both particle numbers and time scales and include general chemical reactions between the species. Their work shows how coupled mathematical models involving atomistic detail for species with low, discrete stochastic dynamical models for species with intermediate, and (stochastic) continuum models for abundant species can systematically be derived from fundamental many-particle representations. Three alternative derivations are presented, two of which address the involved combinatorial relations for multi-species reactions directly while the third utilizes a multi-particle basis representation and quantum-mechanical creation and annihilation operators to achieve the same. The second paper “A porous-media model for reactive fluid–rock interaction in a dehydrating rock” by Zafferi et al.²⁰ frames models for porous media flows within the well-established framework of the General Equation for Non-Equilibrium Reversible–Irreversible Coupling (GENERIC).^8,9 This approach yields a model with a strong mathematical underpinning that enables both future rigorous partial differential equations (PDE) analysis and the design of reliable numerical methods.

The three papers in this group describe recent exciting progress in three different directions of scientific computing for multiscale systems.

The first by Ji et al. on “An asymptotic-preserving and energy-conserving particle-in-cell method for Vlasov–Maxwell equations”²¹ combines ideas from the construction of well-balanced numerical approximations of multiscale systems with techniques from the area of mimetic schemes. The latter reproduce exact relations of vector calculus used to prove derived secondary conservation properties of PDE systems at the discrete level. The combination of both approaches yields novel robust approximations with favorable approximation quality across a broad range of scales. The second paper by Achatz et al. on “Multi-scale dynamics of the interaction between waves and mean flows: From nonlinear WKB theory to gravity-wave parameterizations in weather and climate models”²² addresses the transfer of asymptotic multiscale analysis into unresolved-scale closures for multiscale PDEs. The authors first explain a generalized WKB theory for internal waves with solutions in an extended six-dimensional physical/wavenumber space. This is then utilized in designing a ray-tracing scheme for the propagation of high-wavenumber internal gravity waves in the atmosphere. The resulting innovative subgrid scale closure has been implemented in the German weather and climate computational model ICON.²³ The third paper by E et al. on “Machine learning-assisted multi-scale modeling”²⁴ provides an overview and critical appraisal of recent efforts at combining physics-based multiscale modeling with techniques of machine learning. The underlying motivation for this approach is to address the fact that in many situations satisfactory analytical descriptions of small-scale effects are not available, whereas there are often (a limited number of) very high-resolution simulations that contain a large amount of explicit information, albeit not in closed form, on small-scale processes. The idea then is to combine a well-resolved numerical discretization of the coarse-scale dynamics with subgrid scale models of a rather generic functional structure whose concrete parameters are determined by machine learning methods. Three quite different applications from the molecular and material sciences elucidate the approach.

Multiple scales analyses often require relatively restrictive regularity assumptions for the small scales. This group of papers describe two classes of problems for which these restrictions can be lifted and provide an overview of associated novel mathematical techniques.

The paper by Mielke on “Non-equilibrium steady states as saddle points and energy-dissipation principle (EDP)-convergence for slow-fast gradient systems”²⁵ reviews the theory of slow-fast gradient systems that lead in a natural way to non-equilibrium steady states. The key is that, on the slow time scale, the fast subsystems stay in steady states that are controlled by the interaction with the slow system. Using the theory of convergence of gradient systems depending on a small parameter ε (here the ratio between the slow and the fast time scale) in the sense of the energy-dissipation principle (EDP)²⁶ shows that there is a natural characterization of these non-equilibrium steady states as saddle points of a so-called B-function when the slow variables are fixed. Applications to slow-fast reaction-diffusion systems with cosh-type gradient structure for reactions and a reaction-diffusion system with a thin embedded membrane-like layer are discussed. The paper by Bella et al. on “Γ–convergence for nearly incompressible fluids”²⁷ deals with a domain homogenization problem and considers the flow of a compressible viscous fluid in the low Mach number regime under rough microscale domain perturbations. The largest class of microscale domain perturbations that have no impact on the incompressible limit is identified. Applying the concept of “Mosco convergence,” the convergence to the incompressible Navier–Stokes equations on the limiting domain is proven for perturbations of this class.

Whereas many-particle molecular systems certainly provide first-principles descriptions of inert and reactive gases, liquids, and solids, we are seldomly interested in their full atomistic detail. Instead, what counts in most applications are overall statistical descriptions of steady or quasi-steady states. And yet, the only solid information available about such systems are particle masses, interaction potentials, reactivities, and the like. As a consequence, we can build mathematical models for what we are not really interested in and are faced with the challenge of extracting the aggregated interesting information from intelligently designed ensembles of atomistic simulations. The fourth group of papers addresses mathematical approaches to these challenges. Before our overview of these contributions, one word of caution should be in order:

The first paper by Donati et al. on “A review of Girsanov reweighting and of square root approximation for building molecular Markov state models”²⁸ reviews fundamental problems from this area related to importance sampling: Dynamical reweighting methods estimate kinetic observables in a stochastic process based on a certain (tricky) target potential using simulated detailed trajectories of a different system based on a more convenient potential. As such, this approach provides an efficient tool to explore rare events otherwise not detectable by standard means. The paper discusses Girsanov reweighting and the square-root approximation. The former utilizes Girsanov’s theorem on the ratio of probability densities in the state space of systems with different interaction potentials to infer statistics of the “tricky” system from those of the benign one. The Square Root Approximation of the Fokker–Planck operator, in turn, allows the user to derive the transition rate matrix of a coarse-grained Markov state stochastic process, thereby eliminating the need for detailed atomistic simulations. The second paper by Quer and Ribeira Borrell on “Connecting stochastic optimal control and reinforcement learning”²⁹ draws a connection between stochastic optimal control and reinforcement learning. The motivation of this work is again importance sampling. Here, the efficient sampling strategy is rephrased as an optimal control problem that can be solved by a stochastic optimization scheme. The paper addresses scalability in high-dimensions and how to deal with intrinsic metastabilities of such systems. Optimal control is related to reinforcement learning, both of which are based on a Markov Decision Process (MDP). Two related reinforcement learning algorithms are formulated and critically compared. The third paper by Sikorski et al. on “Learning Koopman eigenfunctions of stochastic diffusions with optimal importance sampling and ISOKANN”³⁰ presents an iterative algorithm that approximates the dominant eigenfunctions of the Koopman (or probability density evolution) operator on the state space of a molecular system. The goal is to efficiently observe the statistically dominant events in a dynamical system. A proof of convergence of ISOKANN, restricted to bistable systems, is presented. Its convergence rate and approaches for its improvement based on optimal control and importance sampling are developed. An implementation of the scheme in a molecular dynamics framework greatly enhances the construction of effective Markov state models. The paper thus provides a formal framework for computational scale analysis in molecular simulations.

The contributions to this special collection of the Journal of Mathematical Physics address tough multiscale problems from diverse application areas including reacting many particles, plasmas, fluid mechanics, porous media flows, and more. They present, develop, or utilize mathematical concepts, approaches, and methods that are motivated by such concrete applications but are generic in that they are transferable to structurally similar problems in other disciplines as well. Specifically, we find

• Quantum-mechanical annihilation and creation operators adapted and applied to reacting classical particle systems,

• The GENERIC framework as a backbone to modeling reactive fluid–solid interactions in dehydrating rock,

• A novel combination of asymptotic-preserving and energy-conserving mimetic discretizations for the Vlasov–Maxwell system from plasma physics,

• A weakly nonlinear WKBJ theory in a hybrid physical/wavenumber space yielding an innovative closure scheme for short internal waves in geophysical flows,

• Machine learning technology deployed to “learning” effective unresolved-scale closures for computational multiscale modeling,

• Gamma-convergence and the Energy-Dissipation-Principle (EDP) and how their combination supports rigorous analyses of rough multiscale problems,

• A mesh- and metric-free numerical scheme, the “square-root approximation,” for the state-space evolution (or Koopman) operator of many-particle systems,

• How Girsanov reweighting and ideas combining optimal control and reinforcement learning lead to efficient importance sampling strategies for complex molecular systems.

May these examples of “thinking out-of-the-box” inspire junior and senior readers alike to freely look for problem solving ideas across established disciplinary and methodological boundaries.

The team of guest editors thanks all contributing authors for sharing their exciting research results; Edriss Titi for suggesting the Journal of Mathematical Physics (JMP) as a home to this special collection as well as for his contributions to managing the editorial process; the editorial team of JMP for their highly professional support and for their patience with us; and the production staff at AIP Publishing for prompt and excellent realization of the individual papers and the online special collection. We also acknowledge Deutsche Forschungsgemeinschaft for their financial support through TRR/CRC 146 “Multiscale Simulation Methods for Soft Matter Systems,” Project No. 233630050 (M.L.-M.) and CRC 1114 “Scaling Cascades in Complex Systems,” Project No. 235221301 (L.D.S., E.S.T., and R.K.).

Luigi Delle Site: Conceptualization (equal); Writing – review & editing (equal). Rupert Klein: Conceptualization (equal); Writing – original draft (lead). Mária Lukáčová-Medvid’ová: Conceptualization (equal); Writing – review & editing (equal). Edriss S. Titi: Conceptualization (equal); Writing – review & editing (equal).

Data sharing is not applicable to this article as no new data were created or analyzed in this study.