Chaos, patterns, coherent structures, and turbulence: Reflections on nonlinear science

The discovery of chaotic dynamics and especially the realization that many physical systems display characteristics of nonlinear dynamical systems sparked a revolution in our understanding of the real world.² At about the same time, the notion of solitons³ (and many insights into chaotic dynamics) emerged from the analysis of one of the first numerical computations ever performed, namely, the Fermi-Pasta-Ulam (FPU) problem.⁴ The convergence of these concepts in the 1970s and early 1980s birthed the nascent field of nonlinear science from the name “chaos”⁵ to the discovery by Feigenbaum^6,7 of universality in the period-doubling and quasiperiodic routes to chaos. Critical to the success in establishing nonlinear science as an important emerging area was the realization in physical systems of chaotic dynamics.^8–11 

Numerous centers and institutes emerged to focus on these early successes, most notably at Los Alamos where the Center for Nonlinear Studies (CNLS) was created in 1980 through the efforts of David Campbell together with Alan Bishop, Darryl Holm, Mac Hyman, and Basil Nicolaenko. I was fortunate to be a postdoc at CNLS starting in 1983, working jointly with John Wheatley on experimental low temperature cryogenic convection and with Doyne Farmer on computational chaotic dynamical systems. I had the great opportunity to participate in the application of many of the theoretical concepts of nonlinear science to real experimental systems and also began to see the limitations of these approaches for some of the hard problems that arise from strongly nonlinear interactions. In particular, the early hopes that universality and chaotic dynamics (and related nonlinear science concepts) would provide the foundations for understanding far harder problems such as fluid turbulence have generally not materialized but much progress has been made in many areas of nonlinear science. Indeed, so well established are many of the concepts and methodologies, e.g., fractals (and fractal dimension), chaos, solitons, universal routes to chaos, bifurcation theory, amplitude equations, pattern formation, etc., that they have become, in several decades, standard tools in the characterization and understanding of complex systems rather than research topics in themselves. The grouping of many of these ideas into broad classes by Campbell¹ helped define the field of nonlinear science and will guide me in reviewing some of the key ideas that I encountered over the past several decades.

In 1990, I helped organize, with David Campbell and Mac Hyman, a CNLS Annual Conference entitled “Nonlinear Science: The Next Decade.”¹² Almost three decades later, I will present some examples from my own research that I think are relevant in looking back on nonlinear science over that interval, identifying some successes from the past and challenges for the future. In Sec. II, I will discuss bifurcation theory and chaotic dynamics with some recent applications of finite time Lyapunov exponent analysis of two-dimensional turbulence. In Sec. III, I will talk about amplitude equations and pattern formation and the power of weakly nonlinear analysis around special bifurcation points. Then, in Sec. IV, I will discuss fully developed fluid turbulence and describe some of the challenges that this strongly interacting nonlinear system poses for modern science. In Sec. V, I will reflect on the overall lessons learned from my three decades in nonlinear science. I do not pretend that this is a comprehensive review of these areas, an effort far beyond the scope of this focus issue of Chaos, and I apologize in advance for the many important contributions in these areas that I will not have time to properly credit. I was also encouraged to be speculative and perhaps provocative, tasks that seem like plenty of fun. So, with all caveats accepted, off we go on some “nonlinear science” appetizers.

Critical phenomena in statistical physics have special points in parameter space, i.e., critical points, at which changes in phase occur and around which certain universality principles have been successfully established.¹³ In the 1970s and 1980s, concepts from dynamical systems and applied mathematics combined with extended notions of universality to treat a wide range of nonlinear physical systems. For such nonlinear, non-equilibrium systems, the analog of thermodynamic phase transitions is bifurcation theory.¹⁴ A simple example is a supercritical bifurcation (analogous to a mean-field ferromagnetic phase transition), which is the generic continuous transition from a quiescent state with zero amplitude to one with finite amplitude. The corresponding amplitude equation is of Ginzurg-Landau type with no spatial dependence

whose solution has several distinct features: a characteristic time for transients $τ∼ϵ−1$ and a steady state relationship $A2∼ϵ$⁠.

To illustrate some examples of dynamical systems behavior from bifurcation theory to chaos, I will refer to a set of experiments performed in the 1980s on cryogenic convection in ³He-superfluid-⁴He mixtures near 1 K.¹⁵ The state of Rayleigh-Bénard convection is governed by the non-dimensional Rayleigh number R which is proportional to the temperature difference between the top and bottom plates of the convection cell. The cell had a small ratio of height to width, i.e., a small aspect ratio, and the states of convection proceeded from an onset value R_c between conduction and time-independent convection through a transition at R₀ to a time periodic limit cycle state and finally at still higher R to quasiperiodic convection with two incommensurate frequencies and transitions to chaos via mode-locking. Details can be found elsewhere.^15–20 

In Fig. 1, I show the relaxation time for transients near a supercritical Hopf bifurcation—from no time dependence to a limit-cycle state—in ³He-superfluid-⁴He convection.^16,17 The expected “critical slowing down” with exponent −1 is observed over more than two orders of magnitude in $ϵ=R/R0−1$⁠. In the upper right inset, one obtains the linear dependence of $| A |2$⁠, and the lower left inset shows the dramatic increase in time scales on a linear scale. Note that to within the precision of the experiment there is no perceptible rounding of the transition in either the steady state amplitude or the relaxation time. This type of behavior is characteristic of a supercritical Hopf bifurcation and is very similar in its properties to a mean-field magnetic phase transition. Nevertheless, it is important to appreciate that bifurcations and thermodynamic phase transitions are similar not identical.

There are some points I would like to make about the differences between thermodynamic phase transitions and bifurcations/transitions far from equilibrium.

• Phase transitions are only perfectly sharp in the thermodynamic limit, i.e., $N→∞$ whereas bifurcations do not rely on that limit for sharpness.

• There are constraints on the relationships among scaling exponents based on known thermodynamic relationships that do not exist for general non-equilibrium phenomena.

• Despite some similarities, bifurcations or other transitions in non-equilibrium systems are not the same as thermodynamic phase transitions and the use of the term “phase transition” for all such behavior can be highly misleading.

In the cryogenic Rayleigh-Bénard (RB) system described above, the state evolves to one of increasing complexity as R increases. From a limit cycle oscillation, the system undergoes another bifurcation to a quasiperiodic two-frequency state with frequencies f₁ and f₂. This state undergoes the nonlinear phenomena of mode-locking where the winding number $W=f1/f2$ scans through a series of mode-locked intervals with rational values P/Q as some parameter is varied. Such a structure is seen in the experimental results as $R/Rc$ is varied as shown in Fig. 2 where prominent rational values of 3/20, 2/13, 3/19, and 4/15 are labeled. The dynamics of two frequency states can be thought to lie on a torus in phase space such that a Poincaré section (in this case through the small radius of the torus) yields a set of points that form a circle-like mapping as successive trajectories intersect the plane of the section. An example from the experimental data is shown in the inset of Fig. 2. The resulting dynamics can be represented by a mapping of the angle for each point at successive crossings of the general form: $θn+1=θn+Ω+(K/2π)F(2πθn)$⁠, where F(x) is a nonlinear periodic function (⁠$F(x)=sin(x)$ in the standard circle map²¹).

Something that is possible in cryogenic convection is to consider the transient relaxation within mode-locked intervals in a similar manner to the supercritical bifurcation described above. The reason for this capability is that the boundaries equilibrate on a fast time scale compared to the fluid, something difficult to achieve at room temperature owing to the thermal diffusivity of the boundary material, i.e., copper or aluminum. So consider a mode-locked state where the representation in the Poincaré section is a set of P points. A sudden change of control parameter, i.e., $R/Rc$⁠, induces a change in the positions of these points, and one can consider the transient relaxation from one state to the next. The sequence of points on the transient Poincaré section gives insight into the stability eigenvalues of the steady state solution. At the edge of the locked interval, one has a pair of real eigenvalues $λ1,2$ such that their product is one, i.e., $λ1=1/λ2$⁠. For sufficiently high nonlinearity, there is a transition to a spiral focus solution with a complex conjugate eigenvalue pair with $λ1,2=λr±iλI$⁠. In Fig. 3(a), one sees the sequence of points describing the decay of the transient state to the steady state in which the real part of the eigenvalue is determined by the exponential decay of the radial distance to the steady solution, i.e., $r∼e−λrn$ and the imaginary part $λI∼Δθ/Δn$⁠. The variation of λ_r and λ_I are shown in Figs. 3(b) and 3(c) as $R/Rc$ is varied and the state moves into and out of a mode locked interval. Experimentally, one can only measure the smaller real eigenvalue pair because the contribution from the larger one quickly decays.²⁰ 

From the structure of the mode-locked tongues in the cryogenic convection system, it is possible to estimate the critical line for the universal quasiperiodic transition to chaos and to probe the dynamical structure of the universal chaotic state.^18–20 In most other systems for which the quasiperiodic transition to chaos was investigated, for example, in convection in mercury,²² there was one natural internal mode with frequency f_i modulated by a parametric oscillation with adjustable amplitude and frequency f_e (for a nice review of experiments and application of theory, see Ref. 23). These controls provide adjustable winding number and nonlinear coupling to allow fine tuning to the golden mean ratio $Wg=(5−1)/2$⁠, a special point at which the convergence to universal properties is most rapid. In our system, the interval of ratios was naturally determined and thus the analysis was more complicated. On the other hand, the signal to noise ratio was about 1000, so precise measurements could be made.

The signature of the chaotic state at the transition to chaos is the nonlinear dynamics that segments the unit circle to produce a fractal distribution of sizes. This process is schematically illustrated in Fig. 4(a) where the temporal sequence of points on the line is labeled from 0 to 11. Segment lengths are determined from the positions of the various points so for example, at level zero one has one segment of size $Δ00=|x0−x1|$ (here the superscript refers to the level). The dynamics creates other segments systematically, and these are ordered upon successive iterations by an understanding of the universal quasiperiodic dynamics. So at level 1 there are two intervals $Δ01=|x0−x2|$ and $Δ11=|x1−x3|$ and at level 2 there are three intervals $Δ02=|x0−x3|, Δ12=|x1−x4|$⁠, and $Δ22=|x2−x5|$⁠; arrows show the connectivity defining these segments. The ratios of segments of the same type at successively higher levels are labeled $σji=Δji+1/Δji$ or, for example, $σ02=Δ02/Δ01$⁠. Upon many iterations of the map, one accumulates many segments that can be evaluated using the multi-fractal formalism²⁴ with a representation using the multi-fractal $f(α)$ function. In Fig. 4(b), I show the computed $f(α)$ distribution for experimental data and from predictions of the universal quasiperiodic theory (solid line). The agreement is excellent, providing a necessary but insufficient demonstration of the correspondence of the theory with the empirical evidence because many possible dynamics can produce the same $f(α)$ distribution. A much sharper test is to evaluate the coupling constants $σji$ which order the segments according to the dynamics. In Fig. 4(c), we show such a “function” with a direct comparison of the experimental data with the theoretical prediction. The excellent correspondence strongly supports the conclusion that the experimental system is in the universality class of the quasiperiodic route to chaos. This finer point was largely missed by the intersecting math, physics, and engineering communities because $f(α)$ curves were easy to compute, whereas one really had to be careful and understand the application of the theory to real experiment which few had the combined tools to attack. Fortunately for me, Ronnie Mainieri was one of those individuals and mastered many of the subtleties associated with the analysis.

Finally, with respect to chaos and fractals, much attention in the 1980s focused upon the direct calculation of the fractal dimension of chaotic attractors.²⁵ One of the first such direct computations of a fractal dimension was for a fluid system,¹¹ a demonstration that helped establish the physical relevance of the mathematical notions of chaos and low dimensional dynamical systems. Despite the conceptual interest and easy calculation of the fractal dimension, its utility in describing the physics of a real system has been disappointing. In Fig. 5, I show the variation of the fractal dimension computed from data in the quasiperiodic regime of the cryogenic convection experiment.¹⁸ The trend shows the mode-locking and quasiperiodic states with fractal dimension D_f in the neighborhood of 1 and 2, respectively. At higher values of $R/Rc$⁠, D_f increases towards 3 indicating chaotic states with more degrees of freedom but it is hard to gain much insight from these data. In retrospect, I think that fractal dimension is a pretty crude tool with which to look at dynamical systems and never provided the kind of impact of other measures of dynamical behavior such as Lyapunov exponents which is considerably harder to measure.

Recently, there has been renewed interest in a nonlinear dynamics approach to Lagrangian trajectories of quasi two-dimensional fluid flows, in particular, the elucidation of the mixing manifolds of a passive scalar field. This “Lagrangian coherent structure” approach^26,27 has significant power in quantitatively describing the mixing dynamics of periodically driven quasi two-dimensional flows with spatially random forcing,²⁸ where one can measure the largest Lyapunov exponent of the flow and identify the stable and unstable manifolds of the stretching field. In an extension of this approach to non-periodic flows,²⁹ one needs to consider finite time Lyapunov exponents and stretching fields that depend on the time considered. In Fig. 6, we show in (a) an example of a quasi two-dimension turbulent flow where the vorticity field of a flowing soap film is represented³⁰ and in (b) a stretching field for an electromagnetically forced stratified flow with the same random field forcing arrangement used previously²⁸ but at higher forcing amplitude, i.e., higher turbulent Reynolds number $Re=urmsLo/ν$ (L_o is the injection scale). Interestingly, it was found²⁹ that the largest Lyapunov exponent $λ1≈σrms/3$ where σ_rms is the root-mean-square rate of strain in the fluid, a considerably easier quantity to measure, see Fig. 6(c). So some lessons learned about chaotic dynamical systems:

• The transient dynamics in Poincaré sections to measure the stability of fixed points of the stable manifolds of the attractor can be highly informative if a separation of times scales can be obtained between the control parameter response and the dynamic response of the nonlinear state.

• Physical scientists like simple diagnostics such as $f(α)$ curves and fractal dimension, whereas more sophisticated analysis at the intersection of physics and mathematics is less accessible: there is a sweet spot between ease and sophistication that tends towards ease and away from more complex analysis.

• Fractal dimension does not tell you very much about the nonlinear dynamics of a system— it is a blunt instrument without much physical utility.

• Lyapunov exponents are harder to measure but give one useful information about the dynamics. There may, however, be easier ways to get equivalent information.

Pattern formation far from equilibrium is one of the great successes of the nonlinear system's era³¹ with diverse types of instabilities and resultant patterns. The theory built on a tradition of linear stability analysis of nonlinear systems dating back to Lord Rayleigh³² and summarized for many fluid problems by Chandrasekhar.³³ For convection, the pattern-forming state at finite amplitude³⁴ shows significant variation depending on fluid parameters and forcings (e.g., rotation, magnetic field, etc.). Here, I give a few examples of pattern formation in thermal convection: a one-dimensional traveling wave mode describable by the complex Ginzburg-Landau (CGL) equation^35,36 and two-dimensional patterns with defects including dislocations,³⁷ focus defects,³⁸ and spiral defect chaos.^39,40

In the 1D example, the stability and dynamics of the experimental state—a traveling wave of the sidewall mode of rotating Rayleigh-Bénard convection,⁴¹ an example of which is shown in the inset of Fig. 7—are very well described by the complex Ginzburg Landau equation

where real coefficients are: time scale τ₀, spatial scale ξ₀, linear bifurcation parameter $ϵ=ΔT/ΔTc−1$ (ΔT_c is the critical temperature difference for the onset of convection with critical wave number k_c), and nonlinear parameter g. The coefficients c_i control the dependence of the frequency ω on wave number k and ϵ. Azimuthally periodic conditions constrain solutions to a discrete set of k and the marginal stability solution yields a parabolic relationship $ϵM=ξ02(k−kc)2$ or $ΔT(k)/ΔTc=1+ξ02(k−kc)2$ as indicated by the solid line in Fig. 7(a), where the open circles are the experimental data. The dashed line and solid squares in Fig. 7(a) denote the nonlinear Eckhaus-Benjamin-Feir stability boundary given by $ϵE≈3ϵM$⁠. If ϵ is reduced for fixed k, the state becomes unstable at ϵ_E and a transition occurs towards a state with a k within the stable wave number band through the formation of space-time defects, see Fig. 7(b). After the transition, the disturbance in A and k decays according to a linear phase equation where the phase of the state $ϕ$ is related to a small wave number disturbance p such that $p=∂xϕ$⁠. In Fig. 7(c), we show the decaying (long-wavelength) frequency (top) and amplitude (bottom) modulation of the phase disturbances (we demodulate the state using the steady state values of ω and k). Overall, the description of the experimental data in terms of the CGL equation is excellent with small deviations owing to higher order terms.^35,36 In general,³¹ this type of analysis works quite well for 1D systems including for Rayleigh-Bénard convection in restricted geometry and for Taylor-Couette flow.

In large-aspect ratio RB convection, and in large-aspect ratio systems in general, the situation is considerably more complicated. In Fig. 8, some examples of 2D patterns are shown for convection in compressed gases with Prandtl number about 1.^37,38,40 Close to the onset of convection, the infinite layer solution is straight parallel rolls³¹ which can be realized in cylindrical containers if the side-wall boundary forcing is weak as in Fig. 8(a), whereas concentric rolls that respect the symmetry of the container are observed for stronger sidewall forcing, see Fig. 8(b). As the Rayleigh number increases above its critical value, dislocation and sidewall focus defects are nucleated and generate a complex pattern dynamics for which there is only qualitative understanding.^37,38 Even more challenging is the fascinating spiral defect chaos dynamics that occurs in low-Prandtl number convection for $R/Rc>1.5$⁠.^39,40 Part of the mechanism for this state is the mean flow which is both hard to measure and is large at low Prandtl number. A reflection of the importance of the mean flow is that an external rotation preferentially selects spirals with the handedness of the rotation.⁴² Spiral defect chaos is amenable to dynamical systems analysis through computation of the Lyapunov spectrum for high-resolution numerical simulations of convection.⁴³ An important conclusion of that work is that spiral defect chaos is extensive, i.e., the degrees of freedom grow linearly with the systems size. This is a nice example where, when used selectively, dynamical systems analysis yields important diagnostics about a complex dynamic state with many spatial degrees of freedom.

Some lessons learned about pattern formation:

• Weakly nonlinear analysis using amplitude equations provides powerful descriptors of physically realizable 1D patterns and some simpler 2D patterns. Symmetry considerations play a large role in determining the appropriate amplitude equation.

• Two-dimensional patterns quickly evolve with increasing forcing away from special points with weakly nonlinear solutions to states with complex dynamics that have not yielded to general quantitative understanding.

• The regions where analysis is effective are small and the extension to more highly nonlinear states appears very limited.

• The practical applications of pattern formation have been few, perhaps owing to the small range of applicability of weakly nonlinear solutions.

Turbulence provides one of the most challenging problems in classical physics owing to the nonlinearity of the system and the strong interactions across a large range of spatial and temporal scales.^44,45 Turbulence is not a single problem but rather a large set of problems involving strong nonlinearity. A few features of isotropic, homogeneous turbulence in three spatial dimensions for sufficiently high Reynolds number are worth noting: energy moves, on average, from larger spatial scales to smaller scales where it is dissipated by viscous forces, and the power spectrum of energy is $E(k)∼k−5/3$⁠, which reflects conservation of energy over a range of scales where forcing and dissipation are weak, the so-called “inertial range.” Turbulence is generated by body forces or at boundaries and plays a critical role in almost every aspect of life including drag on cars and airplanes, mixing in internal combustion engines, and transport of heat and salinity/moisture in oceans/atmospheres.⁴⁵ Here, I will illustrate several very different examples of turbulence where the motivations, challenges and analysis are all quite different.

Spatial dimension has a large effect on turbulent flows. For example, turbulence in two dimensions⁴⁶ is quite different from 3D turbulence in that energy flows to larger spatial scales while mean-squared vorticity, i.e., enstrophy, cascades to smaller scales where it is dissipated by viscosity. The vorticity field for a quasi 2D turbulent flow, generated by electromagnetic forcing of a thin layer of salt water, is shown in Fig. 9(a) where blue (red) is positive (negative) vorticity and circles denote regions of coherent structures of the flow.⁴⁷ For flowing soap films, one can statistically correlate regions of strain, i.e., hyperbolic coherent structures, with the forward cascade of enstrophy,^30,48 whereas the dynamics in elliptic regions characterized by strong vorticity plays little role in that cascade. This correlation quantitatively demonstrates the mechanism for the turbulent enstrophy cascade as the stretching of vorticity gradients in regions of strong strain. On the other hand, for the inverse energy cascade, no obvious correlation with coherent structures provides insights into the mechanism for the turbulent cascade. In both cases described, a technique for measuring the spatial structure of the turbulent cascade quantity based on filtering was employed.^30,46–48

Another example of coherent structures in turbulent flow occurs in rotating Rayleigh-Bénard convection in the geostrophic regime.⁴⁹ In Fig. 9(b), an organized array of upwelling thermal columns, imaged using optical shadowgraph, is shown.⁵⁰ These columns have strong swirling character with cyclonic vorticity and contribute significantly to the transport of heat in rotating convection.^51,52

In the previous two examples of turbulent flow, coherent structures are useful diagnostics of the turbulence, providing insight into their properties. On the other hand, many turbulent fluid flows have structure where coherent structures are much less obvious. In Fig. 9(c), the density field (obtained using laser-induced fluorescence) of an unstable stratified shear flow⁵³ shows minimal evidence of coherent structures while the density field for late-time Rayleigh-Taylor turbulence reveals even less structure.⁵⁴ 

Some lessons learned about turbulence:

• Turbulence is not just one problem but rather many manifestations of strongly nonlinear, multiscale dynamics. In some cases, the idealized notions of isotropic, homogeneous turbulence are useful, whereas in other cases more phenomenological or empirical concepts prove more helpful.

• Dimensionality has a large influence on turbulence, similar in spirit, if not in detail, to dimensionality effects on thermodynamic phase transitions. For example, the very direction of the mean flux of energy reverses upon a reduction of spatial dimension from three to two.

• Coherent structures can be useful in characterizing turbulence but may also be a distraction where the human eye, which is very attuned to detecting patterns, attributes more significance to the structure than it perhaps deserves.

• The impact of chaos, patterns, and coherent structure concepts on turbulence has been less than what was hoped for but provides significant stimulation for thinking about the challenging problem of fluid turbulence.

This brief sojourn through core thematic areas of nonlinear science reflects both the richness of the underlying phenomena and the tremendous advances in understanding that 30 years of theoretical, numerical, and experimental research has provided. The interplay between concepts of universality on the one hand and the appreciation that details matter, especially for applications, has been fascinating. Nonlinear science was a perfect background for “interdisciplinary” science because of the notions of universality but it never found a comfortable home anywhere because applications depend on disciplinary detail and disciplines are slow to evolve. For example, the “modern physics” taught today in most physics departments had their origins almost a century ago whereas much of the “classical physics” of the 19th century has reemerged as highly relevant in many areas of active research from geophysics and astrophysics to both hard and especially soft matter. Other disciplines are similarly slow to change their emphasis. Nevertheless, the nonlinear science of many complex problems will become of increasing importance in the 21st century and the lessons of nonlinear dynamics, chaos, patterns, and coherent structures will be rediscovered and applied by new generations of scientists. Thus, in my opinion, the paradigms articulated so nicely by David Campbell continue to have resonance but have become too narrow to encompass the spectrum of exciting nonlinear, non-equilibrium science that drive today's research agenda. It will be exciting and interesting to see whither goest Nonlinear Science or as David might have said memores acti prudentes futuri.

This work described here was accomplished in collaboration with many scientific colleagues including Guenter Ahlers, Jun Chen, Brent Daniel, David Egolf, Doyne Farmer, Yuchou Hu, Ioannis Kevrekidis, Ning Li, Yuanming Liu, Ronnie Mainieri, Philippe Odier, Michael Rivera, Victor Steinberg, Tim Sullivan, Michael Twardos, Peter Vorobieff, John Wheatley, and Fang Zhong. Thanks to Greg Swift for keeping me honest with hard questions and sound advice. David Campbell and the Center for Nonlinear Studies provided me a scientific home when the nonlinear science I was doing seemed outside the scope of much of the disciplinary focus of the Laboratory. The work reviewed in this paper was performed at Los Alamos National Laboratory under DOE Contract Nos. W-7405-ENG-36 and DE-AC52-06NA25396. Funding included the support of the LANL LDRD Program and the DOE BES/DMR Program.