Many nonequilibrium systems, such as biochemical reactions and socioeconomic interactions, can be described by reaction–diffusion equations that demonstrate a wide variety of complex spatiotemporal patterns. The diversity of the morphology of these patterns makes it difficult to classify them quantitatively, and they are often described visually. Hence, searching through a large parameter space for patterns is a tedious manual task. We discuss how convolutional neural networks can be used to scan the parameter space, investigate existing patterns in more detail, and aid in finding new groups of patterns. As an example, we consider the Gray–Scott model for which training data are easy to obtain. Due to the popularity of machine learning in many scientific fields, well maintained open source toolkits are available that make it easy to implement the methods that we discuss in advanced undergraduate and graduate computational physics projects.
REFERENCES
1.
M.
Cross
and
H.
Greenside
, Pattern Formation and Dynamics in Nonequilibrium Systems
(
Cambridge U. P
.,
Cambridge
, 2009
).2.
P.
Gray
and
S. K.
Scott
, “
Autocatalytic reactions in the isothermal, continuous stirred tank reactor: isolas and other forms of multistability
,” Chem. Eng. Sci.
38
, 29
–43
(1983
).3.
J. D.
Murray
, Mathematical Biology II: Spatial Models and Biomedical Applications
, 3rd ed. (
Springer Science + Business Media, LCC
,
New York
, 2003
).4.
J. E.
Pearson
, “
Complex patterns in a simple system
,” Science
261
, 189
–192
(1993
).5.
W.
Mazin
et al, “
Pattern formation in the bistable Gray-Scott model
,” Math. Comput. Simul.
40
, 371
–396
(1996
).6.
V.
Castets
,
E.
Dulos
,
J.
Boissonade
, and
P.
De Kepper
, “
Experimental evidence of a sustained standing Turing-type nonequilibrium chemical pattern
,” Phys. Rev. Lett.
64
, 2953
–2956
(1990
).7.
K. J.
Lee
,
W. D.
McCormick
,
Q.
Ouyang
, and
H. L.
Swinney
, “
Pattern formation by interacting chemical fronts
,” Science
261
, 192
–194
(1993
).8.
Q.
Ouyang
and
H. L.
Swinney
, “
Transition to chemical turbulence
,” Chaos
1
, 411
–420
(1991
).9.
K. R.
Mecke
, “
Morphological characterization of patterns in reaction-diffusion systems
,” Phys. Rev. E
53
, 4794
–4800
(1996
).10.
J.
Guiu-Souto
,
J.
Carballido-Landeira
, and
A. P.
Muñuzuri
, “
Characterizing topological transitions in a Turing-pattern-forming reaction-diffusion system
,” Phys. Rev. E
85
, 056205
(2012
).11.
12.
F.
Monti
et al, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(
IEEE
, 2017
), pp. 5115
–5124
.13.
C.
Scholz
and
S.
Scholz
, see https://github.com/coscholz1984/GS_CNN for “CNNs for Gray-Scott Pattern Classification-Python Scripts and Pretrained Models, 2021
.”14.
C.
Scholz
and
S.
Scholz
, see https://osf.io/byrzm/ for “CNNs for Gray-Scott Pattern Classification-Raw Datasets, 2021
.”15.
J. D.
Murray
, Mathematical Biology I. An Introduction
, 3rd ed. (
Springer Science + Business Media, LCC
,
New York
, 2002
).16.
R. P.
Munafo
, “
Stable localized moving patterns in the 2D Gray-Scott model
,” arXiv:1501.01990 (2014
).17.
See supplementary material at https://www.scitation.org/doi/suppl/10.1119/5.0065458 for Extended explanation of Gray-Scott model and movie of dynamic patterns, detailed explanation of convolution features, minimal code examples, saliency maps, and 3D convolution filter example (
2021
).18.
R.
Munafo
, see http://www.mrob.com/pub/comp/xmorphia/ogl/index.html for “WebGL Gray-Scott Explorer, 2021
.”19.
W. H.
Press
,
S. A.
Teukolsky
,
W. T.
Vetterling
, and
B. P.
Flannery
, Numerical Recipes with Source Code CD-ROM 3rd Edition: The Art of Scientific Computing
(
Cambridge U. P
.,
Cambridge
, 2007
).20.
Y. C.
Ji
and
F. H.
Fenton
, “
Numerical solutions of reaction-diffusion equations: Application to neural and cardiac models
,” Am. J. Phys.
84
, 626
–638
(2016
).21.
J.
Wei
and
M.
Winter
, “
Existence and stability of multiple-spot solutions for the Gray-Scott model in R2
,” Physica D
176
, 147
–180
(2003
).22.
S.
Wolfram
, “
Universality and complexity in cellular automata
,” Physica D
10
, 1
–35
(1984
).23.
G. E.
Schröder-Turk
et al, “
Tensorial Minkowski functionals and anisotropy measures for planar patterns
,” J. Microsc.
238
, 57
–74
(2010
).24.
C.
Scholz
,
G. E.
Schröder-Turk
, and
K.
Mecke
, “
Pattern-fluid interpretation of chemical turbulence
,” Phys. Rev. E
91
, 042907
(2015
).25.
D. C.
Montgomery
, Design and Analysis of Experiments
(
John Wiley & Sons
,
New York
, 2017
).26.
For linear models this is done by computing the minimum norm solution to a set of linear equations. For nonlinear models we typically need to approximate the solution iteratively starting from an initial estimate of the parameters.
27.
For
predictor variables, even if we include only zeroth and first order terms (1+49152) and two factor interactions (all possible products of two different predictor variables
), we would need to fit more than 1.2 billion parameters.
28.
J. J.
Hopfield
, “
Neural networks and physical systems with emergent collective computational abilities
,” Proc. Natl. Acad. Sci. U. S. A.
79
, 2554
–2558
(1982
).29.
P.
Ramachandran
,
B.
Zoph
, and
Q. V.
Le
, “
Searching for activation functions
,” arXiv:1710.05941 (2017
).30.
31.
A.
Paszke
et al, in Advances in Neural Information Processing Systems
32 (
Curran Associates, Inc
.,
New York
, 2019
), pp. 8024
–8035
.32.
The MathWorks
, MATLAB Deep Learning Toolbox
(
The MathWorks
,
Natick
, 2020
).33.
M.
Abadi
et al, “
TensorFlow: Large-scale machine learning on heterogeneous systems
,” arXiv:1603.04467 (2015
).34.
There are also some constants that affect the optimization procedure. For instance the learning rate, i.e., the step size of the gradient descent. Such parameters are called hyperparameters. Often these hyperparameters are automatically determined or can be kept at default values, but sometimes adjustments might be necessary. For instance, the network might not have enough feature maps, neurons and layers to fit the data. A frequent problem is also low quality of training data due to human error, which requires manual revision. For example, patterns might have been mislabeled. Or the training data might include patterns that do not represent a class very well.
35.
T.
Kadir
and
M.
Brady
, “
Saliency, scale and image description
,” Int. J. Comput. Vision
45
, 83
–105
(2001
).36.
K.
Simonyan
,
A.
Vedaldi
, and
A.
Zisserman
, “
Deep inside convolutional networks: Visualising image classification models and saliency maps
,” arXiv:1312.6034 (2013
).37.
B.
Haibe-Kains
et al, “
Transparency and reproducibility in artificial intelligence
,” Nature
586
, E14
–E16
(2020
).38.
S.
Lapuschkin
et al, “
Unmasking clever Hans predictors and assessing what machines really learn
,” Nat. Commun.
10
, 1096
(2019
).39.
J.
Han
,
A.
Jentzen
, and
E.
Weinan
, “
Solving high-dimensional partial differential equations using deep learning
,” Proc. Natl. Acad. Sci. U. S. A.
115
, 8505
–8510
(2018
).40.
M.
Raissi
,
P.
Perdikaris
, and
G. E.
Karniadakis
, “
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
,” J. Comput. Phys.
378
, 686
–707
(2019
).41.
A.
Li
,
R.
Chen
,
A. B.
Farimani
, and
Y. J.
Zhang
, “
Reaction diffusion system prediction based on convolutional neural network
,” Sci. Rep.
10
, 1
–9
(2020
).© 2022 Author(s).
2022
Author(s)
AAPT members receive access to the American Journal of Physics and The Physics Teacher as a member benefit. To learn more about this member benefit and becoming an AAPT member, visit the Joining AAPT page.
