The gold-standard definition of the Direct Simulation Monte Carlo (DSMC) method is given in the 1994 book by Bird [Molecular Gas Dynamics and the Direct Simulation of Gas Flows (Clarendon Press, Oxford, UK, 1994)], which refined his pioneering earlier papers in which he first formulated the method. In the intervening 25 years, DSMC has become the method of choice for modeling rarefied gas dynamics in a variety of scenarios. The chief barrier to applying DSMC to more dense or even continuum flows is its computational expense compared to continuum computational fluid dynamics methods. The dramatic (nearly billion-fold) increase in speed of the largest supercomputers over the last 30 years has thus been a key enabling factor in using DSMC to model a richer variety of flows, due to the method’s inherent parallelism. We have developed the open-source SPARTA DSMC code with the goal of running DSMC efficiently on the largest machines, both current and future. It is largely an implementation of Bird’s 1994 formulation. Here, we describe algorithms used in SPARTA to enable DSMC to operate in parallel at the scale of many billions of particles or grid cells, or with billions of surface elements. We give a few examples of the kinds of fundamental physics questions and engineering applications that DSMC can address at these scales.
REFERENCES
1.
S. I.
Abarzhi
, “Review of theoretical modelling approaches of Rayleigh-Taylor instabilities and turbulent mixing
,” Philos. Trans. R. Soc., A
368
, 1809
–1828
(2010
).2.
J. L.
Barber
, K.
Kadau
, T. C.
Germann
, P. S.
Lomdahl
, B. L.
Holian
, and B. J.
Alder
, “Atomistic simulation of the Rayleigh-Taylor instability
,” J. Phys.: Conf. Ser.
46
, 58
–62
(2006
).3.
T. J.
Bartel
, S. J.
Plimpton
, and C. R.
Justiz
, “Direct Monte Carlo simulation of ionized rarefied flows on large MIMD parallel supercomputers
,” in Proceedings of the 18th International Symposium on Rarefied Gas Dynamics
(AIAA
, Vancouver, Canada
, 1992
), Vol. A94-30156, pp. 155
–165
.4.
M. J.
Berger
and S. H.
Bokhari
, “A partitioning strategy for nonuniform problems on multiprocessors
,” IEEE Trans. Comput.
C-36
(5
), 570
–580
(1987
).5.
G. A.
Bird
, “Approach to translational equilibrium in a rigid sphere gas
,” Phys. Fluids
6
, 1518
(1963
).6.
7.
G. A.
Bird
, Molecular Gas Dynamics and the Direct Simulation of Gas Flows
(Clarendon Press
, Oxford, UK
, 1994
).8.
G. A.
Bird
, The DSMC Method, Version 1.1, 2013
.9.
A.
Borner
, F.
Panerai
, and N. N.
Mansour
, “High temperature permeability of fibrous materials using direct simulation Monte Carlo
,” Int. J. Heat Mass Transfer
106
, 1318
–1326
(2017
).10.
A. B.
Bortz
, M. H.
Kalos
, and J. L.
Lebowitz
, “New algorithm for Monte Carlo simulation of Ising spin systems
,” J. Comput. Phys.
17
(1
), 10
–18
(1975
).11.
See http://software.nasa.gov/software/ARC-17920-1 for Porous Microstructure Analysis (PuMA) code page on the NASA software website.
12.
See https://www.openfoam.com for OpenFoam CFD code website.
13.
See http://sparta.sandia.gov for SPARTA DSMC code website.
14.
COMSOL AB
, COMSOL Multiphysics User’s Guide, COMSOL AB, Stockholm, Sweden
, 2008
.15.
L.
Custodio
, T.
Etiene
, S.
Pesco
, and C.
Silva
, “Practical considerations on marching cubes 33 topological correctness
,” Comput. Graphics
37
(7
), 840
–850
(2013
).16.
17.
R. E.
Duff
, F. H.
Harlow
, and C. W.
Hirt
, “Effects of diffusion on interface instability between gases
,” Phys. Fluids
5
(4
), 417
–425
(1962
).18.
H. C.
Edwards
, C. R.
Trott
, and D.
Sunderland
, “Kokkos: Enabling manycore performance portability through polymorphic memory access patterns
,” J. Parallel Distrib. Comput.
74
(12
), 3202
–3216
(2014
).19.
J. C.
Ferguson
, F.
Panerai
, A.
Borner
, and N. N.
Mansour
, “PuMA: The porous microstructure analysis software
,” SoftwareX
7
, 81
–87
(2018
).20.
J. C.
Ferguson
, F.
Panerai
, F.
Lachaud
, A.
Martin
, S.
Bailey
, and N. N.
Mansour
, “Modeling the oxidation of low-density carbon fiber material based on micro-tomography
,” Carbon
96
, 57
–65
(2016
).21.
E.
Fermi
and J.
von Neumann
, “Taylor instability of incompressible liquids. Part 1. Taylor instability of an incompressible liquid. Part 2. Taylor instability at the boundary of two incompressible liquids
,” Technical Report AECU-2979, Los Alamos Scientific Laboratory
, Los Alamos, NM
, 1953
.22.
U.
Fey
, M.
König
, and H.
Eckelmann
, “A new Strouhal-Reynolds-number relationship for the circular cylinder in the range 47 < Re < 2 × 105
,” Phys. Fluids
10
(7
), 1547
–1549
(1998
).23.
24.
M. A.
Gallis
, T. P.
Koehler
, J. R.
Torczynski
, and S. J.
Plimpton
, “Direct simulation Monte Carlo investigation of the Richtmyer-Meshkov instability
,” Phys. Fluids
27
(8
), 084105
(2015
).25.
M. A.
Gallis
, T. P.
Koehler
, J. R.
Torczynski
, and S. J.
Plimpton
, “Direct simulation Monte Carlo investigation of the Rayleigh-Taylor instability
,” Phys. Rev. Fluids
1
(4
), 043403
(2016
).26.
M. A.
Gallis
, T. P.
Koehler
, J. R.
Torczynski
, S. J.
Plimpton
, and G.
Papadakis
, “Molecular-level simulations of turbulence and its decay
,” Phys. Rev. Lett.
118
(6
), 064501
(2017
).27.
M. A.
Gallis
, T. P.
Koehler
, J. R.
Torczynski
, S. J.
Plimpton
, and G.
Papadakis
, “Gas-kinetic simulation of sustained turbulence in minimal Couette flow
,” Phys. Rev. Fluids
3
(7
), 071402(R)
(2018
).28.
M. A.
Gallis
and J. R.
Torczynski
, “Investigation of the ellipsoidal-statistical Bhatnagar-Gross-Krook kinetic model applied to gas-phase transport of heat and tangential momentum between parallel walls
,” Phys. Fluids
23
(3
), 030601
(2011
).29.
M. A.
Gallis
, J. R.
Torczynski
, S. J.
Plimpton
, D. J.
Rader
, and T.
Koehler
, “Direct simulation Monte Carlo: The quest for speed
,” in Proceedings of the 29th International Symposium on Rarefied Gas Dynamics
(AIP
, Xi’an, China
, 2014
), Vol. 1628, pp. 27
–36
.30.
M. A.
Gallis
, J. R.
Torczynski
, and D. J.
Rader
, “Molecular gas dynamics observations of Chapman-Enskog behavior and departures therefrom in nonequilibrium gases
,” Phys. Rev. E
69
(4
), 042201
(2004
).31.
D. T.
Gillespie
, “A general method for numerically simulating the stochastic time evolution of coupled chemical reactions
,” J. Chem. Phys.
22
, 403
–434
(1976
).32.
D. T.
Gillespie
, “Exact stochastic simulation of coupled chemical reactions
,” J. Phys. Chem.
81
(25
), 2340
–2361
(1977
).33.
P. A.
Gnoffo
, “CFD validation studies for hypersonic flow prediction
,” AIAA Paper 2001-1025, 2001
.34.
J. K.
Harvey
, “A review of a validation exercise on the use of the DSMC method to compute viscous/inviscid interactions in hypersonic flow
,” AIAA Paper 2003-3643, 2003
.35.
M. S.
Holden
, T. P.
Wadhams
, G. V.
Candler
, and J. K.
Harvey
, “Measurements in regions of low density laminar shock wave/boundary layer interaction in hypervelocity flows and comparison with Navier-Stokes predictions
,” AIAA Paper 2003-1131, 2003
.36.
M. S.
Ivanov
, S. F.
Gimelshein
, and G. N.
Markelov
, “Statistical simulation of the transition between regular and Mach reflection in steady flows
,” Comput. Math. Appl.
35
, 113
–125
(1998
).37.
M. S.
Ivanov
, A. V.
Kashkovsky
, S. F.
Gimelshein
, G. N.
Markelov
, A. A.
Alexeenko
, Y. A.
Bondar
, G. A.
Zhukova
, S. B.
Nikiforov
, and P. V.
Vaschenkov
, “SMILE system for 2D/3D DSMC computations
,” in Proceedings of the 25th International Symposium on Rarefied Gas Dynamics
(AIP
, St. Petersburg, Russian Federation
, 2006
).38.
K.
Kadau
, J. L.
Barber
, T. C.
Germann
, B. L.
Holian
, and B. J.
Alder
, “Atomistic methods in fluid simulation
,” Philos. Trans. R. Soc., A
368
, 1547
–1560
(2010
).39.
K.
Kadau
, T. C.
Germann
, N. G.
Hadjiconstantinou
, P. S.
Lomdahl
, G.
Dimonte
, B. L.
Holian
, and B. J.
Alder
, “Nanohydrodynamics simulations: An atomistic view of the Rayleigh-Taylor instability
,” Proc. Natl. Acad. Sci. U. S. A.
101
(16
), 5851
–5855
(2004
).40.
A.
Kashkovsky
, “3D DSMC computations on a heterogeneous CPU-GPU cluster with a large number of GPUs
,” AIP Conf. Proc.
1628
, 192
(2014
).41.
A. G.
Klothakis
, I. K.
Nikolos
, T. P.
Koehler
, M. A.
Gallis
, and S. J.
Plimpton
, “Validation simulations of the DSMC code SPARTA
,” in Proceedings of the 30th International Symposium on Rarefied Gas Dynamics
(AIP
, 2016
), Vol. 1786, p. 050016
.42.
F.
Kull
, “Theory of Rayleigh-Taylor instability
,” Phys. Rep.
206
(5
), 197
–325
(1991
).43.
D.
Layzer
, “On the instability of superposed fluids in a gravitational field
,” Astrophys. J.
122
(1
), 1
–12
(1955
).44.
G. J.
LeBeau
and F. E.
Lumpkin
III, “Application highlights of the DSMC analysis code (DAC) software for simulating rarefied flows
,” Comput. Methods Appl. Mech. Eng.
191
, 595
–609
(2001
).45.
T.
Lewiner
, H.
Lopes
, A.
Vieira
, and G.
Tavares
, “Efficient implementation of marching cubes’ cases with topological guarantees
,” J. Graphics Tools
8
(2
), 1
–15
(2003
).46.
D. J.
Lewis
, “The instability of liquid surfaces when accelerated in a direction perpendicular to their planes. II
,” Proc. R. Soc. London, Ser. A
202
(1068
), 81
–96
(1950
).47.
See https://github.com/kokkos/kokkos for Kokkos library website.
48.
See http://www.top500.org for Top 500 list website.
49.
W. E.
Lorensen
and H. E.
Cline
, “Marching cubes: A high resolution 3D surface construction algorithm
,” ACM SIGGRAPH Comput. Graphics
21
, 163
–169
(1987
).50.
N. N.
Mansour
, F.
Panerai
, A.
Martin
, D. Y.
Parkinson
, A.
MacDowell
, T.
Fast
, G.
Vignoles
, and J.
Lachaud
, “A new approach to light-weight ablators analysis: From micro-tomography measurements to statistical analysis and modeling
,” in 44th AIAA Thermophysics Conference
(AIAA
, 2013
), p. 2768
.51.
C.
Maple
, “Geometric design and space planning using the marching squares and marching cube algorithms
,” in Proceedings of the 2003 International Conference on Geometric Modeling and Graphics
, 2003
.52.
J.
Mościński
, W.
Alda
, M.
Bubak
, W.
Dzwinel
, J.
Kitowski
, M.
Pogoda
, and D. A.
Yuen
, Molecular Dynamics Simulations of Rayleigh-Taylor Instability
, edited by D.
Stauffer
(World Scientific Publishing Company
, Singapore
, 1997
).53.
J. N.
Moss
and G. A.
Bird
, “Direct simulation Monte Carlo simulations of hypersonic flows with shock interactions
,” AIAA J.
43
, 2565
–2573
(2005
).54.
I.
Nompelis
and T. E.
Schwartzentruber
, “Strategies for parallelization of the DSMC method
,” AIAA Paper 2013-1204, 2013
.55.
S. J.
Plimpton
, B.
Hendrickson
, and J. R.
Stewart
, “A parallel rendezvous algorithm for interpolation between multiple grids
,” J. Parallel Distrib. Comput.
64
, 266
–276
(2004
).56.
T. J.
Scanlon
, E.
Roohi
, C.
White
, M.
Darbandi
, and J. M.
Reese
, “An open source, parallel DSMC code for rarefied gas flows in arbitrary geometries
,” Comput. Fluids
39
, 2078
–2089
(2010
).57.
D. H.
Sharp
, “An overview of Rayleigh-Taylor instability
,” Physica D
12
(1-3
), 3
–18
(1984
).58.
E. C.
Stern
, S.
Poovathingal
, I.
Nompelis
, T. E.
Schwartzentruber
, and G. V.
Candler
, “Nonequilibrium flow through porous thermal protection materials, Part I: Numerical methods
,” J. Comput. Phys.
380
, 408
–426
(2019
).59.
G. I.
Taylor
, “The instability of liquid surfaces when accelerated in a direction perpendicular to their planes
,” Proc. R. Soc. London, Ser. A
201
(1065
), 192
–196
(1950
).60.
D. J.
Tritton
, Physical Fluid Dynamics
(Oxford Science Publications; Clarendon Press
, Oxford
, 1988
).61.
W. M.
Trott
, D. J.
Rader
, J. N.
Castaneda
, J. R.
Torczynski
, and M. A.
Gallis
, “Measurements of gas-surface accommodation
,” Rev. Sci. Instrum.
82
(3
), 035120
(2011
).62.
W.
Wagner
, “A convergence proof for Bird’s direct simulation Monte Carlo method for the Boltzmann equation
,” J. Stat. Phys.
66
(3-4
), 1011
(1992
).63.
K.
Weiler
and P.
Atherton
, “Hidden surface removal using polygon area sorting
,” ACM SIGGRAPH Comput. Graphics
11
, 214
–222
(1977
).64.
C.
White
, M. K.
Borg
, T. J.
Scanlon
, S. M.
Longshaw
, B.
John
, D. R.
Emerson
, and J. M.
Reese
, “dsmcFoam+: An OpenFOAM based direct simulation Monte Carlo solver
,” Comput. Phys. Commun.
224
, 22
–43
(2018
).65.
C.
Zhang
and T. E.
Schwartzentruber
, “Robust cut-cell algorithms for DSMC implementations employing multi-level Cartesian grids
,” Comput. Fluids
69
, 122
–135
(2012
).66.
Y.
Zhou
, “Rayleigh-Taylor and Richtmyer-Meshkov instability induced flow, turbulence, and mixing. I
,” Phys. Rep.
720-722
, 1
–136
(2017
).67.
Y.
Zhou
, “Rayleigh-Taylor and Richtmyer-Meshkov instability induced flow, turbulence, and mixing. II
,” Phys. Rep.
723-725
, 1
–160
(2017
).© 2019 Author(s).
2019
Author(s)
You do not currently have access to this content.
