We present a scheme that spatially couples two gyrokinetic codes using first-principles. Coupled equations are presented and a necessary and sufficient condition for ensuring accuracy is derived. This new scheme couples both the field and the particle distribution function. The coupling of the distribution function is only performed once every few time-steps, using a five-dimensional (5D) grid to communicate the distribution function between the two codes. This 5D grid interface enables the coupling of different types of codes and models, such as particle and continuum codes, or delta-f and total-f models. Transferring information from the 5D grid to the marker particle weights is achieved using a new resampling technique. Demonstration of the coupling scheme is shown using two XGC gyrokinetic simulations for both the core and edge. We also apply the coupling scheme to two continuum simulations for a one-dimensional advection–diffusion problem.
References
1.
J.
Dominski
, S.
Ku
, C.-S.
Chang
, J.
Choi
, E.
Suchyta
, S.
Parker
, S.
Klasky
, and A.
Bhattacharjee
, “A tight-coupling scheme sharing minimum information across a spatial interface between gyrokinetic turbulence codes
,” Phys. Plasmas
25
, 072308
(2018
).2.
H.
Childs
, E.
Brugger
, B.
Whitlock
, J.
Meredith
, S.
Ahern
, D.
Pugmire
, K.
Biagas
, M.
Miller
, C.
Harrison
, G. H.
Weber
, H.
Krishnan
, T.
Fogal
, A.
Sanderson
, C.
Garth
, E. W.
Bethel
, D.
Camp
, O.
Rübel
, M.
Durant
, J. M.
Favre
, and P.
Navrátil
, “VisIt: An end-user tool for visualizing and analyzing very large data
,” in High Performance Visualization–Enabling Extreme-Scale Scientific Insight
(2012
), pp. 357
–372
.3.
M.
Kim
, J.
Kress
, J.
Choi
, N.
Podhorszki
, S.
Klasky
, M.
Wolf
, K.
Mehta
, K.
Huck
, B.
Geveci
, S.
Phillip
, R.
Maynard
, H.
Guo
, T.
Peterka
, K.
Moreland
, C.-S.
Chang
, J.
Dominski
, M.
Churchill
, and D.
Pugmire
, “In situ analysis and visualization of fusion simulations: Lessons learned
,” in High Performance Computing
, edited by R.
Yokota
, M.
Weiland
, J.
Shalf
, and S.
Alam
(Springer International Publishing
, Cham
, 2018
), pp. 230
–242
.4.
F.
Jenko
, W.
Dorland
, M.
Kotschenreuther
, and B. N.
Rogers
, “Electron temperature gradient driven turbulence
,” Phys. Plasmas
7
, 1904
–1910
(2000
).5.
T.
Görler
, X.
Lapillonne
, S.
Brunner
, T.
Dannert
, F.
Jenko
, F.
Merz
, and D.
Told
, “The global version of the gyrokinetic turbulence code GENE
,” J. Comput. Phys.
230
, 7053
–7071
(2011
).6.
Y.
Chen
and S. E.
Parker
, “Electromagnetic gyrokinetic particle-in-cell turbulence simulation with realistic equilibrium profiles and geometry
,” J. Comput. Phys.
220
, 839
–855
(2007
).7.
S.
Ku
, C. S.
Chang
, R.
Hager
, R. M.
Churchill
, G. R.
Tynan
, I.
Cziegler
, M.
Greenwald
, J.
Hughes
, S. E.
Parker
, M. F.
Adams
, E.
D'Azevedo
, and P.
Worley
, “A fast low-to-high confinement mode bifurcation dynamics in the boundary-plasma gyrokinetic code XGC1
,” Phys. Plasmas
25
, 056107
(2018
).8.
D.
Faghihi
, V.
Carey
, C.
Michoski
, R.
Hager
, S.
Janhunen
, C.
Chang
, and R.
Moser
, “Moment preserving constrained resampling with applications to particle-in-cell methods
,” J. Comput. Phys.
409
, 109317
(2020
).9.
Take the example of an edge simulation that does not include the core region where a heat source is localized. This edge simulation will receive the information about the heat source from the core simulation in its buffer region.
10.
Equivalent to the condition we discuss in Sec. III D for gyrokinetic simulations.
11.
Note that if we use higher order stencils, the error would propagate more rapidly, in which case one needs to synchronize the buffer more frequently.
12.
A. J.
Brizard
and T. S.
Hahm
, “Foundations of nonlinear gyrokinetic theory
,” Rev. Mod. Phys.
79
, 421
–468
(2007
).13.
A.
Mishchenko
, A.
Koenies
, and R.
Hatzky
, “Particle simulations with a generalized gyrokinetic solver
,” Phys. Plasmas
12
, 062305
(2005
).14.
J.
Dominski
, B. F.
McMillan
, S.
Brunner
, G.
Merlo
, T.-M.
Tran
, and L.
Villard
, “An arbitrary wavelength solver for global gyrokinetic simulations. Application to the study of fine radial structures on microturbulence due to non-adiabatic passing electron dynamics
,” Phys. Plasmas
24
, 022308
(2017
).15.
A.
Mishchenko
, R.
Hatzky
, E.
Sonnendrücker
, R.
Kleiber
, and A.
Könies
, “An iterative approach to an arbitrarily short-wavelength solver in global gyrokinetic simulations
,” J. Plasma Phys.
85
, 905850116
(2019
).16.
L.
Qing
, L.
Jeremy
, T.
Yuan
, A.
Hasan
, P.
Norbert
, C. J.
Youl
, K.
Scott
, T.
Roselyne
, L.
Jay
, O.
Ron
, P.
Manish
, S.
Nagiza
, S.
Karsten
, S.
Arie
, W.
Matthew
, W.
Kesheng
, and Y.
Weikuan
, “Hello ADIOS: The challenges and lessons of developing leadership class I/O frameworks
,” Concurrency Comput.: Pract. Exp.
26
, 1453
–1473
(2014
).17.
W. F.
Godoy
, N.
Podhorszki
, R.
Wang
, C.
Atkins
, G.
Eisenhauer
, J.
Gu
, P.
Davis
, J.
Choi
, K.
Germaschewski
, K.
Huck
, A.
Huebl
, M.
Kim
, J.
Kress
, T.
Kurc
, Q.
Liu
, J.
Logan
, K.
Mehta
, G.
Ostrouchov
, M.
Parashar
, F.
Poeschel
, D.
Pugmire
, E.
Suchyta
, K.
Takahashi
, N.
Thompson
, S.
Tsutsumi
, L.
Wan
, M.
Wolf
, K.
Wu
, and S.
Klasky
, “ADIOS 2: The adaptable input output system. A framework for high-performance data management
,” SoftwareX
12
, 100561
(2020
).18.
In the total-f version of XGC, the marker weight is evolved by using the “direct delta-f” method such that the weight evolution is deduced from conservation of phase space volume and (collisionless gyrokinetic equation). Collision effects are computed separately on the weights with a collision operator.
19.
R.
Hager
, V.
Carey
, J.
Dominski
, S.
Ku
, and C. S.
Chang
, “Realization of moment-preserving resampling in a particle-in-cell code
,” in
International Conference for the Numerical Simulation of Plasma (2019
).20.
B.
McMillan
, S.
Jolliet
, A.
Bottino
, P.
Angelino
, T.
Tran
, and L.
Villard
, “Rapid Fourier space solution of linear partial integro-differential equations in toroidal magnetic confinement geometries
,” Comput. Phys. Commun.
181
, 715
–719
(2010
).21.
Communicating the distribution function between the core and edge in narrow buffer regions is necessary when one simulation does not cover the full domain. This can be easily understood in a case where a source of
heat is in the core region, i.e., outside of the edge simulation domain. The knowledge of the source term is then communicated by the core simulation to the edge simulation in the interface buffer.22.
I.
Foster
, M.
Ainsworth
, B.
Allen
, J.
Bessac
, F.
Cappello
, J. Y.
Choi
, E.
Constantinescu
, P. E.
Davis
, S.
Di
, W.
Di
et al., “Computing just what you need: Online data analysis and reduction at extreme scales
,” in European Conference on Parallel Processing
(Springer
, Cham
, 2017
), pp. 3
–19
.23.
G.
Merlo
, J.
Dominski
, A.
Bhattacharjee
, C. S.
Chang
, F.
Jenko
, S.
Ku
, E.
Lanti
, and S.
Parker
, “Cross-verification of the global gyrokinetic codes GENE and XGC
,” Phys. Plasmas
25
, 062308
(2018
).24.
G.
Merlo
, S.
Janhunen
, F.
Jenko
, A.
Bhattacharjee
, C.
Chang
, J.
Cheng
, P.
Davis
, J.
Dominski
, K.
Germaschewski
, R.
Hager
, S.
Klasky
, S.
Parker
, and E.
Suchyta
, “First coupled GENE-XGC microturbulence simulations
,” Phys. Plasmas
28
, 012303
(2020
).25.
J.
Cheng
, J.
Dominski
, Y.
Chen
, H.
Chen
, G.
Merlo
, S.
Ku
, R.
Hager
, C.
Chang
, E.
Suchyta
, E.
D'azevedo
, E.
Suchyta
, S.
Sreepathi
, S.
Klasky
, F.
Jenko
, A.
Bhattacharjee
, and S.
Parker
, “Spatial core-edge coupling of the particle-in-cell gyrokinetic codes GEM and XGC
,” Phys. Plasmas
27
, 122510
(2020
).26.
J.
Dominski
, J.
Cheng
, G.
Merlo
, V.
Carey
, R.
Hager
, L.
Ricketson
, J.
Choi
, K.
Germaschewski
, S.
Ku
, A.
Mollen
, N.
Podhorszki
, D.
Pugmire
, E.
Suchyta
, P.
Trivedi
, R.
Wang
, C. S.
Chang
, J.
Hittinger
, F.
Jenko
, S.
Klasky
, S. E.
Parker
, and A.
Bhattacharjee
(2020
). “Data from figures in ‘Spatial coupling of gyrokinetic simulations, a generalized scheme based on first-principles
,’” Dataset.© 2021 Author(s).
2021
Author(s)
You do not currently have access to this content.
