We present a Graphics Processing Unit (GPU)-accelerated version of the real-space SPARC electronic structure code for performing Kohn–Sham density functional theory calculations within the local density and generalized gradient approximations. In particular, we develop a modular math-kernel based implementation for NVIDIA architectures wherein the computationally expensive operations are carried out on the GPUs, with the remainder of the workload retained on the central processing units (CPUs). Using representative bulk and slab examples, we show that relative to CPU-only execution, GPUs enable speedups of up to 6× and 60× in node and core hours, respectively, bringing time to solution down to less than 30 s for a metallic system with over 14 000 electrons and enabling significant reductions in computational resources required for a given wall time.
REFERENCES
1.
P.
Hohenberg
and W.
Kohn
, Phys. Rev.
136
, B864
(1964
).2.
W.
Kohn
and L. J.
Sham
, Phys. Rev.
140
, A1133
(1965
).3.
K.
Burke
, J. Chem. Phys.
136
, 150901
(2012
).4.
A. D.
Becke
, J. Chem. Phys.
140
, 18A301
(2014
).5.
R.
Martin
, Electronic Structure: Basic Theory and Practical Methods
(Cambridge University Press
, 2004
).6.
G.
Kresse
and J.
Furthmüller
, Phys. Rev. B
54
, 11169
(1996
).7.
S. J.
Clark
, M. D.
Segall
, C. J.
Pickard
, P. J.
Hasnip
, M. I. J.
Probert
, K.
Refson
, and M. C.
Payne
, Z. Kristallogr. Cryst. Mater.
220
, 567
(2005
).8.
X.
Gonze
, J.-M.
Beuken
, R.
Caracas
, F.
Detraux
, M.
Fuchs
, G.-M.
Rignanese
, L.
Sindic
, M.
Verstraete
, G.
Zerah
, F.
Jollet
et al, Comput. Mater. Sci.
25
, 478
(2002
).9.
P.
Giannozzi
, S.
Baroni
, N.
Bonini
, M.
Calandra
, R.
Car
, C.
Cavazzoni
, D.
Ceresoli
, G. L.
Chiarotti
, M.
Cococcioni
, I.
Dabo
et al, J. Phys.: Condens. Matter
21
, 395502
(2009
).10.
11.
S.
Ismail-Beigi
and T. A.
Arias
, Comput. Phys. Commun.
128
, 1
(2000
).12.
F.
Gygi
, IBM J. Res. Dev.
52
, 137
(2008
).13.
M.
Valiev
, E. J.
Bylaska
, N.
Govind
, K.
Kowalski
, T. P.
Straatsma
, H. J. J.
Van Dam
, D.
Wang
, J.
Nieplocha
, E.
Apra
, T. L.
Windus
et al, Comput. Phys. Commun.
181
, 1477
(2010
).14.
S.
Goedecker
, Rev. Mod. Phys.
71
, 1085
(1999
).15.
D. R.
Bowler
and T.
Miyazaki
, Rep. Prog. Phys.
75
, 036503
(2012
).16.
J.
Aarons
, M.
Sarwar
, D.
Thompsett
, and C.-K.
Skylaris
, J. Chem. Phys.
145
, 220901
(2016
).17.
A. D.
Becke
, Int. J. Quantum Chem.
36
, 599
(1989
).18.
J. R.
Chelikowsky
, N.
Troullier
, and Y.
Saad
, Phys. Rev. Lett.
72
, 1240
(1994
).19.
L.
Genovese
, A.
Neelov
, S.
Goedecker
, T.
Deutsch
, S. A.
Ghasemi
, A.
Willand
, D.
Caliste
, O.
Zilberberg
, M.
Rayson
, A.
Bergman
et al, J. Chem. Phys.
129
, 014109
(2008
).20.
A. P.
Seitsonen
, M. J.
Puska
, and R. M.
Nieminen
, Phys. Rev. B
51
, 14057
(1995
).21.
S. R.
White
, J. W.
Wilkins
, and M. P.
Teter
, Phys. Rev. B
39
, 5819
(1989
).22.
J.-I.
Iwata
, D.
Takahashi
, A.
Oshiyama
, T.
Boku
, K.
Shiraishi
, S.
Okada
, and K.
Yabana
, J. Comput. Phys.
229
, 2339
(2010
).23.
E.
Tsuchida
and M.
Tsukada
, Phys. Rev. B
52
, 5573
(1995
).24.
Q.
Xu
, P.
Suryanarayana
, and J. E.
Pask
, J. Chem. Phys.
149
, 094104
(2018
).25.
P.
Suryanarayana
, K.
Bhattacharya
, and M.
Ortiz
, J. Comput. Phys.
230
, 5226
(2011
).26.
P.
Suryanarayana
, V.
Gavini
, T.
Blesgen
, K.
Bhattacharya
, and M.
Ortiz
, J. Mech. Phys. Solids
58
, 256
(2010
).27.
C.-K.
Skylaris
, P. D.
Haynes
, A. A.
Mostofi
, and M. C.
Payne
, J. Chem. Phys.
122
, 084119
(2005
).28.
D. R.
Bowler
, R.
Choudhury
, M. J.
Gillan
, and T.
Miyazaki
, Phys. Status Solidi B
243
, 989
(2006
).29.
S.
Das
, P.
Motamarri
, V.
Subramanian
, D. M.
Rogers
, and V.
Gavini
, Comput. Phys. Commun.
280
, 108473
(2022
).30.
A.
Castro
, H.
Appel
, M.
Oliveira
, C. A.
Rozzi
, X.
Andrade
, F.
Lorenzen
, M. A. L.
Marques
, E. K. U.
Gross
, and A.
Rubio
, Phys. Status Solidi B
243
, 2465
(2006
).31.
E. L.
Briggs
, D. J.
Sullivan
, and J.
Bernholc
, Phys. Rev. B
54
, 14362
(1996
).32.
J.-L.
Fattebert
, J. Comput. Phys.
149
, 75
(1999
).33.
F.
Shimojo
, R. K.
Kalia
, A.
Nakano
, and P.
Vashishta
, Comput. Phys. Commun.
140
, 303
(2001
).34.
S.
Ghosh
and P.
Suryanarayana
, Comput. Phys. Commun.
216
, 109
(2017
).35.
T. A.
Arias
, Rev. Mod. Phys.
71
, 267
(1999
).36.
J. E.
Pask
and P. A.
Sterne
, Model. Simul. Mater. Sci. Eng.
13
, R71
(2005
).37.
L.
Lin
, J.
Lu
, L.
Ying
, and W.
E
, J. Comput. Phys.
231
, 2140
(2012
).38.
T. L.
Beck
, Rev. Mod. Phys.
72
, 1041
(2000
).39.
Y.
Saad
, J. R.
Chelikowsky
, and S. M.
Shontz
, SIAM Rev.
52
, 3
(2010
).40.
A.
Sharma
and P.
Suryanarayana
, Phys. Rev. B
103
, 035101
(2021
).41.
S.
Ghosh
, A. S.
Banerjee
, and P.
Suryanarayana
, Phys. Rev. B
100
, 125143
(2019
).42.
Y.
Hasegawa
, J.-I.
Iwata
, M.
Tsuji
, D.
Takahashi
, A.
Oshiyama
, K.
Minami
, T.
Boku
, F.
Shoji
, A.
Uno
, M.
Kurokawa
et al, in Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
(ACM
, 2011
), p. 1
.43.
D.
Osei-Kuffuor
and J.-L.
Fattebert
, Phys. Rev. Lett.
112
, 046401
(2014
).44.
P.
Suryanarayana
, P. P.
Pratapa
, A.
Sharma
, and J. E.
Pask
, Comput. Phys. Commun.
224
, 288
(2018
).45.
V.
Gavini
, S.
Baroni
, V.
Blum
, D. R.
Bowler
, A.
Buccheri
, J. R.
Chelikowsky
, S.
Das
, W.
Dawson
, P.
Delugas
, M.
Dogan
, et al, arXiv:2209.12747 (2022
).46.
Q.
Xu
, A.
Sharma
, B.
Comer
, H.
Huang
, E.
Chow
, A. J.
Medford
, J. E.
Pask
, and P.
Suryanarayana
, SoftwareX
15
, 100709
(2021
).47.
S.
Ghosh
and P.
Suryanarayana
, Comput. Phys. Commun.
212
, 189
(2017
).48.
R. C.
Walker
and A. W.
Goetz
, Electronic Structure Calculations on Graphics Processing Units: From Quantum Chemistry to Condensed Matter Physics
(John Wiley and Sons
, 2016
).49.
X.
Gonze
, F.
Jollet
, F. A.
Araujo
, D.
Adams
, B.
Amadon
, T.
Applencourt
, C.
Audouze
, J.-M.
Beuken
, J.
Bieder
, A.
Bokhanchuk
et al, Comput. Phys. Commun.
205
, 106
(2016
).50.
L.
Genovese
, M.
Ospici
, T.
Deutsch
, J.-F.
Méhaut
, A.
Neelov
, and S.
Goedecker
, J. Chem. Phys.
131
, 034103
(2009
).51.
L.
Genovese
, B.
Videau
, D.
Caliste
, J.-F.
Méhaut
, S.
Goedecker
, and T.
Deutsch
, Electronic Structure Calculations on Graphics Processing Units: From Quantum Chemistry to Condensed Matter Physics
(2016
), pp. 115
–134
.52.
P.
Manninen
and P.
Öster
, Applied Parallel and Scientific Computing: 11th International Conference, PARA 2012
(Springer
, Helsinki, Finland
, 2013
), Vol. 7782
.53.
S.
Maintz
, B.
Eck
, and R.
Dronskowski
, Comput. Phys. Commun.
182
, 1421
(2011
).54.
M.
Hacene
, A.
Anciaux-Sedrakian
, X.
Rozanska
, D.
Klahr
, T.
Guignon
, and P.
Fleurat-Lessard
, J. Comput. Chem.
33
, 2581
(2012
).55.
W.
Jia
, J.
Wang
, X.
Chi
, and L.-W.
Wang
, Comput. Phys. Commun.
211
, 8
(2017
).56.
X.
Andrade
, J.
Alberdi-Rodriguez
, D. A.
Strubbe
, M. J. T.
Oliveira
, F.
Nogueira
, A.
Castro
, J.
Muguerza
, A.
Arruabarrena
, S. G.
Louie
, A.
Aspuru-Guzik
et al, J. Phys.: Condens. Matter
24
, 233202
(2012
).57.
K.
Wilkinson
and C.-K.
Skylaris
, J. Comput. Chem.
34
, 2446
(2013
).58.
W.
Jia
, J.
Fu
, Z.
Cao
, L.
Wang
, X.
Chi
, W.
Gao
, and L.-W.
Wang
, J. Comput. Phys.
251
, 102
(2013
).59.
J.
Romero
, E.
Phillips
, G.
Ruetsch
, M.
Fatica
, F.
Spiga
, and P.
Giannozzi
, in International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems
(Springer
, 2018
), pp. 67
–87
.60.
W. P.
Huhn
, B.
Lange
, V. W.-z.
Yu
, M.
Yoon
, and V.
Blum
, Comput. Phys. Commun.
254
, 107314
(2020
).61.
Y.
Zhou
, Y.
Saad
, M. L.
Tiago
, and J. R.
Chelikowsky
, J. Comput. Phys.
219
, 172
(2006
).62.
Y.
Zhou
, Y.
Saad
, M. L.
Tiago
, and J. R.
Chelikowsky
, Phys. Rev. E
74
, 066704
(2006
).63.
A.
Sharma
and P.
Suryanarayana
, J. Chem. Phys.
149
, 194104
(2018
).64.
P.
Micikevicius
, in Proceedings of 2nd Workshop on General Purpose Processing on Graphics Processing Units
(Association for Computing Machinery
, 2009
), pp. 79
–84
.65.
S. J.
Sahoo
, X.
Jing
, P.
Suryanarayana
, and A. J.
Medford
, J. Phys. Chem. C
126
, 2121
(2022
).66.
J. P.
Perdew
and A.
Zunger
, Phys. Rev. B
23
, 5048
(1981
).67.
J. P.
Perdew
, K.
Burke
, and M.
Ernzerhof
, Phys. Rev. Lett.
77
, 3865
(1996
).68.
H. J.
Monkhorst
and J. D.
Pack
, Phys. Rev. B
13
, 5188
(1976
).69.
Y.
Zhang
and W.
Yang
, Phys. Rev. Lett.
80
, 890
(1998
).70.
D.
Hamann
, Phys. Rev. B
88
, 085117
(2013
).71.
M. F.
Shojaei
, J. E.
Pask
, A. J.
Medford
, and P.
Suryanarayana
, Comput. Phys. Commun.
283
, 108594
(2023
).72.
P. P.
Pratapa
and P.
Suryanarayana
, Chem. Phys. Lett.
635
, 69
(2015
).73.
A. S.
Banerjee
, P.
Suryanarayana
, and J. E.
Pask
, Chem. Phys. Lett.
647
, 31
(2016
).74.
G. P.
Kerker
, Phys. Rev. B
23
, 3082
(1981
).75.
S.
Kumar
, Q.
Xu
, and P.
Suryanarayana
, Chem. Phys. Lett.
739
, 136983
(2020
).76.
P.
Suryanarayana
, P. P.
Pratapa
, and J. E.
Pask
, Comput. Phys. Comm.
234
, 278
(2019
).77.
P. P.
Pratapa
, P.
Suryanarayana
, and J. E.
Pask
, J. Comput. Phys.
306
, 43
(2016
).78.
Lawrence Livermore National Laboratory (LLNL) high performance computing systems: https://hpc.llnl.gov/hardware/compute-platforms, accessed 06 January 2023.
79.
H.
Huang
and E.
Chow
, in Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
,2022
.80.
P.
Suryanarayana
, Chem. Phys. Lett.
584
, 182
(2013
).81.
P. P.
Pratapa
, P.
Suryanarayana
, and J. E.
Pask
, “Spectral quadrature method for accurate O(N) electronic structure calculations of metals and insulators
,” Comp. Phys. Commun.
200
, 96
–107
(2016
).82.
© 2023 Author(s). Published under an exclusive license by AIP Publishing.
2023
Author(s)
You do not currently have access to this content.
