We introduce a machine learning method in which energy solutions from the Schrödinger equation are predicted using symmetry adapted atomic orbital features and a graph neural-network architecture. OrbNet is shown to outperform existing methods in terms of learning efficiency and transferability for the prediction of density functional theory results while employing low-cost features that are obtained from semi-empirical electronic structure calculations. For applications to datasets of drug-like molecules, including QM7b-T, QM9, GDB-13-T, DrugBank, and the conformer benchmark dataset of Folmsbee and Hutchison [Int. J. Quantum Chem. (published online) (2020)], OrbNet predicts energies within chemical accuracy of density functional theory at a computational cost that is 1000-fold or more reduced.
REFERENCES
1.
A. P.
Bartók
, M. C.
Payne
, R.
Kondor
, and G.
Csányi
, “Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons
,” Phys. Rev. Lett.
104
, 136403
(2010
).2.
M.
Rupp
, A.
Tkatchenko
, K.-R.
Müller
, and O. A.
von Lilienfeld
, “Fast and accurate modeling of molecular atomization energies with machine learning
,” Phys. Rev. Lett.
108
, 58301
(2012
).3.
A. S.
Christensen
, L. A.
Bratholm
, F. A.
Faber
, and O. A.
von Lilienfeld
, “FCHL revisited: Faster and more accurate quantum machine learning
,” J. Chem. Phys.
152
, 044107
(2020
).4.
A. S.
Christensen
and O. A.
von Lilienfeld
, “Operator quantum machine learning: Navigating the chemical space of response properties
,” CHIMIA
73
, 1028
–1031
(2019
).5.
R.
Ramakrishnan
, P. O.
Dral
, M.
Rupp
, and O. A.
von Lilienfeld
, “Big data meets quantum chemistry approximations: The Δ-machine learning approach
,” J. Chem. Theory Comput.
11
, 2087
(2015
).6.
T. T.
Nguyen
, E.
Székely
, G.
Imbalzano
, J.
Behler
, G.
Csányi
, M.
Ceriotti
, A. W.
Götz
, and F.
Paesani
, “Comparison of permutationally invariant polynomials, neural networks, and Gaussian approximation potentials in representing water interactions through many-body expansions
,” J. Chem. Phys.
148
, 241725
(2018
).7.
S.
Fujikake
, V. L.
Deringer
, T. H.
Lee
, M.
Krynski
, S. R.
Elliott
, and G.
Csányi
, “Gaussian approximation potential modeling of lithium intercalation in carbon nanostructures
,” J. Chem. Phys.
148
, 241714
(2018
).8.
A.
Grisafi
, A.
Fabrizio
, B.
Meyer
, D. M.
Wilkins
, C.
Corminboeuf
, and M.
Ceriotti
, “Transferable machine-learning model of the electron density
,” ACS Cent. Sci.
5
, 57
–64
(2019
).9.
Y.
Zhai
, A.
Caruso
, S.
Gao
, and F.
Paesani
, “Active learning of many-body configuration space: Application to the Cs+–water MB-nrg potential energy function as a case study
,” J. Chem. Phys.
152
, 144103
(2020
).10.
J. S.
Smith
, O.
Isayev
, and A. E.
Roitberg
, “ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
,” Chem. Sci.
8
, 3192
–3203
(2017
).11.
J. S.
Smith
, B. T.
Nebgen
, R.
Zubatyuk
, N.
Lubbers
, C.
Devereux
, K.
Barros
, S.
Tretiak
, O.
Isayev
, and A. E.
Roitberg
, “Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning
,” Nat. Commun.
10
, 1
–8
(2019
).12.
N.
Lubbers
, J. S.
Smith
, and K.
Barros
, “Hierarchical modeling of molecular energies using a deep neural network
,” J. Chem. Phys.
148
, 241715
(2018
).13.
G.
Montavon
, M.
Rupp
, V.
Gobre
, A.
Vazquez-Mayagoitia
, K.
Hansen
, A.
Tkatchenko
, K.-R.
Müller
, and O.
Anatole von Lilienfeld
, “Machine learning of molecular electronic properties in chemical compound space
,” New J. Phys.
15
, 095003
(2013
).14.
K.
Hansen
, G.
Montavon
, F.
Biegler
, S.
Fazli
, M.
Rupp
, M.
Scheffler
, O. A.
von Lilienfeld
, A.
Tkatchenko
, and K.-R.
Müller
, “Assessment and validation of machine learning methods for predicting molecular atomization energies
,” J. Chem. Theory Comput.
9
, 3404
(2013
).15.
P.
Gasparotto
and M.
Ceriotti
, “Recognizing molecular patterns by machine learning: An agnostic structural definition of the hydrogen bond
,” J. Chem. Phys.
141
, 174110
(2014
).16.
J.
Behler
, “Perspective: Machine learning potentials for atomistic simulations
,” J. Chem. Phys.
145
, 170901
(2016
).17.
S.
Kearnes
, K.
McCloskey
, M.
Berndl
, V.
Pande
, and P.
Riley
, “Molecular graph convolutions: Moving beyond fingerprints
,” J. Comput. Aided Mol. Des.
30
, 595
(2016
).18.
K. T.
Schütt
, F.
Arbabzadah
, S.
Chmiela
, K.-R.
Müller
, and A.
Tkatchenko
, “Quantum-chemical insights from deep tensor neural networks
,” Nat. Commun.
8
, 13890
(2017
).19.
F.
Brockherde
, L.
Vogt
, L.
Li
, M. E.
Tuckerman
, K.
Burke
, and K.-R.
Müller
, “Bypassing the Kohn-Sham equations with machine learning
,” Nat. Commun.
8
, 872
(2017
).20.
Z.
Wu
, B.
Ramsundar
, E. N.
Feinberg
, J.
Gomes
, C.
Geniesse
, A. S.
Pappu
, K.
Leswing
, and V.
Pande
, “MoleculeNet: A benchmark for molecular machine learning
,” Chem. Sci.
9
, 513
(2018
).21.
K.
Yao
, J. E.
Herr
, D. W.
Toth
, R.
McKintyre
, and J.
Parkhill
, “The TensorMol-0.1 model chemistry: A neural network augmented with long-range physics
,” Chem. Sci.
9
, 2261
–2269
(2018
).22.
H.
Li
, C.
Collins
, M.
Tanha
, G. J.
Gordon
, and D. J.
Yaron
, “A density functional tight binding layer for deep learning of chemical Hamiltonians
,” J. Chem. Theory Comput.
14
, 5764
–5776
(2018
).23.
L.
Zhang
, J.
Han
, H.
Wang
, R.
Car
, and W.
E
, “Deep potential molecular dynamics: A scalable model with the accuracy of quantum mechanics
,” Phys. Rev. Lett.
120
, 143001
(2018
).24.
M.
Welborn
, L.
Cheng
, and T. F.
Miller
III, “Transferability in machine learning for electronic structure via the molecular orbital basis
,” J. Chem. Theory Comput.
14
, 4772
–4779
(2018
).25.
L.
Cheng
, M.
Welborn
, A. S.
Christensen
, and T. F.
Miller
III, “A universal density matrix functional from molecular orbital-based machine learning: Transferability across organic molecules
,” J. Chem. Phys.
150
, 131103
(2019
).26.
L.
Cheng
, N. B.
Kovachki
, M.
Welborn
, and T. F.
Miller
III, “Regression clustering for improved accuracy and training costs with molecular-orbital-based machine learning
,” J. Chem. Theory Comput.
15
, 6668
–6677
(2019
).27.
S.
Dick
and M.
Fernandez-Serra
, “Machine learning accurate exchange and correlation functionals of the electronic density
,” Nat. Commun.
11
, 3509
(2020
).28.
Y.
Chen
, L.
Zhang
, H.
Wang
, and W.
E
, “Ground state energy functional with Hartree-Fock efficiency and chemical accuracy
,” J. Phys. Chem. A
124
(35
), 7155
–7165
(2020
).29.
T. N.
Kipf
and M.
Welling
, “Semi-supervised classification with graph convolutional networks
,” in International Conference on Learning Representations
, 2017
.30.
P.
Veličković
, G.
Cucurull
, A.
Casanova
, A.
Romero
, P.
Liò
, and Y.
Bengio
, “Graph attention networks
,” in International Conference on Learning Representations
, 2018
.31.
K.
Yang
, K.
Swanson
, W.
Jin
, C.
Coley
, P.
Eiden
, H.
Gao
, A.
Guzman-Perez
, T.
Hopper
, B.
Kelley
, M.
Mathea
, A.
Palmer
, V.
Settels
, T.
Jaakkola
, K.
Jensen
, and R.
Barzilay
, “Analyzing learned molecular representations for property prediction
,” J. Chem. Inf. Model.
59
, 3370
–3388
(2019
).32.
K.
Schütt
, P.-J.
Kindermans
, H. E. S.
Felix
, S.
Chmiela
, A.
Tkatchenko
, and K.-R.
Müller
, “Schnet: A continuous-filter convolutional neural network for modeling quantum interactions
,” in Advances in Neural Information Processing Systems
(Neural Information Processing Systems Foundation, Inc.
, 2017
), pp. 991
–1001
.33.
O. T.
Unke
and M.
Meuwly
, “PhysNet: A neural network for predicting energies, forces, dipole moments, and partial charges
,” J. Chem. Theory Comput.
15
, 3678
–3693
(2019
).34.
J.
Klicpera
, J.
Groß
, and S.
Günnemann
, “Directional message passing for molecular graphs
,” in International Conference on Learning Representations
, 2019
.35.
Z.
Liu
, L.
Lin
, Q.
Jia
, Z.
Cheng
, Y.
Jiang
, Y.
Guo
, and J.
Ma
, “Transferable multi-level attention neural network for accurate prediction of quantum chemistry properties via multi-task learning
,” ChemRxiv:12588170.v1 (2020
).36.
S.
Grimme
, “A simplified Tamm-Dancoff density functional approach for the electronic excitation spectra of very large molecules
,” J. Chem. Phys.
138
, 244104
(2013
).37.
S.
Grimme
and C.
Bannwarth
, “Ultra-fast computation of electronic spectra for large systems by tight-binding based simplified Tamm-Dancoff approximation (sTDA-xTB)
,” J. Chem. Phys.
145
, 054103
(2016
).38.
T.
Risthaus
, A.
Hansen
, and S.
Grimme
, “Excited states using the simplified Tamm–Dancoff-approach for range-separated hybrid density functionals: Development and application
,” Phys. Chem. Chem. Phys.
16
, 14408
–14419
(2014
).39.
K.
He
, X.
Zhang
, S.
Ren
, and J.
Sun
, “Deep residual learning for image recognition
,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(IEEE
, 2016
), pp. 770
–778
.40.
A.
Vaswani
, N.
Shazeer
, N.
Parmar
, J.
Uszkoreit
, L.
Jones
, A. N.
Gomez
, Ł.
Kaiser
, and I.
Polosukhin
, “Attention is all you need
,” in Advances in Neural Information Processing Systems
(Neural Information Processing Systems Foundation, Inc.
, 2017
), pp. 5998
–6008
.41.
R. T.
McGibbon
, A. G.
Taube
, A. G.
Donchev
, K.
Siva
, F.
Hernández
, C.
Hargus
, K.-H.
Law
, J. L.
Klepeis
, and D. E.
Shaw
, “Improving the accuracy of Møller-Plesset perturbation theory with neural networks
,” J. Chem. Phys.
147
, 161725
(2017
).42.
J. T.
Margraf
and K.
Reuter
, “Making the coupled cluster correlation energy machine-learnable
,” J. Phys. Chem. A
122
, 6343
–6348
(2018
).43.
J.
Behler
and M.
Parrinello
, “Generalized neural-network representation of high-dimensional potential-energy surfaces
,” Phys. Rev. Lett.
98
, 146401
(2007
).44.
L.
Cheng
, M.
Welborn
, A. S.
Christensen
, and T. F.
Miller
, Thermalized (350k) QM7b, GDB-13, water, and short alkane quantum chemistry dataset including MOB-ML features, 2019
, https://data.caltech.edu/records/1177.45.
R.
Ramakrishnan
, P. O.
Dral
, M.
Rupp
, and O. A.
von Lilienfeld
, “Quantum chemistry structures and properties of 134 kilo molecules
,” Sci. Data
1
, 1
–7
(2014
).46.
L. C.
Blum
and J.-L.
Reymond
, “970 million druglike small molecules for virtual screening in the chemical universe database GDB-13
,” J. Am. Chem. Soc.
131
, 8732
(2009
).47.
V.
Law
, C.
Knox
, Y.
Djoumbou
, T.
Jewison
, A. C.
Guo
, Y.
Liu
, A.
Maciejewski
, D.
Arndt
, M.
Wilson
, V.
Neveu
, A.
Tang
, G.
Gabriel
, C.
Ly
, S.
Adamjee
, Z. T.
Dame
, B.
Han
, Y.
Zhou
, and D. S.
Wishart
, “DrugBank 4.0: Shedding new light on drug metabolism
,” Nucl. Acids Res.
42
, D1091
–D1097
(2014
).48.
D.
Folmsbee
and G.
Hutchison
, “Assessing conformer energies using electronic structure and machine learning methods
,” Int. J. Quantum Chem.
(published online).49.
S. H.
Vosko
, L.
Wilk
, and M.
Nusair
, “Accurate spin-dependent electron liquid correlation energies for local spin density calculations: A critical analysis
,” Can. J. Phys.
58
, 1200
(1980
).50.
C.
Lee
, W.
Yang
, and R. G.
Parr
, “Development of the Colle-Salvetti correlation-energy formula into a functional of the electron density
,” Phys. Rev. B
37
, 785
(1988
).51.
A. D.
Becke
, “Density-functional thermochemistry. III. The role of exact exchange
,” J. Chem. Phys.
98
, 5648
(1993
).52.
P. J.
Stephens
, F. J.
Devlin
, C. F.
Chabalowski
, and M. J.
Frisch
, “Ab initio calculation of vibrational absorption and circular dichroism spectra using density functional force fields
,” J. Phys. Chem.
98
, 11623
(1994
).53.
P. C.
Hariharan
and J. A.
Pople
, “The influence of polarization functions on molecular orbital hydrogenation energies
,” Theor. Chim. Acta
28
, 213
(1973
).54.
G.
Bussi
and M.
Parrinello
, “Accurate sampling using Langevin dynamics
,” Phys. Rev. E
75
, 056707
(2007
).55.
Y.-S.
Lin
, G.-D.
Li
, S.-P.
Mao
, and J.-D.
Chai
, “Long-range corrected hybrid density functionals with improved dispersion corrections
,” J. Chem. Theory Comput.
9
, 263
–272
(2013
).56.
F.
Weigend
and R.
Ahlrichs
, “Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence quality for H to Rn: Design and assessment of accuracy
,” Phys. Chem. Chem. Phys.
7
, 3297
(2005
).57.
R.
Polly
, H.-J.
Werner
, F. R.
Manby
, and P. J.
Knowles
, “Fast Hartree-Fock theory using local density fitting approximations
,” Mol. Phys.
102
, 2311
–2321
(2004
).58.
F.
Weigend
, “Hartree-Fock exchange fitting basis sets for H to Rn
,” J. Comput. Chem.
29
, 167
–175
(2008
).59.
D. G. A.
Smith
, L. A.
Burns
, A. C.
Simmonett
, R. M.
Parrish
, M. C.
Schieber
, R.
Galvelis
, P.
Kraus
, H.
Kruse
, R.
Di Remigio
, A.
Alenaizan
, A. M.
James
, S.
Lehtola
, J. P.
Misiewicz
, M.
Scheurer
, R. A.
Shaw
, J. B.
Schriber
, Y.
Xie
, Z. L.
Glick
, D. A.
Sirianni
, J. S.
O’Brien
, J. M.
Waldrop
, A.
Kumar
, E. G.
Hohenstein
, B. P.
Pritchard
, B. R.
Brooks
, H. F.
Schaefer
, A. Y.
Sokolov
, K.
Patkowski
, A. E.
DePrince
, U.
Bozkaya
, R. A.
King
, F. A.
Evangelista
, J. M.
Turney
, T. D.
Crawford
, and C. D.
Sherrill
, “Psi4 1.4: Open-source software for high-throughput quantum chemistry
,” J. Chem. Phys.
152
, 184108
(2020
).60.
S.
Grimme
, C.
Bannwarth
, and P.
Shushkov
, “A robust and accurate tight-binding quantum chemical method for structures, vibrational frequencies, and noncovalent interactions of large molecular systems parametrized for all spd-block elements (Z = 1–86)
,” J. Chem. Theory Comput.
13
, 1989
–2009
(2017
).61.
F.
Manby
, T.
Miller
, P.
Bygrave
, F.
Ding
, T.
Dresselhaus
, F.
Batista-Romero
, A.
Buccheri
, C.
Bungey
, S.
Lee
, R.
Meli
, K.
Miyamoto
, C.
Steinmann
, T.
Tsuchiya
, M.
Welborn
, T.
Wiles
, and Z.
Williams
, “Entos: A quantum molecular simulation package
,” ChemRxiv:7762646.v2 (2019
).62.
D. P.
Kingma
and J.
Ba
, “Adam: A method for stochastic optimization
,” in 3rd International Conference for Learning Representations, San Diego, 2015.63.
L. N.
Smith
and N.
Topin
, “Super-convergence: Very fast training of neural networks using large learning rates
,” in Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications
(International Society for Optics and Photonics
, 2019
), Vol. 11006, p. 1100612
.64.
S.
Ioffe
and C.
Szegedy
, “Batch normalization: Accelerating deep network training by reducing internal covariate shift
,” in International Conference on Machine Learning
(International Machine Learning Society
, 2015
), pp. 448
–456
.65.
P.
Pracht
, E.
Caldeweyher
, S.
Ehlert
, and S.
Grimme
, “A robust non-self-consistent tight-binding quantum chemistry method for large molecules
,” ChemRxiv:8326202.v1 (2019
).66.
C.
Bannwarth
, S.
Ehlert
, and S.
Grimme
, “GFN2-xTB—An accurate and broadly parametrized self-consistent tight-binding quantum chemical method with multipole electrostatics and density-dependent dispersion contributions
,” J. Chem. Theory Comput.
15
, 1652
–1671
(2019
).67.
H.
Jiang
, X.
Tao
, M.
Kammler
, F.
Ding
, A. M.
Wodtke
, A.
Kandratsenka
, T. F.
Miller
III, and O.
Bünermann
, “Nuclear quantum effects in scattering of H and D from graphene
,” arXiv:2007.03372 (2020
).68.
Semiempirical extended tight-binding program package 2020, accessed 14 July 2020, https://github.com/grimme-lab/xtb.
69.
G.
Zhou
, B.
Nebgen
, N.
Lubbers
, W. F.
Malone
, A. M.
Niklasson
, and S.
Tretiak
, “Graphics processing unit-accelerated semiempirical Born Oppenheimer molecular dynamics using PyTorch
,” J. Chem. Theory Comput.
16
, 4951
–4962
(2020
).© 2020 Author(s).
2020
Author(s)
You do not currently have access to this content.