We introduce a machine learning method in which energy solutions from the Schrödinger equation are predicted using symmetry adapted atomic orbital features and a graph neural-network architecture. OrbNet is shown to outperform existing methods in terms of learning efficiency and transferability for the prediction of density functional theory results while employing low-cost features that are obtained from semi-empirical electronic structure calculations. For applications to datasets of drug-like molecules, including QM7b-T, QM9, GDB-13-T, DrugBank, and the conformer benchmark dataset of Folmsbee and Hutchison [Int. J. Quantum Chem. (published online) (2020)], OrbNet predicts energies within chemical accuracy of density functional theory at a computational cost that is 1000-fold or more reduced.

1.
A. P.
Bartók
,
M. C.
Payne
,
R.
Kondor
, and
G.
Csányi
, “
Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons
,”
Phys. Rev. Lett.
104
,
136403
(
2010
).
2.
M.
Rupp
,
A.
Tkatchenko
,
K.-R.
Müller
, and
O. A.
von Lilienfeld
, “
Fast and accurate modeling of molecular atomization energies with machine learning
,”
Phys. Rev. Lett.
108
,
58301
(
2012
).
3.
A. S.
Christensen
,
L. A.
Bratholm
,
F. A.
Faber
, and
O. A.
von Lilienfeld
, “
FCHL revisited: Faster and more accurate quantum machine learning
,”
J. Chem. Phys.
152
,
044107
(
2020
).
4.
A. S.
Christensen
and
O. A.
von Lilienfeld
, “
Operator quantum machine learning: Navigating the chemical space of response properties
,”
CHIMIA
73
,
1028
1031
(
2019
).
5.
R.
Ramakrishnan
,
P. O.
Dral
,
M.
Rupp
, and
O. A.
von Lilienfeld
, “
Big data meets quantum chemistry approximations: The Δ-machine learning approach
,”
J. Chem. Theory Comput.
11
,
2087
(
2015
).
6.
T. T.
Nguyen
,
E.
Székely
,
G.
Imbalzano
,
J.
Behler
,
G.
Csányi
,
M.
Ceriotti
,
A. W.
Götz
, and
F.
Paesani
, “
Comparison of permutationally invariant polynomials, neural networks, and Gaussian approximation potentials in representing water interactions through many-body expansions
,”
J. Chem. Phys.
148
,
241725
(
2018
).
7.
S.
Fujikake
,
V. L.
Deringer
,
T. H.
Lee
,
M.
Krynski
,
S. R.
Elliott
, and
G.
Csányi
, “
Gaussian approximation potential modeling of lithium intercalation in carbon nanostructures
,”
J. Chem. Phys.
148
,
241714
(
2018
).
8.
A.
Grisafi
,
A.
Fabrizio
,
B.
Meyer
,
D. M.
Wilkins
,
C.
Corminboeuf
, and
M.
Ceriotti
, “
Transferable machine-learning model of the electron density
,”
ACS Cent. Sci.
5
,
57
64
(
2019
).
9.
Y.
Zhai
,
A.
Caruso
,
S.
Gao
, and
F.
Paesani
, “
Active learning of many-body configuration space: Application to the Cs+–water MB-nrg potential energy function as a case study
,”
J. Chem. Phys.
152
,
144103
(
2020
).
10.
J. S.
Smith
,
O.
Isayev
, and
A. E.
Roitberg
, “
ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
,”
Chem. Sci.
8
,
3192
3203
(
2017
).
11.
J. S.
Smith
,
B. T.
Nebgen
,
R.
Zubatyuk
,
N.
Lubbers
,
C.
Devereux
,
K.
Barros
,
S.
Tretiak
,
O.
Isayev
, and
A. E.
Roitberg
, “
Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning
,”
Nat. Commun.
10
,
1
8
(
2019
).
12.
N.
Lubbers
,
J. S.
Smith
, and
K.
Barros
, “
Hierarchical modeling of molecular energies using a deep neural network
,”
J. Chem. Phys.
148
,
241715
(
2018
).
13.
G.
Montavon
,
M.
Rupp
,
V.
Gobre
,
A.
Vazquez-Mayagoitia
,
K.
Hansen
,
A.
Tkatchenko
,
K.-R.
Müller
, and
O.
Anatole von Lilienfeld
, “
Machine learning of molecular electronic properties in chemical compound space
,”
New J. Phys.
15
,
095003
(
2013
).
14.
K.
Hansen
,
G.
Montavon
,
F.
Biegler
,
S.
Fazli
,
M.
Rupp
,
M.
Scheffler
,
O. A.
von Lilienfeld
,
A.
Tkatchenko
, and
K.-R.
Müller
, “
Assessment and validation of machine learning methods for predicting molecular atomization energies
,”
J. Chem. Theory Comput.
9
,
3404
(
2013
).
15.
P.
Gasparotto
and
M.
Ceriotti
, “
Recognizing molecular patterns by machine learning: An agnostic structural definition of the hydrogen bond
,”
J. Chem. Phys.
141
,
174110
(
2014
).
16.
J.
Behler
, “
Perspective: Machine learning potentials for atomistic simulations
,”
J. Chem. Phys.
145
,
170901
(
2016
).
17.
S.
Kearnes
,
K.
McCloskey
,
M.
Berndl
,
V.
Pande
, and
P.
Riley
, “
Molecular graph convolutions: Moving beyond fingerprints
,”
J. Comput. Aided Mol. Des.
30
,
595
(
2016
).
18.
K. T.
Schütt
,
F.
Arbabzadah
,
S.
Chmiela
,
K.-R.
Müller
, and
A.
Tkatchenko
, “
Quantum-chemical insights from deep tensor neural networks
,”
Nat. Commun.
8
,
13890
(
2017
).
19.
F.
Brockherde
,
L.
Vogt
,
L.
Li
,
M. E.
Tuckerman
,
K.
Burke
, and
K.-R.
Müller
, “
Bypassing the Kohn-Sham equations with machine learning
,”
Nat. Commun.
8
,
872
(
2017
).
20.
Z.
Wu
,
B.
Ramsundar
,
E. N.
Feinberg
,
J.
Gomes
,
C.
Geniesse
,
A. S.
Pappu
,
K.
Leswing
, and
V.
Pande
, “
MoleculeNet: A benchmark for molecular machine learning
,”
Chem. Sci.
9
,
513
(
2018
).
21.
K.
Yao
,
J. E.
Herr
,
D. W.
Toth
,
R.
McKintyre
, and
J.
Parkhill
, “
The TensorMol-0.1 model chemistry: A neural network augmented with long-range physics
,”
Chem. Sci.
9
,
2261
2269
(
2018
).
22.
H.
Li
,
C.
Collins
,
M.
Tanha
,
G. J.
Gordon
, and
D. J.
Yaron
, “
A density functional tight binding layer for deep learning of chemical Hamiltonians
,”
J. Chem. Theory Comput.
14
,
5764
5776
(
2018
).
23.
L.
Zhang
,
J.
Han
,
H.
Wang
,
R.
Car
, and
W.
E
, “
Deep potential molecular dynamics: A scalable model with the accuracy of quantum mechanics
,”
Phys. Rev. Lett.
120
,
143001
(
2018
).
24.
M.
Welborn
,
L.
Cheng
, and
T. F.
Miller
 III
, “
Transferability in machine learning for electronic structure via the molecular orbital basis
,”
J. Chem. Theory Comput.
14
,
4772
4779
(
2018
).
25.
L.
Cheng
,
M.
Welborn
,
A. S.
Christensen
, and
T. F.
Miller
 III
, “
A universal density matrix functional from molecular orbital-based machine learning: Transferability across organic molecules
,”
J. Chem. Phys.
150
,
131103
(
2019
).
26.
L.
Cheng
,
N. B.
Kovachki
,
M.
Welborn
, and
T. F.
Miller
 III
, “
Regression clustering for improved accuracy and training costs with molecular-orbital-based machine learning
,”
J. Chem. Theory Comput.
15
,
6668
6677
(
2019
).
27.
S.
Dick
and
M.
Fernandez-Serra
, “
Machine learning accurate exchange and correlation functionals of the electronic density
,”
Nat. Commun.
11
,
3509
(
2020
).
28.
Y.
Chen
,
L.
Zhang
,
H.
Wang
, and
W.
E
, “
Ground state energy functional with Hartree-Fock efficiency and chemical accuracy
,”
J. Phys. Chem. A
124
(
35
),
7155
7165
(
2020
).
29.
T. N.
Kipf
and
M.
Welling
, “
Semi-supervised classification with graph convolutional networks
,” in
International Conference on Learning Representations
,
2017
.
30.
P.
Veličković
,
G.
Cucurull
,
A.
Casanova
,
A.
Romero
,
P.
Liò
, and
Y.
Bengio
, “
Graph attention networks
,” in
International Conference on Learning Representations
,
2018
.
31.
K.
Yang
,
K.
Swanson
,
W.
Jin
,
C.
Coley
,
P.
Eiden
,
H.
Gao
,
A.
Guzman-Perez
,
T.
Hopper
,
B.
Kelley
,
M.
Mathea
,
A.
Palmer
,
V.
Settels
,
T.
Jaakkola
,
K.
Jensen
, and
R.
Barzilay
, “
Analyzing learned molecular representations for property prediction
,”
J. Chem. Inf. Model.
59
,
3370
3388
(
2019
).
32.
K.
Schütt
,
P.-J.
Kindermans
,
H. E. S.
Felix
,
S.
Chmiela
,
A.
Tkatchenko
, and
K.-R.
Müller
, “
Schnet: A continuous-filter convolutional neural network for modeling quantum interactions
,” in
Advances in Neural Information Processing Systems
(
Neural Information Processing Systems Foundation, Inc.
,
2017
), pp.
991
1001
.
33.
O. T.
Unke
and
M.
Meuwly
, “
PhysNet: A neural network for predicting energies, forces, dipole moments, and partial charges
,”
J. Chem. Theory Comput.
15
,
3678
3693
(
2019
).
34.
J.
Klicpera
,
J.
Groß
, and
S.
Günnemann
, “
Directional message passing for molecular graphs
,” in
International Conference on Learning Representations
,
2019
.
35.
Z.
Liu
,
L.
Lin
,
Q.
Jia
,
Z.
Cheng
,
Y.
Jiang
,
Y.
Guo
, and
J.
Ma
, “
Transferable multi-level attention neural network for accurate prediction of quantum chemistry properties via multi-task learning
,” ChemRxiv:12588170.v1 (
2020
).
36.
S.
Grimme
, “
A simplified Tamm-Dancoff density functional approach for the electronic excitation spectra of very large molecules
,”
J. Chem. Phys.
138
,
244104
(
2013
).
37.
S.
Grimme
and
C.
Bannwarth
, “
Ultra-fast computation of electronic spectra for large systems by tight-binding based simplified Tamm-Dancoff approximation (sTDA-xTB)
,”
J. Chem. Phys.
145
,
054103
(
2016
).
38.
T.
Risthaus
,
A.
Hansen
, and
S.
Grimme
, “
Excited states using the simplified Tamm–Dancoff-approach for range-separated hybrid density functionals: Development and application
,”
Phys. Chem. Chem. Phys.
16
,
14408
14419
(
2014
).
39.
K.
He
,
X.
Zhang
,
S.
Ren
, and
J.
Sun
, “
Deep residual learning for image recognition
,” in
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(
IEEE
,
2016
), pp.
770
778
.
40.
A.
Vaswani
,
N.
Shazeer
,
N.
Parmar
,
J.
Uszkoreit
,
L.
Jones
,
A. N.
Gomez
,
Ł.
Kaiser
, and
I.
Polosukhin
, “
Attention is all you need
,” in
Advances in Neural Information Processing Systems
(
Neural Information Processing Systems Foundation, Inc.
,
2017
), pp.
5998
6008
.
41.
R. T.
McGibbon
,
A. G.
Taube
,
A. G.
Donchev
,
K.
Siva
,
F.
Hernández
,
C.
Hargus
,
K.-H.
Law
,
J. L.
Klepeis
, and
D. E.
Shaw
, “
Improving the accuracy of Møller-Plesset perturbation theory with neural networks
,”
J. Chem. Phys.
147
,
161725
(
2017
).
42.
J. T.
Margraf
and
K.
Reuter
, “
Making the coupled cluster correlation energy machine-learnable
,”
J. Phys. Chem. A
122
,
6343
6348
(
2018
).
43.
J.
Behler
and
M.
Parrinello
, “
Generalized neural-network representation of high-dimensional potential-energy surfaces
,”
Phys. Rev. Lett.
98
,
146401
(
2007
).
44.
L.
Cheng
,
M.
Welborn
,
A. S.
Christensen
, and
T. F.
Miller
, Thermalized (350k) QM7b, GDB-13, water, and short alkane quantum chemistry dataset including MOB-ML features,
2019
, https://data.caltech.edu/records/1177.
45.
R.
Ramakrishnan
,
P. O.
Dral
,
M.
Rupp
, and
O. A.
von Lilienfeld
, “
Quantum chemistry structures and properties of 134 kilo molecules
,”
Sci. Data
1
,
1
7
(
2014
).
46.
L. C.
Blum
and
J.-L.
Reymond
, “
970 million druglike small molecules for virtual screening in the chemical universe database GDB-13
,”
J. Am. Chem. Soc.
131
,
8732
(
2009
).
47.
V.
Law
,
C.
Knox
,
Y.
Djoumbou
,
T.
Jewison
,
A. C.
Guo
,
Y.
Liu
,
A.
Maciejewski
,
D.
Arndt
,
M.
Wilson
,
V.
Neveu
,
A.
Tang
,
G.
Gabriel
,
C.
Ly
,
S.
Adamjee
,
Z. T.
Dame
,
B.
Han
,
Y.
Zhou
, and
D. S.
Wishart
, “
DrugBank 4.0: Shedding new light on drug metabolism
,”
Nucl. Acids Res.
42
,
D1091
D1097
(
2014
).
48.
D.
Folmsbee
and
G.
Hutchison
, “
Assessing conformer energies using electronic structure and machine learning methods
,”
Int. J. Quantum Chem.
(published online).
49.
S. H.
Vosko
,
L.
Wilk
, and
M.
Nusair
, “
Accurate spin-dependent electron liquid correlation energies for local spin density calculations: A critical analysis
,”
Can. J. Phys.
58
,
1200
(
1980
).
50.
C.
Lee
,
W.
Yang
, and
R. G.
Parr
, “
Development of the Colle-Salvetti correlation-energy formula into a functional of the electron density
,”
Phys. Rev. B
37
,
785
(
1988
).
51.
A. D.
Becke
, “
Density-functional thermochemistry. III. The role of exact exchange
,”
J. Chem. Phys.
98
,
5648
(
1993
).
52.
P. J.
Stephens
,
F. J.
Devlin
,
C. F.
Chabalowski
, and
M. J.
Frisch
, “
Ab initio calculation of vibrational absorption and circular dichroism spectra using density functional force fields
,”
J. Phys. Chem.
98
,
11623
(
1994
).
53.
P. C.
Hariharan
and
J. A.
Pople
, “
The influence of polarization functions on molecular orbital hydrogenation energies
,”
Theor. Chim. Acta
28
,
213
(
1973
).
54.
G.
Bussi
and
M.
Parrinello
, “
Accurate sampling using Langevin dynamics
,”
Phys. Rev. E
75
,
056707
(
2007
).
55.
Y.-S.
Lin
,
G.-D.
Li
,
S.-P.
Mao
, and
J.-D.
Chai
, “
Long-range corrected hybrid density functionals with improved dispersion corrections
,”
J. Chem. Theory Comput.
9
,
263
272
(
2013
).
56.
F.
Weigend
and
R.
Ahlrichs
, “
Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence quality for H to Rn: Design and assessment of accuracy
,”
Phys. Chem. Chem. Phys.
7
,
3297
(
2005
).
57.
R.
Polly
,
H.-J.
Werner
,
F. R.
Manby
, and
P. J.
Knowles
, “
Fast Hartree-Fock theory using local density fitting approximations
,”
Mol. Phys.
102
,
2311
2321
(
2004
).
58.
F.
Weigend
, “
Hartree-Fock exchange fitting basis sets for H to Rn
,”
J. Comput. Chem.
29
,
167
175
(
2008
).
59.
D. G. A.
Smith
,
L. A.
Burns
,
A. C.
Simmonett
,
R. M.
Parrish
,
M. C.
Schieber
,
R.
Galvelis
,
P.
Kraus
,
H.
Kruse
,
R.
Di Remigio
,
A.
Alenaizan
,
A. M.
James
,
S.
Lehtola
,
J. P.
Misiewicz
,
M.
Scheurer
,
R. A.
Shaw
,
J. B.
Schriber
,
Y.
Xie
,
Z. L.
Glick
,
D. A.
Sirianni
,
J. S.
O’Brien
,
J. M.
Waldrop
,
A.
Kumar
,
E. G.
Hohenstein
,
B. P.
Pritchard
,
B. R.
Brooks
,
H. F.
Schaefer
,
A. Y.
Sokolov
,
K.
Patkowski
,
A. E.
DePrince
,
U.
Bozkaya
,
R. A.
King
,
F. A.
Evangelista
,
J. M.
Turney
,
T. D.
Crawford
, and
C. D.
Sherrill
, “
Psi4 1.4: Open-source software for high-throughput quantum chemistry
,”
J. Chem. Phys.
152
,
184108
(
2020
).
60.
S.
Grimme
,
C.
Bannwarth
, and
P.
Shushkov
, “
A robust and accurate tight-binding quantum chemical method for structures, vibrational frequencies, and noncovalent interactions of large molecular systems parametrized for all spd-block elements (Z = 1–86)
,”
J. Chem. Theory Comput.
13
,
1989
2009
(
2017
).
61.
F.
Manby
,
T.
Miller
,
P.
Bygrave
,
F.
Ding
,
T.
Dresselhaus
,
F.
Batista-Romero
,
A.
Buccheri
,
C.
Bungey
,
S.
Lee
,
R.
Meli
,
K.
Miyamoto
,
C.
Steinmann
,
T.
Tsuchiya
,
M.
Welborn
,
T.
Wiles
, and
Z.
Williams
, “
Entos: A quantum molecular simulation package
,” ChemRxiv:7762646.v2 (
2019
).
62.
D. P.
Kingma
and
J.
Ba
, “
Adam: A method for stochastic optimization
,” in 3rd International Conference for Learning Representations, San Diego, 2015.
63.
L. N.
Smith
and
N.
Topin
, “
Super-convergence: Very fast training of neural networks using large learning rates
,” in
Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications
(
International Society for Optics and Photonics
,
2019
), Vol. 11006, p.
1100612
.
64.
S.
Ioffe
and
C.
Szegedy
, “
Batch normalization: Accelerating deep network training by reducing internal covariate shift
,” in
International Conference on Machine Learning
(
International Machine Learning Society
,
2015
), pp.
448
456
.
65.
P.
Pracht
,
E.
Caldeweyher
,
S.
Ehlert
, and
S.
Grimme
, “
A robust non-self-consistent tight-binding quantum chemistry method for large molecules
,” ChemRxiv:8326202.v1 (
2019
).
66.
C.
Bannwarth
,
S.
Ehlert
, and
S.
Grimme
, “
GFN2-xTB—An accurate and broadly parametrized self-consistent tight-binding quantum chemical method with multipole electrostatics and density-dependent dispersion contributions
,”
J. Chem. Theory Comput.
15
,
1652
1671
(
2019
).
67.
H.
Jiang
,
X.
Tao
,
M.
Kammler
,
F.
Ding
,
A. M.
Wodtke
,
A.
Kandratsenka
,
T. F.
Miller
 III
, and
O.
Bünermann
, “
Nuclear quantum effects in scattering of H and D from graphene
,” arXiv:2007.03372 (
2020
).
68.
Semiempirical extended tight-binding program package 2020, accessed 14 July 2020, https://github.com/grimme-lab/xtb.
69.
G.
Zhou
,
B.
Nebgen
,
N.
Lubbers
,
W. F.
Malone
,
A. M.
Niklasson
, and
S.
Tretiak
, “
Graphics processing unit-accelerated semiempirical Born Oppenheimer molecular dynamics using PyTorch
,”
J. Chem. Theory Comput.
16
,
4951
4962
(
2020
).

Supplementary Material

You do not currently have access to this content.