Molecular-orbital-based machine learning (MOB-ML) provides a general framework for the prediction of accurate correlation energies at the cost of obtaining molecular orbitals. The application of Nesbet’s theorem makes it possible to recast a typical extrapolation task, training on correlation energies for small molecules and predicting correlation energies for large molecules, into an interpolation task based on the properties of orbital pairs. We demonstrate the importance of preserving physical constraints, including invariance conditions and size consistency, when generating the input for the machine learning model. Numerical improvements are demonstrated for different datasets covering total and relative energies for thermally accessible organic and transition-metal containing molecules, non-covalent interactions, and transition-state energies. MOB-ML requires training data from only 1% of the QM7b-T dataset (i.e., only 70 organic molecules with seven and fewer heavy atoms) to predict the total energy of the remaining 99% of this dataset with sub-kcal/mol accuracy. This MOB-ML model is significantly more accurate than other methods when transferred to a dataset comprising of 13 heavy atom molecules, exhibiting no loss of accuracy on a size intensive (i.e., per-electron) basis. It is shown that MOB-ML also works well for extrapolating to transition-state structures, predicting the barrier region for malonaldehyde intramolecular proton-transfer to within 0.35 kcal/mol when only trained on reactant/product-like structures. Finally, the use of the Gaussian process variance enables an active learning strategy for extending the MOB-ML model to new regions of chemical space with minimal effort. We demonstrate this active learning strategy by extending a QM7b-T model to describe non-covalent interactions in the protein backbone–backbone interaction dataset to an accuracy of 0.28 kcal/mol.
REFERENCES
1.
A. P.
Bartók
, M. C.
Payne
, R.
Kondor
, and G.
Csányi
, “Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons
,” Phys. Rev. Lett.
104
, 136403
(2010
).2.
M.
Rupp
, A.
Tkatchenko
, K.-R.
Müller
, and O. A.
von Lilienfeld
, “Fast and accurate modeling of molecular atomization energies with machine learning
,” Phys. Rev. Lett.
108
, 058301
(2012
).3.
K.
Hansen
, G.
Montavon
, F.
Biegler
, S.
Fazli
, M.
Rupp
, M.
Scheffler
, O. A.
von Lilienfeld
, A.
Tkatchenko
, and K.-R.
Müller
, “Assessment and validation of machine learning methods for predicting molecular atomization energies
,” J. Chem. Theory Comput.
9
, 3404
(2013
).4.
R.
Ramakrishnan
, P. O.
Dral
, M.
Rupp
, and O. A.
von Lilienfeld
, “Big data meets quantum chemistry approximations: The δ-machine learning approach
,” J. Chem. Theory Comput.
11
, 2087
–2096
(2015
).5.
J.
Behler
, “Perspective: Machine learning potentials for atomistic simulations
,” J. Chem. Phys.
145
, 170901
(2016
).6.
F.
Brockherde
, L.
Vogt
, L.
Li
, M. E.
Tuckerman
, K.
Burke
, and K.-R.
Müller
, “Bypassing the Kohn–Sham equations with machine learning
,” Nat. Commun.
8
, 872
(2017
).7.
K.
Schütt
, P.-J.
Kindermans
, H. E.
Sauceda
, S.
Chmiela
, A.
Tkatchenko
, and K.-R.
Müller
, “SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
,” in Advances in Neural Information Processing Systems 30
, edited by I.
Guyon
, U. V.
Luxburg
, S.
Bengio
, H.
Wallach
, R.
Fergus
, S.
Vishwanathan
, and R.
Garnett
(Curran Associates, Inc.
, 2017
), pp. 991
–1001
.8.
K. T.
Schütt
, F.
Arbabzadah
, S.
Chmiela
, K.-R.
Müller
, and A.
Tkatchenko
, “Quantum-chemical insights from deep tensor neural networks
,” Nat. Commun.
8
, 13890
(2017
).9.
J. S.
Smith
, O.
Isayev
, and A. E.
Roitberg
, “ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
,” Chem. Sci.
8
, 3192
–3203
(2017
).10.
R. T.
McGibbon
, A. G.
Taube
, A. G.
Donchev
, K.
Siva
, F.
Hernández
, C.
Hargus
, K.-H.
Law
, J. L.
Klepeis
, and D. E.
Shaw
, “Improving the accuracy of Møller–Plesset perturbation theory with neural networks
,” J. Chem. Phys.
147
, 161725
(2017
).11.
C. R.
Collins
, G. J.
Gordon
, O. A.
von Lilienfeld
, and D. J.
Yaron
, “Constant size descriptors for accurate machine learning models of molecular properties
,” J. Chem. Phys.
148
, 241718
(2018
).12.
S.
Fujikake
, V. L.
Deringer
, T. H.
Lee
, M.
Krynski
, S. R.
Elliott
, and G.
Csányi
, “Gaussian approximation potential modeling of lithium intercalation in carbon nanostructures
,” J. Chem. Phys.
148
, 241714
(2018
).13.
N.
Lubbers
, J. S.
Smith
, and K.
Barros
, “Hierarchical modeling of molecular energies using a deep neural network
,” J. Chem. Phys.
148
, 241715
(2018
).14.
T. T.
Nguyen
, E.
Székely
, G.
Imbalzano
, J.
Behler
, G.
Csányi
, M.
Ceriotti
, A. W.
Götz
, and F.
Paesani
, “Comparison of permutationally invariant polynomials, neural networks, and Gaussian approximation potentials in representing water interactions through many-body expansions
,” J. Chem. Phys.
148
, 241725
(2018
).15.
M.
Welborn
, L.
Cheng
, and T. F.
Miller
III, “Transferability in machine learning for electronic structure via the molecular orbital basis
,” J. Chem. Theory Comput.
14
, 4772
–4779
(2018
).16.
Z.
Wu
, B.
Ramsundar
, E. N.
Feinberg
, J.
Gomes
, C.
Geniesse
, A. S.
Pappu
, K.
Leswing
, and V.
Pande
, “MoleculeNet: A benchmark for molecular machine learning
,” Chem. Sci.
9
, 513
–530
(2018
).17.
K.
Yao
, J. E.
Herr
, D. W.
Toth
, R.
Mckintyre
, and J.
Parkhill
, “The TensorMol-0.1 model chemistry: A neural network augmented with long-range physics
,” Chem. Sci.
9
, 2261
–2269
(2018
).18.
L.
Cheng
, M.
Welborn
, A. S.
Christensen
, and T. F.
Miller
III, “A universal density matrix functional from molecular orbital-based machine learning: Transferability across organic molecules
,” J. Chem. Phys.
150
, 131103
(2019
).19.
L.
Cheng
, N. B.
Kovachki
, M.
Welborn
, and T. F.
Miller
III, “Regression clustering for improved accuracy and training costs with molecular-orbital-based machine learning
,” J. Chem. Theory Comput.
15
, 6668
–6677
(2019
).20.
A. S.
Christensen
, F. A.
Faber
, and O. A.
von Lilienfeld
, “Operators in quantum machine learning: Response properties in chemical space
,” J. Chem. Phys.
150
, 064105
(2019
).21.
A.
Grisafi
, A.
Fabrizio
, B.
Meyer
, D. M.
Wilkins
, C.
Corminboeuf
, and M.
Ceriotti
, “Transferable machine-learning model of the electron density
,” ACS Cent. Sci.
5
, 57
–64
(2019
).22.
J. S.
Smith
, B. T.
Nebgen
, R.
Zubatyuk
, N.
Lubbers
, C.
Devereux
, K.
Barros
, S.
Tretiak
, O.
Isayev
, and A. E.
Roitberg
, “Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning
,” Nat. Commun.
10
, 2903
(2019
).23.
O. T.
Unke
and M.
Meuwly
, “PhysNet: A neural network for predicting energies, forces, dipole moments and partial charges
,” J. Chem. Theory Comput.
15
, 3678
–3693
(2019
).24.
A.
Fabrizio
, B.
Meyer
, and C.
Corminboeuf
, “Machine learning models of the energy curvature vs particle number for optimal tuning of long-range corrected functionals
,” J. Chem. Phys.
152
, 154103
(2020
).25.
Y.
Chen
, L.
Zhang
, H.
Wang
, and W. E
, “Ground state energy functional with Hartree–Fock efficiency and chemical accuracy
,” J. Phys. Chem. A
124
, 7155
–7165
(2020
).26.
A. S.
Christensen
, L. A.
Bratholm
, F. A.
Faber
, and O.
Anatole von Lilienfeld
, “FCHL revisited: Faster and more accurate quantum machine learning
,” J. Chem. Phys.
152
, 044107
(2020
).27.
S.
Dick
and M.
Fernandez-Serra
, “Machine learning accurate exchange and correlation functionals of the electronic density
,” Nat. Commun.
11
, 3509
(2020
).28.
Z.
Liu
, L.
Lin
, Q.
Jia
, Z.
Cheng
, Y.
Jiang
, Y.
Guo
, and J.
Ma
, “Transferable multi-level attention neural network for accurate prediction of quantum chemistry properties via multi-task learning
,” chemRxiv:12588170.v1 (2020
).29.
Z.
Qiao
, M.
Welborn
, A.
Anandkumar
, F. R.
Manby
, and T. F.
Miller
III, “OrbNet: Deep learning for quantum chemistry using symmetry-adapted atomic-orbital features
,” J. Chem. Phys.
153
, 124111
(2020
).30.
S.
Manzhos
, “Machine learning for the solution of the Schrödinger equation
,” Mach. Learn.: Sci. Technol.
1
, 013002
(2020
).31.
R. K.
Nesbet
, “‘Brueckner’s theory and the method of superposition of configurations
,” Phys. Rev.
109
, 1632
–1638
(1958
).32.
C.
Møller
and M. S.
Plesset
, “Note on an approximation treatment for many-electron systems
,” Phys. Rev.
46
, 618
(1934
).33.
C. E.
Rasmussen
and C. K. I.
Williams
, Gaussian Processes for Machine Learning
(MIT Press
, Cambridge, MA
, 2006
), Publication Title: Gaussian processes for machine learning.34.
E.
Kapuy
, Z.
Csépes
, and C.
Kozmutza
, “Application of the many-body perturbation theory by using localized orbitals
,” Int. J. Quantum Chem.
23
, 981
(1983
).35.
R. J.
Bartlett
, “Many-Body perturbation theory and coupled cluster theory for electron correlation in molecules
,” Annu. Rev. Phys. Chem.
32
, 359
–401
(1981
).36.
A.
Szabo
and N. S.
Ostlund
, Modern Quantum Chemistry
(Dover
, Mineola
, 1996
), pp. 261
–265
.37.
S.
Grimme
, A.
Hansen
, J. G.
Brandenburg
, and C.
Bannwarth
, “Dispersion-corrected mean-field electronic structure methods
,” Chem. Rev.
116
, 5105
–5154
(2016
).38.
L.
Cheng
, M.
Welborn
, A. S.
Christensen
, and T. F.
Miller
III, “Thermalized (350K) QM7b, GDB-13, water, and short alkane quantum chemistry dataset including MOB-ML features
,” CaltechDATA, Dataset
.39.
L. A.
Burns
, J. C.
Faver
, Z.
Zheng
, M. S.
Marshall
, D. G. A.
Smith
, K.
Vanommeslaeghe
, A. D.
MacKerell
, K. M.
Merz
, and C. D.
Sherrill
, “The BioFragment Database (BFDb): An open-data platform for computational chemistry analysis of noncovalent interactions
,” J. Chem. Phys.
147
, 161727
(2017
).40.
J. P.
Janet
and H. J.
Kulik
, “Predicting electronic structure properties oftransition metal complexes with neural networks
,” Chem. Sci.
8
, 5137
–5152
(2017
).41.
T.
Husch
, J.
Sun
, L.
Cheng
, S. J. R.
Lee
, and T. F.
Miller
III (2020
). “QM7b-T, GDB-13-T, TM-T, malonaldehyde, BBI, and short alkanes quantumBBI, and short alkanes quantum chemistry dataset including MOB-ML features
,” CaltechDATA, Dataset. .42.
F. R.
Manby
, T. F.
Miller
III, P.
Bygrave
, F.
Ding
, T.
Dresselhaus
, F.
Batista-Romero
, A.
Buccheri
, C.
Bungey
, S. J. R.
Lee
, R.
Meli
, K.
Miyamoto
, C.
Steinmann
, T.
Tsuchiya
, M.
Welborn
, T.
Wiles
, and Z.
Williams
, “Entos: A quantum molecular simulation package
,” chemRxiv:7762646.v2 (2019
).43.
T. H.
Dunning
, “Gaussian basis sets for use in correlated molecular calculations. I. The atoms boron through neon and hydrogen
,” J. Chem. Phys.
90
, 1007
–1023
(1989
).44.
F.
Weigend
and R.
Ahlrichs
, “Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence quality for H to Rn: Design and assessment of accuracy
,” Phys. Chem. Chem. Phys.
7
, 3297
–3305
(2005
).45.
F.
Weigend
, “A fully direct RI-HF algorithm: Implementation, optimised auxiliary basis sets, demonstration of accuracy and efficiency
,” Phys. Chem. Chem. Phys.
4
, 4285
–4291
(2002
).46.
F.
Weigend
, “Accurate coulomb-fitting basis sets for H to Rn
,” Phys. Chem. Chem. Phys.
8
, 1057
–1065
(2006
).47.
S. F.
Boys
, “Construction of some molecular orbitals to Be approximately invariant for changes from one molecule to another
,” Rev. Mod. Phys.
32
, 296
–299
(1960
).48.
J. M.
Foster
and S. F.
Boys
, “Canonical configurational interaction procedure
,” Rev. Mod. Phys.
32
, 300
–302
(1960
).49.
G.
Knizia
, “Intrinsic atomic orbitals: An unbiased bridge between quantum theory and chemical concepts
,” J. Chem. Theory Comput.
9
, 4834
(2013
).50.
H.-J.
Werner
, P. J.
Knowles
, G.
Knizia
, F. R.
Manby
, M.
Schütz
, and others
, MOLPRO, Version 2018.3, A Package of Ab Initio Programs, Cardiff
, UK
, 2018
.51.
H.-J.
Werner
, P. J.
Knowles
, F. B.
Manby
, J. A.
Black
, K.
Doll
, A.
Hesselmann
, D.
Kats
, A.
Köhn
, T.
Korona
, D. A.
Kreplin
, Q.
Ma
, T. F.
Miller
III, A.
Mitrushchenkov
, Peterson
, I.
Polyak
, M.
Rauhut
, and G.
Sibaev
, “The MOLPRO quantum chemistry package
,” J. Chem. Phys.
152
, 144107
(2020
).52.
H.-J.
Werner
, F. R.
Manby
, and P. J.
Knowles
, “Fast linear scaling second-order Møller–Plesset perturbation theory (MP2) using local and density fitting approximations
,” J. Chem. Phys.
118
, 8149
(2003
).53.
G. E.
Scuseria
and T. J.
Lee
, “Comparison of coupled-cluster methods which include the effects of connected triple excitations
,” J. Chem. Phys.
93
, 5851
–5855
(1990
).54.
C.
Hampel
and H. J.
Werner
, “Local treatment of electron correlation in coupled cluster theory
,” J. Chem. Phys.
104
, 6286
–6297
(1996
).55.
M.
Schütz
and H.-J.
Werner
, “Local perturbative triples correction (t) with linear cost scaling
,” Chem. Phys. Lett.
318
, 370
–378
(2000
).56.
F.
Pedregosa
, G.
Varoquaux
, A.
Gramfort
, V.
Michel
, B.
Thirion
, O.
Grisel
, M.
Blondel
, P.
Prettenhofer
, R.
Weiss
, V.
Dubourg
, J.
Vanderplas
, A.
Passos
, D.
Cournapeau
, M.
Brucher
, M.
Perrot
, and E.
Duchesnay
, “Scikit-learn: Machine learning in python (v0.21.2)
,” J. Mach. Learn. Res.
12
, 2825
(2011
).57.
GPy, GPy: A Gaussian Process Framework in Python,
2012
.58.
A.
Shapeev
, K.
Gubaev
, E.
Tsymbalov
, and E.
Podryabinkin
, “Active learning and uncertainty estimation
,” in Machine Learning Meets Quantum Physics
, edited by K. T.
Schütt
, S.
Chmiela
, O. A.
von Lilienfeld
, A.
Tkatchenko
, K.
Tsuda
and K.-R.
Müller
(Springer Nature Switzerland AG.
, 2020
), pp. 309
–329
.59.
L.
Hirschfeld
, K.
Swanson
, K.
Yang
, R.
Barzilay
, and C. W.
Coley
, “Uncertainty quantification using neural networks for molecular property prediction
,” J. Chem. Inf. Model.
60
, 3770
–3780
(2020
).60.
J.-L.
Reymond
, “The chemical space project
,” Acc. Chem. Res.
48
, 722
–730
(2015
).61.
L. C.
Blum
and J.-L.
Reymond
, “970 million druglike small molecules for virtual screening in the chemical universe database GDB-13
,” J. Am. Chem. Soc.
131
, 8732
(2009
).62.
S. J. R.
Lee
, T.
Husch
, F.
Ding
, and T. F.
Miller
III, “Analytical gradients for molecular-orbital-based machine learning
,” arXiv:2012.08899 (2020
).63.
T.
Weymuth
, E. P. A.
Couzijn
, P.
Chen
, and M.
Reiher
, “New benchmark set of transition-metal coordination reactions for the assessment of density functionals
,” J. Chem. Theory Comput.
10
, 3092
–3103
(2014
).64.
T.
Husch
, L.
Freitag
, and M.
Reiher
, “Calculation of ligand dissociation energies in large transition-metal complexes
,” J. Chem. Theory Comput.
14
, 2456
–2468
(2018
).65.
R. J.
Bartlett
and D. S.
Ranasinghe
, “The power of exact conditions in electronic structure theory
,” Chem. Phys. Lett.
669
, 54
–70
(2017
).© 2021 Author(s).
2021
Author(s)
You do not currently have access to this content.
