It is well known in the field of machine learning that committee models improve accuracy, provide generalization error estimates, and enable active learning strategies. In this work, we adapt these concepts to interatomic potentials based on artificial neural networks. Instead of a single model, multiple models that share the same atomic environment descriptors yield an average that outperforms its individual members as well as a measure of the generalization error in the form of the committee disagreement. We not only use this disagreement to identify the most relevant configurations to build up the model’s training set in an active learning procedure but also monitor and bias it during simulations to control the generalization error. This facilitates the adaptive development of committee neural network potentials and their training sets while keeping the number of ab initio calculations to a minimum. To illustrate the benefits of this methodology, we apply it to the development of a committee model for water in the condensed phase. Starting from a single reference ab initio simulation, we use active learning to expand into new state points and to describe the quantum nature of the nuclei. The final model, trained on 814 reference calculations, yields excellent results under a range of conditions, from liquid water at ambient and elevated temperatures and pressures to different phases of ice, and the air–water interface—all including nuclear quantum effects. This approach to committee models will enable the systematic development of robust machine learning models for a broad range of systems.

1.
J.
Behler
, “
Perspective: Machine learning potentials for atomistic simulations
,”
J. Chem. Phys.
145
,
170901
(
2016
).
2.
A. P.
Bartók
,
S.
De
,
C.
Poelking
,
N.
Bernstein
,
J. R.
Kermode
,
G.
Csányi
, and
M.
Ceriotti
, “
Machine learning unifies the modeling of materials and molecules
,”
Sci. Adv.
3
,
e1701816
(
2017
).
3.
K. T.
Butler
,
D. W.
Davies
,
H.
Cartwright
,
O.
Isayev
, and
A.
Walsh
, “
Machine learning for molecular and materials science
,”
Nature
559
,
547
555
(
2018
).
4.
V. L.
Deringer
,
M. A.
Caro
, and
G.
Csányi
, “
Machine learning interatomic potentials as emerging tools for materials science
,”
Adv. Mater.
31
,
1902765
(
2019
).
5.
T.
Mueller
,
A.
Hernandez
, and
C.
Wang
, “
Machine learning for interatomic potential models
,”
J. Chem. Phys.
152
,
050902
(
2020
).
6.
J.
Behler
and
M.
Parrinello
, “
Generalized neural-network representation of high-dimensional potential-energy surfaces
,”
Phys. Rev. Lett.
98
,
146401
(
2007
).
7.
J.
Behler
, “
First principles neural network potentials for reactive simulations of large molecular and condensed systems
,”
Angew. Chem,. Int. Ed.
56
,
12828
12840
(
2017
).
8.
J.
Behler
, “
Atom-centered symmetry functions for constructing high-dimensional neural network potentials
,”
J. Chem. Phys.
134
,
074106
(
2011
).
9.
S. A.
Ghasemi
,
A.
Hofstetter
,
S.
Saha
, and
S.
Goedecker
, “
Interatomic potentials for ionic systems with density functional accuracy based on charge densities obtained by a neural network
,”
Phys. Rev. B
92
,
045131
(
2015
); arXiv:1501.07344.
10.
A.
Khorshidi
and
A. A.
Peterson
, “
Amp: A modular approach to machine learning in atomistic simulations
,”
Comput. Phys. Commun.
207
,
310
324
(
2016
).
11.
K. T.
Schütt
,
F.
Arbabzadah
,
S.
Chmiela
,
K. R.
Müller
, and
A.
Tkatchenko
, “
Quantum-chemical insights from deep tensor neural networks
,”
Nat. Commun.
8
,
13890
(
2017
).
12.
N.
Artrith
,
A.
Urban
, and
G.
Ceder
, “
Efficient and accurate machine-learning interpolation of atomic energies in compositions with many species
,”
Phys. Rev. B
96
,
014112
(
2017
); arXiv:1706.06293.
13.
J. S.
Smith
,
O.
Isayev
, and
A. E.
Roitberg
, “
ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
,”
Chem. Sci.
8
,
3192
3203
(
2017
); arXiv:1610.08935.
14.
L.
Zhang
,
J.
Han
,
H.
Wang
,
R.
Car
, and
E.
Weinan
, “
Deep potential molecular dynamics: A scalable model with the accuracy of quantum mechanics
,”
Phys. Rev. Lett.
120
,
143001
(
2018
); arXiv:1707.09571.
15.
O. T.
Unke
and
M.
Meuwly
, “
PhysNet: A neural network for predicting energies, forces, dipole moments, and partial charges
,”
J. Chem. Theory Comput.
15
,
3678
3693
(
2019
).
16.
Y.
Shao
,
M.
Hellström
,
P. D.
Mitev
,
L.
Knijff
, and
C.
Zhang
, “
PiNN: A python library for building atomic neural networks of molecules and materials
,”
J. Chem. Inf. Model.
60
,
1184
1193
(
2020
); arXiv:1910.03376.
17.
A. P.
Bartók
,
M. C.
Payne
,
R.
Kondor
, and
G.
Csányi
, “
Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons
,”
Phys. Rev. Lett.
104
,
136403
(
2010
).
18.
M.
Rupp
,
A.
Tkatchenko
,
K. R.
Müller
, and
O. A.
von Lilienfeld
, “
Fast and accurate modeling of molecular atomization energies with machine learning
,”
Phys. Rev. Lett.
108
,
058301
(
2012
).
19.
A. P.
Thompson
,
L. P.
Swiler
,
C. R.
Trott
,
S. M.
Foiles
, and
G. J.
Tucker
, “
Spectral neighbor analysis method for automated generation of quantum-accurate interatomic potentials
,”
J. Comput. Phys.
285
,
316
330
(
2015
).
20.
A. V.
Shapeev
, “
Moment tensor potentials: A class of systematically improvable interatomic potentials
,”
Multiscale Model. Simul.
14
,
1153
1173
(
2015
).
21.
Z.
Li
,
J. R.
Kermode
, and
A.
De Vita
, “
Molecular dynamics with on-the-fly machine learning of quantum-mechanical forces
,”
Phys. Rev. Lett.
114
,
096405
(
2015
).
22.
S.
Chmiela
,
A.
Tkatchenko
,
H. E.
Sauceda
,
I.
Poltavsky
,
K. T.
Schütt
, and
K.-R.
Müller
, “
Machine learning of accurate energy-conserving molecular force fields
,”
Sci. Adv.
3
,
e1603015
(
2017
).
23.
T.
Morawietz
,
A.
Singraber
,
C.
Dellago
, and
J.
Behler
, “
How van der Waals interactions determine the unique properties of water
,”
Proc. Natl. Acad. Sci. U. S. A.
113
,
8368
8373
(
2016
).
24.
T.
Morawietz
,
O.
Marsalek
,
S. R.
Pattenaude
,
L. M.
Streacker
,
D.
Ben-Amotz
, and
T. E.
Markland
, “
The interplay of structure and dynamics in the Raman spectrum of liquid water over the full frequency and temperature range
,”
J. Phys. Chem. Lett.
9
,
851
857
(
2018
).
25.
B.
Cheng
,
E. A.
Engel
,
J.
Behler
,
C.
Dellago
, and
M.
Ceriotti
, “
Ab initio thermodynamics of liquid and solid water
,”
Proc. Natl. Acad. Sci. U. S. A.
116
,
1110
1115
(
2019
); arXiv:1811.08630.
26.
N.
Artrith
and
A.
Urban
, “
An implementation of artificial neural-network potentials for atomistic materials simulations: Performance for TiO2
,”
Comput. Mater. Sci.
114
,
135
150
(
2016
).
27.
A.
Singraber
,
T.
Morawietz
,
J.
Behler
, and
C.
Dellago
, “
Parallel multistream training of high-dimensional neural network potentials
,”
J. Chem. Theory Comput.
15
,
3075
3092
(
2019
).
28.
S. K.
Natarajan
and
J.
Behler
, “
Neural network molecular dynamics simulations of solid-liquid interfaces: Water at low-index copper surfaces
,”
Phys. Chem. Chem. Phys.
18
,
28704
28725
(
2016
).
29.
M.
Hellström
and
J.
Behler
, “
Structure of aqueous NaOH solutions: Insights from neural-network-based molecular dynamics simulations
,”
Phys. Chem. Chem. Phys.
19
,
82
96
(
2017
).
30.
V.
Quaranta
,
M.
Hellström
, and
J.
Behler
, “
Proton-transfer mechanisms at the water-ZnO interface: The role of presolvation
,”
J. Phys. Chem. Lett.
8
,
1476
1483
(
2017
).
31.
M.
Hellström
,
V.
Quaranta
, and
J.
Behler
, “
One-dimensional vs. two-dimensional proton transport processes at solid-liquid zinc-oxide-water interfaces
,”
Chem. Sci.
10
,
1232
1243
(
2019
).
32.
R. T.
Clemen
, “
Combining forecasts: A review and annotated bibliography
,”
Int. J. Forecasting
5
,
559
583
(
1989
).
33.
L. K.
Hansen
and
P.
Salamon
, “
Neural network ensembles
,”
IEEE Trans. Pattern Anal. Mach. Intell.
12
,
993
1001
(
1990
).
34.
A.
Krogh
and
J.
Vedelsby
, “
Neural network ensembles, cross validation, and active learning
,”
Adv. Neural Inf. Process. Syst.
7
,
231
238
(
1995
).
35.
Y.
Zhao
,
J.
Gao
, and
X.
Yang
, “
A survey of neural network ensembles
,” in
Proceedings of the 2005 International Conference on Neural Networks and Brain (ICNNB’05)
(
2005
), Vol. 1, pp.
438
442
.
36.
L.
Rokach
, “
Ensemble-based classifiers
,”
Artif. Intell. Rev.
33
,
1
39
(
2010
).
37.
S.
Hashem
and
B.
Schmeiser
, “
Improving model accuracy using optimal linear combinations of trained neural networks
,”
IEEE Trans. Neural Networks
6
,
792
794
(
1995
).
38.
L.
Breiman
, “
Bagging predictors
,”
Mach. Learn.
24
,
123
140
(
1996
).
39.
S.
Hashem
, “
Optimal linear combinations of neural networks
,”
Neural Networks
10
,
599
614
(
1997
).
40.
D. H.
Wolpert
, “
Stacked generalization
,”
Neural Networks
5
,
241
259
(
1992
).
41.
N.
Ueda
and
R.
Nakano
, “
Generalization error of ensemble estimators
,” in
IEEE International Conference on Neural Networks
(
IEEE
,
1996
), Vol. 1, pp.
90
95
.
42.
P.
Sollich
and
A.
Krogh
, “
Learning with ensembles: How overfitting can be useful
,” in
Proceedings of the 1995 Conference
(
MIT Press
,
1996
), pp.
4
10
.
43.
H. S.
Seung
,
M.
Opper
, and
H.
Sompolinsky
, “
Query by committee
,” in
Proceedings of the Fifth Annual Workshop on Computational Learning Theory (COLT’92)
(
ACM Press
,
New York, NY, USA
,
1992
), pp.
287
294
.
44.
N.
Artrith
and
J.
Behler
, “
High-dimensional neural network potentials for metal surfaces: A prototype study for copper
,”
Phys. Rev. B
85
,
045439
(
2012
).
45.
L.
Chen
,
I.
Sukuba
,
M.
Probst
, and
A.
Kaiser
, “
Iterative training set refinement enables reactive molecular dynamics via machine learned forces
,”
RSC Adv.
10
,
4293
4299
(
2020
).
46.
J.
Behler
, “
Neural network potential-energy surfaces in chemistry: A tool for large-scale simulations
,”
Phys. Chem. Chem. Phys.
13
,
17930
17955
(
2011
).
47.
M.
Gastegger
,
J.
Behler
, and
P.
Marquetand
, “
Machine learning molecular dynamics for the simulation of infrared spectra
,”
Chem. Sci.
8
,
6924
6935
(
2017
);
M.
Gastegger
and
P.
Marquetand
, “
Molecular Dynamics with Neural Network Potentials
,” in (
Springer
,
2020
), vol. 968.
48.
N.
Raimbault
,
A.
Grisafi
,
M.
Ceriotti
, and
M.
Rossi
, “
Using Gaussian process regression to simulate the vibrational Raman spectra of molecular crystals
,”
New J. Phys.
21
,
105001
(
2019
); arXiv:1906.07485.
49.
C.
Schran
,
J.
Behler
, and
D.
Marx
, “
Automated fitting of neural network potentials at coupled cluster accuracy: Protonated water clusters as testing ground
,”
J. Chem. Theory Comput.
16
,
88
99
(
2020
); arXiv:1908.08734.
50.
F.
Musil
,
M. J.
Willatt
,
M. A.
Langovoy
, and
M.
Ceriotti
, “
Fast and accurate uncertainty estimation in chemical machine learning
,”
J. Chem. Theory Comput.
15
,
906
915
(
2019
); arXiv:1809.07653.
51.
E. V.
Podryabinkin
and
A. V.
Shapeev
, “
Active learning of linearly parametrized interatomic potentials
,”
Comput. Mater. Sci.
140
,
171
180
(
2017
).
52.
L.
Zhang
,
D.-Y.
Lin
,
H.
Wang
,
R.
Car
, and
W.
E
, “
Active learning of uniformly accurate interatomic potentials for materials simulation
,”
Phys. Rev. Mater.
3
,
023804
(
2019
); arXiv:1810.11890.
53.
J. S.
Smith
,
B.
Nebgen
,
N.
Lubbers
,
O.
Isayev
, and
A. E.
Roitberg
, “
Less is more: Sampling chemical space with active learning
,”
J. Chem. Phys.
148
,
241733
(
2018
); arXiv:1801.09319.
54.
V. L.
Deringer
,
C. J.
Pickard
, and
G.
Csányi
, “
Data-driven learning of total and local energies in elemental boron
,”
Phys. Rev. Lett.
120
,
156001
(
2018
).
55.
M.
Ceriotti
,
G. A.
Tribello
, and
M.
Parrinello
, “
Demonstrating the transferability and the descriptive power of sketch-map
,”
J. Chem. Theory Comput.
9
,
1521
1532
(
2013
).
56.
F.
Musil
,
S.
De
,
J.
Yang
,
J. E.
Campbell
,
G. M.
Day
, and
M.
Ceriotti
, “
Machine learning for the structure-energy-property landscapes of molecular crystals
,”
Chem. Sci.
9
,
1289
1300
(
2018
).
57.
D.
Marx
and
J.
Hutter
,
Ab Initio Molecular Dynamics: Basic Theory and Advanced Methods
(
Cambridge University Press
,
Cambridge
,
2009
).
58.
See https://www.cp2k.org for CP2K, released under GPL license,
2020
.
59.
J.
Behler
 et al, RuNNer: A Neural Network Code for High-Dimensional Potential-Energy Surfaces, Universität Göttingen 2020 (GPL3 license),
2020
.
60.
Y.
Zhai
,
A.
Caruso
,
S.
Gao
, and
F.
Paesani
, “
Active learning of many-body configuration space: Application to the Cs+-water MB-nrg potential energy function as a case study
,”
J. Chem. Phys.
152
,
144103
(
2020
).
61.
Q.
Lin
,
Y.
Zhang
,
B.
Zhao
, and
B.
Jiang
, “
Automatically growing global reactive neural network potential energy surfaces: A trajectory-free active learning strategy
,”
J. Chem. Phys.
152
,
154104
(
2020
); arXiv:2002.05912.
62.
C.
Schran
,
F.
Uhl
,
J.
Behler
, and
D.
Marx
, “
High-dimensional neural network potentials for solvation: The case of protonated water clusters in helium
,”
J. Chem. Phys.
148
,
102310
(
2018
).
63.
T. E.
Markland
and
M.
Ceriotti
, “
Nuclear quantum effects enter the mainstream
,”
Nat. Rev. Chem.
2
,
0109
(
2018
); arXiv:1803.01037.
64.
M.
Ceriotti
,
W.
Fang
,
P. G.
Kusalik
,
R. H.
McKenzie
,
A.
Michaelides
,
M. A.
Morales
, and
T. E.
Markland
, “
Nuclear quantum effects in water and aqueous systems: Experiment, theory, and current challenges
,”
Chem. Rev.
116
,
7529
7550
(
2016
).
65.
O.
Marsalek
and
T. E.
Markland
, “
Quantum dynamics and spectroscopy of ab initio liquid water: The interplay of nuclear and electronic quantum effects
,”
J. Phys. Chem. Lett.
8
,
1545
1551
(
2017
).
66.
M. D.
Baer
,
C. J.
Mundy
,
M. J.
McGrath
,
I.-F. W.
Kuo
,
J. I.
Siepmann
, and
D. J.
Tobias
, “
Re-examining the properties of the aqueous vapor–liquid interface using dispersion corrected density functional theory
,”
J. Chem. Phys.
135
,
124712
(
2011
).
67.
S.
Shah
,
F.
Palmieri
, and
M.
Datum
, “
Optimal filtering algorithms for fast learning in feedforward neural networks
,”
Neural Networks
5
,
779
787
(
1992
).
68.
T. B.
Blank
and
S. D.
Brown
, “
Adaptive, global, extended Kalman filters for training feedforward neural networks
,”
J. Chemom.
8
,
391
407
(
1994
).
69.
J.
Hutter
,
M.
Iannuzzi
,
F.
Schiffmann
, and
J.
Vandevondele
, “
CP2K: Atomistic simulations of condensed matter systems
,”
Wiley Interdiscip. Rev.: Comput. Mol. Sci.
4
,
15
25
(
2014
).
70.
J.
Vandevondele
,
M.
Krack
,
F.
Mohamed
,
M.
Parrinello
,
T.
Chassaing
, and
J.
Hutter
, “
QUICKSTEP: Fast and accurate density functional calculations using a mixed Gaussian and plane waves approach
,”
Comput. Phys. Commun.
167
,
103
128
(
2005
).
71.
J. P.
Perdew
,
K.
Burke
, and
M.
Ernzerhof
, “
Generalized gradient approximation made simple
,”
Phys. Rev. Lett.
77
,
3865
3868
(
1996
).
72.
Y.
Zhang
and
W.
Yang
, “
Comment on “Generalized gradient approximation made simple”
,”
Phys. Rev. Lett.
80
,
890
(
1998
).
73.
C.
Adamo
and
V.
Barone
, “
Toward reliable density functional methods without adjustable parameters: The PBE0 model
,”
J. Chem. Phys.
110
,
6158
6170
(
1999
).
74.
S.
Grimme
,
J.
Antony
,
S.
Ehrlich
, and
H.
Krieg
, “
A consistent and accurate ab initio parametrization of density functional dispersion correction (DFT-D) for the 94 elements H-Pu
,”
J. Chem. Phys.
132
,
154104
(
2010
).
75.
G.
Bussi
,
D.
Donadio
, and
M.
Parrinello
, “
Canonical sampling through velocity rescaling
,”
J. Chem. Phys.
126
,
014101
(
2007
); arXiv:0803.4060.
76.
M.
Ceriotti
,
M.
Parrinello
,
T. E.
Markland
, and
D. E.
Manolopoulos
, “
Efficient stochastic thermostatting of path integral molecular dynamics
,”
J. Chem. Phys.
133
,
124104
(
2010
); arXiv:1009.1045.
77.
C.
Schran
,
F.
Brieuc
, and
D.
Marx
, “
Converged colored noise path integral molecular dynamics study of the Zundel cation down to ultralow temperatures at coupled cluster accuracy
,”
J. Chem. Theory Comput.
14
,
5068
5078
(
2018
).
78.
C.
Schran
and
D.
Marx
, “
Quantum nature of the hydrogen bond from ambient conditions down to ultra-low temperatures
,”
Phys. Chem. Chem. Phys.
21
,
24967
24975
(
2019
).
79.
R.
Vácha
,
O.
Marsalek
,
A. P.
Willard
,
D. J.
Bonthuis
,
R. R.
Netz
, and
P.
Jungwirth
, “
Charge transfer between water molecules as the possible origin of the observed charging at the surface of pure water
,”
J. Phys. Chem. Lett.
3
,
107
111
(
2011
).
80.
M.
Rossi
,
M.
Ceriotti
, and
D. E.
Manolopoulos
, “
How to remove the spurious resonances from ring polymer molecular dynamics
,”
J. Chem. Phys.
140
,
234116
(
2014
); arXiv:1406.1074v1.
81.
V.
Kapil
,
J.
Behler
, and
M.
Ceriotti
, “
High order path integrals made easy
,”
J. Chem. Phys.
145
,
234103
(
2016
).
82.
M.
Buchowiecki
and
J.
Vaníček
, “
Monte Carlo evaluation of the equilibrium isotope effects using the Takahashi-Imada factorization of the Feynman path integral
,”
Chem. Phys. Lett.
588
,
11
16
(
2013
).

Supplementary Material

You do not currently have access to this content.