We describe a method for the post-hoc interpretation of a neural network (NN) trained on the global and local minima of neutral water clusters. We use the structures recently reported in a newly published database containing over 5 × 106 unique water cluster networks (H2O)N of size N = 3–30. The structural properties were first characterized using chemical descriptors derived from graph theory, identifying important trends in topology, connectivity, and polygon structure of the networks associated with the various minima. The code to generate the molecular graphs and compute the descriptors is available at https://github.com/exalearn/molecular-graph-descriptors, and the graphs are available alongside the original database at https://sites.uw.edu/wdbase/. A Continuous-Filter Convolutional Neural Network (CF-CNN) was trained on a subset of 500 000 networks to predict the potential energy, yielding a mean absolute error of 0.002 ± 0.002 kcal/mol per water molecule. Clusters of sizes not included in the training set exhibited errors of the same magnitude, indicating that the CF-CNN protocol accurately predicts energies of networks for both smaller and larger sizes than those used during training. The graph-theoretical descriptors were further employed to interpret the predictive power of the CF-CNN. Topological measures, such as the Wiener index, the average shortest path length, and the similarity index, suggested that all networks from the test set were within the range of values as the ones from the training set. The graph analysis suggests that larger errors appear when the mean degree and the number of polygons in the cluster lie further from the mean of the training set. This indicates that the structural space, and not just the chemical space, is an important factor to consider when designing training sets, as predictive errors can result when the structural composition is sufficiently different from the bulk of those in the training set. To this end, the developed descriptors are quite effective in explaining the results of the CF-CNN (a.k.a. the “black box”) model.
REFERENCES
1.
X.
Yang
, Y.
Wang
, R.
Byrne
, G.
Schneider
, and S.
Yang
, “Concepts of artificial intelligence for computer-assisted drug discovery
,” Chem. Rev.
119
, 10520
–10594
(2019
).2.
B.
Sanchez-Lengeling
and A.
Aspuru-Guzik
, “Inverse molecular design using machine learning: Generative models for matter engineering
,” Science
361
, 360
–365
(2018
).3.
C. W.
Coley
, W. H.
Green
, and K. F.
Jensen
, “Machine learning in computer-aided synthesis planning
,” Acc. Chem. Res.
51
, 1281
–1289
(2018
).4.
J.
Behler
, “First principles neural network potentials for reactive simulations of large molecular and condensed systems
,” Angew. Chem., Int. Ed.
56
, 12828
–12840
(2017
).5.
J. S.
Smith
, O.
Isayev
, and A. E.
Roitberg
, “ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
,” Chem. Sci.
8
, 3192
–3203
(2017
).6.
N.
Lubbers
, J. S.
Smith
, and K.
Barros
, “Hierarchical modeling of molecular energies using a deep neural network
,” J. Chem. Phys.
148
, 241715
(2018
).7.
K.
Simonyan
, A.
Vedaldi
, and A.
Zisserman
, “Deep inside convolutional networks: Visualising image classification models and saliency maps
,” in Workshop at International Conference on Learning Representations
, 2014
.8.
M. T.
Ribeiro
, S.
Singh
, and C.
Guestrin
, ““Why should i trust you?” Explaining the predictions of any classifier
,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
(ACM
, 2016
), pp. 1135
–1144
.9.
Q.
Zhang
, Y.
Nian Wu
, and S.-C.
Zhu
, “Interpretable convolutional neural networks
,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(IEEE
, 2018
).10.
M.
Raissi
, P.
Perdikaris
, and G. E.
Karniadakis
, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
,” J. Comput. Phys.
378
, 686
–707
(2019
).11.
A.
Rakshit
, P.
Bandyopadhyay
, J. P.
Heindel
, and S. S.
Xantheas
, “Atlas of putative minima and low-lying energy networks of water clusters n = 3–25
,” J. Chem. Phys.
151
, 214307
(2019
).12.
E.
Aprà
, A. P.
Rendell
, R. J.
Harrison
, V.
Tipparaju
, W. A.
deJong
, and S. S.
Xantheas
, “Liquid water: Obtaining the right answer for the right reasons
,” in Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC’09
(ACM
, New York, NY, USA
, 2009
), pp. 66-1
–66-7
.13.
S.
Yoo
and S. S.
Xantheas
, “Structures, energetics and spectroscopic fingerprints of water clusters n = 2–24
,” in Handbook of Computational Chemistry
, 2nd ed., edited by J.
Leszczynski
(Springer International Publishing Switzerland
, 2017
), Chap. 26, pp. 1139
–1175
.14.
C. J.
Burnham
and S. S.
Xantheas
, “Development of transferable interaction models for water. IV. A flexible, all-atom polarizable potential (TTM2-F) based on geometry dependent charges derived from an ab initio monomer dipole moment surface
,” J. Chem. Phys.
116
, 5115
–5124
(2002
).15.
G. S.
Fanourgakis
and S. S.
Xantheas
, “The flexible, polarizable, thole-type interaction potential for water (TTM2-F) revisited
,” J. Phys. Chem. A
110
, 4100
–4106
(2006
).16.
G. S.
Fanourgakis
and S. S.
Xantheas
, “Development of transferable interaction potentials for water. V. Extension of the flexible, polarizable, thole-type model potential (TTM3-F, v. 3.0) to describe the vibrational spectra of water clusters and liquid water
,” J. Chem. Phys.
128
, 074506
(2008
).17.
Y.
Wang
and J. M.
Bowman
, “Towards an ab initio flexible potential for water, and post-harmonic quantum vibrational analysis of water clusters
,” Chem. Phys. Lett.
491
, 1
–10
(2010
).18.
G. R.
Medders
, V.
Babin
, and F.
Paesani
, “Development of a “first-principles” water potential with flexible monomers. III. Liquid phase properties
,” J. Chem. Theory Comput.
10
, 2906
–2910
(2014
).19.
G. S.
Fanourgakis
, G. K.
Schenter
, and S. S.
Xantheas
, “A quantitative account of quantum effects in liquid water
,” J. Chem. Phys.
125
, 141102
(2006
).20.
F.
Paesani
, S.
Iuchi
, and G. A.
Voth
, “Quantum effects in liquid water from an ab initio-based polarizable force field
,” J. Chem. Phys.
127
, 074506
(2007
).21.
F.
Paesani
, S. S.
Xantheas
, and G. A.
Voth
, “Infrared spectroscopy and hydrogen-bond dynamics of liquid water from centroid molecular dynamics with an ab initio-based force field
,” J. Phys. Chem. B
113
, 13118
–13130
(2009
).22.
F.
Paesani
, S.
Yoo
, H. J.
Bakker
, and S. S.
Xantheas
, “Nuclear quantum effects in the reorientation of water
,” J. Phys. Chem. Lett.
1
, 2316
(2010
).23.
S.
Izvekov
, M.
Parrinello
, C. J.
Burnham
, and G. A.
Voth
, “Effective force fields for condensed phase systems from ab initio molecular dynamics simulation: A new method for force-matching
,” J. Chem. Phys.
120
, 10896
–10913
(2004
).24.
A.
Davtyan
, J. F.
Dama
, G. A.
Voth
, and H. C.
Andersen
, “Dynamic force matching: A method for constructing dynamical coarse-grained models with realistic time dependence
,” J. Chem. Phys.
142
, 154104
(2015
).25.
S.
Chmiela
, A.
Tkatchenko
, H. E.
Sauceda
, I.
Poltavsky
, K. T.
Schütt
, and K.-R.
Müller
, “Machine learning of accurate energy-conserving molecular force fields
,” Sci. Adv.
3
, e1603015
(2017
).26.
P. O.
Dral
, A.
Owens
, S. N.
Yurchenko
, and W.
Thiel
, “Structure-based sampling and self-correcting machine learning for accurate calculations of potential energy surfaces and vibrational levels
,” J. Chem. Phys.
146
, 244108
(2017
).27.
N.
Bernstein
, G.
Csányi
, and V. L.
Deringer
, “De novo exploration and self-guided learning of potential-energy surfaces
,” npj Comput. Mater.
5
, 99
(2019
).28.
T.
Morawietz
and J.
Behler
, “A density-functional theory-based neural network potential for water clusters including van der Waals corrections
,” J. Phys. Chem. A
117
, 7356
–7366
(2013
).29.
S.
Kondati Natarajan
, T.
Morawietz
, and J.
Behler
, “Representing the potential-energy surface of protonated water clusters by high-dimensional neural network potentials
,” Phys. Chem. Chem. Phys.
17
, 8356
–8371
(2015
).30.
C.
Schran
, F.
Uhl
, J.
Behler
, and D.
Marx
, “High-dimensional neural network potentials for solvation: The case of protonated water clusters in helium
,” J. Chem. Phys.
148
, 102310
(2018
).31.
J.
Behler
and M.
Parrinello
, “Generalized neural-network representation of high-dimensional potential-energy surfaces
,” Phys. Rev. Lett.
98
, 146401
(2007
).32.
J.
Behler
, “Atom-centered symmetry functions for constructing high-dimensional neural network potentials
,” J. Chem. Phys.
134
, 074106
(2011
).33.
A.
Singraber
, J.
Behler
, and C.
Dellago
, “Library-based lammps implementation of high-dimensional neural network potentials
,” J. Chem. Theory Comput.
15
, 1827
–1840
(2019
).34.
A.
Singraber
, T.
Morawietz
, J.
Behler
, and C.
Dellago
, “Parallel multistream training of high-dimensional neural network potentials
,” J. Chem. Theory Comput.
15
, 3075
–3092
(2019
).35.
C.
Schran
, J.
Behler
, and D.
Marx
, “Automated fitting of neural network potentials at coupled cluster accuracy: Protonated water clusters as testing ground
,” J. Chem. Theor. Comput.
16
(1
) , 88
–99
(2019
).36.
J.
Behler
, “Constructing high-dimensional neural network potentials: A tutorial review
,” Int. J. Quantum Chem.
115
, 1032
–1050
(2015
).37.
K. T.
Schütt
, F.
Arbabzadah
, S.
Chmiela
, K. R.
Müller
, and A.
Tkatchenko
, “Quantum-chemical insights from deep tensor neural networks
,” Nat. Commun.
8
, 13890
(2017
).38.
K. T.
Schütt
, H. E.
Sauceda
, P.-J.
Kindermans
, A.
Tkatchenko
, and K.-R.
Müller
, “SchNet—A deep learning architecture for molecules and materials
,” J. Chem. Phys.
148
, 241722
(2018
).39.
K. T.
Schütt
, P.
Kessel
, M.
Gastegger
, K. A.
Nicoli
, A.
Tkatchenko
, and K.-R.
Müller
, “SchNetPack: A deep learning toolbox for atomistic systems
,” J. Chem. Theory Comput.
15
, 448
–455
(2019
).40.
D. J.
Wales
and J. P. K.
Doye
, “Global optimization by basin-hopping and the lowest energy structures of Lennard-Jones clusters containing up to 110 atoms
,” J. Phys. Chem. A
101
, 5111
–5116
(1997
).41.
A.
Rakshit
and P.
Bandyopadhyay
, “Finding low energy minima of (H2O)25 and (H2O)30 with temperature basin paving Monte Carlo method with effective fragment potential: New ‘global minimum’and graph theoretical characterization of low energy structures
,” Comput. Theor. Chem.
1021
, 206
–214
(2013
).42.
A.
Rakshit
, T.
Yamaguchi
, T.
Asada
, and P.
Bandyopadhyay
, “Understanding the structure and hydrogen bonding network of (H2O)32 and (H2O)33: An improved Monte Carlo temperature basin paving (MCTBP) method and quantum theory of atoms in molecules (QTAIM) analysis
,” RSC Adv.
7
, 18401
–18417
(2017
).43.
P.
Battaglia
, J. B. C.
Hamrick
, V.
Bapst
, A.
Sanchez
, V.
Zambaldi
, M.
Malinowski
, A.
Tacchetti
, D.
Raposo
, A.
Santoro
, R.
Faulkner
, C.
Gulcehre
, F.
Song
, A.
Ballard
, J.
Gilmer
, G. E.
Dahl
, A.
Vaswani
, K.
Allen
, C.
Nash
, V. J.
Langston
, C.
Dyer
, N.
Heess
, D.
Wierstra
, P.
Kohli
, M.
Botvinick
, O.
Vinyals
, Y.
Li
, and R.
Pascanu
, “Relational inductive biases, deep learning, and graph networks
,” arXiv:1806.01261 (2018
).44.
R.
Kumar
, J. R.
Schmidt
, and J. L.
Skinner
, “Hydrogen bonding definitions and dynamics in liquid water
,” J. Chem. Phys.
126
, 204107
(2007
).45.
J.-L.
Kuo
, J. V.
Coe
, S. J.
Singer
, Y. B.
Band
, and L.
Ojamäe
, “On the use of graph invariants for efficiently generating hydrogen bond topologies and predicting physical properties of water clusters and ice
,” J. Chem. Phys.
114
, 2527
–2540
(2001
).46.
A. M.
Tokmachev
, A. L.
Tchougréeff
, and R.
Dronskowski
, “Hydrogen-bond networks in water clusters (H2O)20: An exhaustive quantum-chemical analysis
,” ChemPhysChem
11
, 384
–388
(2010
).47.
A. M.
Tokmachev
, A. L.
Tchougréeff
, and R.
Dronskowski
, “Benchmarks of graph invariants for hydrogen-bond networks in water clusters of different topology
,” in Péter R. Surján
(Springer
, 2016
), pp. 157
–164
.48.
S.
Yoo
, M. V.
Kirov
, and S. S.
Xantheas
, “Lowest energy networks of the T-cage (H2O)24 cluster and their use in constructing unit cells of the structure I (sI) hydrate lattice
,” J. Am. Chem. Soc.
131
, 7564
(2009
).49.
D.
Koutra
, A.
Parikh
, A.
Ramdas
, and J.
Xiang
, “Algorithms for graph similarity and subgraph matching
,” in Proceedings of Ecology Inference Conference
(2011
), Vol. 17.50.
D. J.
Watts
and S. H.
Strogatz
, “Collective dynamics of ‘small-world’ networks
,” Nature
393
, 440
–442
(1998
).51.
S. S.
Xantheas
, “Cooperativity and hydrogen bonding network in water clusters
,” Chem. Phys.
258
, 225
–231
(2000
).52.
A.
Rahman
and F. H.
Stillinger
, “Hydrogen-bond patterns in liquid water
,” J. Am. Chem. Soc.
95
, 7943
–7948
(1973
).53.
T. H.
Dunning
, Jr., R. J.
Harrison
, D.
Feller
, and S. S.
Xantheas
, “Promise and challenge of high-performance computing, with examples from molecular modeling
,” Philos. Trans. R. Soc. London, Ser. A
360
, 1079
(2002
).54.
G. S.
Fanourgakis
, E.
Aprà
, and S. S.
Xantheas
, “High-level ab initio calculations for the four low-lying families of minima of (H2O)20. I. Estimates of MP2/CBS binding energies and comparison with empirical potentials
,” J. Chem. Phys.
121
, 2655
(2004
).55.
J. P.
Heindel
and S. S.
Xantheas
, “Database of water cluster minima
,” https://sites.uw.edu/wdbase/, 2019
.2020
U.S. Government
You do not currently have access to this content.
