Density functional theory (DFT) is the most successful and widely used approach for computing the electronic structure of matter. However, for tasks involving large sets of candidate molecules, running DFT separately for every possible compound of interest is forbiddingly expensive. In this paper, we propose a neural network based machine learning algorithm which, assuming a sufficiently large training sample of actual DFT results, can instead learn to predict certain properties of molecules purely from their molecular graphs. Our algorithm is based on the recently proposed covariant compositional networks framework and involves tensor reduction operations that are covariant with respect to permutations of the atoms. This new approach avoids some of the representational limitations of other neural networks that are popular in learning from molecular graphs and yields promising results in numerical experiments on the Harvard Clean Energy Project and QM9 molecular datasets.

1.
P.
Hohenberg
and
W.
Kohn
,
Phys. Rev.
136
,
B864
(
1964
).
2.
Y.
LeCun
,
Y.
Bengio
, and
G.
Hinton
,
Nature
521
,
436
444
(
2015
).
3.
R.
Ramakrishnan
,
P. O.
Dral
,
M.
Rupp
, and
O. A.
von Lilienfeld
,
Sci. Data
1
,
140022
(
2014
).
4.
J.
Hachmann
,
R.
Olivares-Amaya
,
S.
Atahan-Evrenk
,
C.
Amador-Bedolla
,
R. S.
Sanchez-Carrera
,
A.
Gold-Parker
,
L.
Vogt
,
A. M.
Brockway
, and
A.
Aspuru-Guzik
,
J. Phys. Chem. Lett.
2
,
2241
2251
(
2011
).
5.
S.
Kirklin
,
J. E.
Saal
,
B.
Meredig
,
A.
Thompson
,
J. W.
Doak
,
M.
Aykol
,
S.
Rühl
, and
C.
Wolverton
,
npj Comput. Mater.
1
,
15010
(
2015
).
6.
K.
Hansen
,
F.
Biegler
,
R.
Ramakrishnan
,
W.
Pronobis
,
O. A.
von Lilienfeld
,
K.-R.
Müller
, and
A.
Tkatchenko
,
J. Phys. Chem. Lett.
6
,
2326
(
2015
).
7.
B.
Huang
and
O. A.
von Lilienfeld
,
J. Chem. Phys.
145
,
161102
(
2016
).
8.
F. A.
Faber
,
L.
Hutchison
,
B.
Huang
,
J.
Gilmer
,
S. S.
Schoenholz
,
G. E.
Dahl
,
O.
Vinyals
,
S.
Kearnes
,
P. F.
Riley
, and
O. A.
von Lilienfeld
,
J. Chem. Theory Comput.
13
,
5255
5264
(
2017
).
9.
O. A.
von Lilienfeld
,
R.
Ramakrishnan
,
M.
Rupp
, and
A.
Knoll
,
Int. J. Quantum Chem.
115
,
1084
(
2015
).
10.
M.
Rupp
,
A.
Tkatchenko
,
K. R.
Müller
, and
O. A.
von Lilienfeld
,
Phys. Rev. Lett.
108
,
058301
(
2012
).
11.
K.
Hansen
,
G.
Montavon
,
F.
Biegler
,
S.
Fazli
,
M.
Rupp
,
M.
Scheffler
,
O. A.
von Lilienfeld
,
A.
Tkatchenko
, and
K.-R.
Müller
,
J. Chem. Theory Comput.
9
,
3404
(
2013
).
12.
G.
Montavon
,
M.
Rupp
,
V.
Gobre
,
A.
Vazquez-Mayagoitia
,
K.
Hansen
,
A.
Tkatchenko
,
K.-R.
Müller
, and
O. A.
von Lilienfeld
,
New J. Phys.
15
,
095003
(
2013
).
13.
A. P.
Bartók
,
R.
Kondor
, and
G.
Csányi
,
Phys. Rev. B
87
,
184115
(
2013
).
14.
A. P.
Bartók
,
S.
De
,
C.
Poelking
,
N.
Bernstein
,
J. R.
Kermode
,
G.
Csányi
, and
M.
Ceriotti
,
Sci. Adv.
3
,
e1701816
(
2017
).
15.
G.
Ferré
,
J.-B.
Maillet
, and
G.
Stoltz
,
J. Chem. Phys.
143
,
104114
(
2015
).
16.
J.
Behler
and
M.
Parrinello
,
Phys. Rev. Lett.
98
,
146401
(
2007
).
17.
J.
Behler
,
J. Chem. Phys.
134
,
074106
(
2011
).
18.
A. V.
Shapeev
,
Multiscale Model. Simul.
14
,
1153
(
2016
).
19.
M.
Hirn
,
S.
Mallat
, and
N.
Poilvert
,
Multiscale Model. Simul.
15
,
827
(
2017
).
20.
J.
Bruna
,
W.
Zaremba
,
A.
Szlam
, and
Y.
LeCun
, in
Proceedings of International Conference on Learning Representations (ICLR)
(
PMLR
,
2014
), Vol. 3.
21.
D. K.
Duvenaud
,
D.
Maclaurin
,
J.
Iparraguirre
,
R.
Bombarell
,
T.
Hirzel
,
A.
Aspuru-Guzik
, and
R. P.
Adams
, in
Advances in Neural Information Processing Systems
(
NIPS
,
2015
), Vol. 28, p.
2224
.
22.
S.
Kearns
,
K.
McCloskey
,
M.
Brendl
,
V.
Pande
, and
P.
Riley
,
J. Comput.-Aided Mol. Des.
30
,
595
(
2016
).
23.
M. M.
Bronstein
,
J.
Bruna
,
Y.
LeCun
,
A.
Szlam
, and
P.
Vandergheynst
,
IEEE Signal Process. Mag.
34
,
18
(
2017
).
24.
K. T.
Schütt
,
F.
Arbabzadah
,
S.
Chmiela
,
K. R.
Müller
, and
A.
Tkatchenko
,
Nat. Commun.
8
,
13890
(
2017
).
25.
J.
Gilmer
,
S. S.
Schoenholz
,
P. F.
Riley
,
O.
Vinyals
, and
G. E.
Dahl
, in
Proceedings of International Conference on Machine Learning (ICML)
(
PMLR
,
2017
), Vol. 70.
26.
K.
Schütt
,
P.-J.
Kindermans
,
H. E.
Sauceda Felix
,
S.
Chmiela
,
A.
Tkatchenko
, and
K.-R.
Müller
, in
Proceedings of NIPS
,
2017
.
27.
R.
Kondor
,
T. S.
Hy
,
H.
Pan
,
S.
Trivedi
, and
B. M.
Anderson
, in
Proceedings of International Conference on Machine Learning (ICLR)
,
2018
.
28.
T.
Gärtner
, in
NIPS 2002 Workshop on Unreal Data
,
2002
.
29.
K. M.
Borgwardt
and
H. P.
Kriegel
, in
Proceedings of IEEE International Conference on Data Mining
(
IEEE
,
2005
), Vol. 5, p.
74
.
30.
A.
Feragen
,
N.
Kasenburg
,
J.
Peterson
,
M.
de Bruijne
, and
K. M.
Borgwardt
, in
Advances in Neural Information Processing Systems
(
NIPS
,
2013
), Vol. 26.
31.
N.
Shervashidze
,
S. V. N.
Vishwanathan
,
T.
Petri
,
K.
Mehlhorn
, and
K. M.
Borgwardt
, in
Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS)
(
PMLR
,
2009
), Vol. 12, p.
488
.
32.
S. V. N.
Vishwanathan
,
N. N.
Schraudolf
,
R.
Kondor
, and
K. M.
Bogwardt
,
J. Mach. Learn. Res.
11
,
1201
(
2010
).
33.
N.
Shervashidze
,
P.
Schweitzer
,
E. J.
van Leeuwan
,
K.
Mehlhorn
, and
K. M.
Borgwardt
,
J. Mach. Learn. Res.
12
,
2539
(
2011
).
34.
M.
Neumann
,
R.
Garnett
,
C.
Baukhage
, and
K.
Kersting
,
Mach. Learn.
102
,
209
(
2016
).
35.
R.
Kondor
and
K. M.
Borgwardt
, in
Proceedings of International Conference on Machine Learning (ICML)
(
PMLR
,
2008
), Vol. 25, p.
496
.
36.
F.
Scarselli
,
M.
Gori
,
A. C.
Tsoi
,
M.
Hagenbuchner
, and
G.
Monfardini
,
IEEE Trans. Neural Networks
20
,
61
(
2009
).
37.
M.
Niepert
,
M.
Ahmed
, and
K.
Kutzkov
, in
Proceedings of International Conference on Machine Learning (ICML)
(
PMLR
,
2016
), Vol. 33, p.
2014
.
38.
Y.
LeCun
,
Y.
Bengio
, and
P.
Haffner
,
Proc. IEEE
86
,
2278
(
1998
).
39.
A.
Krizhevsky
,
I.
Sutskever
, and
G. E.
Hinton
, in
Advances in Neural Information Processing Systems
(
NIPS
,
2012
), Vol. 25, p.
1097
.
40.
Y.
Li
,
D.
Tarlow
,
M.
Brockschmidt
, and
R.
Zemel
, in
Proceedings of International Conference on Learning Representations
(
PMLR
,
2016
), Vol. 4.
41.
P.
Battaglia
,
R.
Pascanu
,
M.
Lai
,
D. J.
Rezende
, and
K.
Kavukcuoglu
, in
Advances in Neural Information Processing Systems
(
NIPS
,
2016
), Vol. 29, p.
4502
.
42.
T. N.
Kipf
and
M.
Welling
, in
Proceedings of International Conference on Learning Representations
(
PMLR
,
2017
), Vol. 5.
43.
M.
Fischler
and
R.
Elschlager
,
IEEE Trans. Comput.
C-22
,
67
(
1973
).
44.
Y.
Ohta
,
T.
Kanade
, and
T.
Sakai
, in
Proceedings of IJCPR
(
Institute of Electrical and Electronics Engineers
,
1978
), Vol. 4, p.
752
.
45.
Z. W.
Tu
,
X. R.
Chen
,
A. L.
Yuille
, and
S. C.
Zhu
,
Int. J. Comput. Vision
63
,
113
(
2005
).
46.
P. F.
Felzenszwalb
and
D. P.
Huttenlocher
,
Int. J. Comput. Vision
61
,
55
(
2005
).
47.
S.
Zhu
and
D.
Mumford
,
Found. Trends Comput. Graphics Vision
2
,
259
(
2006
).
48.
P. F.
Felzenszwalb
,
R. B.
Girshick
,
D.
McAllester
, and
D.
Ramanan
,
IEEE Trans. Pattern Anal. Mach. Intell.
32
,
1627
(
2010
).
49.
B.
Weisfeiler
and
A. A.
Lehman
, Nauchno-Tekh. Inf.
2
(
9
),
12
(
1968
).
50.
R. C.
Read
and
D. G.
Corneil
,
J. Graph Theory
1
,
339
(
1977
).
51.
J. Y.
Cai
,
M.
Furer
, and
N.
Immerman
,
Combinatorica
12
,
389
(
1992
).
52.
T.
Cohen
and
M.
Welling
, in
Proceedings of International Conference on Machine Learning (ICML)
(
PMLR
,
2016
), Vol. 33, p.
2990
.
53.
D. E.
Worrall
,
S.
Garbin
,
D.
Turmukhambetov
, and
G. J.
Brostow
, in
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(
IEEE
,
2017
).
54.
W. T.
Freeman
and
E. H.
Adelson
,
IEEE Trans. Pattern Anal. Mach. Intell.
13
,
891
(
1991
).
55.
E. P.
Simoncelli
,
W. T.
Freeman
,
E. H.
Adelson
, and
D. J.
Heeger
,
IEEE Trans. Inf. Theory
38
,
587
(
1992
).
56.
P.
Perona
,
IEEE Trans. Pattern Anal. Mach. Intell.
17
,
488
(
1995
).
57.
P. C.
Teo
and
Y.
Hel-Or
,
Pattern Recognit. Lett.
19
,
7
(
1998
).
58.
R.
Manduchi
,
P.
Perona
, and
D.
Shy
,
IEEE Trans. Signal Process.
46
,
1168
(
1998
).
59.
T.
Cohen
and
M.
Welling
, in
Proceedings of International Conference on Learning Representations (ICLR)
(
PMLR
,
2017
), Vol. 5.
60.
J.-P.
Serre
,
Linear Representations of Finite Groups
, Graduate Texts in Mathamatics (
Springer-Verlag
,
1977
), Vol. 42.
61.
B. E.
Sagan
,
The Symmetric Group
, Graduate Texts in Mathamatics (
Springer
,
2001
).
62.
R.
Kondor
and
H.
Pan
, in
Advances in Neural Information Processing Systems
(
NIPS
,
2016
), Vol. 29, p.
2982
.
63.
N. M.
Kriege
,
P.
Giscard
, and
R.
Wilson
,
Advances in Neural Information Processing Systems
(
NIPS
,
2016
), Vol. 20, p.
1623
.
64.
T. S.
Hy
, “
GraphFlow: A C++ deep learning framework
,” https://github.com/HyTruongSon/GraphFlow,
2017
.
65.
M.
Abadi
,
A.
Agarwal
,
P.
Barham
,
E.
Brevdo
,
Z.
Chen
,
C.
Citro
,
G. S.
Corrado
,
A.
Davis
,
J.
Dean
,
M.
Devin
,
S.
Ghemawat
,
I.
Goodfellow
,
A.
Harp
,
G.
Irving
,
M.
Isard
,
Y.
Jia
,
R.
Jozefowicz
,
L.
Kaiser
,
M.
Kudlur
,
J.
Levenberg
,
D.
Mané
,
R.
Monga
,
S.
Moore
,
D.
Murray
,
C.
Olah
,
M.
Schuster
,
J.
Shlens
,
B.
Steiner
,
I.
Sutskever
,
K.
Talwar
,
P.
Tucker
,
V.
Vanhoucke
,
V.
Vasudevan
,
F.
Viégas
,
O.
Vinyals
,
P.
Warden
,
M.
Wattenberg
,
M.
Wicke
,
Y.
Yu
, and
X.
Zheng
, “
TensorFlow: Large-scale machine learning on heterogeneous systems
,”
2015
, software available from tensorflow.org.
66.
A.
Paszke
,
S.
Gross
,
S.
Chintala
,
G.
Chanan
,
E.
Yang
,
Z.
DeVito
,
Z.
Lin
,
A.
Desmaison
,
L.
Antiga
, and
A.
Lerer
, in
Advances in Neural Information Processing Systems
(
NIPS
,
2017
), Vol. 30.
67.
T.
Chen
,
M.
Li
,
Y.
Li
,
M.
Lin
,
N.
Wang
,
M.
Wang
,
T.
Xiao
,
B.
Xu
,
C.
Zhang
, and
Z.
Zhang
, in
Conference and Workshop on Neural Information Processing Systems (NIPS)
,
2016
.
68.
S.
Trivedi
,
T. S.
Hy
, and
H.
Pan
, “
CCN in PyTorch
,” https://github.com/horacepan/CCN,
2017
.
69.
D. P.
Kingma
and
J.
Ba
, in
Proceedings of International Conference on Learning Representations (ICLR)
,
San Diego
,
2015
.
70.
A. K.
Debnat
,
R. L. L.
de Compadre
,
G.
Debnath
,
A. J.
Shusterman
, and
C.
Hansch
,
J. Med. Chem.
34
,
786
(
1991
).
71.
H.
Toivonen
,
A.
Srinivasan
,
R. D.
King
,
S.
Kramer
, and
C.
Helma
,
Bioinformatics
19
,
1183
(
2003
).
72.
N.
Wale
,
I. A.
Watson
, and
G.
Karypis
,
Knowl. Inf. Syst.
14
,
347
(
2008
).
You do not currently have access to this content.