Molecular simulations are widely applied in the study of chemical and bio-physical problems. However, the accessible timescales of atomistic simulations are limited, and extracting equilibrium properties of systems containing rare events remains challenging. Two distinct strategies are usually adopted in this regard: either sticking to the atomistic level and performing enhanced sampling or trading details for speed by leveraging coarse-grained models. Although both strategies are promising, either of them, if adopted individually, exhibits severe limitations. In this paper, we propose a machine-learning approach to ally both strategies so that simulations on different scales can benefit mutually from their crosstalks: Accurate coarse-grained (CG) models can be inferred from the fine-grained (FG) simulations through deep generative learning; in turn, FG simulations can be boosted by the guidance of CG models via deep reinforcement learning. Our method defines a variational and adaptive training objective, which allows end-to-end training of parametric molecular models using deep neural networks. Through multiple experiments, we show that our method is efficient and flexible and performs well on challenging chemical and bio-molecular systems.

1.
D.
Frenkel
and
B.
Smit
,
Understanding Molecular Simulation: From Algorithms to Applications
(
Elsevier
,
2001
).
2.
M. E.
Tuckerman
,
J. Phys.: Condens. Matter
14
(
50
),
R1297
(
2002
).
3.
G.
Fiorin
,
M. L.
Klein
, and
J.
Hénin
,
Mol. Phys.
111
(
22-23
),
3345
3362
(
2013
).
4.
G. A.
Voth
,
Coarse-Graining of Condensed Phase and Biomolecular Systems
(
CRC Press
,
2008
).
5.
W. G.
Noid
,
J.-W.
Chu
,
G. S.
Ayton
,
V.
Krishna
,
S.
Izvekov
,
G. A.
Voth
,
A.
Das
, and
H. C.
Andersen
,
J. Chem. Phys.
128
(
24
),
244114
(
2008
).
6.
G. M.
Torrie
and
J. P.
Valleau
,
J. Comput. Phys.
23
(
2
),
187
199
(
1977
).
7.
B. W.
Silverman
,
Density Estimation for Statistics and Data Analysis
(
Routledge
,
2018
).
8.
R.
Bellman
, “
Adaptive control processes: a guided tour
,” (
Princeton
,
New Jersey, USA,
1961
).
9.
J. J.
Crabbe
, “
Handling the curse of dimensionality in multivariate kernel density estimation
,”
Ph.D. dissertation, Oklahoma State University
,
2013
.
10.
Y.
LeCun
,
Y.
Bengio
, and
G.
Hinton
,
Nature
521
(
7553
),
436
(
2015
).
11.
I.
Goodfellow
,
Y.
Bengio
, and
A.
Courville
,
Deep Learning
(
MIT Press
,
2016
).
12.
E.
Schneider
,
L.
Dai
,
R. Q.
Topper
,
C.
Drechsel-Grau
, and
M. E.
Tuckerman
,
Phys. Rev. Lett.
119
(
15
),
150601
(
2017
).
13.
J.
Wang
,
S.
Olsson
,
C.
Wehmeyer
,
A.
Pérez
,
N. E.
Charron
,
G.
De Fabritiis
,
F.
Noé
, and
C.
Clementi
,
ACS Cent. Sci.
5
,
755
(
2019
).
14.
V. N.
Vapnik
,
IEEE Trans. Neural Networks
10
(
5
),
988
999
(
1999
).
15.
D. P.
Kingma
and
M.
Welling
, “
Auto-encoding variational bayes
,” 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada (
2014
).
16.
A.
Van den Oord
,
N.
Kalchbrenner
,
L.
Espeholt
,
O.
Vinyals
, and
A.
Graves
, presented at the
Advances in Neural Information Processing Systems
,
2016
.
17.
L.
Dinh
,
D.
Krueger
, and
Y.
Bengio
, preprint arXiv:1410.8516 (
2014
).
18.
D. J.
Rezende
and
S.
Mohamed
, “
Variational inference with normalizing flows
,” in International Conference on Machine Learning (ICML, 2015), pp. 1530–1538.
19.
M. E.
Abbasnejad
,
Q.
Shi
,
A. v. d.
Hengel
, and
L.
Liu
, presented at the
The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
.
20.
R.
Salakhutdinov
,
A.
Mnih
, and
G.
Hinton
, presented at the
Proceedings of the 24th International Conference on Machine Learning
,
2007
.
21.
R.
Salakhutdinov
and
G.
Hinton
, in
Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics
, edited by
D.
David van
and
W.
Max
(
PMLR, Proceedings of Machine Learning Research
,
2009
), Vol. 5, pp.
448
455
.
22.
J. J.
Hopfield
,
Proc. Natl. Acad. Sci. U. S. A.
79
(
8
),
2554
2558
(
1982
).
23.
D.
Wales
,
Energy Landscapes: Applications to Clusters, Biomolecules and Glasses
(
Cambridge University Press
,
2003
).
24.
Y.
Du
and
I.
Mordatch
, preprint arXiv:1903.08689 (
2019
).
25.
A. P.
Lyubartsev
,
Eur. Biophys. J.
35
(
1
),
53
(
2005
).
26.
M. S.
Shell
,
J. Chem. Phys.
129
(
14
),
144108
(
2008
).
27.
D.
Reith
,
M.
Pütz
, and
F.
Müller-Plathe
,
J. Comput. Chem.
24
(
13
),
1624
1636
(
2003
).
28.
I.
Goodfellow
,
J.
Pouget-Abadie
,
M.
Mirza
,
B.
Xu
,
D.
Warde-Farley
,
S.
Ozair
,
A.
Courville
, and
Y.
Bengio
, presented at the
Advances in Neural Information Processing Systems
,
2014
.
29.
S.
Kullback
and
R. A.
Leibler
,
Ann. Math. Stat.
22
(
1
),
79
86
(
1951
).
30.
M.
Arjovsky
,
S.
Chintala
, and
L.
Bottou
, presented at the
International Conference on Machine Learning
,
2017
.
31.
A.
Chaimovich
and
M. S.
Shell
,
J. Chem. Phys.
134
(
9
),
094112
(
2011
).
32.
O.
Valsson
and
M.
Parrinello
,
Phys. Rev. Lett.
113
(
9
),
090601
(
2014
).
33.
J.
Zhang
,
Y. I.
Yang
, and
F.
Noé
,
J. Phys. Chem. Lett.
10
(
19
),
5791
5797
(
2019
).
34.
D. M.
Blei
,
A.
Kucukelbir
, and
J. D.
McAuliffe
,
J. Am. Stat. Assoc.
112
(
518
),
859
877
(
2017
).
35.
D.
Wu
,
L.
Wang
, and
P.
Zhang
,
Phys. Rev. Lett.
122
(
8
),
080602
(
2019
).
36.
F.
Noé
,
S.
Olsson
,
J.
Köhler
, and
H.
Wu
,
Science
365
(
6457
),
eaaw1147
(
2019
).
37.
L.
Dinh
,
J.
Sohl-Dickstein
, and
S.
Bengio
, in
International Conference on Learning Representations
,
2017
.
38.
G.
Papamakarios
,
T.
Pavlakou
, and
I.
Murray
, in
Neural Information Processing Systems
(
NIPS
,
2017
), pp.
2338
2347
.
39.
T. Q.
Chen
,
Y.
Rubanova
,
J.
Bettencourt
, and
D.
Duvenaud
, in
Neural Information Processing Systems
(
NIPS
,
2018
), pp.
6572
6583
.
40.
W.
Grathwohl
,
R. T. Q.
Chen
,
J.
Bettencourt
,
I.
Sutskever
, and
D.
Duvenaud
, in
International Conference on Learning Representations
,
2019
.
41.
M. E.
Abbasnejad
,
Q.
Shi
,
A. v. d.
Hengel
, and
L.
Liu
, in
Computer Vision and Pattern Recognition
(
IEEE
,
2019
), pp.
10782
10791
.
42.
C.
Abrams
and
G.
Bussi
,
Entropy
16
(
1
),
163
199
(
2014
).
43.
Y.
Okamoto
,
J. Mol. Graphics Modell.
22
(
5
),
425
439
(
2004
).
44.
Y. I.
Yang
,
Q.
Shao
,
J.
Zhang
,
L.
Yang
, and
Y. Q.
Gao
,
J. Chem. Phys.
151
(
7
),
070902
(
2019
).
45.
A.
Laio
and
M.
Parrinello
,
Proc. Natl. Acad. Sci. U. S. A.
99
(
20
),
12562
12566
(
2002
).
46.
R. S.
Sutton
and
A. G.
Barto
,
Reinforcement Learning: An Introduction
(
MIT Press
,
2018
).
47.
I.
Grondman
,
L.
Busoniu
,
G. A. D.
Lopes
, and
R.
Babuska
,
IEEE Trans. Syst. Man, Cybernetics, Part C
42
(
6
),
1291
1307
(
2012
).
48.
A.
Barducci
,
G.
Bussi
, and
M.
Parrinello
,
Phys. Rev. Lett.
100
(
2
),
020603
(
2008
).
49.
M.
Heusel
,
H.
Ramsauer
,
T.
Unterthiner
,
B.
Nessler
, and
S.
Hochreiter
, presented at the
Advances in Neural Information Processing Systems
,
2017
.
50.
H.
Van Hasselt
,
A.
Guez
, and
D.
Silver
, presented at the
Thirtieth AAAI Conference on Artificial Intelligence
,
2016
.
51.
A. v. d.
Oord
,
Y.
Li
,
I.
Babuschkin
,
K.
Simonyan
,
O.
Vinyals
,
k.
kavukcuoglu
,
G. v. d.
Driessche
,
E.
Lockhart
,
L.
Cobo
,
F.
Stimberg
,
N.
Casagrande
,
D.
Grewe
,
S.
Noury
,
S.
Dieleman
,
E.
Elsen
,
N.
Kalchbrenner
,
H.
Zen
,
A.
Graves
,
H.
King
,
T.
Walters
,
D.
Belov
, and
D.
Hassabis
, in
International Conference on Machine Learning
(
ICML
,
2018
), pp.
3915
3923
.
52.
L.
Bonati
,
Y.-Y.
Zhang
, and
M.
Parrinello
,
Proc. Natl. Acad. Sci. U. S. A.
116
(
36
),
17641
17647
(
2019
).
53.
P.
Tiwary
and
B. J.
Berne
,
J. Chem. Phys.
147
(
15
),
152701
(
2017
).
54.
S.
Singer
and
J.
Nelder
,
Scholarpedia
4
(
7
),
2928
(
2009
).
55.
S.
Kmiecik
,
D.
Gront
,
M.
Kolinski
,
L.
Wieteska
,
A. E.
Dawid
, and
A.
Kolinski
,
Chem. Rev.
116
(
14
),
7898
7936
(
2016
).
56.
K.
Lindorff-Larsen
,
S.
Piana
,
R. O.
Dror
, and
D. E.
Shaw
,
Science
334
(
6055
),
517
,
520
(
2011
).
57.
S.
Piana
and
A.
Laio
,
J. Phys. Chem. B
111
(
17
),
4553
4559
(
2007
).
58.
C. M.
Davis
,
S.
Xiao
,
D. P.
Raleigh
, and
R. B.
Dyer
,
J. Am. Chem. Soc.
134
(
35
),
14476
14482
(
2012
).
59.
H.
Nguyen
,
J.
Maier
,
H.
Huang
,
V.
Perrone
, and
C.
Simmerling
,
J. Am. Chem. Soc.
136
(
40
),
13959
13962
(
2014
).
60.
P.
Shaffer
,
O.
Valsson
, and
M.
Parrinello
,
Proc. Natl. Acad. Sci. U. S. A.
113
(
5
),
1150
1155
(
2016
).
61.
J.
Zhang
,
Y.-K.
Lei
,
X.
Che
,
Z.
Zhang
,
Y. I.
Yang
, and
Y. Q.
Gao
,
J. Phys. Chem. Lett.
10
(
18
),
5571
5576
(
2019
).
62.
F.
Palazzesi
,
O.
Valsson
, and
M.
Parrinello
,
J. Phys. Chem. Lett.
8
(
19
),
4752
4756
(
2017
).
63.
H.
Wu
,
A.
Mardt
,
L.
Pasquali
, and
F.
Noe
, presented at the
Advances in Neural Information Processing Systems
,
2018
.
64.
H.
Sidky
,
W.
Chen
, and
A. L.
Ferguson
,
Chem. Sci.
11
(
35
),
9459
9467
(
2020
).
65.
J. M. L.
Ribeiro
,
P.
Bravo
,
Y.
Wang
, and
P.
Tiwary
,
J. Chem. Phys.
149
(
7
),
072301
(
2018
).
66.
L.
Zhang
,
H.
Wang
, and
W.
E
,
J. Chem. Phys.
148
(
12
),
124113
(
2018
).

Supplementary Material

You do not currently have access to this content.