Extending spatio-temporal scale limitations of models for complex atomistic systems considered in biochemistry and materials science necessitates the development of enhanced sampling methods. The potential acceleration in exploring the configurational space by enhanced sampling methods depends on the choice of collective variables (CVs). In this work, we formulate the discovery of CVs as a Bayesian inference problem and consider the CVs as hidden generators of the full-atomistic trajectory. The ability to generate samples of the fine-scale atomistic configurations using limited training data allows us to compute estimates of observables as well as our probabilistic confidence on them. The methodology is based on emerging methodological advances in machine learning and variational inference. The discovered CVs are related to physicochemical properties which are essential for understanding mechanisms especially in unexplored complex systems. We provide a quantitative assessment of the CVs in terms of their predictive ability for alanine dipeptide (ALA-2) and ALA-15 peptide.
REFERENCES
1.
J. R.
Perilla
, B. C.
Goh
, C. K.
Cassidy
, B.
Liu
, R. C.
Bernardi
, T.
Rudack
, H.
Yu
, Z.
Wu
, and K.
Schulten
, Curr. Opin. Struct. Biol.
31
, 64
(2015
).2.
P.
Koutsourelakis
, N.
Zabaras
, and M.
Girolami
, J. Comput. Phys.
321
, 1252
(2016
).3.
A.
Barducci
, M.
Bonomi
, and M.
Parrinello
, Wiley Interdiscip. Rev.: Comput. Mol. Sci.
1
, 826
(2011
).4.
F.
Pietrucci
and W.
Andreoni
, Phys. Rev. Lett.
107
, 085504
(2011
).5.
A. L.
Ferguson
, A. Z.
Panagiotopoulos
, P. G.
Debenedetti
, and I. G.
Kevrekidis
, J. Chem. Phys.
134
, 135103
(2011
).6.
W.
Zheng
, M. A.
Rohrdanz
, and C.
Clementi
, J. Phys. Chem. B
117
, 12769
(2013
).7.
O.
Valsson
and M.
Parrinello
, Phys. Rev. Lett.
113
, 090601
(2014
).8.
9.
P.-Y.
Chen
and M. E.
Tuckerman
, J. Chem. Phys.
148
, 024106
(2018
).10.
A.
Mitsutake
, Y.
Mori
, and Y.
Okamoto
, “Enhanced sampling algorithms
,” in Biomolecular Simulations: Methods and Protocols
, edited by L.
Monticelli
and E.
Salonen
(Humana Press
, Totowa, NJ
, 2013
), pp. 153
–195
.11.
C.
Bierig
and A.
Chernov
, J. Comput. Phys.
314
, 661
(2016
).12.
Physico-Chemical and Computational Approaches to Drug Discovery, RSC Drug Discovery
, edited by J.
Luque
and X.
Barril
(The Royal Society of Chemistry
, 2012
), pp. FP001–418
.13.
M. A.
Rohrdanz
, W.
Zheng
, and C.
Clementi
, Annu. Rev. Phys. Chem.
64
, 295
(2013
).14.
G.
Torrie
and J.
Valleau
, J. Comput. Phys.
23
, 187
(1977
).15.
A. F.
Voter
, J. Chem. Phys.
106
, 4665
(1997
).16.
D.
Hamelberg
, J.
Mongan
, and J. A.
McCammon
, J. Chem. Phys.
120
, 11919
(2004
).17.
T.
Huber
, A. E.
Torda
, and W. F.
van Gunsteren
, J. Comput.-Aided Mol. Des.
8
, 695
(1994
).18.
H.
Grubmüller
, Phys. Rev. E
52
, 2893
(1995
).19.
A.
Laio
and M.
Parrinello
, Proc. Natl. Acad. Sci. U. S. A.
99
, 12562
(2002
).20.
A.
Barducci
, G.
Bussi
, and M.
Parrinello
, Phys. Rev. Lett.
100
, 020603
(2008
).21.
E.
Darve
, D.
Rodríguez-Gómez
, and A.
Pohorille
, J. Chem. Phys.
128
, 144120
(2008
).22.
J.
Hénin
, G.
Fiorin
, C.
Chipot
, and M. L.
Klein
, J. Chem. Theory Comput.
6
, 35
(2010
).23.
F.
Pietrucci
, Rev. Phys.
2
, 32
(2017
).24.
A. C.
Pan
, T. M.
Weinreich
, Y.
Shan
, D. P.
Scarpazza
, and D. E.
Shaw
, J. Chem. Theory Comput.
10
, 2860
(2014
).25.
C. D.
Fu
, L. F. L.
Oliveira
, and J.
Pfaendtner
, J. Chem. Theory Comput.
13
, 968
(2017
).26.
H.
Hotelling
, J. Educ. Psychol.
24
, 498
(1933
).27.
R. T.
McGibbon
, B. E.
Husic
, and V. S.
Pande
, J. Chem. Phys.
146
, 044109
(2017
).28.
A.
Amadei
, A. B. M.
Linssen
, and H. J. C.
Berendsen
, Proteins: Struct., Funct., Bioinf.
17
, 412
(1993
).29.
K.
Pearson
, Philos. Mag.
2
, 559
(1901
).30.
J. M.
Troyer
and F. E.
Cohen
, Proteins: Struct., Funct., Bioinf.
23
, 97
(1995
).31.
W.
Härdle
and L.
Simar
, Applied Multivariate Statistical Analysis
(Springer Berlin Heidelberg
, 2007
).32.
M.
Ceriotti
, G. A.
Tribello
, and M.
Parrinello
, Proc. Natl. Acad. Sci. U. S. A.
108
, 13023
(2011
).33.
J. B.
Tenenbaum
, V. d.
Silva
, and J. C.
Langford
, Science
290
, 2319
(2000
).34.
M. A.
Rohrdanz
, W.
Zheng
, M.
Maggioni
, and C.
Clementi
, J. Chem. Phys.
134
, 124116
(2011
).35.
M.
Balasubramanian
and E. L.
Schwartz
, Science
295
, 7
(2002
).36.
D. L.
Donoho
and C.
Grimes
, Proc. Natl. Acad. Sci. U. S. A.
100
, 5591
(2003
).37.
H.
Risken
and T.
Frank
, The Fokker-Planck Equation: Methods of Solution and Applications
, Springer Series in Synergetics (Springer
, 1996
).38.
R. R.
Coifman
, S.
Lafon
, A. B.
Lee
, M.
Maggioni
, B.
Nadler
, F.
Warner
, and S. W.
Zucker
, Proc. Natl. Acad. Sci. U. S. A.
102
, 7426
(2005
).39.
R. R.
Coifman
, S.
Lafon
, A. B.
Lee
, M.
Maggioni
, B.
Nadler
, F.
Warner
, and S. W.
Zucker
, Proc. Natl. Acad. Sci. U. S. A.
102
, 7432
(2005
).40.
A. L.
Ferguson
, A. Z.
Panagiotopoulos
, I. G.
Kevrekidis
, and P. G.
Debenedetti
, Chem. Phys. Lett.
509
, 1
(2011
).41.
B.
Nadler
, S.
Lafon
, R. R.
Coifman
, and I. G.
Kevrekidis
, Appl. Comput. Harmonic Anal.
21
, 113
(2006
), special Issue: Diffusion Maps and Wavelets.42.
R. R.
Coifman
, I. G.
Kevrekidis
, S.
Lafon
, M.
Maggioni
, and B.
Nadler
, Multiscale Model. Simul.
7
, 842
(2008
).43.
M. A.
Rohrdanz
, W.
Zheng
, B.
Lambeth
, J.
Vreede
, and C.
Clementi
, PLoS Comput. Biol.
10
, e1003797
(2014
).44.
W.
Zheng
, A. V.
Vargiu
, M. A.
Rohrdanz
, P.
Carloni
, and C.
Clementi
, J. Chem. Phys.
139
, 145102
(2013
).45.
F.
Noé
and F.
Nüske
, Multiscale Model. Simul.
11
, 635
(2013
).46.
J.
McCarty
and M.
Parrinello
, J. Chem. Phys.
147
, 204109
(2017
).47.
F.
Noé
and C.
Clementi
, J. Chem. Theory Comput.
11
, 5002
(2015
).48.
F.
Noé
, R.
Banisch
, and C.
Clementi
, J. Chem. Theory Comput.
12
, 5620
(2016
).49.
M.
Duan
, J.
Fan
, M.
Li
, L.
Han
, and S.
Huo
, J. Chem. Theory Comput.
9
, 2490
(2013
).50.
Learning in Graphical Models
, edited by M. I.
Jordan
(MIT Press
, Cambridge, MA, USA
, 1999
).51.
M.
Schöberl
, N.
Zabaras
, and P.-S.
Koutsourelakis
, J. Comput. Phys.
333
, 49
(2017
).52.
L.
Felsberger
and P.
Koutsourelakis
, “Communications in computational physics
” (to be published); e-print arXiv:1802.03824.53.
54.
D. J.
Rezende
, S.
Mohamed
, and D.
Wierstra
, in Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21–26 June 2014
(PMLR
, 2014
), pp. 1278
–1286
, http://proceedings.mlr.press/v32/rezende14.html.55.
S.
Kmiecik
, D.
Gront
, M.
Kolinski
, L.
Wieteska
, A. E.
Dawid
, and A.
Kolinski
, Chem. Rev.
116
, 7898
(2016
).56.
W. G.
Noid
, J.-W.
Chu
, G. S.
Ayton
, and G. A.
Voth
, J. Phys. Chem. B
111
, 4116
(2007
).57.
M. S.
Shell
, J. Chem. Phys.
129
, 144108
(2008
).58.
C.
Peter
and K.
Kremer
, Soft Matter
5
, 4357
(2009
).59.
J.
Trashorras
and D.
Tsagkarogiannis
, SIAM J. Numer. Anal.
48
, 1647
(2010
).60.
E.
Kalligiannaki
, M. A.
Katsoulakis
, P.
Plecháč
, and D. G.
Vlachos
, J. Comput. Phys.
231
, 2599
(2012
).61.
V.
Harmandaris
, E.
Kalligiannaki
, M.
Katsoulakis
, and P.
Plecháč
, J. Comput. Phys.
314
, 355
(2016
).62.
I.
Bilionis
and N.
Zabaras
, J. Chem. Phys.
138
, 044313
(2013
).63.
J. F.
Dama
, A. V.
Sinitskiy
, M.
McCullagh
, J.
Weare
, B.
Roux
, A. R.
Dinner
, and G. A.
Voth
, J. Chem. Theory Comput.
9
, 2466
(2013
).64.
W. G.
Noid
, J. Chem. Phys.
139
, 090901
(2013
).65.
T. T.
Foley
, M. S.
Shell
, and W. G.
Noid
, J. Chem. Phys.
143
, 243104
(2015
).66.
M.
Langenberg
, N. E.
Jackson
, J. J.
de Pablo
, and M.
Müller
, J. Chem. Phys.
148
, 094112
(2018
).67.
C. X.
Hernández
, H. K.
Wayment-Steele
, M. M.
Sultan
, B. E.
Husic
, and V. S.
Pande
, J. Chem. Theory Comput.
14
, 1887
(2017
).68.
C.
Wehmeyer
and F.
Noé
, J. Chem. Phys.
148
, 241703
(2018
).69.
M. M.
Sultan
, H. K.
Wayment-Steele
, and V. S.
Pande
, J. Chem. Theory Comput.
14
, 1887
(2018
).70.
M. J.
Beal
, “Variational algorithms for approximate Bayesian inference
,” Ph.D. thesis, Gatsby Computational Neuroscience Unit, University College London
, 2003
.71.
B. J.
Alder
and T. E.
Wainwright
, J. Chem. Phys.
31
, 459
(1959
).72.
D.
Landau
and K.
Binder
, A Guide to Monte Carlo Simulations in Statistical Physics
(Cambridge University Press
, New York, NY, USA
, 2005
).73.
Y.
LeCun
, Y.
Bengio
, and G.
Hinton
, Nature
521
, 436
(2015
).74.
Z.
Ghahramani
, Nature
521
, 452
(2015
).75.
W.
von der Linden
, V.
Dose
, and U.
von Toussaint
, Bayesian Probability Theory: Applications in the Physical Sciences
(Cambridge University Press
, 2014
), p. 649
.76.
A. Y.
Ng
and M. I.
Jordan
, in Advances in Neural Information Processing Systems 14
, edited by T. G.
Dietterich
, S.
Becker
, and Z.
Ghahramani
(MIT Press
, 2002
), pp. 841
–848
.77.
D. J. C.
MacKay
, Information Theory, Inference, and Learning Algorithms
(Cambridge University Press
, 2003
).78.
79.
A.
Cichocki
and S.-i.
Amari
, Entropy
12
, 1532
(2010
).80.
S.-H.
Cha
, Int. J. Math. Mod. Meth. Appl. Sci.
1
, 300
(2007
), http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.154.8446.81.
Inference on the generalized α-divergence is addressed in Ref. 137.
82.
D. J. C.
MacKay
, Neural Comput.
4
, 448
(1992
).83.
A.
Gelman
, J. B.
Carlin
, H. S.
Stern
, D. B.
Dunson
, A.
Vehtari
, and D. B.
Rubin
, Bayesian Data Analysis, 3rd Edition, Chapman and Hall/CRC 2013, ISBN 9781439840955.84.
E. T.
Jaynes
, Math. Intell.
27
, 83
(2005
).85.
M. D.
Hoffman
, D. M.
Blei
, C.
Wang
, and J.
Paisley
, J. Mach. Learn. Res.
14
, 1303
(2013
).86.
R.
Ranganath
, S.
Gerrish
, and D.
Blei
, in Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics
[PMLR 33, 814–822 (2014
)], http://proceedings.mlr.press/v33/ranganath14.html.87.
J. W.
Paisley
, D. M.
Blei
, and M. I.
Jordan
, in International Conference on Machine Learning
(Omnipress
, 2012
).88.
A. P.
Dempster
, N. M.
Laird
, and D. B.
Rubin
, J. R. Stat. Soc., Ser. B: Methodol.
39
, 1
(1977
).89.
R. M.
Neal
and G. E.
Hinton
, “A view of the EM algorithm that justifies incremental, sparse, and other variants
,” in Learning in Graphical Models
(MIT Press
, Cambridge, MA, USA
, 1999
), pp. 355
–368
.90.
T. M.
Ruiz
, J. R.
Francisco
, and D.
Blei
, Advances in Neural Information Processing Systems
(Curran Associates, Inc.
, 2016
), pp. 460
–468
.91.
P.
Zhao
and T.
Zhang
, “Accelerating minibatch stochastic gradient descent using stratified sampling
,” e-print arXiv:1405.3080 (2014
).92.
L.
Bottou
, F.
Curtis
, and J.
Nocedal
, SIAM Rev.
60
, 223
(2018
).93.
D. P.
Kingma
and J.
Ba
, “Adam: A method for stochastic optimization
,” e-print arXiv:1412.6980 (2014
).94.
P.-A.
Mattei
and J.
Frellsen
, Adv. Neural Info. Proc. Sys.
31
, 3859
(2018
), https://papers.nips.cc/paper/7642-leveraging-the-exact-likelihood-of-deep-latent-variable-models.95.
N.
Metropolis
, A. W.
Rosenbluth
, M. N.
Rosenbluth
, A. H.
Teller
, and E.
Teller
, J. Chem. Phys.
21
, 1087
(1953
).96.
W. K.
Hastings
, Biometrika
57
, 97
(1970
).97.
L. L.
Cam
, Int. Stat. Rev.
58
, 153
(1990
).98.
99.
M. A.
Figueiredo
and S.
Member
, IEEE Trans. Pattern Anal. Mach. Intell.
25
, 1150
(2003
).100.
D. J. C.
MacKay
and R. M.
Neal
, “Automatic relevance determination for neural networks
,” Technical Report, University of Cambridge
, 1994
(unpublished).101.
102.
H.
Ritter
, A.
Botev
, and D.
Barber
, in International Conference on Learning Representations
(ICLR
, 2018
), https://iclr.cc/Conferences/2018/Schedule?showEvent=224.103.
P. E.
Smith
, J. Chem. Phys.
111
, 5568
(1999
).104.
J.
Hermans
, Proc. Natl. Acad. Sci. U. S. A.
108
, 3095
(2011
).105.
G.
Ramachandran
, C.
Ramakrishnan
, and V.
Sasisekharan
, J. Mol. Biol.
7
, 95
(1963
).106.
R.
Vargas
, J.
Garza
, B. P.
Hay
, and D. A.
Dixon
, J. Phys. Chem. A
106
, 3213
(2002
).107.
S. P.
Carmichael
and M. S.
Shell
, J. Phys. Chem. B
116
, 8383
(2012
).108.
E. J.
Sorin
and V. S.
Pande
, Biophys. J.
88
, 2472
(2005
).109.
A. J.
DePaul
, E. J.
Thompson
, S. S.
Patel
, K.
Haldeman
, and E. J.
Sorin
, Nucleic Acids Res.
38
, 4856
(2010
).110.
M. P.
Allen
and D. J.
Tildesley
, Computer Simulation of Liquids
(Clarendon Press
, New York, NY, USA
, 1989
).111.
A.
Onufriev
, D.
Bashford
, and D. A.
Case
, Proteins: Struct., Funct., Bioinf.
55
, 383
(2004
).112.
W. C.
Still
, A.
Tempczyk
, R. C.
Hawley
, and T.
Hendrickson
, J. Am. Chem. Soc.
112
, 6127
(1990
).113.
H.
Berendsen
, D.
van der Spoel
, and R.
van Drunen
, Comput. Phys. Commun.
91
, 43
(1995
).114.
E.
Lindahl
, B.
Hess
, and D.
van der Spoel
, Mol. Model. Annu.
7
, 306
(2001
).115.
D. V. D.
Spoel
, E.
Lindahl
, B.
Hess
, G.
Groenhof
, A. E.
Mark
, and H. J. C.
Berendsen
, J. Comput. Chem.
26
, 1701
(2005
).116.
B.
Hess
, C.
Kutzner
, D.
van der Spoel
, and E.
Lindahl
, J. Chem. Theory Comput.
4
, 435
(2008
).117.
S.
Pronk
, S.
Páll
, R.
Schulz
, P.
Larsson
, P.
Bjelkmar
, R.
Apostolov
, M. R.
Shirts
, J. C.
Smith
, P. M.
Kasson
, D.
van der Spoel
, B.
Hess
, and E.
Lindahl
, Bioinformatics
29
, 845
(2013
).118.
S.
Páll
, M. J.
Abraham
, C.
Kutzner
, B.
Hess
, and E.
Lindahl
, in Solving Software Challenges for Exascale
, edited by S.
Markidis
and E.
Laure
(Springer International Publishing
, Cham
, 2015
), pp. 3
–27
.119.
M. J.
Abraham
, T.
Murtola
, R.
Schulz
, S.
Páll
, J. C.
Smith
, B.
Hess
, and E.
Lindahl
, SoftwareX
1-2
, 19
(2015
).120.
D. E.
Rumelhart
, G. E.
Hinton
, and R. J.
Williams
, Learning Internal Representations by Error Propagation
(MIT Press
, Cambridge, MA, USA
, 1986
), pp. 318
–362
.121.
C.
Van Der Malsburg
, in Brain Theory
, edited by G.
Palm
and A.
Aertsen
(Springer Berlin Heidelberg
, Berlin, Heidelberg
, 1986
), pp. 245
–248
.122.
S.
Haykin
, Neural Networks: A Comprehensive Foundation
, 2nd ed. (Prentice Hall PTR
, Upper Saddle River, NJ, USA
, 1998
).123.
D. J.
Rezende
and S.
Mohamed
, in Proceedings of the 32nd International Conference on Machine Learning
[PMLR 37, 1530–1538 (2015
)], http://proceedings.mlr.press/v37/rezende15.html.124.
D. P.
Kingma
, T.
Salimans
, R.
Jozefowicz
, X.
Chen
, I.
Sutskever
, and M.
Welling
, Adv. Neural Info. Proc. Syst.
29
, 4743
(2016
), http://papers.nips.cc/paper/6581-improved-variational-inference-with-inverse-autoregressive-flow.125.
J. C.
Pinheiro
and D. M.
Bates
, Stat. Comput.
6
, 289
(1996
).126.
G.
Klambauer
, T.
Unterthiner
, A.
Mayr
, and S.
Hochreiter
, in Advances in Neural Information Processing Systems 30
, edited by I.
Guyon
, U. V.
Luxburg
, S.
Bengio
, H.
Wallach
, R.
Fergus
, S.
Vishwanathan
, and R.
Garnett
(Curran Associates, Inc.
, 2017
), pp. 971
–980
.127.
P.
Ramachandran
, B.
Zoph
, and Q. V.
Le
“Searching for activation functions
” (2017
), arXiv:1710.05941.128.
W.
Humphrey
, A.
Dalke
, and K.
Schulten
, J. Mol. Graphics
14
, 33
(1996
).129.
A. M.
Fluitt
and J. J.
de Pablo
, Biophys. J.
109
, 1009
(2015
).130.
Y.
Sugita
and Y.
Okamoto
, Chem. Phys. Lett.
314
, 141
(1999
).131.
A.
Marini
and R. Y.
Dong
, Phys. Rev. E
83
, 041712
(2011
).132.
Y.
Zhou
, A.
Kloczkowski
, E.
Faraggi
, and Y.
Yang
, “Prediction of protein secondary structure
,” in Methods in Molecular Biology
(Springer
, New York
, 2016
).133.
I. J.
Goodfellow
, J.
Pouget-Abadie
, M.
Mirza
, B.
Xu
, D.
Warde-Farley
, S.
Ozair
, A.
Courville
, and Y.
Bengio
, Adv. Neural Info. Proc. Syst.
27
, 2672
(2014
), https://papers.nips.cc/paper/5423-generative-adversarial-nets.134.
Y.
Saatchi
and A. G.
Wilson
, Adv. Neural Info. Proc. Syst.
30
, 3622
(2017
), https://papers.nips.cc/paper/6953-bayesian-gan.135.
T.
Salimans
, I.
Goodfellow
, W.
Zaremba
, V.
Cheung
, A.
Radford
, X.
Chen
, and X.
Chen
, in Advances in Neural Information Processing Systems 29
, edited by D. D.
Lee
, M.
Sugiyama
, U. V.
Luxburg
, I.
Guyon
, and R.
Garnett
(Curran Associates, Inc.
, 2016
), pp. 2234
–2242
.136.
R.
Galvelis
and Y.
Sugita
, J. Chem. Theory Comput.
13
, 2489
(2017
).137.
J.
Hernandez-Lobato
, Y.
Li
, M.
Rowland
, T.
Bui
, D.
Hernandez-Lobato
, and R.
Turner
, in Proceedings of the 33rd International Conference on Machine Learning
, PMLR, Vol. 48, edited by M. F.
Balcan
and K. Q.
Weinberger
(PMLR
, New York, New York, USA
, 2016
), pp. 1511
–1520
, http://proceedings.mlr.press/v48/hernandez-lobatob16.html.© 2019 Author(s).
2019
Author(s)
You do not currently have access to this content.
