Markov state models of molecular kinetics (MSMs), in which the long-time statistical dynamics of a molecule is approximated by a Markov chain on a discrete partition of configuration space, have seen widespread use in recent years. This approach has many appealing characteristics compared to straightforward molecular dynamics simulation and analysis, including the potential to mitigate the sampling problem by extracting long-time kinetic information from short trajectories and the ability to straightforwardly calculate expectation values and statistical uncertainties of various stationary and dynamical molecular observables. In this paper, we summarize the current state of the art in generation and validation of MSMs and give some important new results. We describe an upper bound for the approximation error made by modeling molecular dynamics with a MSM and we show that this error can be made arbitrarily small with surprisingly little effort. In contrast to previous practice, it becomes clear that the best MSM is not obtained by the most metastable discretization, but the MSM can be much improved if non-metastable states are introduced near the transition states. Moreover, we show that it is not necessary to resolve all slow processes by the state space partitioning, but individual dynamical processes of interest can be resolved separately. We also present an efficient estimator for reversible transition matrices and a robust test to validate that a MSM reproduces the kinetics of the molecular dynamics data.

1.
M.
Jäger
,
Y.
Zhang
,
J.
Bieschke
,
H.
Nguyen
,
M.
Dendle
,
M. E.
Bowman
,
J. P.
Noel
,
M.
Gruebele
, and
J. W.
Kelly
,
Proc. Natl. Acad. Sci. USA
103
,
10648
(
2006
).
2.
A. Y.
Kobitski
,
A.
Nierth
,
M.
Helm
,
A.
Jäschke
, and
G. U.
Nienhaus
,
Nucleic Acids Res.
35
,
2047
(
2007
).
3.
S.
Fischer
,
B.
Windshuegel
,
D.
Horak
,
K. C.
Holmes
, and
J. C.
Smith
,
Proc. Natl. Acad. Sci. U.S.A.
102
,
6873
(
2005
).
4.
F.
Noé
,
D.
Krachtus
,
J. C.
Smith
, and
S.
Fischer
,
J. Chem. Theo. Comp.
2
,
840
(
2006
).
5.
A.
Ostermann
,
R.
Waschipky
,
F. G.
Parak
, and
U. G.
Nienhaus
,
Nature (London)
404
,
205
(
2000
).
6.
H.
Frauenfelder
,
S. G.
Sligar
, and
P. G.
Wolynes
,
Science
254
,
1598
(
1991
).
7.
A.
Gansen
,
A.
Valeri
,
F.
Hauger
,
S.
Felekyan
,
S.
Kalinin
,
K.
Tóth
,
J.
Langowski
, and
C. A. M.
Seidel
,
Proc. Natl. Acad. Sci. U.S.A.
106
,
15308
(
2009
).
8.
H.
Neubauer
,
N.
Gaiko
,
S.
Berger
,
J.
Schaffer
,
C.
Eggeling
,
J.
Tuma
,
L.
Verdier
,
C. A.
Seidel
,
C.
Griesinger
, and
A.
Volkmer
,
J. Am. Chem. Soc.
129
,
12746
(
2007
).
9.
W.
Min
,
G.
Luo
,
B. J.
Cherayil
,
S. C.
Kou
, and
X. S.
Xie
,
Phys. Rev. Lett.
94
,
198302
(
2005
).
10.
E. Z.
Eisenmesser
,
O.
Millet
,
W.
Labeikovsky
,
D. M.
Korzhnev
,
M.
Wolf-Watz
,
D. A.
Bosco
,
J. J.
Skalicky
,
L. E.
Kay
, and
D.
Kern
,
Nature (London)
438
,
117
(
2005
).
11.
Y.
Santoso
,
C. M.
Joyce
,
O.
Potapova
,
L.
Le Reste
,
J.
Hohlbein
,
J. P.
Torella
,
N. D. F.
Grindley
, and
A. N.
Kapanidis
,
Proc. Natl. Acad. Sci. U.S.A.
107
,
715
(
2010
).
12.
Gebhardt
,
T.
Bornschlögl
, and
M.
Rief
,
Proc. Natl. Acad. Sci. U.S.A.
107
,
2013
(
2010
).
13.
B. G.
Wensley
,
S.
Batey
,
F. A.C.
Bone
,
Z. M.
Chan
,
N. R.
Tumelty
,
A.
Steward
,
L. G.
Kwa
,
A.
Borgia
, and
J.
Clarke
,
Nature (London)
463
,
685
(
2010
).
14.
B. P.
English
,
W.
Min
,
A. M.
van Oijen
,
K. T.
Lee
,
G.
Luo
,
H.
Sun
,
B. J.
Cherayil
,
S. C.
Kou
, and
X. S.
Xie
,
Nat. Chem. Bio.
2
,
87
(
2006
).
15.
M. O.
Lindberg
and
M.
Oliveberg
,
Curr. Opin. Struct. Biol.
17
,
21
(
2007
).
16.
K.
Sridevi
,
J. Mol. Biol.
302
,
479
(
2000
).
17.
R. A.
Goldbeck
,
Y. G.
Thomas
,
E.
Chen
,
R. M.
Esquerra
, and
D. S.
Kliger
,
Proc. Natl. Acad. Sci. U.S.A.
96
,
2782
(
1999
).
18.
A.
Matagne
,
S. E.
Radford
, and
C. M.
Dobson
,
J. Mol. Biol.
267
,
1068
(
1997
).
19.
C. C.
Mello
and
D.
Barrick
,
Proc. Natl. Acad. Sci. U.S.A.
101
,
14102
(
2004
).
20.
S. A.
Waldauer
,
O.
Bakajin
,
T.
Ball
,
Y.
Chen
,
S. J.
DeCamp
,
M.
Kopka
,
M.
Jäger
,
V. R.
Singh
,
W. J.
Wedemeyer
,
S.
Weiss
,
S.
Yao
, and
L. J.
Lapidus
,
HFSP J.
2
,
388
(
2006
).
21.
D. D.
Schaeffer
,
A.
Fersht
, and
V.
Daggett
,
Curr. Opin. Struct. Biol.
18
,
4
(
2008
).
22.
F.
Noé
,
C.
Schütte
,
E.
Vanden-Eijnden
,
L.
Reich
, and
T. R.
Weikl
,
Proc. Natl. Acad. Sci. U.S.A.
106
,
19011
(
2009
).
23.
W.
van Gunsteren
,
J.
Dolenc
, and
A.
Mark
,
Curr. Opin. Struct. Biol.
18
,
149
(
2008
).
24.
S. V.
Krivov
and
M.
Karplus
,
Proc. Nat. Acad. Sci. U.S.A.
101
,
14766
(
2004
).
25.
F.
Noé
and
S.
Fischer
,
Curr. Opin. Struc. Biol.
18
,
154
(
2008
).
26.
S.
Muff
and
A.
Caflisch
,
Proteins
70
,
1185
(
2007
).
27.
D. J.
Wales
,
Energy Landscapes
(
Cambridge University Press
,
Cambridge
,
2003
).
28.
M. E.
Karpen
,
D. J.
Tobias
, and
C. L.
Brooks
,
Biochemistry
32
,
412
(
1993
).
29.
I. A.
Hubner
,
E. J.
Deeds
, and
E. I.
Shakhnovich
,
Proc. Natl. Acad. Sci. U.S.A.
103
,
17747
(
2006
).
30.
M.
Weber
, ZIB Report 03-04 (
2003
).
31.
N. V.
Buchete
and
G.
Hummer
,
J. Phys. Chem. B
112
,
6057
(
2008
).
32.
F.
Rao
and
A.
Caflisch
,
J. Mol. Bio.
342
,
299
(
2004
).
33.
B.
de Groot
,
X.
Daura
,
A.
Mark
, and
H.
Grubmüller
,
J. Mol. Bio.
301
,
299
(
2001
).
34.
V.
Schultheis
,
T.
Hirschberger
,
H.
Carstens
, and
P.
Tavan
,
J. Chem. Theory Comp.
1
,
515
(
2005
).
35.
A. C.
Pan
and
B.
Roux
,
J. Chem. Phys.
129
,
064107
(
2008
).
36.
M.
Sarich
,
F.
Noé
, and
C.
Schütte
,
SIAM Multiscale Model. Simul.
8
,
1154
(
2010
).
37.
F.
Noé
,
M.
Oswald
,
G.
Reinelt
,
S.
Fischer
, and
J. C.
Smith
,
Multiscale Model. Simul.
5
,
393
(
2006
).
38.
F.
Noé
,
I.
Horenko
,
C.
Schütte
, and
J. C.
Smith
,
J. Chem. Phys.
126
,
155102
(
2007
).
39.
J. D.
Chodera
,
K. A.
Dill
,
N.
Singhal
,
V. S.
Pande
,
W. C.
Swope
, and
J. W.
Pitera
,
J. Chem. Phys.
126
,
155101
(
2007
).
40.
W. C.
Swope
,
J. W.
Pitera
, and
F.
Suits
,
J. Phys. Chem. B
108
,
6571
(
2004
).
41.
N.
Singhal
,
C.
Snow
, and
V. S.
Pande
,
J. Chem. Phys.
121
,
415
(
2004
).
42.
C.
Schütte
,
A.
Fischer
,
W.
Huisinga
, and
P.
Deuflhard
,
J. Comput. Phys.
151
,
146
(
1999
).
43.
M.
Weber
, ZIB Report 09-27-rev (
2009
).
44.
M.
Jäger
,
H.
Nguyen
,
J. C.
Crane
,
J. W.
Kelly
, and
M.
Gruebele
,
J. Mol. Biol.
311
,
373
(
2001
).
45.
J. D.
Chodera
and
F.
Noé
,
J. Chem. Phys.
133
,
105102
(
2010
).
46.
V. A.
Voelz
,
G. R.
Bowman
,
K.
Beauchamp
, and
V. S.
Pande
,
J. Am. Chem. Soc.
132
,
1526
(
2010
).
47.
N. S.
Hinrichs
and
V. S.
Pande
,
J. Chem. Phys.
126
,
244101
(
2007
).
48.
N.
Singhal
and
V. S.
Pande
,
J. Chem. Phys.
123
,
204909
(
2005
).
49.
G. R.
Bowman
,
K. A.
Beauchamp
,
G.
Boxer
, and
V. S.
Pande
,
J. Chem. Phys.
131
,
124101
(
2009
).
50.
P.
Metzner
,
C.
Schütte
, and
E. V.
Eijnden
,
Multiscale Model. Simul.
7
,
1192
(
2009
).
51.
F.
Noé
,
J. Chem. Phys.
128
,
244103
(
2008
).
52.
J. D.
Chodera
,
W. C.
Swope
,
J. W.
Pitera
, and
K. A.
Dill
,
Multiscale Model. Simul.
5
,
1214
(
2006
).
53.
S.
Bacallado
,
J. D.
Chodera
, and
V.
Pande
,
J. Chem. Phys.
131
,
045106
(
2009
).
54.
N. G.
van Kampen
,
Stochastic Processes in Physics and Chemistry
, 4th ed. (
Elsevier
,
Amsterdam
,
2006
).
55.
S.
Park
and
V. S.
Pande
,
J. Chem. Phys.
124
,
054118
(
2006
).
56.
J. D.
Chodera
,
W. C.
Swope
,
F.
Noé
,
J.-H.
Prinz
,
M. R.
Shirts
, and
V. S.
Pande
, “Dynamical reweighting: Improved estimates of dynamical properties from simulations at multiple temperatures,”
J. Phys. Chem.
(in press).
57.
H. C.
Andersen
,
J. Chem. Phys.
72
,
2384
(
1980
).
58.
S.
Duane
,
Phys. Lett. B
195
,
216
(
1987
).
59.
D. L.
Ermak
and
Y.
Yeh
,
Chem. Phys. Lett.
24
,
243
(
1974
).
60.
D. L.
Ermak
,
J. Chem. Phys.
62
,
4189
(
1975
).
61.
W. C.
Swope
,
H. C.
Andersen
,
P. H.
Berens
, and
K. R.
Wilson
,
J. Chem. Phys.
76
,
637
(
1982
).
62.
B.
Cooke
and
S. C.
Schmidler
,
J. Chem. Phys.
129
,
164112
(
2008
).
63.
C.
Schütte
,
F.
Noé
,
E.
Meerbach
,
P.
Metzner
, and
C.
Hartmann
, in
Proceedings of the International Congress on Industrial and Applied Mathematics (ICIAM)
, edited by
R.
Jeltsch
and
G.
Wanner
(
EMS Publishing House
,
Zurich
,
2009
), pp.
297
336
.
64.
S.
Sriraman
,
I. G.
Kevrekidis
, and
G.
Hummer
,
J. Phys. Chem. B
109
,
6479
(
2005
).
65.
See supplementary material at http://dx.doi.org/10.1063/1.3565032 for a practical approach to Markov model analysis, the model systems setup, and details about rate matrices and transition matrix estimations.
67.
M.
Weber
, “
Meshless methods in conformation dynamics
,” Ph.D. thesis (
Free University Berlin
,
2006
).
68.
M. G.
Voronoi
,
J. Reine Angew. Math.
134
,
198
(
1908
).
69.
S.
Kube
and
M.
Weber
,
J. Chem. Phys.
126
,
024103
(
2007
).
70.
P.
Metzner
,
I.
Horenko
, and
C.
Schütte
,
Phys. Rev. E
76
,
066702
(
2007
).
71.
D.
Crommelin
and
E. V.
Eijnden
,
Multiscale Model. Simul.
7
,
1751
(
2009
).
72.
B.
Keller
,
P.
Hünenberger
, and
W.
van Gunsteren
, “An Analysis of the Validity of Markov State Models for Emulating the Dynamics of Classical Molecular Systems and Ensembles,”
J. Chem. Theo. Comput.
(submitted).
73.
E.
Meerbach
,
C.
Schütte
,
I.
Horenko
, and
B.
Schmidt
, “
Metastable conformational structure and dynamics: Peptides between gas phase and aqueous solution
”, in
Analysis and Control of Ultrafast Photoinduced Reactions
,
Series in Chemical Physics
, Vol.
87
(
Springer
,
Berlin
,
2007
), pp.
796
806
.
74.
W. C.
Swope
,
J. W.
Pitera
,
F.
Suits
,
M.
Pitman
, and
M.
Eleftheriou
,
J. Phys. Chem. B
108
,
6582
(
2004
).
75.
G. H.
Golub
and
C. F.
van Loan
,
Matrix Computations
, 3rd ed. (
Johns Hopkins University Press
,
Baltimore, MD
,
1996
).
76.
C.-K.
Chan
,
Y.
Hu
,
S.
Takahashi
,
D. L.
Rousseau
,
W. A.
Eaton
, and
J.
Hofrichter
,
Proc. Natl. Acad. Sci. U.S.A.
94
,
1779
(
1997
).
77.
O.
Bieri
,
J.
Wirz
,
B.
Hellrung
,
M.
Schutkowski
,
M.
Drewello
, and
T.
Kiefhaber
,
Proc. Natl. Acad. Sci. U.S.A.
96
,
9597
(
1999
).
78.
H.
Neuweiler
,
S.
Doose
, and
M.
Sauer
,
Proc. Natl. Acad. Sci. U.S.A.
102
,
16650
(
2005
).
79.
N.
Djurdjevac
,
M.
Sarich
, and
C.
Schütte
, “Estimating the eigenvalue error of Markov state models,”
Multiscale Model. Simul.
(submitted).
80.
A.
Amadei
,
A. B.
Linssen
, and
H. J.C.
Berendsen
,
Proteins
17
,
412
(
1993
).
81.
B.
Keller
,
X.
Daura
, and
W. F.
van Gunsteren
,
J. Chem. Phys.
132
(
2010
).
82.
Y.
Yao
,
J.
Sun
,
X.
Huang
,
G. R.
Bowman
,
G.
Singh
,
M.
Lesnick
,
L. J.
Guibas
,
V. S.
Pande
, and
G.
Carlsson
,
J. Chem. Phys.
130
,
144115
(
2009
).
83.
S.
Dasgupta
and
P.
Long
,
J. Comput. Syst. Sci.
70
,
555
(
2005
).
84.
A.
Laio
and
M.
Parrinello
,
Proc Natl. Acad. Sci. U.S.A.
99
,
12562
(
2002
).
85.
Y.
Sugita
and
Y.
Okamoto
,
Chem. Phys. Lett.
314
,
141
(
1999
).
86.
S.
Röblitz
, “
Statistical error estimation and grid-free hierarchical refinement in conformation dynamics
, Ph.D. thesis (
Free University Berlin
,
2009
).
87.
T. W.
Anderson
and
L. A.
Goodman
,
Ann. Math. Statist.
28
,
89
(
1957
).
88.
J.-H.
Prinz
,
M.
Held
,
J. C.
Smith
, and
F.
Noé
, “Efficient computation, sensitivity and error analysis of committor probabilites for complex dynamical processes,”
SIAM Multiscale Model. Simul.
(submitted).
89.
F.
Noé
,
M.
Oswald
, and
G.
Reinelt
, in
Operations Research Proceedings
, edited by
J.
Kalcsics
and
S.
Nickel
(
Springer
,
New York
,
2007
), pp.
435
440
.
90.
P.
Metzner
,
F.
Noé
, and
C.
Schütte
,
Phys. Rev. E
80
,
021106
(
2009
).
91.
O. F.
Lange
and
H.
Grubmüller
,
J. Chem. Phys.
124
,
214903
(
2006
).
92.
H.
Grubmüller
,
Phys. Rev. E
52
,
2893
(
1995
).
93.
G. M.
Torrie
and
J. P.
Valleau
,
J. Comp. Phys.
23
,
187
(
1977
).
94.
J.
Schlitter
,
M.
Engels
, and
P.
Krüger
,
J. Mol. Graphics
12
,
84
(
1994
).
95.
G. R.
Bowman
,
D. L.
Ensign
, and
V. S.
Pande
,
J. Chem. Theory Comput.
6
,
787
(
2010
).
96.
D. J.
Wales
,
Science
271
,
925
(
1996
).
97.
R.
Hegger
and
G.
Stock
,
J. Chem. Phys.
130
,
034106
(
2009
).
98.
C.
Micheletti
,
G.
Bussi
, and
A.
Laio
,
J. Chem. Phys.
129
,
074105
(
2008
).

Supplementary Material

You do not currently have access to this content.