Markov state models (MSMs) have been successful in computing metastable states, slow relaxation timescales and associated structural changes, and stationary or kinetic experimental observables of complex molecules from large amounts of molecular dynamics simulation data. However, MSMs approximate the true dynamics by assuming a Markov chain on a clusters discretization of the state space. This approximation is difficult to make for high-dimensional biomolecular systems, and the quality and reproducibility of MSMs has, therefore, been limited. Here, we discard the assumption that dynamics are Markovian on the discrete clusters. Instead, we only assume that the full phase-space molecular dynamics is Markovian, and a projection of this full dynamics is observed on the discrete states, leading to the concept of Projected Markov Models (PMMs). Robust estimation methods for PMMs are not yet available, but we derive a practically feasible approximation via Hidden Markov Models (HMMs). It is shown how various molecular observables of interest that are often computed from MSMs can be computed from HMMs/PMMs. The new framework is applicable to both, simulation and single-molecule experimental data. We demonstrate its versatility by applications to educative model systems, a 1 ms Anton MD simulation of the bovine pancreatic trypsin inhibitor protein, and an optical tweezer force probe trajectory of an RNA hairpin.

1.
K. A.
Beauchamp
,
R.
McGibbon
,
Y.-S.
Lin
, and
V. S.
Pande
, “
Simple few-state models reveal hidden complexity in protein folding
,”
Proc. Natl. Acad. Sci. U.S.A.
109
,
17807
17813
(
2012
).
2.
G. R.
Bowman
, “
Improved coarse-graining of Markov state models via explicit consideration of statistical uncertainty
,”
J. Chem. Phys.
137
,
134111
(
2012
).
3.
G. R.
Bowman
,
D. L.
Ensign
, and
V. S.
Pande
, “
Enhanced modeling via network theory: Adaptive sampling of Markov state models
,”
J. Chem. Theory Comput.
6
(
3
),
787
794
(
2010
).
4.
G. R.
Bowman
and
P. L.
Geissler
, “
Equilibrium fluctuations of a single folded protein reveal a multitude of potential cryptic allosteric sites
,”
Proc. Natl. Acad. Sci. U.S.A.
109
,
11681
11686
(
2012
).
5.
N. V.
Buchete
and
G.
Hummer
, “
Coarse master equations for peptide folding dynamics
,”
J. Phys. Chem. B
112
,
6057
6069
(
2008
).
6.
J. D.
Chodera
,
K. A.
Dill
,
N.
Singhal
,
V. S.
Pande
,
W. C.
Swope
, and
J. W.
Pitera
, “
Automatic discovery of metastable states for the construction of Markov models of macromolecular conformational dynamics
,”
J. Chem. Phys.
126
,
155101
(
2007
).
7.
J. D.
Chodera
,
P.
Elms
,
F.
Noé
,
B.
Keller
,
C. M.
Kaiser
,
A.
Ewall-Wice
,
S.
Marqusee
,
C.
Bustamante
, and
N.
Singhal Hinrichs
, “
Bayesian hidden Markov model analysis of single-molecule force spectroscopy: Characterizing kinetics under measurement uncertainty
,” e-print arXiv:1108.1430.
8.
P.
Deuflhard
and
M.
Weber
, “
Robust Perron cluster analysis in conformation dynamics
,” in
Linear Algebra Applications
, edited by
M.
Dellnitz
,
S.
Kirkland
,
M.
Neumann
, and
C.
Schütte
(
Elsevier
,
New York
,
2005
), Vol.
398C
, pp.
161
184
.
9.
P.
Deulfhard
,
W.
Huisinga
,
A.
Fischer
, and
C.
Schütte
, “
Identification of almost invariant aggregates in reversibly nearly uncoupled Markov chains
,”
Linear Algebr. Appl.
315
,
39
59
(
2000
).
10.
N.
Djurdjevac
,
M.
Sarich
, and
C.
Schütte
, “
Estimating the eigenvalue error of Markov State Models
,”
Multiscale Model. Simul.
10
,
61
81
(
2012
).
11.
E. Z.
Eisenmesser
,
O.
Millet
,
W.
Labeikovsky
,
D. M.
Korzhnev
,
M.
Wolf-Watz
,
D. A.
Bosco
,
J. J.
Skalicky
,
L. E.
Kay
, and
D.
Kern
, “
Intrinsic dynamics of an enzyme underlies catalysis
,”
Nature (London)
438
(
7064
),
117
121
(
2005
).
12.
P. J.
Elms
,
J. D.
Chodera
,
C.
Bustamante
, and
S.
Marqusee
, “
The limitations of constant-force-feedback experiments
,”
Biophys. J.
103
,
1490
(
2012
).
13.
A.
Gansen
,
A.
Valeri
,
F.
Hauger
,
S.
Felekyan
,
S.
Kalinin
,
K.
Tóth
,
J.
Langowski
, and
C. A. M.
Seidel
, “
Nucleosome disassembly intermediates characterized by single-molecule FRET
,”
Proc. Natl. Acad. Sci. U.S.A.
106
(
36
),
15308
15313
(
2009
).
14.
J. C.
Gebhardt
,
T.
Bornschlögl
, and
M.
Rief
, “
Full distance-resolved folding energy landscape of one single protein molecule
,”
Proc. Natl. Acad. Sci. U.S.A.
107
(
5
),
2013
2018
(
2010
).
15.
N. S.
Hinrichs
and
V. S.
Pande
, “
Calculation of the distribution of eigenvalues and eigenvectors in Markovian state models for molecular dynamics
,”
J. Chem. Phys.
126
,
244101
(
2007
).
16.
B.
Keller
,
J.-H.
Prinz
, and
F.
Noé
, “
Markov models and dynamical fingerprints: Unraveling the complexity of molecular kinetics
,”
Chem. Phys.
396
,
92
107
(
2012
).
17.
S.
Kube
and
M.
Weber
, “
A coarse graining method for the identification of transition rates between molecular conformations
,”
J. Chem. Phys.
126
(
2
),
024103
(
2007
).
18.
G.
Soules
,
L. E.
Baum
,
T.
Petrie
, and
N.
Weiss
, “
A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains
,”
Ann. Math. Stat.
41
,
164
171
(
1970
).
19.
B.
Lindner
,
Z.
Yi
,
J.-H.
Prinz
,
J. C.
Smith
, and
F.
Noé
, “
Dynamic neutron scattering from conformational dynamics I: Theory and Markov models
,”
J. Chem. Phys.
139
,
175101
(
2013
).
20.
K.
Lindorff-Larsen
,
S.
Piana
,
R. O.
Dror
, and
D. E.
Shaw
, “
How fast-folding proteins fold
,”
Science
334
,
517
520
(
2011
).
21.
P.
Metzner
,
C.
Schütte
, and
E.
Vanden-Eijnden
, “
Illustration of transition path theory on a collection of simple examples
,”
J. Chem. Phys.
125
(
8
),
084110
(
2006
).
22.
L.
Molgedey
and
H. G.
Schuster
, “
Separation of a mixture of independent signals using time delayed correlations
,”
Phys. Rev. Lett.
72
,
3634
3637
(
1994
).
23.
F.
Noé
,
S.
Doose
,
I.
Daidone
,
M.
Löllmann
,
J. D.
Chodera
,
M.
Sauer
, and
J. C.
Smith
, “
Dynamical fingerprints for probing individual relaxation processes in biomolecular dynamics with simulations and kinetic experiments
,”
Proc. Natl. Acad. Sci. U.S.A.
108
,
4822
4827
(
2011
).
24.
F.
Noé
,
I.
Horenko
,
C.
Schütte
, and
J. C.
Smith
, “
Hierarchical analysis of conformational dynamics in biomolecules: Transition networks of metastable states
,”
J. Chem. Phys.
126
,
155102
(
2007
).
25.
F.
Noé
and
F.
Nüske
, “
A variational approach to modeling slow processes in stochastic dynamical systems
,”
SIAM Multiscale Model. Simul.
11
,
635
655
(
2013
).
26.
F.
Noé
,
C.
Schütte
,
E.
Vanden-Eijnden
,
L.
Reich
, and
T. R.
Weikl
, “
Constructing the full ensemble of folding pathways from short off-equilibrium simulations
,”
Proc. Natl. Acad. Sci. U.S.A.
106
,
19011
19016
(
2009
).
27.
G.
Perez-Hernandez
,
F.
Paul
,
T.
Giorgino
,
G.
de Fabritiis
, and
Frank
Noé
, “
Identification of slow molecular order parameters for Markov model construction
,”
J. Chem. Phys.
139
,
015102
(
2013
).
28.
J.-H.
Prinz
,
J. D.
Chodera
, and
F.
Noé
, “
Spectral rate theory for projected two-state kinetics
,”
Phys. Rev. X
(in press); e-print arXiv:1207.0225.
29.
J.-H.
Prinz
,
H.
Wu
,
M.
Sarich
,
B.
Keller
,
M.
Senne
,
M.
Held
,
J. D.
Chodera
,
C.
Schütte
, and
F.
Noé
, “
Markov models of molecular kinetics: Generation and validation
,”
J. Chem. Phys.
134
,
174105
(
2011
).
30.
L. R.
Rabiner
, “
A tutorial on hidden Markov models and selected applications in speech recognition
,”
Proceedings of the IEEE
77
,
257
286
(
1989
).
31.
S.
Röblitz
, “
Statistical error estimation and grid-free hierarchical refinement in conformation dynamics
,” Ph.D. thesis (FU Berlin,
2009
).
32.
Y.
Santoso
,
C. M.
Joyce
,
O.
Potapova
,
L.
Le Reste
,
J.
Hohlbein
,
J. P.
Torella
,
N. D. F.
Grindley
, and
A. N.
Kapanidis
, “
Conformational transitions in DNA polymerase I revealed by single-molecule FRET
,”
Proc. Natl. Acad. Sci. U.S.A.
107
(
2
),
715
720
(
2010
).
33.
M.
Sarich
,
F.
Noé
, and
C.
Schütte
, “
On the approximation error of Markov state models
,”
SIAM Multiscale Model. Simul.
8
,
1154
1177
(
2010
).
34.
C.
Schütte
,
A.
Fischer
,
W.
Huisinga
, and
P.
Deuflhard
, “
A direct approach to conformational dynamics based on hybrid Monte Carlo
,”
J. Comput. Phys.
151
,
146
168
(
1999
).
35.
C. R.
Schwantes
and
V. S.
Pande
, “
Improvements in Markov state model construction reveal many non-native interactions in the folding of NTL9
,”
J. Chem. Theory Comput.
9
,
2000
2009
(
2013
).
36.
M.
Senne
,
B.
Trendelkamp-Schroer
,
A. S. J. S.
Mey
,
C.
Schütte
, and
F.
Noé
, “
EMMA - A software package for Markov model building and analysis
,”
J. Chem. Theory Comput.
8
,
2223
2238
(
2012
).
37.
D. E.
Shaw
,
P.
Maragakis
,
K.
Lindorff-Larsen
,
S.
Piana
,
R. O.
Dror
,
M. P.
Eastwood
,
J. A.
Bank
,
J. M.
Jumper
,
J. K.
Salmon
,
Y.
Shan
, and
W.
Wriggers
, “
Atomic-level characterization of the structural dynamics of proteins
,”
Science
330
(
6002
),
341
346
(
2010
).
38.
N.
Singhal
and
V. S.
Pande
, “
Error analysis and efficient sampling in Markovian state models for molecular dynamics
,”
J. Chem. Phys.
123
,
204909
(
2005
).
39.
W. C.
Swope
,
J. W.
Pitera
,
F.
Suits
,
M.
Pitman
, and
M.
Eleftheriou
, “
Describing protein folding kinetics by molecular dynamics simulations: 2. Example applications to alanine dipeptide and beta-hairpin peptide
,”
J. Phys. Chem. B
108
,
6582
6594
(
2004
).
40.
V. A.
Voelz
,
G. R.
Bowman
,
K. A.
Beauchamp
, and
V. S.
Pande
, “
Molecular simulation of ab initio protein folding for a millisecond folder NTL9
,”
J. Am. Chem. Soc.
132
(
5
),
1526
1528
(
2010
).
41.
W.
E.
and
E.
Vanden-Eijnden
, “
Towards a theory of transition paths
,”
J. Stat. Phys.
123
(
3
),
503
523
(
2006
).
42.
M.
Weber
, “
Meshless methods in conformation dynamics
,” Ph.D. thesis (
FU Berlin
,
2006
).
43.
L. R.
Welch
, “
Hidden Markov models and the Baum-Welch algorithm
,”
IEEE Inf. Theory Soc. Newsl.
53
,
1
13
(
2003
).
44.
B. G.
Wensley
,
S.
Batey
,
F. A. C.
Bone
,
Z. M.
Chan
,
N. R.
Tumelty
,
A.
Steward
,
L. G.
Kwa
,
A.
Borgia
,
A.
Garrido
, and
J.
Clarke
, “
Experimental evidence for a frustrated energy landscape in a three-helix-bundle protein family
,”
Nature (London)
463
(
7281
),
685
688
(
2010
).
You do not currently have access to this content.