Markov chain Monte Carlo algorithms are invaluable tools for exploring stationary properties of physical systems, especially in situations where direct sampling is unfeasible. Common implementations of Monte Carlo algorithms employ reversible Markov chains. Reversible chains obey detailed balance and thus ensure that the system will eventually relax to equilibrium, though detailed balance is not necessary for convergence to equilibrium. We review nonreversible Markov chains, which violate detailed balance and yet still relax to a given target stationary distribution. In particular cases, nonreversible Markov chains are substantially better at sampling than the conventional reversible Markov chains with up to a square root improvement in the convergence time to the steady state. One kind of nonreversible Markov chain is constructed from the reversible ones by enlarging the state space and by modifying and adding extra transition rates to create non-reversible moves. Because of the augmentation of the state space, such chains are often referred to as lifted Markov Chains. We illustrate the use of lifted Markov chains for efficient sampling on several examples. The examples include sampling on a ring, sampling on a torus, the Ising model on a complete graph, and the one-dimensional Ising model. We also provide a pseudocode implementation, review related work, and discuss the applicability of such methods.

1.
See, for example,
J.
Steward
,
Calculus: Early Transcendentals
, 6th ed. (
Thompson Brooks/Cole
,
Belmont, CA
,
2008
).
2.
A.
Sokal
,
Monte Carlo Methods in Statistical Mechanics: Foundations and New Algorithms
(
Springer
,
New York, NY
,
1997
).
3.
K.
Turitsyn
,
M.
Chertkov
, and
M.
Vucelja
, “
Irreversible Monte Carlo algorithms for efficient sampling
,”
Physica D
240
,
410
414
(
2011
).
4.
F.
Chen
,
L.
Lovasz
, and
I.
Pak
, “
Lifting Markov chains to speed up mixing
,” in
Proceedings of the ACM symposium on Theory of Computing
(
1999
), pp.
275
281
.
5.
Thomas P.
Hayes
and
Alistair
Sinclair
, “
Liftings of tree-structured Markov chains
,”
Lect. Notes Comp. Sci.
6302
,
602
616
(
2010
).
6.
D. A.
Levin
,
Y.
Peres
, and
E. L.
Wilmer
,
Markov Chains and Mixing Times
(
American Mathematical Society
,
Providence, RI
,
2009
). This book, which we strongly recommend to the mathematically inclined novice, provides a highly comprehensible source of knowledge on Markov chains, stochastic processes, and mixing.
7.
N.
Metropolis
,
A.
Rosenbluth
,
M.
Rosenbluth
,
A.
Teller
, and
E.
Teller
, “
Equation of state calculations by fast computing machines
,”
J. Chem. Phys.
21
,
1087
1092
(
1953
).
8.
W. K.
Hastings
, “
Monte Carlo sampling methods using Markov chains and their applications
,”
Biometrika
57
,
97
109
(
1970
).
9.
W.
Krauth
,
Statistical Mechanics: Algorithms and Computations
(
Oxford U.P.
,
Oxford
,
2006
).
10.
P.
Diaconis
,
S.
Holmes
, and
R. M.
Neal
, “
Analysis of a nonreversible Markov chain sampler
,”
Ann. Appl. Prob.
10
,
726
752
(
2000
).
11.
Y.
Sakai
and
K.
Hukushima
, “
Dynamics of one-dimensional Ising model without detailed balance condition
,”
J. Phys. Soc. Japan
82
,
064003
(
2013
).
12.
L.
Colonna-Romano
,
H.
Gould
, and
W.
Klein
, “
Anomalous mean-field behavior of the fully connected Ising model
,”
Phys. Rev. E
90
,
042111
(
2014
).
13.
H.
Gould
and
J.
Tobochnik
,
Statistical and Thermal Physics with Computer Applications
(
Princeton U.P.
,
Princeton, NJ
,
2010
).
14.
J.
Bierkens
and
G.
Roberts
, “
A piecewise deterministic scaling limit of Lifted Metropolis-Hastings in the Curie-Weiss model
,” e-print arXiv:1500.00302.
15.
R. J.
Glauber
, “
Time dependent statistics of the Ising model
,”
J. Math. Phys.
4
,
294
307
(
1963
).
16.
H. C.
Fernandes
and
M.
Weigel
, “
Non-reversible Monte Carlo simulations of spin models
,”
Comput. Phys. Commun.
182
,
1856
1859
(
2011
).
17.
A.
Ichiki
and
M.
Ohzeki
, “
Violation of detailed balance accelerates relaxation
,”
Phys. Rev. E
88
,
020101
(
2013
).
18.
H.
Suwa
and
S.
Todo
, “
Markov chain Monte Carlo method without detailed balance
,”
Phys. Rev. Lett.
105
,
120603
(
2010
).
19.
E. P.
Bernard
,
W.
Krauth
, and
D. B.
Wilson
, “
Event-chain algorithms for hard-sphere systems
,”
Phys. Rev. E
80
,
056704
(
2009
).
20.
E. P.
Bernard
and
W.
Krauth
, “
Addendum to event-chain Monte Carlo algorithms for hard-sphere systems
,”
Phys. Rev. E
86
,
017701
(
2012
).
21.
R. D.
Schram
and
G. T.
Barkema
, “
Monte Carlo methods beyond detailed balance
,”
Physica A
418
,
88
93
(
2015
).
22.
T.
Hayes
, private communication (2013).
23.
J.
Bierkens
, “
Non-reversible metropolis-hastings
,”
Stat. Comput.
26
,
1213
1228
(
2016
).
24.
C.
Godrèche
and
A. J.
Bray
, “
Nonequilibrium stationary states and phase transitions in directed Ising
,”
J. Stat. Mech.
2009
,
P12016-1
19
.
25.
T.
Lelièvre
,
F.
Nier
, and
G.
Pavliotis
, “
Optimal non-reversible linear drift for the convergence to equilibrium of a diffusion
,”
J. Stat. Phys.
152
,
237
274
(
2013
).
26.
L.
Rey-Bellet
and
K.
Spiliopoulos
, “
Irreversible Langevin samplers and variance reduction: A large deviation approach
,”
Nonlinearity
28
,
2081
2103
(
2015
).