The minimum heat cost of computation is subject to bounds arising from Landauer’s principle. Here, I derive bounds on finite modeling—the production or anticipation of patterns (time-series data)—by devices that model the pattern in a piecewise manner and are equipped with a finite amount of memory. When producing a pattern, I show that the minimum dissipation is proportional to the information in the model’s memory about the pattern’s history that never manifests in the device’s future behavior and must be expunged from memory. I provide a general construction of a model that allows this dissipation to be reduced to zero. By also considering devices that consume or effect arbitrary changes on a pattern, I discuss how these finite models can form an information reservoir framework consistent with the second law of thermodynamics.
REFERENCES
1.
R.
Landauer
, “Irreversibility and heat generation in the computer process
,” IBM J. Res. Dev.
5
, 183
(1961
). 2.
C. H.
Bennett
, “The thermodynamics of computation—A review
,” Int. J. Theor. Phys.
21
, 905
(1982
). 3.
D.
Mandal
and C.
Jarzynski
, “Work and information processing in a solvable model of Maxwell’s demon
,” Proc. Natl. Acad. Sci. U.S.A.
109
, 11641
(2012
). 4.
A. B.
Boyd
, D.
Mandal
, and J. P.
Crutchfield
, “Identifying functional thermodynamics in autonomous Maxwellian ratchets
,” New J. Phys.
18
, 023049
(2016
). 5.
A. B.
Boyd
, D.
Mandal
, and J. P.
Crutchfield
, “Leveraging environmental correlations: The thermodynamics of requisite variety
,” J. Stat. Phys.
167
, 1555
(2017
). 6.
A. J. P.
Garner
, J.
Thompson
, V.
Vedral
, and M.
Gu
, “Thermodynamics of complexity and pattern manipulation
,” Phys. Rev. E
95
, 042140
(2017
). 7.
A. B.
Boyd
, D.
Mandal
, and J. P.
Crutchfield
, “Thermodynamics of modularity: Structural costs beyond the Landauer bound
,” Phys. Rev. X
8
, 031036
(2018
).8.
K.
Wiesner
, M.
Gu
, E.
Rieper
, and V.
Vedral
, “Information-theoretic lower bound on energy cost of stochastic computation
,” Proc. R. Soc. A: Math. Phys. Eng. Sci.
468
, 4058
(2012
).9.
S.
Still
, D. A.
Sivak
, A. J.
Bell
, and G. E.
Crooks
, “Thermodynamics of prediction
,” Phys. Rev. Lett.
109
, 120604
(2012
). 10.
S.
Deffner
and C.
Jarzynski
, “Information processing and the second law of thermodynamics: An inclusive, Hamiltonian approach
,” Phys. Rev. X
3
, 041003
(2013
).11.
P.
Strasberg
, “Thermodynamics and information processing at the nanoscale,” Ph.D. thesis (Technische Universität, Berlin, 2015).12.
A. B.
Boyd
, D.
Mandal
, and J. P.
Crutchfield
, “Correlation-powered information engines and the thermodynamics of self-correction
,” Phys. Rev. E
95
, 012152
(2017
). 13.
Z.
Lu
and C.
Jarzynski
, “A programmable mechanical Maxwell’s demon
,” Entropy
21
, 65
(2019
). 14.
J. P.
Crutchfield
and K.
Young
, “Inferring statistical complexity
,” Phys. Rev. Lett.
63
, 105
(1989
). 15.
C. R.
Shalizi
and J. P.
Crutchfield
, “Computational mechanics: Pattern and prediction, structure and simplicity
,” J. Stat. Phys.
104
, 817
(2001
). 16.
J. P.
Crutchfield
, C. J.
Ellison
, and J. R.
Mahoney
, “Time’s barbed arrow: Irreversibility, crypticity, and stored information
,” Phys. Rev. Lett.
103
, 094101
(2009
). 17.
C. J.
Ellison
, J. R.
Mahoney
, R. G.
James
, J. P.
Crutchfield
, and J.
Reichardt
, “Information symmetries in irreversible processes
,” Chaos
21
, 037107
(2011
). 18.
J. R.
Mahoney
, C. J.
Ellison
, R. G.
James
, and J. P.
Crutchfield
, “How hidden are hidden processes? A primer on crypticity and entropy convergence
,” Chaos
21
, 037112
(2011
). 19.
N.
Barnett
and J. P.
Crutchfield
, “Computational mechanics of input–output processes: Structured transformations and the -transducer
,” J. Stat. Phys.
161
, 404
(2015
). 20.
J. P.
Crutchfield
and D. P.
Feldman
, “Statistical complexity of simple one-dimensional spin systems
,” Phys. Rev. E
55
, R1239
(1997
). 21.
W. Y.
Suen
, J.
Thompson
, A. J. P.
Garner
, V.
Vedral
, and M.
Gu
, “The classical-quantum divergence of complexity in modelling spin chains
,” Quantum
1
, 25
(2017
). 22.
K.
Lindgren
, “Microscopic and macroscopic entropy
,” Phys. Rev. A
38
, 4794
(1988
). 23.
R.
Landauer
, “Information is physical
,” Phys. Today
44
(5
), 23
(1991
). 24.
R.
Alicki
, “The quantum open system as a model of the heat engine
,” J. Phys. A: Math. Gen.
12
, L103
(1979
). 25.
R.
Alicki
, M.
Horodecki
, P.
Horodecki
, and R.
Horodecki
, “Thermodynamics of quantum information systems—Hamiltonian description
,” Open Syst. Inf. Dyn.
11
, 205
(2004
). 26.
J.
Åberg
, “Truly work-like work extraction via a single-shot analysis
,” Nat. Commun.
4
, 1925
(2013
). 27.
C.
Jarzynski
, “Nonequilibrium equality for free energy differences
,” Phys. Rev. Lett.
78
, 2690
(1997
). 28.
G. E.
Crooks
, “Entropy production fluctuation theorem and the nonequilibrium work relation for free energy differences
,” Phys. Rev. E
60
, 2721
(1999
). 29.
T. M.
Cover
and J. A.
Thomas
, Elements of Information Theory
(John Wiley & Sons
, New York
, 1991
).30.
N.
Tishby
, F. C.
Pereira
, and W.
Bialek
, “The information bottleneck method,” in 37th Annual Allerton Conference, Monticello, IL (University of Illinois, 1999), pp. 368–377.31.
S.
Still
, “Information bottleneck approach to predictive inference
,” Entropy
16
, 968
(2014
). 32.
J. B.
Ruebeck
, R. G.
James
, J. R.
Mahoney
, and J. P.
Crutchfield
, “Prediction and generation of binary Markov processes: Can a finite-state fox catch a Markov mouse?
,” Chaos
28
, 013109
(2018
). 33.
J. P.
Crutchfield
, C. J.
Ellison
, R. G.
James
, and J. R.
Mahoney
, “Synchronization and control in intrinsic and designed computation: An information-theoretic analysis of competing models of stochastic computation
,” Chaos
20
, 037105
(2010
). 34.
M.
Gu
, K.
Wiesner
, E.
Rieper
, and V.
Vedral
, “Quantum mechanics can reduce the complexity of classical models
,” Nat. Commun.
3
, 762
(2012
). 35.
J. R.
Mahoney
, C.
Aghamohammadi
, and J. P.
Crutchfield
, “Occam’s quantum strop: Synchronizing and compressing classical cryptic processes via a quantum channel
,” Sci. Rep.
6
, 20495
(2016
). 36.
F. C.
Binder
, J.
Thompson
, and M.
Gu
, “Practical unitary simulator for non-Markovian complex processes
,” Phys. Rev. Lett.
120
, 240502
(2018
). 37.
S. P.
Loomis
and J. P.
Crutchfield
, “Thermal efficiency of quantum memory compression
,” Phys. Rev. Lett.
125
, 020601
(2020
). 38.
J.
Thompson
, A. J. P.
Garner
, V.
Vedral
, and M.
Gu
, “Using quantum theory to simplify input-output processes
,” npj Quantum Inf.
3
, 817
(2017
). 39.
K. T.
Hu
, “On the amount of information
,” Theory Probab. Appl.
7
, 439
(1962
). 40.
R. G.
James
, C. J.
Ellison
, and J. P.
Crutchfield
, “Anatomy of a bit: Information in a time series observation
,” Chaos
21
, 037109
(2011
). 41.
N. F.
Travers
and J. P.
Crutchfield
, “Exact synchronization for finite-state sources
,” J. Stat. Phys.
145
, 1181
(2011
). 42.
N. F.
Travers
and J. P.
Crutchfield
, “Asymptotic synchronization for finite-state sources
,” J. Stat. Phys.
145
, 1202
(2011
). 43.
© 2021 Author(s). Published under an exclusive license by AIP Publishing.
2021
Author(s)
You do not currently have access to this content.