Almost 25 years ago, Rolf Landauer argued in the pages of this magazine that information is physical (see Physics Today, May 1991, page 23). It is stored in physical systems such as books and memory sticks, transmitted by physical means—for instance, via electrical or optical signals—and processed in physical devices. Therefore, he concluded, it must obey the laws of physics, in particular the laws of thermodynamics.
But what is information? A simple, intuitive answer is “what you don’t already know.” If someone tells you that Earth is spherical, you surely would not learn much; the message has low information content. However, if you are told that the price of oil will double tomorrow, then, assuming that to be true, you would learn a great deal; the message has high information content.
Mathematically, a system’s information content can be quantified by the so-called information entropy H, introduced by Claude Shannon in 1948. The larger the information entropy, the greater the information content.1 Consider the simplest possible information-storage device: a system with two distinct states—for example, up and down, left and right, or magnetized and unmagnetized. If the system is known with certainty to be in a particular state, then no new information can be gained by probing the system, and the information entropy is zero.
On the other hand, if the two states can be occupied with equal probability, the actual state is initially undetermined, and an examination of the system will provide information about its state. In that case, the information entropy is equal to ln(2), where ln denotes the natural logarithm. That value corresponds to the information associated with the simplest uncertain situation, a random binary choice, and is called a bit. A two-state system can thus store up to one bit of information.
In deriving his information entropy, Shannon took a cue from the second law of thermodynamics. As formulated by Rudolf Clausius in 1850, the second law is based on the empirical observation that spontaneous processes have a preferred direction.2 Everyone who’s left a cup of hot tea on a table has noted that heat flows spontaneously from a hotter body (the cup) to a colder one (the room) but never the other way around. Clausius characterized the irreversibility of such natural macroscopic processes by defining the thermodynamic entropy S, a quantity that, unlike energy, is not conserved and can only increase in isolated systems; it may decrease only in systems that exchange energy with the environment. That asymmetry imposes restrictions on the types of physical phenomena that are possible. In 1851, for instance, William Thomson (Lord Kelvin) devised an equivalent formulation of the second law that limits the amount of work that can be produced by a cyclic engine.
An interesting question, then, is whether the thermodynamic consequences of the second law extend to information. Is it possible to extract useful mechanical work from a system just by observing its state? If so, how much? And at a more fundamental level, are the thermodynamic and information entropies related?
Maxwell and Szilard
The first hint of a connection between information and thermodynamics may be traced back to James Clerk Maxwell’s now-famous demon,3 introduced in 1867. The demon is an intelligent creature able to monitor individual molecules of a gas contained in two neighboring chambers, as shown in figure 1.
Figure 1. Maxwell’s demon. By detecting the positions and velocities of gas molecules in two neighboring chambers and using that information to time the opening and closing of a trapdoor that separates them, a tiny, intelligent being could, in theory, sort molecules by velocity. By doing so, it could create a temperature difference across the chambers that could be used to perform mechanical work. If the trapdoor is frictionless, the sorting requires no work from the demon himself, in apparent violation of the second law of thermodynamics. (Image from ref. 3, H. S. Leff, A. F. Rex.)
Figure 1. Maxwell’s demon. By detecting the positions and velocities of gas molecules in two neighboring chambers and using that information to time the opening and closing of a trapdoor that separates them, a tiny, intelligent being could, in theory, sort molecules by velocity. By doing so, it could create a temperature difference across the chambers that could be used to perform mechanical work. If the trapdoor is frictionless, the sorting requires no work from the demon himself, in apparent violation of the second law of thermodynamics. (Image from ref. 3, H. S. Leff, A. F. Rex.)
Initially, the two chambers are at the same temperature, defined by the mean kinetic energy of the molecules and proportional to their mean-square velocity. Some of the particles, however, travel faster than others. By opening and closing a molecule-sized trapdoor in the partitioning wall, the demon can collect the faster molecules in one chamber and the slower ones in the other. The two chambers then contain gases with different temperatures, and that temperature difference may be used to power a heat engine and produce mechanical work.
By gathering information about the particles’ positions and velocities and using that knowledge to sort them, the demon is able to decrease the entropy of the system and convert information into energy. Assuming the trapdoor is frictionless, the demon is able to do all that without performing any work himself—an apparent violation of the second law of thermodynamics.
The proper resolution of the paradox wouldn’t come for another 115 years, but in 1929 a second thought experiment, contrived by Leo Szilard, provided crucial insight. Szilard’s take on the demon involves a gas consisting of just a single molecule.4 The wall separating the identical chambers is replaced by a movable piston, held in place with a pin. The result is a two-state system analogous to a bit: Initially, the particle occupies each chamber with probability one-half. By looking into the container, the demon acquires information about the actual state of the system. If the molecule is found in the left chamber, the demon attaches a weight to the left side of the piston, as illustrated in figure 2, and releases the pin. As the gas expands, the piston is pushed rightward and the weight is pulled upward against gravity. If the molecule is found in the right chamber, the weight is attached to the right-hand side of the piston.
Figure 2. Szilard’s engine. A crafty observer can turn a single particle in a box into an engine that converts information into mechanical work. If, say, the particle is found on the box’s left-hand side, the observer inserts a movable wall and attaches a weight to its left side. The free expansion of the one-particle gas pushes the wall to the right, lifts the weight, and thereby performs work against gravity. (Adapted from ref. 12, J. V. Koski et al.)
Figure 2. Szilard’s engine. A crafty observer can turn a single particle in a box into an engine that converts information into mechanical work. If, say, the particle is found on the box’s left-hand side, the observer inserts a movable wall and attaches a weight to its left side. The free expansion of the one-particle gas pushes the wall to the right, lifts the weight, and thereby performs work against gravity. (Adapted from ref. 12, J. V. Koski et al.)
The second law of thermodynamics limits the maximum amount of work that can be produced by the engine to kBTln(2). Here, kB is Boltzmann’s constant and T is the temperature of the gas. The “2” arises because there are two chambers in Szilard’s system. Thus, kBTln(2) is the maximum amount of energy that can be obtained from one bit of information. Historically, that result is the first clear link between information and energy. In modern language, it suggests that the thermodynamic and information entropies are essentially equivalent, S = kBH, with the factor kB introduced for dimensional reasons. (The information entropy H is dimensionless.)
The equivalence might appear strange at first glance, since S is related to the amount of heat reversibly exchanged with a reservoir and H characterizes the information content of a message sent through a communication channel. In 1957, however, Edwin Jaynes established a general relationship between the two using statistical inference. His approach can be illustrated by considering one of the classic problems of thermodynamics: determining the maximum work Wmax that can be extracted from a heat engine.
According to the second law, processes occurring at constant temperature T have Wmax = −ΔF = −(ΔE − TΔS), where F is free energy and E is energy. In other words, not all thermodynamic energy can be converted to work, as can mechanical energy; there exists a nonusable part given by TΔS. That result can be readily understood from the point of view of information. Any thermodynamic system can be described either microscopically or macroscopically. The microstate, characterized by the positions and velocities of all of a system’s constituent particles, contains complete information about that system. But typical systems contain so many particles—on the order of 1024—that the microstate isn’t experimentally accessible. By contrast, the macrostate—defined by macroscopic parameters such as temperature, pressure, and volume—is measurable but contains only partial information.
Jaynes recognized that entropy essentially quantifies the degree of ignorance about the state of the system—that is, the amount of microstate information that’s lost when one monitors the system macroscopically. Put another way, energy can be entirely converted to work only if the microstate, and hence the complete information about the system, is available. Jaynes’s observation provided a firm theoretical basis for Szilard’s notion of an equivalence between thermodynamic and information entropies. And it showed that the equivalence should, in principle, hold for any arbitrary system at equilibrium. The fact that thermodynamic entropy can only increase in an isolated system could be understood to imply that the information content of the system can spontaneously only decrease.
Landauer and Bennett
It is useful to distinguish two complementary concepts: information gain and information erasure. Let us again consider a two-state system, and let us assume that the two states are occupied with equal probability one-half, so that the system initially stores one bit of information.
Gaining, or writing, information is akin to copying information from one place to another—mapping the system’s left and right states to the left and right states of a storage device, for example. Such one-to-one mapping can be realized, in principle, without dissipating any heat. In statistical mechanics parlance, one would say that the mapping conserves volume in phase space.
By contrast, erasing a bit of information involves mapping two states (say, left and right) onto one (right), which is then occupied with probability one. That process does not conserve the volume in phase space and is therefore dissipative. Applying the second law of thermodynamics, Landauer demonstrated in 1961 that the erasure of one bit of information is necessarily accompanied by the release of at least kBTln(2) of heat into the environment. (See the box at left.) That theoretical result, known as Landauer’s erasure principle, illustrates a fundamental difference between the process of writing and erasing information.
Landauer’s principle was central to solving the paradox of Maxwell’s demon. In 1982 Charles Bennett noted that the demon has to memorize the information he acquires about the gas molecules.5 Bennett argued that after a full cycle of information gathering and energy production, the demon’s memory has to be reset to its initial state to allow for a new iteration. According to Landauer’s principle, the erasure process will always dissipate more energy than the demon produces during one cycle, in full agreement with the second law of thermodynamics. Oliver Penrose had put forward a similar argument about a decade earlier.5
Before Bennett’s resolution, it was widely believed—following arguments put forward by Leon Brillouin and Dennis Gabor—that the energetic price of the measurement would save the second law.6 Bennett showed, however, that there is no fundamental energetic limitation on the measurement process.
Dancing with the demon
For almost a century and a half, Maxwell’s demon belonged to the realm of gedanken experiments; tracking and manipulating individual microscopic particles were impossible. As Erwin Schrödinger famously wrote:
We never experiment with just one electron or atom or (small) molecule. In thought-experiments we sometimes assume that we do; this invariably entails ridiculous consequences…. In the first place it is fair to state that we are not experimenting with single particles, any more than we can raise Ichthyosauria in the zoo.7
However, owing to remarkable technological progress achieved in recent decades, experiments with atoms and small particles have now become feasible. Maxwell’s demon, Szilard’s engine, and Landauer’s erasure principle can now be rigorously studied in lab experiments.
One of the first such experiments was performed by Mark Raizen and coworkers at the University of Texas at Austin.8 They confined an ensemble of atoms in a magnetic trap, as shown schematically in figure 3. Initially, all the atoms are in the same internal state. The group then introduced a one-way optical barrier, composed of two laser beams arranged side by side: One beam promotes atoms to an excited state, and the other is tuned such that it has no effect on excited atoms but repels atoms in the ground state. An atom (red) approaching from the excitation-beam side gets promoted to an excited state, passes unimpeded through the second beam, and then relaxes to the ground state by emitting a photon. An atom approaching from the other side, by contrast, encounters the repelling beam first and is turned around—it can’t get through. The two beams behave as an atom diode.
Figure 3. Bringing Maxwell’s demon to life. A pair of laser beams can be tuned to atomic transitions and configured to create a one-way potential barrier; atoms may cross unimpeded in one direction—right to left in this figure—but not in the other. (a) When the barrier is introduced at the periphery of a V-shaped magnetic trap, the atoms that cross the barrier will be those that have converted nearly all their kinetic energy to potential energy—in other words, the cold ones. (b–c) By slowly sweeping the barrier across the trap, one can sort cold atoms (blue) from hot ones (red), reminiscent of James Clerk Maxwell’s famous thought experiment, or cool an entire atomic ensemble. Because the cold atoms do work against the optical barrier as it moves, their kinetic energy remains small even as they return to the deep portion of the potential well. (Adapted from ref. 8, M. G. Raizen.)
Figure 3. Bringing Maxwell’s demon to life. A pair of laser beams can be tuned to atomic transitions and configured to create a one-way potential barrier; atoms may cross unimpeded in one direction—right to left in this figure—but not in the other. (a) When the barrier is introduced at the periphery of a V-shaped magnetic trap, the atoms that cross the barrier will be those that have converted nearly all their kinetic energy to potential energy—in other words, the cold ones. (b–c) By slowly sweeping the barrier across the trap, one can sort cold atoms (blue) from hot ones (red), reminiscent of James Clerk Maxwell’s famous thought experiment, or cool an entire atomic ensemble. Because the cold atoms do work against the optical barrier as it moves, their kinetic energy remains small even as they return to the deep portion of the potential well. (Adapted from ref. 8, M. G. Raizen.)
In the schematic shown in figure 3, the barrier is configured to allow atoms to cross only from right to left. When it is introduced from the far left, where the trap potential is very large, atoms reaching the barrier will have converted almost all their kinetic energy into potential energy and will therefore get trapped in a very cool state. Because the atoms do work on the optical barrier as it moves, they remain cool as the barrier is swept from left to right.
In the above scenario, the optical potential serves as the demon. If an atom is determined to be moving from right to left—that is, if it encounters the excitation beam first, the trapdoor is opened. For all other atoms, the trapdoor is closed. Information about the position and internal state of the atoms is stored in the photons scattered by the atoms.
The optical demon is beholden to thermodynamic constraints. Each time an atom scatters a photon, the entropy of the optical beam increases, because a photon that was propagating coherently with the beam gets scattered in an uncertain direction. Raizen’s group was able to show that this entropy increase is always larger than the reduction of entropy produced by the cooling of the atomic cloud. David Leigh and colleagues at the University of Edinburgh also realized a chemical version of Maxwell’s demon using a rotaxane molecule.9
Revisiting Szilard and Landauer
In 2010 Masaki Sano’s group at the University of Tokyo realized a Szilard engine with a single, micron-sized colloidal particle bathed in a fluid of uniform temperature.10 The particle was trapped in a tilted periodic optical potential, which can be schematically understood as a spiral staircase, as shown in figure 4. Driven by thermal fluctuations, a particle moves erratically up and down the staircase. Because the staircase potential is inclined, downward steps are more frequent than upward ones, and the particle will, on average, move downward.
Figure 4. Building Szilard’s engine with light and flow. (a) Left to its own devices, a colloidal particle will, on average, move down a staircase potential, though energy fluctuations occasionally will push the particle upward. (b) An intelligent demon can exploit the upward fluctuations to do work: Each time the demon observes the particle making an upward step, it inserts a wall that prevents the particle from regressing. (c) In an actual experiment, the staircase is implemented with a tilted periodic optical potential (blue), and the insertion of the barrier is realized with π phase shifts of that potential (dotted line). When fluctuations steer the particle into a so-called switching zone (shaded gray), the phase shift promotes the particle to a higher-energy well. (Adapted from ref. 10.)
Figure 4. Building Szilard’s engine with light and flow. (a) Left to its own devices, a colloidal particle will, on average, move down a staircase potential, though energy fluctuations occasionally will push the particle upward. (b) An intelligent demon can exploit the upward fluctuations to do work: Each time the demon observes the particle making an upward step, it inserts a wall that prevents the particle from regressing. (c) In an actual experiment, the staircase is implemented with a tilted periodic optical potential (blue), and the insertion of the barrier is realized with π phase shifts of that potential (dotted line). When fluctuations steer the particle into a so-called switching zone (shaded gray), the phase shift promotes the particle to a higher-energy well. (Adapted from ref. 10.)
By gathering and then acting on information about the fluctuations, however, one can coax the particle to climb. Sano and company achieved that feat by monitoring the position of the particle in real time with the help of a CCD camera. Each time they saw the particle jump upward, they inserted an optical potential barrier—a virtual wall—that prevented the particle from falling back down. Experimentally, the insertion of the wall is realized by switching the phase of the optical potential. When the procedure is repeated, the particle moves upward, on average, and does work against the force exerted by the staircase potential. The Tokyo group’s system is the first example of a device that uses information to extract energy from a system coupled to a uniform thermal environment.
The year following the Tokyo experiment, the group of one of us (Ciliberto) at the École Normale Supérieure (ENS) in Lyon, France, realized Landauer’s thought experiment using a colloidal particle trapped in the double-well potential produced by two strongly focused laser beams.11 Such a system has two distinct states—the particle may be in the right or left well—and can thus be used to store one bit of information. The bit can be erased by implementing a protocol proposed by Bennett and illustrated in figure 5. First, the potential barrier between the two wells is lowered by varying the laser intensity. Next, the particle is pushed to the right by a gentle flow induced with a piezoelectric motor. In effect, the flow inclines the trapping potential. Finally, the potential is brought back to its initial shape. The particle, regardless of which well it started off in, will end up in the right well with unit probability.
Figure 5. Testing Landauer’s erasure principle. A colloidal particle confined with equal probability to one of two optical potential wells constitutes one bit of information. Modulating (a–b), tilting (c–e), and then restoring (f) the barrier between the wells places the particle in the right well with probability close to one, irrespective of the particle’s initial position. The final configuration corresponds to zero bits of information. In the limit of long erasure cycles, the heat dissipated during the erasure process approaches, but does not fall below, kBTln(2), as predicted by Rolf Landauer. (Adapted from ref. 11.)
Figure 5. Testing Landauer’s erasure principle. A colloidal particle confined with equal probability to one of two optical potential wells constitutes one bit of information. Modulating (a–b), tilting (c–e), and then restoring (f) the barrier between the wells places the particle in the right well with probability close to one, irrespective of the particle’s initial position. The final configuration corresponds to zero bits of information. In the limit of long erasure cycles, the heat dissipated during the erasure process approaches, but does not fall below, kBTln(2), as predicted by Rolf Landauer. (Adapted from ref. 11.)
For a full erasure cycle, the average heat dissipated into the environment is equal to the average work needed to modulate the form of the double-well potential. The ENS group evaluated that quantity from the measured particle trajectory and the known force applied to the particle. (As in the Tokyo experiment, the position of the particle was recorded with the help of a camera.) In the limit of long erasure times, heat dissipation approaches, but is always larger than, the Landauer bound.
The above examples of gedanken-turned-real experiments provide a firm empirical foundation for the physics of information and tangible evidence of the intimate connection between information and energy. They have been followed by additional experiments and simulations along similar lines.12 (See, for example, Physics Today, August 2014, page 60.) Collectively, that body of experimental work further demonstrates the equivalence of information and thermodynamic entropies at thermal equilibrium.
Modern theory
The experiments with demons and colloidal bits coincided with a surge of theoretical activity.13 In particular, the insights from Maxwell’s demon and Szilard’s engine were generalized to other types of thermodynamic systems. In 2008 Takahiro Sagawa and Masahito Ueda formalized the idea that information gained through microlevel measurements can be used to extract added work from a heat engine.14 Their formula for the maximum extractable work, Wmax = −(ΔE − TΔS) + kBTI, includes an extra term representing the so-called mutual information I. Roughly speaking, I quantifies the amount of information obtained via measurements.
Sagawa and Ueda’s result extends the second law to explicitly incorporate information; it shows that information, entropy, and energy should be treated on equal footings. Applied to the theory of heat engines with feedback, the formula indicates that the maximum extractable work is Wmax = (1 − T1/T2) Q2 + kBT1I, where T1 and T2 are the temperatures of the cold and hot reservoirs and Q2 is the heat absorbed from the hot reservoir.
In the absence of information gain, when I = 0, the formula reduces to the familiar expression for the efficiency η = Wmax/Q2 of a Carnot engine. With feedback, however, the engine can produce additional mechanical work and, remarkably, can even produce work from a single reservoir (T1 = T2), which is impossible in standard thermodynamics. The prospect of generic information engines in which the entropy-decreasing function of the cold reservoir is performed instead by the feedback process—which can be regarded as an information reservoir—is one of the more surprising predictions of the new formulation.
From experiments to applications
Landauer’s principle applies not only to information erasure but to any logically irreversible procedure that produces fewer outputs than inputs. It predicts, for instance, that any Boolean gate operation that maps multiple input states onto the same output state—such as AND, NAND, and OR—will dissipate kBTln(2) of heat per processed bit.
Landauer’s principle therefore has important technological consequences. Nowadays, laptop computers are a part of everyday experience. But heat production by microprocessors used in modern computers is a major factor hindering their miniaturization; it gets increasingly difficult to evacuate heat as a microprocessor’s size, and therefore surface area, is reduced. Although heat-generation rates in microchips are still several orders of magnitude larger than the Landauer limit, they are steadily decreasing. The switching energy of a complementary metal oxide–semiconductor FET is predicted to reach the Landauer bound by 2035, an indication that engineers will soon face a fundamental physical limitation imposed by the second law of thermodynamics.15 That’s remarkable considering that kBTln(2) is about 3 × 10−21 joules at room temperature and hence 22 orders of magnitude smaller than typical amounts of energy dissipated on our macroscopic scale.
Manmade computers are not the only existing information-processing devices. Scientists have long realized that living biological cells can be viewed as biochemical information processors; some may even outperform our current technology.16 Cells, for instance, can reproduce and copy themselves, acquire and process information from external stimuli, and communicate and exchange information with other cells. Recently, Pankaj Mehta and David Schwab used Landauer’s principle to evaluate the energetic cost a cell must pay to compute the steady-state concentration of a chemical ligand in its surrounding environment.17 That cost arguably sets strong constraints on the design of cellular computing networks, since it implies a tradeoff between a network’s information-processing capability and its energy consumption.
Another important problem is the investigation of ultrasensitive switches in molecular biology. One example is the flagellar motor of Escherichia coli bacteria, which switches between clockwise and counterclockwise rotation depending on the intracellular concentration of a regulator protein. (See the article by Howard Berg, Physics Today, January 2000, page 24.) The switching mechanisms are highly complex and not fully understood. A mathematical framework that treats the flagellar motor as a Maxwell’s demon can be used to calculate the rate of energy consumption needed for both protein sensing and switching and can provide a quantitative description of the switching statistics.18
Some 150 years after its inception, Maxwell’s demon is still vibrant. Together with Landauer’s principle, it continues to influence modern research. Having only recently become an experimental science, the thermodynamics of information has potential to deliver new insights in physics, chemistry, and biology. Conceivably, it may even be extended to the realm of quantum mechanics, where it could pave the way for a full-fledged thermodynamics of quantum information.
The relationship between thermodynamic entropy and information entropy prescribes an energetic cost of erasing a bit. Consider a system (SYS) coupled to a reservoir (RES) at temperature T. According to the second law, any change in the combined entropies of the system and the reservoir must be positive: ΔSTOT = ΔSSYS + ΔSRES ≥ 0. Because the reservoir, owing to its large size, is always at equilibrium, we can invoke the Clausius equality, ΔSRES = QRES/T. In other words, the heat flow to the reservoir satisfies QRES ≥ −TΔSSYS.
For a two-state system that stores one bit of information, the initial information entropy is Hi = ln(2). After erasure, the information entropy vanishes, Hf = 0, so ΔH = −ln(2). Assuming an equivalence between thermodynamic entropy S and information entropy H, we can write ΔSSYS = kBΔH = kBln(2), where kB is Boltzmann’s constant. We hence obtain QRES ≥ kBTln(2). In other words, the heat dissipated into the reservoir during the erasure of one bit of information in the system is always larger than kBTln(2).
We gratefully acknowledge support from the German Research Foundation, the European Union, the French National Research Agency, and the European Research Council.
REFERENCES
Eric Lutz is a professor of theoretical physics at the University of Erlangen–Nuremberg in Erlangen, Germany. Sergio Ciliberto is a research director in the laboratory of physics at the École Normale Supérieure in Lyon, France.
