Insight into entropy

Oral remark by John von Neumann to Claude Shannon, recalled by Shannon. See page 354 of Myron Tribus, “Information theory and thermodynamics,” in Heat Transfer, Thermodynamics, and Education: Boelter Anniversary Volume, edited by Harold A. Johnson (McGraw-Hill, New York, 1964), pp. 348–368.

Karl K.

Darrow

, “,” , – ().

P. G.

Wright

, “,” , – ().

P. W. Atkins, The Second Law (Scientific American Books, New York, 1984).

Bernd

Rodewald

, “,” , – ().

Harvey S. Leff and Andrew F. Rex, Eds., Maxwell’s Demon: Entropy, Information, Computing (Princeton University Press, Princeton, NJ, 1990).

Ralph

Baierlein

, “,” , – ().

Harvey S.

Leff

, “,” , – ().

Thomas A.

Moore

and

Daniel V.

Schroeder

, “,” , – ().

Vinay

Ambegaokar

and

Aashish A.

Clerk

, “,” , – ().

J.

Machta

, “,” , – ().

Roger

Balian

, “,” , – ().

Harvey S.

Leff

, “” , – ().

J. Willard Gibbs, Collected Works (Yale University Press, New Haven, CT, 1928), Vol. 1, p. 418.

John W. Patterson, “Thermodynamics and evolution,” in Scientists Confront Creationism, edited by Laurie R. Godfrey (Norton, New York, 1983), pp. 99–116.

Duane T. Gish, Creation Scientists Answer their Critics [sic] (Institute for Creation Research, El Cajon, California, 1993). An Appendix contribution by D. R. Boylan seeks to split entropy into the usual entropy which is “due to random effects” and a different sort of entropy related to the “order or information in the system” (p. 429). An even greater error appears in Chap. 6 (on pp. 164 and 175) where Gish claims that scientists must show not that evolution is consistent with the second law of thermodynamics, but that evolution is necessary according to the second law of thermodynamics. The moon provides a counterexample.

Detailed discussion of this N! factor and the related “Gibbs paradox” can be found in

David

 

Hestenes

, “,”  , – ();

Barry M.

 

Casper

and

Susan

 

Freier

, “,”  , – ();

and

Peter D.

Pešić

, “,” , – ().

The Sackur-Tetrode formula (2) predicts that $SKr−SAr=(3/2)kBN ln(mKr/mAr).$ The data in Ihsan Barin, Thermochemical Data of Pure Substances, 3rd ed. (VCH Publishers, New York, 1995), pp. 76 and 924, verify this prediction to 1.4% at 300 K, and to 90 parts per million at 2000 K.

The moral of the paradox is given in the body of this paper. The resolution of the paradox is both deeper and more subtle: It hinges on the fact that the proper home of statistical mechanics is phase space, not configuration space, because Liouville’s theorem implies conservation of volume in phase space, not in configuration space. See Ludwig Boltzmann, Vorlesungen über Gastheorie (J. A. Barth, Leipzig, 1896–98), Part II, Chaps. III and VII [translated into English by Stephen G. Brush: Lectures on Gas Theory (University of California Press, Berkeley, 1964)];

J. Willard Gibbs, Elementary Principles in Statistical Mechanics (C. Scribner’s Sons, New York, 1902), p. 3;

and Richard C. Tolman, The Principles of Statistical Mechanics (Oxford University Press, Oxford, U.K., 1938), pp. 45, 51–52.

P. G. de Gennes, The Physics of Liquid Crystals (Oxford University Press, London, 1974).

J. David

Lister

and

Robert J.

Birgeneau

, “,” , – (May).

D.

Guillon

,

P. E.

Cladis

, and

J.

Stamatoff

, “,” , – ().

The phenomenon of reentrance in general, and particularly in liquid crystals, is reviewed in section 6 of

Shri

Singh

, “,” , – ().

Y.

Yeshurun

,

A. P.

Malozemoff

, and

A.

Shaulov

, “,” , – (). See page 916 and Fig. 4b on page 915.

G. W. Castellan, Physical Chemistry, 2nd ed. (Addison-Wesley, Reading, MA, 1971), p. 330.

See, for example, Max Hansen and Kurt Anderko, Constitution of Binary Alloys (McGraw-Hill, New York, 1958);

David A. Young, Phase Diagrams of the Elements (University of California Press, Berkeley, 1991);

Robert J.

 

Birgeneau

, “,”  , – () (Fig. 4);

C.

 

Lobban

,

J. L.

 

Finney

, and

W. F.

 

Kuhs

, “,”  , – ();

and

I-Ming

Chou

,

J. G.

Blank

,

A. F.

Gohcharov

,

H.-k.

Mao

, and

R. J.

Hemley

, “,” , – ().

The argument of this section was invented by Edward M. Purcell and is summarized in Stephen Jay Gould, Bully for Brontosaurus (W. W. Norton, New York, 1991), pp. 265–268, 260–261.

See, for example, H. Eugene Stanley, Introduction to Phase Transitions and Critical Phenomena (Oxford University Press, New York, 1971);

D. C.

Radulescu

and

D. F.

Styer

, “,” , – (); and Ref. 2.

These computer programs, which work under MS-DOS, are available for free downloading through http://www.oberlin.edu/physics/dstyer/.

The model of Fig. 2 is called the “ideal lattice gas,” while the nearest-neighbor-excluding model of Fig. 3 is called the “hard-square lattice gas.” (These are just two of the infinite number of varieties of the lattice gas model.) Although the entropy of the hard-square lattice gas is clearly less than that of the corresponding ideal lattice gas, it is difficult to calculate the exact entropy for either model. Such values can be found (to high accuracy) by extrapolating power series expansions in the activity z: details and results are given in

D. S.

 

Gaunt

and

M. E.

 

Fisher

, “,”  , – ()

and

R. J.

Baxter

,

I. G.

Enting

, and

S. K.

Tsang

, “,” , – ().

Reference 4 promotes the idea that entropy is a measure of homogeneity. (This despite the everyday observation of two-phase coexistence.) To buttress this argument, the book presents six illustrations (on pp. 54, 72, 74, 75, and 77) of “equilibrium lattice gas configurations.” Each configuration has 100 occupied sites on a $40×40$ grid. If the occupied sites had been selected at random, then the probability of any site being occupied would be 100/1600, and the probability of any given pair of sites both being occupied would be $1/(16)2.$ The array contains $2×39×39$ adjacent site pairs, so the expected number of occupied adjacent pairs would be $2(39/16)2=11.88.$ The actual numbers of occupied nearest-neighbor pairs in the six illustrations are 0, 7, 3, 7, 4, and 3. A similar calculation shows that the expected number of empty rows or columns in a randomly occupied array is $(15/16)10×2×40=41.96.$ The actual numbers for the six illustrations are 28, 5, 5, 4, 4, and 0. I am confident that the sites in these illustrations were not occupied at random, but rather to give the impression of uniformity.

Someone might raise the objection: “Yes, but how many configurations would you have to draw from the pool, on average, before you obtained exactly the special configuration of Fig. 5?” The answer is, “Precisely the same number that you would need to draw, on average, before you obtained exactly the special configuration of Fig. 2.” These two configurations are equally special and equally rare.

In this connection it is worth observing that in the canonical ensemble (where all microstates are “accessible”) the microstate most likely to be occupied is the ground state, and that this is true at any positive temperature, no matter how high. The ground state energy is not the most probable energy, nor is the ground state typical, yet the ground state is the most probable microstate. In specific, even at a temperature of 1 000 000 K, a sample of helium is more likely to be in a particular crystalline microstate than in any particular plasma microstate. However, there are so many more plasma than crystalline microstates that (in the thermodynamic limit) the sample occupies a plasma macrostate with probability 1.

The definition of entropy in Eq. (1) is the best starting place for teaching about entropy, but it holds only for the microcanonical ensemble. The definition Eq. (A1) is harder to understand but is also more general, applying to any ensemble. The two definitions are logically equivalent. See, for example, Richard E. Wilde and Surjit Singh, Statistical Mechanics: Fundamentals and Modern Applications (Wiley, New York, 1998), Sec. 1.6.1.