In this paper we introduce and exploit the real replica approach for a minimal generalization of the Hopfield model by assuming the learned patterns to be distributed according to a standard unit Gaussian. We consider the high storage case, when the number of patterns linearly diverges with the number of neurons. We study the infinite volume behavior of the normalized momenta of the partition function. We find a region in the parameter space where the free energy density in the infinite volume limit self-averages around its annealed approximation, as well as the entropy and the internal energy density. Moreover, we evaluate the corrections to their extensive counterparts with respect to their annealed expressions. The fluctuations of properly introduced overlaps, which act as order parameters, are also discussed.
REFERENCES
1.
Albeverio
, S.
, Tirozzi
, B.
, and Zegarlinski
, B.
, “Rigorous results for the free energy in the Hopfield model
,” Commun. Math. Phys.
150
, 337
(1992
).2.
Amit
, D. J.
, Modeling Brain Function: The World of Attractor Neural Network
(Cambridge University Press
, Cambridge
, 1992
).3.
Amit
, D. J.
, Gutfreund
, H.
, and Sompolinsky
, H.
, “Spin glass model of neural networks
,” Phys. Rev. A
32
, 1007
(1985
).4.
Amit
, D. J.
, Gutfreund
, H.
, and Sompolinsky
, H.
, “Storing infinite numbers of patterns in a spin glass model of neural networks
,” Phys. Rev. Lett.
55
, 1530
(1985
).5.
Barra
, A.
, “Irreducible free energy expansion and overlap locking in mean field spin glasses
,” J. Stat. Phys.
123
, 601
(2006
).6.
Barra
, A.
, “The mean field Ising model through interpolating techniques
,” J. Stat. Phys.
132
, 787
(2008
).7.
Bernacchia
, A.
and Amit
, D. J.
, “Impact of spatiotemporally correlated images on the structure of memory
,” Proc. Natl. Acad. Sci. U.S.A.
104
, 3544
(2007
).8.
Bernacchia
, A.
and Naveau
, P.
, “Detecting spatial patterns with the cumulant function: the theory
,” Nonlinear Processes Geophys.
15
, 159
(2008
).9.
Bovier
, A.
and Niederhauser
, B.
, “The spin-glass phase-transition in the Hopfield model with p-spin interactions
,” Adv. Theor. Math. Phys.
5
, 1001
(2001
).10.
Bovier
, A.
, van Enter
, A. C. D.
, and Niederhauser
, B.
, “Stochastic symmetry-breaking in a Gaussian Hopfield-model
,” J. Stat. Phys.
95
, 181
(1999
).11.
Bovier
, A.
and Gayrard
, V.
, “An almost sure central limit theorem for the Hopfield model
,” Markov Processes Relat. Fields
3
, 151
(1997
).12.
Bovier
, A.
, “Self-averaging in a class of generalized Hopfield models
,” J. Phys. A
27
, 7069
(1994
).13.
Akhiezer
, N. I.
, The Classical Moment Problem and Some Related Questions in Analysis
(Oliver-Boyd
, Edinburgh
, 1965
).14.
Comets
, F.
and Neveu
, J.
, “The Sherrington-Kirkpatrick model of spin glasses and stochastic calculus: the high temperature case
,” Commun. Math. Phys.
166
, 549
(1995
).15.
Coolen
, A. C. C.
, Kuehn
, R.
, and Sollich
, P.
, Theory of Neural Information Processing Systems
(Oxford University Press
, Oxford
, 2005
).16.
Ellis
, R. S.
, Large Deviations and Statistical Mechanics
(Springer
, New York
, 1985
).17.
Engel
, A.
and Van den Broeck
, C.
, Statistical Mechanics of Learning
(Cambridge University Press
, Cambridge
, 2001
).18.
Guerra
, F.
, in Mathematical Statistical Physics
, edited by A.
Bovier
, F.
Dunlop
, F.
den Hollander
, A.
van Enter
, and J.
Dalibard
(Elsevier
, Oxford
, 2006
), pp. 243
–271
.19.
Guerra
, F.
, “Broken replica symmetry bounds in the mean field spin glass model
,” Commun. Math. Phys.
233
, 1
(2003
).20.
Guerra
, F.
, “About the overlap distribution in mean field spin glass models
,” Int. J. Mod. Phys. B
10
, 1675
(1996
).21.
Guerra
, F.
, in Mathematical Physics in Mathematics and Physics: Quantum and Operator Algebraic Aspects
, Fields Institute Communications
Vol. 30
(American Mathematical Society
, Providence
, 2001
).22.
Guerra
, F.
and Toninelli
, F. L.
, “The Thermodynamic Limit in Mean Field Spin Glass Models
,” Commun. Math. Phys.
230
, 71
(2002
).23.
Guerra
, F.
and Toninelli
, F. L.
, “The high temperature region of the Viana-Bray diluted spin glass model
,” J. Stat. Phys.
115
, 531
(2004
).24.
Guerra
, F.
and Toninelli
, F. L.
, “The infinite volume limit in generalized mean field disordered models
,” Markov Processes Relat. Fields
9
, 195
(2003
).25.
26.
Artificial Intelligence and Neural Networks: Steps Toward Principled Integration
, edited by V.
Honavar
and L.
Uhr
(Academic
, Boston
, 1994
).27.
Hertz
, J.
, Krogh
, A.
, and Palmer
, R.
, Introduction to the Theory of Neural Computation
, Santa Fe Institute Studies in the Sciences of Complexity
(Addison-Wesley Longman Publishing Co., Inc.
, Boston, MA
, 1991
).28.
Hopfield
, J. J.
, “Neural networks and physical systems with emergent collective computational abilities
,” Proc. Natl. Acad. Sci. U.S.A.
79
, 2554
(1982
).29.
Krein
, M.
and Nudelman
, A.
, The Markov Moment Problem and Extremal Problems: Ideas and Problems of P. L. Chebyshev and A. A. Markov and Their Further Development
(American Mathematical Society
, Providence
, 1977
), Vol. 50
.30.
Mézard
, M.
, Parisi
, G.
, and Virasoro
, M. A.
, Spin Glass Theory and Beyond
(World Scientific
, Singapore
, 1987
).31.
Pastur
, L.
and Shcherbina
, M.
, “The absence of self-averaging of the order parameter in the Sherrington-Kirkpatrick model
,” J. Stat. Phys.
62
, 1
(1991
).32.
Pastur
, L.
, Scherbina
, M.
, and Tirozzi
, B.
, “The replica symmetric solution of the Hopfield model without replica trick
,” J. Stat. Phys.
74
, 1161
(1994
).33.
Pastur
, L.
, Scherbina
, M.
, and Tirozzi
, B.
, “On the replica symmetric equations for the Hopfield model
,” J. Math. Phys.
40
, 3930
(1999
).34.
Talagrand
, M.
, “Rigourous results for the Hopfield model with many patterns
,” Probab. Theory Relat. Fields
110
, 177
(1998
).35.
Talagrand
, M.
, “Exponential inequalities and convergence of moments in the replica-symmetric regime of the Hopfield model
,” Ann. Probab.
28
, 1393
(2000
).36.
Talagrand
, M.
, Spin Glasses: A Challenge for Mathematicians: Cavity and Mean Field Models
(Springer
, Berlin
, 2003
).37.
Aizenman
, M.
, Lebowitz
, J.
, and Ruelle
, D.
, “Some rigorous results on the Sherrington-Kirkpatrick model of spin glasses
,” Commun. Math. Phys.
112
, 3
(1987
).© 2008 American Institute of Physics.
2008
American Institute of Physics
You do not currently have access to this content.
