Traditionally, physical models of associative memory assume conditions of equilibrium. Here, we consider a prototypical oscillator model of associative memory and study how active noise sources that drive the system out of equilibrium, as well as nonlinearities in the interactions between the oscillators, affect the associative memory properties of the system. Our simulations show that pattern retrieval under active noise is more robust to the number of learned patterns and noise intensity than under passive noise. To understand this phenomenon, we analytically derive an effective energy correction due to the temporal correlations of active noise in the limit of short correlation decay time. We find that active noise deepens the energy wells corresponding to the patterns by strengthening the oscillator couplings, where the more nonlinear interactions are preferentially enhanced. Using replica theory, we demonstrate qualitative agreement between this effective picture and the retrieval simulations. Our work suggests that the nonlinearity in the oscillator couplings can improve memory under nonequilibrium conditions.
REFERENCES
1.
N. C.
Keim
, J. D.
Paulsen
, Z.
Zeravcic
, S.
Sastry
, and S. R.
Nagel
, “Memory formation in matter
,” Rev. Mod. Phys.
91
, 035002
(2019
).2.
A. S.
Sokolov
, H.
Abbas
, Y.
Abbas
, and C.
Choi
, “Towards engineering in memristors for emerging memory and neuromorphic computing: A review
,” J. Semicond.
42
, 013101
(2021
).3.
D. J.
Amit
, Modeling Brain Function: The World of Attractor Neural Networks
(Cambridge University Press
, 1989
).4.
K.
Nakano
, “Associatron—A model of associative memory
,” IEEE Trans. Syst., Man, Cybern.
SMC-2
, 380
–388
(1972
).5.
T.
Kohonen
, “Correlation matrix memories
,” IEEE Trans. Comput.
C-21
, 353
–359
(1972
).6.
S.-I.
Amari
, “Learning patterns and pattern sequences by self-organizing nets of threshold elements
,” IEEE Trans. Comput.
C-21
, 1197
–1206
(1972
).7.
J. J.
Hopfield
, “Neural networks and physical systems with emergent collective computational abilities
,” Proc. Natl. Acad. Sci. U. S. A.
79
, 2554
–2558
(1982
).8.
D. J.
Amit
, H.
Gutfreund
, and H.
Sompolinsky
, “Statistical mechanics of neural networks near saturation
,” Ann. Phys.
173
, 30
–67
(1987
).9.
D. J.
Amit
, H.
Gutfreund
, and H.
Sompolinsky
, “Storing infinite numbers of patterns in a spin-glass model of neural networks
,” Phys. Rev. Lett.
55
, 1530
–1533
(1985
).10.
M. V.
Feigelman
and L. B.
Ioffe
, “The statistical properties of the Hopfield model of memory
,” Europhys. Lett.
1
, 197
(1986
).11.
A. C. C.
Coolen
, “Chapter 14. Statistical mechanics of recurrent neural networks I—Statics
,” in Handbook of Biological Physics, Neuro-Informatics and Neural Modelling
, edited by F.
Moss
and S.
Gielen
(North-Holland
, 2001
), Vol. 4
, pp. 553
–618
.12.
F. S.
Gnesotto
, F.
Mura
, J.
Gladrow
, and C. P.
Broedersz
, “Broken detailed balance and non-equilibrium dynamics in living systems: A review
,” Rep. Prog. Phys.
81
, 066601
(2018
).13.
X.
Fang
, K.
Kruse
, T.
Lu
, and J.
Wang
, “Nonequilibrium physics in biology
,” Rev. Mod. Phys.
91
, 045004
(2019
).14.
T.
Vicsek
, A.
Czirók
, E.
Ben-Jacob
, I.
Cohen
, and O.
Shochet
, “Novel type of phase transition in a system of self-driven particles
,” Phys. Rev. Lett.
75
, 1226
–1229
(1995
).15.
J.
Toner
, Y.
Tu
, and S.
Ramaswamy
, “Hydrodynamics and phases of flocks
,” Ann. Phys.
318
, 170
–244
(2005
).16.
F.
Jülicher
, K.
Kruse
, J.
Prost
, and J. F.
Joanny
, “Active behavior of the cytoskeleton
,” Phys. Rep.
449
, 3
–28
(2007
).17.
S.
Banerjee
, M. L.
Gardel
, and U. S.
Schwarz
, “The actin cytoskeleton as an active adaptive material
,” Annu. Rev. Condens. Matter Phys.
11
, 421
–439
(2020
).18.
S.
Ramaswamy
, “The mechanics and statistics of active matter
,” Annu. Rev. Condens. Matter Phys.
1
, 323
–345
(2010
).19.
M. C.
Marchetti
, J. F.
Joanny
, S.
Ramaswamy
, T. B.
Liverpool
, J.
Prost
, M.
Rao
, and R. A.
Simha
, “Hydrodynamics of soft active matter
,” Rev. Mod. Phys.
85
, 1143
–1189
(2013
).20.
C.
Bechinger
, R.
Di Leonardo
, H.
Löwen
, C.
Reichhardt
, G.
Volpe
, and G.
Volpe
, “Active particles in complex and crowded environments
,” Rev. Mod. Phys.
88
, 045006
(2016
).21.
N.
Sepúlveda
, L.
Petitjean
, O.
Cochet
, E.
Grasland-Mongrain
, P.
Silberzan
, and V.
Hakim
, “Collective cell motion in an epithelial sheet can be quantitatively described by a stochastic interacting particle model
,” PLoS Comput. Biol.
9
, e1002944
(2013
).22.
C.
Maggi
, M.
Paoluzzi
, N.
Pellicciotta
, A.
Lepore
, L.
Angelani
, and R.
Di Leonardo
, “Generalized energy equipartition in harmonic oscillators driven by active baths
,” Phys. Rev. Lett.
113
, 238303
(2014
).23.
E.
Fodor
, C.
Nardini
, M. E.
Cates
, J.
Tailleur
, P.
Visco
, and F.
van Wijland
, “How far from equilibrium is active matter?
,” Phys. Rev. Lett.
117
, 038103
(2016
).24.
D.
Martin
, J.
O’Byrne
, M. E.
Cates
, E.
Fodor
, C.
Nardini
, J.
Tailleur
, and F.
van Wijland
, “Statistical mechanics of active Ornstein-Uhlenbeck particles
,” Phys. Rev. E
103
, 032607
(2021
).25.
M. J.
Chacron
, A.
Longtin
, M.
St-Hilaire
, and L.
Maler
, “Suprathreshold stochastic firing dynamics with memory in P-type electroreceptors
,” Phys. Rev. Lett.
85
, 1576
–1579
(2000
).26.
N.
Fourcaud
and N.
Brunel
, “Dynamics of the firing probability of noisy integrate-and-fire neurons
,” Neural Comput.
14
, 2057
–2110
(2002
).27.
A. I.
Selverston
and M.
Moulins
, “Oscillatory neural networks
,” Annu. Rev. Physiol.
47
, 29
–48
(1985
).28.
L. M.
Ward
, “Synchronous neural oscillations and cognitive processes
,” Trends Cognit. Sci.
7
, 553
–559
(2003
).29.
X.-J.
Wang
, “Neurophysiological and computational principles of cortical rhythms in cognition
,” Physiol. Rev.
90
, 1195
–1268
(2010
).30.
H. G.
Schuster
and P.
Wagner
, “A model for neuronal oscillations in the visual cortex
,” Biol. Cybern.
64
, 77
–82
(1990
).31.
A.
Arenas
and C. J. P.
Vicente
, “Phase locking in a network of neural oscillators
,” Europhys. Lett.
26
, 79
(1994
).32.
T.
Fukai
and M.
Shiino
, “Memory encoding by oscillator death
,” Europhys. Lett.
26
, 647
(1994
).33.
T.
Fukai
, “A model of cortical memory processing based on columnar organization
,” Biol. Cybern.
70
, 427
–434
(1994
).34.
K.
Park
and M. Y.
Choi
, “Synchronization in a network of neuronal oscillators with finite storage capacity
,” Phys. Rev. E
52
, 2907
–2911
(1995
).35.
T.
Aonishi
, “Phase transitions of an oscillator neural network with a standard Hebb learning rule
,” Phys. Rev. E
58
, 4865
–4871
(1998
).36.
T.
Aoyagi
and K.
Kitano
, “Retrieval dynamics in oscillator neural networks
,” Neural Comput.
10
, 1527
–1546
(1998
).37.
Y.
Kuramoto
, “Self-entrainment of a population of coupled non-linear oscillators
,” in International Symposium on Mathematical Problems in Theoretical Physics
(Kyoto University; Springer
, Kyoto/Japan
, 1975
), pp. 420
–422
.38.
T.
Nishikawa
, Y.-C.
Lai
, and F. C.
Hoppensteadt
, “Capacity of oscillatory associative-memory networks with error-free retrieval
,” Phys. Rev. Lett.
92
, 108101
(2004
).39.
A.
Maitra
and R.
Voituriez
, “Enhanced orientational ordering induced by an active yet isotropic bath
,” Phys. Rev. Lett.
124
, 048003
(2020
).40.
P.
Jung
and P.
Hänggi
, “Dynamical systems: A unified colored-noise approximation
,” Phys. Rev. A
35
, 4464
–4466
(1987
).41.
C.
Maggi
, U. M. B.
Marconi
, N.
Gnan
, and R.
Di Leonardo
, “Multidimensional stationary probability distribution for interacting active particles
,” Sci. Rep.
5
, 10742
(2015
).42.
M.
Mézard
, G.
Parisi
, and M. A.
Virasoro
, Spin Glass Theory and Beyond: An Introduction to the Replica Method and its Applications
(World Scientific Publishing Company
, 1987
), Vol. 9
.43.
D. O.
Hebb
, The Organization of Behavior: A Neuropsychological Theory
(Psychology Press
, 2005
).44.
A. K.
Behera
, M.
Rao
, S.
Sastry
, and S.
Vaikuntanathan
, “Enhanced associative memory, classification, and learning with active dynamics
,” Phys. Rev. X
13
, 041043
(2023
).45.
D.
Bollé
, T. M.
Nieuwenhuizen
, I. P.
Castillo
, and T.
Verbeiren
, “A spherical Hopfield model
,” J. Phys. A: Math. Gen.
36
, 10269
–10277
(2003
).46.
D.
Krotov
and J. J.
Hopfield
, “Dense associative memory for pattern recognition
,” in Advances in Neural Information Processing Systems
(Curran Associates, Inc.
, 2016
), Vol. 29
.47.
H.
Ramsauer
, B.
Schäfl
, J.
Lehner
, P.
Seidl
, M.
Widrich
, L.
Gruber
, M.
Holzleitner
, T.
Adler
, D.
Kreil
, M.
Kopp
, G.
Klambauer
, J.
Brandstetter
, and S.
Hochreiter
, “Hopfield networks is all you need
,” Int. Conf. Learn. Rep.
(2021
).48.
L.
Albanese
, F.
Alemanno
, A.
Alessandrelli
, and A.
Barra
, “Replica symmetry breaking in dense Hebbian neural networks
,” J. Stat. Phys.
189
, 24
(2022
).49.
S. F.
Edwards
and P. W.
Anderson
, “Theory of spin glasses
,” J. Phys. F: Met. Phys.
5
, 965
(1975
).50.
J.-P.
Naef
and A.
Canning
, “Reetrant spin glass behaviour in the replica symmetric solution of the Hopfield neural network model
,” J. Phys. I
2
, 247
–250
(1992
).51.
A.
Crisanti
, D. J.
Amit
, and H.
Gutfreund
, “Saturation level of the Hopfield model for neural network
,” Europhys. Lett.
2
, 337
(1986
).52.
H.
Steffan
and R.
Kühn
, “Replica symmetry breaking in attractor neural network models
,” Z. Phys. B: Condens. Matter
95
, 249
–260
(1994
).© 2024 Author(s). Published under an exclusive license by AIP Publishing.
2024
Author(s)
You do not currently have access to this content.
