The relativistic Hopfield model constitutes a generalization of the standard Hopfield model that is derived by the formal analogy between the statistical-mechanic framework embedding neural networks and the Lagrangian mechanics describing a fictitious single-particle motion in the space of the tuneable parameters of the network itself. In this analogy, the cost-function of the Hopfield model plays as the standard kinetic-energy term and its related Mattis overlap (naturally bounded by one) plays as the velocity. The Hamiltonian of the relativisitc model, once Taylor-expanded, results in a p-spin series with alternate signs: the attractive contributions enhance the information-storage capabilities of the network, while the repulsive contributions allow for an easier unlearning of spurious states, conferring overall more robustness to the system as a whole. Here, we do not deepen the information processing skills of this generalized Hopfield network, rather we focus on its statistical mechanical foundation. In particular, relying on Guerra’s interpolation techniques, we prove the existence of the infinite-volume limit for the model free-energy and we give its explicit expression in terms of the Mattis overlaps. By extremizing the free energy over the latter, we get the generalized self-consistent equations for these overlaps as well as a picture of criticality that is further corroborated by a fluctuation analysis. These findings are in full agreement with the available previous results.
REFERENCES
Notice that, once the existence of the thermodynamic limit is proved for the free energy, it holds, in a cascade fashion, also for other various quantities of interest, as entropy and internal energy.22
It is worth mentioning that biased patterns (e.g., where positive entries are more likely), beyond being less informative, may also lead to technical issues as spurious configurations would present large overlap with pure memories, so the attracting power of the latter is strongly downsized, see also Ref. 7.
For instance, information can be supplied to the network as an external field to keep a physical jargon.
Actually, we are committing a small abuse of notation as the “standard” free energy is defined as , but, as from the mathematical side and α(β) are (a constant apart) equivalent, we will keep our choice as it makes calculations more transparent, obviously, without affecting the results.
This assumption crucially allows us to use the Wick Theorem, namely, given a Gaussian variable z we can write .
