The exceptional capabilities of the human brain provide inspiration for artificially intelligent hardware that mimics both the function and the structure of neurobiology. In particular, the recent development of nanodevices with biomimetic characteristics promises to enable the development of neuromorphic architectures with exceptional computational efficiency. In this work, we propose biomimetic neurons comprised of domain wall-magnetic tunnel junctions that can be integrated into the first trainable CMOS-free recurrent neural network with biomimetic components. This paper demonstrates the computational effectiveness of this system for benchmark tasks and its superior computational efficiency relative to alternative approaches for recurrent neural networks.

Recurrent neural networks (RNNs) are computationally powerful structures that can solve complex tasks that are challenging for conventional von Neumann systems. Their unique problem-solving capabilities have led to their widespread use in areas such as image recognition, classification, and artificial intelligence. However, as datasets become larger and tasks become harder, the cost to run and train software RNNs increases to impractical levels.1,2

An answer to these issues is to redesign networks using analog circuits instead of the current software implementations that are inefficiently mapped to CMOS von Neumann hardware. Inspiration can be drawn from the brain: an analog machine that efficiently solves problems that are challenging for modern computers.3 Research shows that the brain accomplishes this through a diverse system of neurons woven together in a complex web of specialized bio-circuitry that cascades together to solve tasks.3 By emulating neuronal behavior and cytoarchitectonics, efficient circuits can be designed that bolster the capabilities of neural networks.

As conventional CMOS devices do not naturally mimic the physics of the brain, alternative switching elements have been proposed for neuromorphic computing based on emerging device technologies. Hysteretic devices, such as memristors4–9 and spintronic devices,10–17 have received particular interest as hysteresis is a critical component of neurobiological learning. RNNs have been proposed based on a hybrid of memristors and CMOS,18–23 and fully spintronic reservoir computers have also been designed;18,21,24–34 however, the hybrid systems utilize a significant amount of inefficient CMOS, detracting from the benefits of using emerging device technologies, while the spintronic reservoir utilizes no CMOS elements but does not provide the ability to train the weights. Although memristors and other device technologies exhibit physical characteristics that are promising for alternative neural network structures, there remains a need for a CMOS-free trainable RNN system that exploits the biomimetic properties of emerging technologies.

This paper therefore proposes the first CMOS-free trainable RNN with biomimetic components. Leveraging the domain wall-magnetic tunnel junction (DW-MTJ),35–39 this efficient and flexible RNN system is solely composed of spintronic devices and allows trained weights to be written into the system. This system mirrors the function, structure, and asynchronous nature of biological neurons and is able to recover corrupted versions of patterns with high accuracy at higher speeds and with less power than alternative architectures.

The direct implementation of RNNs in efficient hardware circuits requires the interconnection of hysteretic switching elements that can drive other hysteretic switching elements. Unlike CMOS transistors that do not store information in a non-volatile manner, the DW-MTJ can store a non-volatile state with a programmable domain wall position. This non-volatility enables the DW-MTJ memory to achieve low-energy computing in which power is only dissipated when reading/writing a state from/to a DW-MTJ, in contrast to volatile devices or memory cells that also require static power to maintain a state within volatile memory. Moreover, these devices can be easily chained together without the need for external circuitry to translate between differing output and input formats. These properties make them ideal building blocks for hardware RNNs, with neurons connected in a recurrent manner.

An RNN is a neural network architecture that retains a state that evolves over time in response to varying inputs.40 These architectures are commonly used to solve tasks involving classification of spatial data or temporally related data; however, they suffer from high computational costs, especially as the number of neurons increases.41 While actually consisting of a fixed number of interconnected neurons, the evolution of the network’s state can be visualized as a directed graph of arbitrary length, with one time step separating each layer in the graph.

The earliest example of an RNN is the Hopfield network,42 shown in Fig. 1. Hopfield networks can store and remember patterns by functioning as associative memory, an important type of neural computation.43 However, because the network size scales with the data size41 and the network is fully connected, excessive energy is required to calculate updates for large datasets.

FIG. 1.

Illustration of the Hopfield network structure. Each triangle represents the interconnected neurons, and the dashed lines illustrate how the network steps over time, eventually converging to a final state.

FIG. 1.

Illustration of the Hopfield network structure. Each triangle represents the interconnected neurons, and the dashed lines illustrate how the network steps over time, eventually converging to a final state.

Close modal

A Hopfield network can be represented as a fully connected graph of neurons, where each neuron i has an excitation state (Si): ON (1) or OFF (0). Each neuron i also sends an output (Oij) to every other neuron j in the network based on its current state and the synaptic weight (Wij) between the pair of neurons,

Oij=Si×Wij.
(1)

As the network runs forward in time, each neuron’s state is updated based on its inputs according to the following expressions:

Si=1 if j=0N1Oji>θ,Siif j=0N1Oji=θ,0 otherwise,
(2)

where θ is some threshold value. The network can be trained to remember a certain pattern by updating the weights using the following equation:

Wij=(2Si1)(2Sj1),
(3)

where Si is the state of the ith bit of the training pattern, resulting in Wij = ±1.

The DW-MTJ is a spintronic device that has been used to implement non-volatile logic and neuronal and synaptic functions, offering improvements in efficiency and the ability to cascade into large-scale networks.13,15,16,44,45 As shown in Fig. 2, a four-terminal DW-MTJ is composed of an MTJ atop a DW track separated by an electrically insulating layer.45 The top ferromagnet’s magnetization is fixed, while the bottom ferromagnet is coupled with the DW track’s magnetization directly beneath the MTJ through a magnetic insulating layer separating the DW track and the MTJ free layer.46 Experimental demonstration of exchange coupling through a magnetic insulator of thickness ∼1 nm has been reported by growing native oxide on passivated CoFeB.47 The MTJ resistance is larger when the two ferromagnets are in an anti-parallel state relative to a parallel state. Thus, the resistance of the MTJ is directly controlled by the DW’s position in the track.

FIG. 2.

Four terminal DW-MTJ with the physical structure labeled. Input current flows between DWIN and DWOUT, which can cause the DW to move. The free layer (purple) of the MTJ is coupled to the region of the DW track directly below it and, therefore, switches when the DW passes under it. The MTJ resistance depends on the relative orientation of its two magnets, producing an output current through the MTJ between MTJIN and MTJOUT.

FIG. 2.

Four terminal DW-MTJ with the physical structure labeled. Input current flows between DWIN and DWOUT, which can cause the DW to move. The free layer (purple) of the MTJ is coupled to the region of the DW track directly below it and, therefore, switches when the DW passes under it. The MTJ resistance depends on the relative orientation of its two magnets, producing an output current through the MTJ between MTJIN and MTJOUT.

Close modal

An input excitation, such as a magnetic field, spin-transfer torque, or spin–orbit torque, can move the DW along its track length. When the DW passes the MTJ position, the MTJ free ferromagnet magnetically realigns itself in parallel with the updated magnetization state of the DW track segment beneath the MTJ via exchange coupling through the magnetic insulator layer.46 The MTJ thus switches between high and low resistive states.

While in-plane current through the DW track writes the MTJ state via exchange coupling, it is electrically isolated from the MTJ by the magnetic insulator. Additionally, a perpendicular current through the MTJ stack reads the resistive state in a manner similar to reading a two-terminal spin-transfer torque magnetic random-access memory. Therefore, the tunneling magnetoresistance (TMR) of the proposed four-terminal MTJ device is expected to be similar to that of two-terminal MTJ devices.47,48 This TMR depends on the magnetization configuration of the DW-MTJ: if the magnetization direction of both the DW track and MTJ is in-plane, then the TMR can reach 500%;49,50 if the magnetization directions are perpendicular to the plane, then the TMR can reach 300%.51 

This paper proposes the first CMOS-free trainable RNN with biomimetic components, leveraging the hysteretic and analog properties of DW-MTJs13,15,16,44,45 in a novel biomimetic neuron structure comprising multiple DW-MTJs. We further propose that recurrent networks can be realized by interconnecting synaptic outputs to dendritic inputs in fully spintronic RNNs with high computational efficiency.

In neurobiological systems, a neuron continually stores and updates its state based on input excitations and transmits output signals as a function of that state; our biomimetic neuron is analogous both functionally and structurally to biological neurons, as illustrated in Fig. 3. The state of the neuron is represented in its body—hereafter referred to as soma—and updated by integrating the inputs it receives at its dendrite from pre-synaptic neurons. If the soma is sufficiently excited (i.e., its threshold is surpassed), a signal is fired to the several branches of the neuron’s axon. The signal is modified by the synapses at the end of each axon and propagated to the dendrites of post-synaptic neurons.52 

FIG. 3.

A spintronic neuron with two axon branches. Input signals come through the dendrite, flow into the soma, and then into each axon. Each axon fires a synapse-weighted output of ±Wij.

FIG. 3.

A spintronic neuron with two axon branches. Input signals come through the dendrite, flow into the soma, and then into each axon. Each axon fires a synapse-weighted output of ±Wij.

Close modal

1. Dendrite

In biological neurons, the dendrite is a branching structure that receives the synaptic output, represented as voltage differentials, of spatially distributed neurons.53 In our structure, the communication between neurons is represented as current levels. As such, the dendrite is an electrical node connected to each of the pre-synaptic neurons’ synaptic outputs.

2. Soma

In the proposed biomimetic neuron, the soma is a single DW-MTJ that integrates the input from the neuron’s dendrite and sends signals to its axons through its MTJ. The soma’s excitation state is physically determined by the location of the DW with respect to the MTJ—right of the MTJ corresponds to an OFF state and left corresponds to an ON state. Current through the DW track moves the DW in either direction, dependent on the direction of the input current. In an ON state, the magnetization of the free and fixed ferromagnets is parallel across the MTJ, sending a high current signal to the axons. In an OFF state, the magnetic orientations are anti-parallel across the MTJ, sending a smaller signal and relaxing the axons. When the DW crosses the MTJ from the right to the left direction, the neuron fires by increasing its output current value from low to high, thereby pushing the DWs in cascading axons toward the right end of the DW track. Due to the binary nature of MTJs, this switch acts as a step activation function within the RNN.

When it is directly beneath the MTJ, the output signal is in an intermediate state between its high and low signals; to prevent the DW from staying in this intermediate state, the soma of the artificial neuron employs a leaking factor that relaxes the DW toward one edge of the track when no input is applied. Analogous to the method by which biological neurons maintain homeostasis with ion leakage channels to create a leaky integrate-and-fire system, our biomimetic neuron can perform leaking via a dipolar magnetic field, an anisotropy gradient, or a shape gradient.15,16,44 This leaky integrate-and-fire system helps modulate the neuron’s excitation when insufficiently stimulated and serves as a means of temporal coincidence detection between stimuli.54 

3. Axons

All axon branches are DW-MTJs connected to the soma’s MTJ, as shown in Fig. 3. In order for a change in state to propagate between neurons, a soma’s axons must mirror the soma’s state. While a high current causes the soma to pull its axons into the parallel state, a low current does not push the axons’ DWs toward the anti-parallel state. An appropriate threshold leaking force is used instead15,16,44 that outweighs the low current signal, moving the axon’s DW toward an anti-parallel state. In contrast, a high current signal is greater than the leaking force and, thus, pushes the DW toward a high state. In a high state, the axon fully fires its weighted connection toward another neuron. Similar to somas driving axons, the output current of the axon drives cascaded neurons by pushing their soma DWs.

4. Synapse

The synaptic weight voltages on top of the axons function as the synapses as these are where the connection weights are manipulated. A positive voltage value corresponds to an excitatory relation with the post-synaptic neuron, while a negative voltage value corresponds to an inhibitory relation. This is analogous to the brain, where information is stored not in the neurons themselves, but in the synaptic connections between them.55 As such, the synaptic weight is represented by the MTJ voltage and is set externally during training.

These biomimetic neurons can be directly interconnected to form an RNN, without requiring any CMOS or other circuitry. A simple RNN, the Hopfield network, is illustrated in Fig. 4 with all-to-all connectivity. In this case of all-to-all connectivity, every neuron in an N-neuron network has one soma and N-1 axons. In total, an N-neuron Hopfield network consists of N2 DW-MTJ devices.

FIG. 4.

Schematic of a three-neuron spintronic Hopfield network showing neuron interconnection. Ini is the external input to neuron i, VDW is the soma-to-axon voltage, and Wij is the synaptic weight between neurons i and j. The neurons are connected in an all-to-all scheme, affecting each other over time through their weighted connections and eventually converging to a stable state.

FIG. 4.

Schematic of a three-neuron spintronic Hopfield network showing neuron interconnection. Ini is the external input to neuron i, VDW is the soma-to-axon voltage, and Wij is the synaptic weight between neurons i and j. The neurons are connected in an all-to-all scheme, affecting each other over time through their weighted connections and eventually converging to a stable state.

Close modal

Training is performed offline according to (3), with the resulting weights applied as synapse voltages of ±W. For multiple patterns, the formula is applied individually for each, and then, the weight matrices are summed to form the final set.

To perform a computational task on a particular input, the network states can be initialized to an input pattern using voltage sources of constant ±VC. After the network reaches a relaxed state, these sources are disconnected from the network and the synaptic weight voltages are connected to the axons. The network then converges to the nearest trained pattern.

To demonstrate the computational capabilities of this spintronic biomimetic RNN, three distinct tasks were carried out by this network: convergence to random patterns from random states, restoration of distorted image patterns, and determination of the max-cut division of a graph. In all three cases, the network successfully performed the tasks with accuracy and efficiency superior to the best previously proposed RNNs. Our network was found to generate solutions quicker and with less energy than the alternatives technologies.

The RNNs were simulated in Cadence using a SPICE model39,56,57 of the DW-MTJ that incorporates its electric and magnetic behavior with the experimentally realistic device parameters listed in Table I.14,58–64 In order to ensure that the axon mirrors the state of the soma, the value for VDW was selected with the following design rule:

VDW=LSYNRPk(N1)1+T2.
(4)

As network sizes larger than 20 neurons require prohibitively long simulation times, a simplified simulation technique was developed for those larger networks and validated against SPICE simulations. This simplified simulation approach replicated the behavior of the DW-MTJ with electrical node equations within a multi-threaded Java-based application. Identical simulations were conducted in Cadence and the custom Java application, where the DW positions and current at various locations were tracked as a function of time, verifying the accuracy of the custom application.

TABLE I.

Device parameters used for simulations.

SymbolParameterValueUnits
g Landé factor 2.1 ⋯ 
P Polarization constant 0.7 ⋯ 
MSAT Saturation magnetization 8 × 105 A/m 
A Cross-sectional area 50 nm2 
Len DW track length 100 nm 
MTJW MTJ width 20 nm 
MTJP MTJ placement 50 
LSYN Axon leak speed −5 m/s 
LAXON Soma leak speed 0.2 m/s 
T Tunnel magnetoresistance 300% ⋯ 
RP MTJ parallel resistance 500 Ω 
RAP MTJ antiparallel resistance 2000 Ω 
RM Metal layer resistance 2000 Ω 
VC Charge-up voltage 0.1 
W Synaptic weight magnitude 0.1 
VDW Soma to axon voltage Varies with N 
SymbolParameterValueUnits
g Landé factor 2.1 ⋯ 
P Polarization constant 0.7 ⋯ 
MSAT Saturation magnetization 8 × 105 A/m 
A Cross-sectional area 50 nm2 
Len DW track length 100 nm 
MTJW MTJ width 20 nm 
MTJP MTJ placement 50 
LSYN Axon leak speed −5 m/s 
LAXON Soma leak speed 0.2 m/s 
T Tunnel magnetoresistance 300% ⋯ 
RP MTJ parallel resistance 500 Ω 
RAP MTJ antiparallel resistance 2000 Ω 
RM Metal layer resistance 2000 Ω 
VC Charge-up voltage 0.1 
W Synaptic weight magnitude 0.1 
VDW Soma to axon voltage Varies with N 

When evaluating the accuracy of an RNN, it is critical to note that the weight matrix of a Hopfield network is identical when storing a pattern or its inverse. For example, storing the pattern “110” or its inverse “001” results in identical Hopfield networks.42 Therefore, in simulations, an output was considered accurate if it matched the desired pattern or its inverse. With this criterion and the parameters detailed in Table I, our system was found to exhibit exceptional accuracy in both the recall of distorted memories and the calculation of approximate solutions to the max-cut problem.

1. Random pattern recall

In this task, the Hopfield network is programmed to recall random patterns and then provided with random input patterns; the network is considered successful if it converges to any one of the programmed patterns or their inverse. Networks of up to 29 neurons had their accuracy averaged across 1000 test cases with a randomly selected pattern and input. Due to the number of input and test pattern pairs growing with 22n, a random sampling test procedure was used to determine the accuracy across a large number of neurons.

As shown in the results of Fig. 5, the network achieved perfect accuracy for all single-pattern cases. Accurate recall was also achieved when two patterns were programmed into the network, eventually approaching 99% bit accuracy. With three patterns, an average of more than 96% of the bits in a pattern were resolved for all networks larger than 17 neurons.

FIG. 5.

Bitwise convergence accuracy for random pattern recall.

FIG. 5.

Bitwise convergence accuracy for random pattern recall.

Close modal

2. Distorted image recall

One of the many uses of Hopfield networks is the reconstruction of images distorted by noise. As a test case, the three 10 × 10 binary images shown in Fig. 6(a) were programmed into a network. Each image was then subjected to distortion with varying percentages of flipped bits, as illustrated in Fig. 6(b). These distorted images were then fed into the network; the network was considered successful if it converged to the original image. A distortion of 0.5, or flipping half the bits, is the greatest possible amount of distortion, as inverse patterns are intrinsically programmed.

FIG. 6.

(a) Three distinct 10 × 10 binary images programmed into a spintronic Hopfield network. (b) Sample inputs to the network with varying levels of distortion. (c) Bitwise recall accuracy as a function of the distortion level.

FIG. 6.

(a) Three distinct 10 × 10 binary images programmed into a spintronic Hopfield network. (b) Sample inputs to the network with varying levels of distortion. (c) Bitwise recall accuracy as a function of the distortion level.

Close modal

The average accuracy of the spintronic network as a function of distortion level is shown in Fig. 6(c). Perfect recall was achieved for low levels of distortion, and the spintronic network was able to converge to nearly correct patterns even when provided with extremely high levels of distortion. At distortion levels of 0.33 or more, the pattern can become unrecognizable or closer to a different pattern, causing false convergences and lower accuracy. When distortion increases past 0.35, the “T” pattern accuracy increases, while all other letters’ accuracies decrease; due to the nature of this set of images, the network interprets noise as a “T” pattern.

3. Max-cut

The max-cut problem is a well-known NP-complete problem in which the nodes of a simple, undirected graph are divided into two sets such that the weights of the edges between the sets are maximized.65 It is often useful to provide a fast approximate solution.66 

Our biomimetic RNN architecture was used to find approximate max-cut solutions for undirected graphs. Each neuron corresponds to a graph node such that a network requires the same number of neurons as nodes in the graph. The weights between neurons representing disconnected nodes were set to +W, while weights between connected nodes were set to −1.05W; this 5% difference was chosen empirically to introduce asymmetry and maximize accuracy. The initial states of all the neurons’ DW-MTJs were set to the anti-parallel state before release. After network convergence, the set to which each neuron belongs is represented by the state of its DW-MTJ.

Figure 7 shows the average max-cut score of our algorithm normalized by the total number of connections for a 50% connected graph of varying sizes. This is compared against the optimal solution and that of a naive 0.5-approximation algorithm. Across all sizes, our network greatly outperformed the 0.5-approximation algorithm and averaged a 9.02% difference from the optimal solution.

FIG. 7.

Normalized score of the optimal solution, solution generated by a 0.5-approximation algorithm, and a solution generated by the proposed DW-MTJ architecture. An average based on an input of 20 of the same 50%-connected randomly generated graphs were used for each data point.

FIG. 7.

Normalized score of the optimal solution, solution generated by a 0.5-approximation algorithm, and a solution generated by the proposed DW-MTJ architecture. An average based on an input of 20 of the same 50%-connected randomly generated graphs were used for each data point.

Close modal

In addition to accuracy, the efficiency of a computing system is a critical metric of its utility. While the area is important, this study focuses on the time-to-solution, power dissipation, and energy consumption.

The time-to-solution is calculated as the time between when the network starts to load the initial state and when all of its DWs are stationary post-convergence. We have approximated the energy consumption of the network based on the assumption that half of its MTJs are in the parallel state and half are in the anti-parallel state; all power calculations are then performed by dividing the total energy use by the total run time across all trials.

1. Random pattern recall

The single pattern associative memory task achieved a fast time-to-solution: over 100 trials, a 60-neuron spintronic Hopfield network reached an approximate solution after an average of 81.2 ns, with no trial taking longer than 113 ns to converge. The approximate average power dissipation of the network was 44.3 mW, and the average energy consumption was 3.60 nJ; the worst-case energy consumption was 5.23 nJ.

2. Distorted image recall

While similar to the random pattern recall procedure, the binary image distortion task required a 100-neuron spintronic Hopfield network. While the convergence time was slightly faster, the larger network resulted in greater energy consumption. Over 1000 trials restoring the “U,” “T,” and “D” images with the varying distortion levels shown in Fig. 6, the network consumed an average of 11.04 nJ over 78.14 ns.

3. Max-cut

For max-cut simulations, ten standard 50% connected graphs in the Biq Mac Library67 were used as inputs to a 60-neuron spintronic Hopfield network. Across all runs, the spintronic network found a solution after an average of 344 ns. The spintronic network also consumed minimal energy, with an average of 50.6 mW of power dissipation and 17.4 nJ of energy consumption. Table II summarizes the process time, power, and energy consumption of RNNs with different implementations. This task took longer to converge compared to the memory-based ones primarily due to the uniformly anti-parallel starting state of the neurons: the smaller currents associated with the anti-parallel state result in slower DW motion until the neurons begin to flip into the parallel state.

TABLE II.

Max-cut efficiency comparison.

WorkTime-to-solution (ns)Power (mW)Energy (nJ)
Memristor RNN 600 120 72 
SONOS RNN Unknown Unknown 33 
Spintronic RNN 344 50.6 17.4 
WorkTime-to-solution (ns)Power (mW)Energy (nJ)
Memristor RNN 600 120 72 
SONOS RNN Unknown Unknown 33 
Spintronic RNN 344 50.6 17.4 

To evaluate its utility in neural network applications, this spintronic RNN was compared to alternative architectures, leading to the conclusion that the spintronic RNN reaches a solution of similar quality in less time, with less power, and with less energy. Furthermore, we analyzed the impact of the synaptic weight voltage on energy consumption and convergence time, enabling system optimization for maximum efficiency.

A recent study32 described a Hopfield network that used memristors to find solutions to the max-cut problem (alongside a required co-processor, presumably requiring CMOS). In this work, a switch matrix and driver circuit were used to select rows of a memristor array to perform multiplication. The result was then filtered to produce a threshold function and then fed back into the input/output buffers to be loaded in during the next cycle. They estimate that for a network of 60 neurons with 50% connectivity, 50 cycles of the device would require 72 nJ over 600 ns. These values correspond to a probability of converging to the correct solution of between 30% and 40%.

In contrast, our spintronic network does not include cycles and is solely composed of DW-MTJ devices and voltage sources. This allows for a faster convergence time and less energy use. Compared to the quicker of their two configurations of the memristive Hopfield network, the spintronic network converged 74.4% faster. The spintronic network also uses less energy, consuming 24.4% less energy. The spintronic network’s worst-case energy consumption, out of the ten trials, is an outlier at 47.6 nJ, but still lower than the memristive network’s.

Another work has modeled a Hopfield network based on memristive SONOS transistors.23 For the same Biq Mac library and network size, their energy consumption was 33 nJ, which is 89.7% greater than our proposed spintronic network. Their work also requires external circuitry to read the output of each cycle in order to feed it as input to the next cycle. While their work did not specify the cycle time, it mentions a SONOS ramp-up time of 1 ms for a different demonstration. We therefore suspect that our network is significantly faster.

The efficiency of the spintronic network depends on both device parameters and voltage levels. Device variables, such as the MTJ dimensions, metal track resistance, and TMR, depend on manufacturing technology and are difficult to tune. On the other hand, voltage levels in the inputs and weights are easily modified. The synaptic weight magnitude levels can drastically affect power consumption, as illustrated in Fig. 8. In general, increased synaptic weight voltages benefit the network by increasing convergence speed but require greater power consumption. For the parameters tested, the minimum energy-delay product (EDP) is at approximately W = 0.04 V with a value of 0.16 aJ s.

FIG. 8.

Approximate median convergence time and energy consumption of a 60-neuron network with VC = 0.25 V as a function of synaptic weight voltage W. Inset: The energy-delay product (EDP) for this network.

FIG. 8.

Approximate median convergence time and energy consumption of a 60-neuron network with VC = 0.25 V as a function of synaptic weight voltage W. Inset: The energy-delay product (EDP) for this network.

Close modal

This paper presents a biologically inspired, asynchronous, fully spintronic recurrent neural network built solely from DW-MTJs. Neurons are cascaded into a Hopfield network, demonstrating the first biomimetic, CMOS-free RNN with reconfigurable weights. The network successfully performed associative memory and max-cut problems in SPICE simulations, while converging faster and requiring less energy consumption than previously proposed approaches. This spintronic RNN thus makes effective use of biomimicry with spintronic devices to provide a promising approach for artificially intelligent systems with exceptional efficiency.

The authors thank R. Iyer for editing; E. Laws, J. McConnell, N. Nazir, L. Philoon, and C. Simmons for technical support; and H.-J. Drouhin for arranging the research exchange. This research was sponsored, in part, by the National Science Foundation under CCF Award No. 1910800 and the Texas Analog Center of Excellence undergraduate internship program.

The authors have no conflicts to disclose.

P.O.M., C.B.D., and A.T. contributed equally to this work.

Pranav O. Mathews: Conceptualization (equal); Data curation (equal); Formal analysis (equal); Investigation (equal); Methodology (equal); Validation (equal); Writing – original draft (equal). Christian B. Duffee: Conceptualization (equal); Data curation (lead); Formal analysis (equal); Investigation (equal); Methodology (supporting); Visualization (lead); Writing – original draft (lead). Abel Thayil: Conceptualization (lead); Formal analysis (equal); Investigation (equal); Methodology (supporting). Ty E. Stovall: Conceptualization (supporting); Investigation (supporting); Writing – original draft (supporting). Christopher H. Bennett: Conceptualization (supporting); Investigation (supporting); Methodology (supporting). Felipe Garcia-Sanchez: Investigation (supporting); Methodology (supporting); Software (lead). Matthew J. Marinella: Conceptualization (supporting); Methodology (supporting); Project administration (supporting); Resources (supporting). Jean Anne C. Incorvia: Conceptualization (supporting); Methodology (supporting); Resources (supporting); Supervision (supporting). Naimul Hassan: Methodology (equal); Project administration (lead); Supervision (equal); Validation (equal); Writing – review & editing (supporting). Xuan Hu: Project administration (supporting); Supervision (lead). Joseph S. Friedman: Conceptualization (supporting); Funding acquisition (lead); Project administration (supporting); Resources (lead); Supervision (supporting).

The data that support the findings of this study are openly available in DWMTJ-RNN at http://doi.org/10.5281/zenodo.7275757.68 

2.
J.
Keuper
and
F. J.
Preundt
, in
Proceedings of MLHPC 2016: Machine Learning in HPC Environments - Held in Conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis
(
IEEE
,
2017
), Vol. 19, p.
19
.
3.
V.
Balasubramanian
,
Proc. IEEE
103
,
1346
(
2015
).
4.
M.
Prezioso
,
F.
Merrikh-Bayat
,
B. D.
Hoskins
,
G. C.
Adam
,
K. K.
Likharev
, and
D. B.
Strukov
,
Nature
521
,
61
(
2015
).
5.
C. S.
Thakur
,
J. L.
Molin
,
G.
Cauwenberghs
,
G.
Indiveri
,
K.
Kumar
,
N.
Qiao
,
J.
Schemmel
,
R.
Wang
,
E.
Chicca
,
J.
Olson Hasler
 et al,
Front. Neurosci.
12
,
891
(
2018
).
6.
Y.
Li
,
Z.
Wang
,
R.
Midya
,
Q.
Xia
, and
J. J.
Yang
,
J. Phys. D: Appl. Phys.
51
,
503002
(
2018
).
7.
X.
Zhang
,
W.
Wang
,
Q.
Liu
,
X.
Zhao
,
J.
Wei
,
R.
Cao
,
Z.
Yao
,
X.
Zhu
,
F.
Zhang
,
H.
Lv
 et al,
IEEE Electron Device Lett.
39
,
308
(
2017
).
8.
Z.
Wang
,
S.
Joshi
,
S.
Savel’ev
,
W.
Song
,
R.
Midya
,
Y.
Li
,
M.
Rao
,
P.
Yan
,
S.
Asapu
,
Y.
Zhuo
 et al,
Nat. Electron.
1
,
137
(
2018
).
9.
G.
Pedretti
,
V.
Milo
,
S.
Ambrogio
,
R.
Carboni
,
S.
Bianchi
,
A.
Calderoni
,
N.
Ramaswamy
,
A. S.
Spinelli
, and
D.
Ielmini
,
IEEE J. Emerging Sel. Top. Circuits Syst.
8
,
77
(
2017
).
10.
J.
Grollier
,
D.
Querlioz
,
K. Y.
Camsari
,
K.
Everschor-Sitte
,
S.
Fukami
, and
M. D.
Stiles
,
Nat. Electron.
3
,
360
(
2020
).
11.
S.
Jung
,
H.
Lee
,
S.
Myung
,
H.
Kim
,
S. K.
Yoon
,
S.-W.
Kwon
,
Y.
Ju
,
M.
Kim
,
W.
Yi
,
S.
Han
 et al,
Nature
601
,
211
(
2022
).
12.
S. A.
Siddiqui
,
S.
Dutta
,
A.
Tang
,
L.
Liu
,
C. A.
Ross
, and
M. A.
Baldo
,
Nano Lett.
20
,
1033
(
2019
).
13.
N.
Hassan
,
X.
Hu
,
L.
Jiang-Wei
,
W. H.
Brigner
,
O. G.
Akinola
,
F.
Garcia-Sanchez
,
M.
Pasquale
,
C. H.
Bennett
,
J. A. C.
Incorvia
, and
J. S.
Friedman
,
J. Appl. Phys.
124
,
152127
(
2018
).
14.
W. H.
Brigner
,
N.
Hassan
,
X.
Hu
,
C. H.
Bennett
,
F.
Garcia-Sanchez
,
C.
Cui
,
A.
Velasquez
,
M. J.
Marinella
,
J. A. C.
Incorvia
, and
J. S.
Friedman
,
IEEE Trans. Electron Devices
69
,
2353
(
2022
).
15.
W. H.
Brigner
,
X.
Hu
,
N.
Hassan
,
C. H.
Bennett
,
J. A. C.
Incorvia
,
F.
Garcia-Sanchez
, and
J. S.
Friedman
,
IEEE J. Explor. Solid-State Comput. Devices Circuits
5
,
19
(
2019
).
16.
W. H.
Brigner
,
J. S.
Friedman
,
N.
Hassan
,
L.
Jiang-Wei
,
X.
Hu
,
D.
Saha
,
C. H.
Bennett
,
M. J.
Marinella
,
J. A. C.
Incorvia
, and
F.
Garcia-Sanchez
,
IEEE Trans. Electron Devices
66
,
4970
(
2019
).
17.
A.
Kurenkov
,
S.
Fukami
, and
H.
Ohno
,
J. Appl. Phys.
128
,
010902
(
2020
).
18.
A.
Wu
,
S.
Wen
, and
Z.
Zeng
,
Inf. Sci.
183
,
106
(
2012
).
19.
K.
Smagulova
,
K.
Adam
,
O.
Krestinskaya
, and
A. P.
James
, in
2018 IEEE International Conference on Electron Devices and Solid State Circuits (EDSSC)
(
IEEE
,
2018
), pp.
1
2
.
20.
Y.
Long
,
E. M.
Jung
,
J.
Kung
, and
S.
Mukhopadhyay
, in
2016 International Joint Conference on Neural Networks (IJCNN)
(
IEEE
,
2016
), pp.
939
946
.
21.
X.
Guo
,
F.
Merrikh-Bayat
,
L.
Gao
,
B. D.
Hoskins
,
F.
Alibart
,
B.
Linares-Barranco
,
L.
Theogarajan
,
C.
Teuscher
, and
D. B.
Strukov
,
Front. Neurosci.
9
,
488
(
2015
).
22.
G.
Cauwenberghs
,
IEEE Trans. Neural Networks
7
,
346
(
1996
).
23.
S.-I.
Yi
,
S.
Kumar
, and
R. S.
Williams
,
IEEE Trans. Circuits Syst., I
68
,
4970
(
2021
).
24.
P.
Zhou
,
N. R.
McDonald
,
A. J.
Edwards
,
L.
Loomis
,
C. D.
Thiem
, and
J. S.
Friedman
, arXiv:2003.10948 (
2020
).
25.
C.
Mesaritakis
,
M.
Skontranis
,
G.
Sarantoglou
, and
A.
Bogris
, in
Optics InfoBase Conference Papers Part F174-OFC, 2020
,
2020
.
26.
K.
Vandoorne
,
P.
Mechet
,
T.
Van Vaerenbergh
,
M.
Fiers
,
G.
Morthier
,
D.
Verstraeten
,
B.
Schrauwen
,
J.
Dambre
, and
P.
Bienstman
,
Nat. Commun.
5
,
3541
(
2014
).
27.
S.
Ganguly
,
K. Y.
Camsari
, and
A. W.
Ghosh
, arXiv:1709.10211 (
2017
).
28.
J.
Cao
and
R.
Li
,
Sci. China Inf. Sci.
60
,
032201
(
2017
).
29.
K.
Smagulova
,
O.
Krestinskaya
, and
A. P.
James
,
Analog Integr. Circuits Signal Process.
95
,
467
(
2018
).
30.
X.
Liu
,
Z.
Zeng
, and
D. C.
Wunsch
 II
,
Neural Networks
131
,
300
(
2020
).
31.
C.
Li
,
Z.
Wang
,
M.
Rao
,
D.
Belkin
,
W.
Song
,
H.
Jiang
,
P.
Yan
,
Y.
Li
,
P.
Lin
,
M.
Hu
,
N.
Ge
,
J. P.
Strachan
,
M.
Barnell
,
Q.
Wu
,
R. S.
Williams
,
J. J.
Yang
, and
Q.
Xia
,
Nat. Mach. Intell.
1
,
49
(
2019
).
32.
F.
Cai
,
S.
Kumar
,
T.
Van Vaerenbergh
,
X.
Sheng
,
R.
Liu
,
C.
Li
,
Z.
Liu
,
M.
Foltin
,
S.
Yu
,
Q.
Xia
,
J. J.
Yang
,
R.
Beausoleil
,
W. D.
Lu
, and
J. P.
Strachan
,
Nat. Electron.
3
,
409
(
2020
).
33.
A. R.
Bevi
,
P.
Monurajan
, and
J.
Manjula
,
Neural Comput. Appl.
32
,
9443
(
2020
).
34.
S.
Duan
,
Z.
Dong
,
X.
Hu
,
L.
Wang
, and
H.
Li
,
Neural Comput. Appl.
27
,
837
(
2016
).
35.
J. A.
Currivan-Incorvia
,
S.
Siddiqui
,
S.
Dutta
,
E. R.
Evarts
,
J.
Zhang
,
D.
Bono
,
C.
Ross
, and
M.
Baldo
,
Nat. Commun.
7
,
10275
(
2016
).
36.
M.
Alamdar
,
T.
Leonard
,
C.
Cui
,
B. P.
Rimal
,
L.
Xue
,
O. G.
Akinola
,
T.
Patrick Xiao
,
J. S.
Friedman
,
C. H.
Bennett
,
M. J.
Marinella
, and
J. A. C.
Incorvia
,
Appl. Phys. Lett.
118
,
112401
(
2021
).
37.
S.
Liu
,
T. P.
Xiao
,
C.
Cui
,
J. A. C.
Incorvia
,
C. H.
Bennett
, and
M. J.
Marinella
,
Appl. Phys. Lett.
118
,
202405
(
2021
).
38.
T.
Shibata
,
T.
Shinohara
,
T.
Ashida
,
M.
Ohta
,
K.
Ito
,
S.
Yamada
,
Y.
Terasaki
, and
T.
Sasaki
,
Appl. Phys. Express
13
,
043004
(
2020
).
39.
X.
Hu
,
A.
Timm
,
W. H.
Brigner
,
J. A. C.
Incorvia
, and
J. S.
Friedman
,
IEEE Trans. Electron Devices
66
,
2817
(
2019
).
40.
Z. C.
Lipton
,
J.
Berkowitz
, and
C.
Elkan
, arXiv:1506.00019 (
2015
).
41.
P. Orponen, 6.
42.
J. J.
Hopfield
,
Proc. Natl. Acad. Sci. U. S. A.
79
,
2554
(
1982
).
43.
Z.
Yu
,
A. M.
Abdulghani
,
A.
Zahid
,
H.
Heidari
,
M. A.
Imran
, and
Q. H.
Abbasi
,
IEEE Access
8
,
67085
(
2020
).
44.
W. H.
Brigner
,
X.
Hu
,
N.
Hassan
,
L.
Jiang-Wei
,
C. H.
Bennett
,
F.
Garcia-Sanchez
,
O.
Akinola
,
M.
Pasquale
,
M. J.
Marinella
,
J. A. C.
Incorvia
, and
J. S.
Friedman
,
SPIN
10
,
2040003
(
2020
).
45.
W. H.
Brigner
,
N.
Hassan
,
X.
Hu
,
C. H.
Bennett
,
F.
Garcia-Sanchez
,
M. J.
Marinella
,
J. A. C.
Incorvia
, and
J. S.
Friedman
, “
CMOS-free multilayer perceptron enabled by four-terminal MTJ device
,” arXiv:2002.00862 (
2020
).
46.
D.
Morris
,
D.
Bromberg
,
J.-G.
Zhu
, and
L.
Pileggi
, in
Proceedings of the 49th Annual Design Automation Conference
(
ACM
,
2012
), pp.
486
491
.
47.
D.
Bromberg
,
M.
Moneck
,
V.
Sokalski
,
J.
Zhu
,
L.
Pileggi
, and
J.-G.
Zhu
, in
2014 IEEE International Electron Devices Meeting
(
IEEE
,
2014
), pp.
33
41
.
48.
Y.
Shao
,
V.
Lopez-Dominguez
,
N.
Davila
,
Q.
Sun
,
N.
Kioussis
,
J. A.
Katine
, and
P.
Khalili Amiri
,
Commun. Mater.
3
,
87
(
2022
).
49.
Y. M.
Lee
,
J.
Hayakawa
,
S.
Ikeda
,
F.
Matsukura
, and
H.
Ohno
,
Appl. Phys. Lett.
90
,
212507
(
2007
).
50.
K.
Kunimatsu
,
T.
Tsuchiya
,
T.
Roy
,
K.
Elphick
,
T.
Ichinose
,
M.
Tsujikawa
,
A.
Hirohata
,
M.
Shirai
, and
S.
Mizukami
,
Appl. Phys. Express
13
,
083007
(
2020
).
51.
S. S. P.
Parkin
,
C.
Kaiser
,
A.
Panchula
,
P. M.
Rice
,
B.
Hughes
,
M.
Samant
, and
S.-H.
Yang
,
Nat. Mater.
3
,
862
(
2004
).
52.

While the other terms closely match their biological counterparts in both form and function, soma has been chosen here for lack of a more appropriate term.

53.
J.
Hawkins
and
S.
Ahmad
,
Front. Neural Circuits
10
,
23
(
2016
).
54.
T. P.
Trappenberg
,
Fundamentals of Computational Neuroscience
, 2nd ed. (
Oxford University Press
,
2010
).
55.
56.
T. P.
Xiao
,
M. J.
Marinella
,
C. H.
Bennett
,
X.
Hu
,
B.
Feinberg
,
R.
Jacobs-Gedrim
,
S.
Agarwal
,
J. S.
Brunhaver
,
J. S.
Friedman
, and
J. A. C.
Incorvia
,
IEEE J. Explor. Solid-State Comput. Devices Circuits
5
,
188
(
2019
).
57.
X.
Hu
,
A. J.
Edwards
,
T. P.
Xiao
,
C. H.
Bennett
,
J. A. C.
Incorvia
,
M. J.
Marinella
, and
J. S.
Friedman
,
2020 IEEE International Symposium on Circuits and Systems (ISCAS)
2020, (
2020
).
58.
J.
Ben Youssef
,
N.
Vukadinovic
,
D.
Billet
, and
M.
Labrune
,
Phys. Rev. B
69
,
174402
(
2004
).
59.
E.
Landi Degl’Innocenti
,
Solar Phys.
77
,
285
(
1982
).
60.
L.
Bainsla
,
K. G.
Suresh
,
A. K.
Nigam
,
M.
Manivel Raja
,
B. S. D. C. S.
Varaprasad
,
Y. K.
Takahashi
, and
K.
Hono
,
J. Appl. Phys.
116
,
203902
(
2014
).
61.
M.
Gajek
,
J. J.
Nowak
,
J. Z.
Sun
,
P. L.
Trouilloud
,
E. J.
O’Sullivan
,
D. W.
Abraham
,
M. C.
Gaidis
,
G.
Hu
,
S.
Brown
,
Y.
Zhu
 et al,
Appl. Phys. Lett.
100
,
132408
(
2012
).
62.
V.
Krizakova
,
M.
Perumkunnil
,
S.
Couet
,
P.
Gambardella
, and
K.
Garello
,
J. Magn. Magn. Mater.
562
,
169692
(
2022
).
63.
J. H.
Franken
,
H. J. M.
Swagten
, and
B.
Koopmans
,
Nat. Nanotechnol.
7
,
499
(
2012
).
64.
Y.
Takeuchi
,
C.
Zhang
,
A.
Okada
,
H.
Sato
,
S.
Fukami
, and
H.
Ohno
,
Appl. Phys. Lett.
112
,
192408
(
2018
).
65.
L.-Y.
Wu
,
X.-S.
Zhang
, and
J.-L.
Zhang
, “
Application of discrete Hopfield-type neural network for max-cut problems
,”
Proceedings of ICONIP
1439
1444
(
2001
).
66.
M. X.
Goemans
and
D. P.
Williamson
,
J. ACM
42
,
1115
(
1995
).
67.
A.
Wiegele
, “
Biq Mac Library
,” http://biqmac.uni-klu.ac.at/biqmaclib.html (online),
2007
.
68.
P. O.
Mathews
,
C. B.
Duffee
,
A.
Thayil
,
T. E.
Stovall
,
C. H.
Bennett
,
F. G.-S.
Garcia-Sanchez
,
M. J.
Marinella
,
J. A. C.
Incorvia
,
N.
Hassan
,
X.
Hu
, and
J. S.
Friedman
(
2022
). “
DWMTJ-RNN
,” Zenodo. 10.5281/zenodo.7275757