Spike time dependent plasticity (STDP) enabled learning in spiking neural networks using domain wall based synapses and neurons

Implementing Neural Network (NN) algorithms in specialized neuromorphic hardware, including spintronic hardware, has been a heavily pursued topic of research in recent years.^1–9 Spiking Neural Network (SNN) algorithms are of particular interest in this regard. Power consumption is considered to be very low in hardware implementation of SNN because computation is event based.^10–15 While some SNN implementations involve training a non-spiking NN first and then converting it to SNN^16,17 some other implementations attempt at training the SNN hardware itself using Spike Time Dependent Plasticity (STDP) property of synapses and homoeostasis property of neurons.^18–20 Local and unsupervised nature of such STDP and homoeostasis based learning, coupled with biological plausibility of these mechanisms,^18,22–24 makes such training/learning in hardware SNN very interesting.

Spintronic devices have earlier been proposed as neurons and synapses in such SNN.^19,25–27 In this paper, we have designed and simulated a hardware SNN using spintronic devices (Domain Wall (DW) based devices) as synapses as well as neurons in the same network. We have showed STDP and homoeostasis enabled learning in our designed SNN. In Section II and III, we carry out micromagnetic simulation²⁸ to model the DW devices. Transistor based analog circuits are designed on SPICE circuit simulator that will apply the required current pulses into DW devices to make them function as synapses and neurons.^19–21 In Section IV, we incorporate the neuron and synapse characteristics in a SNN we design. We train the SNN on a popular Machine Learning (ML) dataset (Iris dataset²⁹) both by a completely unsupervised mechanism, enabled by STDP at synapses and homoeostasis at output stage neurons,¹⁸ and a partially supervised mechanism, enabled by STDP at synapses (unsupervised local learning rule) and supervised inhibitory current at the output stage neurons.^30–32 Section V concludes the paper.

Both DW synapse and DW neuron (designed in the next section) utilize the physics of Spin Orbit Torque (SOT) driven DW motion in Heavy Metal (HM)/FerroMagnet (FM) heterostructures. The physics has been studied extensively in the past through experiments and micromagnetic simulations.^33–36 When in-plane current flows through HM layer, DW in the FM layer above it experiences SOT. If the wall has Néel chirality due to Dzyaloshinskii Moriya Interaction (DMI) at the interface, then DW motion can happen even in the absence of magnetic field^33,34 (Fig. 1).

We carry out micromagnetic simulations to model DW motion in two different hetero-structures- Pt (HM)/CoFe (FM)/MgO (for synapse design) and Ta (HM)/CoFe (FM)/MgO (for neuron design). The dynamics of the magnetic moments of the FM layer (in which DW moves), under the influence of vertical spin current injected into it due to in-plane charge current through the heavy metal layer below, is simulated for either system using micromagnetic package “muma×3.”²⁸ Simulation parameters used are from miromagnetic models in previous works,^33,37–39 that are used to benchmark experimental data on DW motion in such systems (See Section 1 of supplementary material). DW velocity is plotted as function of in-plane charge current density for both the systems in Fig. 2. For both Pt/CoFe/MgO and Ta/CoFe/MgO systems, DW velocity increases linearly with current density within a certain range (Fig. 2). However, for the same current density the velocity is much lower for Ta/CoFe/MgO compared to Pt/CoFe/MgO because the magnitude of DMI is much lower for the Ta/CoFe/MgO system (0.06 mJ/m²) than that for the Pt/CoFe/MgO system (1.2 mJ/m²).^37–39 

Utilizing the physics of DW motion, a Magnetic Tunnel Junction (MTJ) device has been proposed as a synapse in analog hardware NN.^7–9 The FM layer in which the DW moves is the free layer of the MTJ device. As the in-plane current flowing through the HM layer below the free layer moves the DW, average magnetization inside the free layer changes and hence conductance of the MTJ changes. Change in conductance (ΔG) of the synaptic device is related to the change in weight the synapse stores (Δw) as follows:

where G_max and G_min are the maximum and minimum conductances of the MTJ. Pt/CoFe/MgO system is chosen for DW synapse.^7–9 Lateral dimensions of the synaptic DW device simulated here are 500 by 50 nm. Based on the values of R-A product and Tunneling Magneto-Resistance (TMR) of the MTJ,^40,41G_max = 2.9 × 10⁻³ Ω⁻¹, G_min = 6.1 × 10⁻³ Ω⁻¹ (Fig. 3). w_max and w_min are the maximum and minimum values that the weight of any synapse in the SNN can take. Antiferromagnetic regions at the edges prevent the DW from getting destroyed.^7,9

Since velocity of the DW is proportional to the current density, when in-plane current pulse of a fixed duration (3 ns), also known as “write” current pulse, is applied on the device, conductance of the MTJ changes by a magnitude proportional to that of the current pulse (Fig. 3). Hence magnitude of the “write” current pulse (I_write) needed to bring about a certain change in conductance (ΔG) is given as follows:

where $∂Iwrite∂G$ = the slope of straight line that fits the conductance vs write current characteristic of the DW device (Fig. 3) = 1.63 × 10⁵μA −Ω.

In the SNN we design in Section IV, a neuron of the input layer (pre-neuron) is connected to a neuron of the output layer (post-neuron) through a synapse and the weight of the synapse is updated by the STDP rule:²² 

where Δw is the change in weight of the synapse, t_pre is the time when the pre-neuron spikes, t_post is the time when the post-neuron spikes, Γ is a dimensionless constant of proportionality equal to 7, and τ₁ and τ₂ are the two time constants for the synapse.

From equation (1), (2) and (3), write current I_write needs to be applied on the DW synapse as follows in order to update the weight and train the SNN:

where I₀ ≈ 5μA (from equation (1) and (2)). For that purpose, the DW synapse is integrated with a transistor based circuit operating in sub-threshold regime, which injects appropriate “write” current pulse into DW synapse, as shown in Fig. 4, Fig. 5 and Fig. 6 ^19,42

I_write in equation (4) can be considered a sum of two components: I_write = I_write,1 + I_write,2 where

In the circuit of Fig. 4, drain current through transistor T4 corresponds to I_write,1 and drain current through T8 corresponds to I_write,2. Based on SPICE simulations of the circuit in Fig. 4 we carry out on Cadence Vurtuoso simulator, we plot I_write,1 and I_write,2 as a function of time for spiking pattern 1 (Fig. 5) and spiking pattern 2 (Fig. 6). Circuit parameters are chosen such that τ₁ = τ₂ = 1.5 μs for all synapses in our designed SNN. For spiking pattern 1, pre-neuron spikes once (t_pre) followed by several post-neuron spikes (t_post). Hence t_post > t_pre here. Since pre-neuron spike corresponds to a sharp drop in gate voltage of T2 followed by its rise to original value (Fig. 5(a)), T2 being pMOS first turns on resetting voltage across capacitor C1 to 0.8 V (lower electrode of C1 in Fig. 4 considered positive terminal) and then turns off. For all time t after t_pre C1 is charged through T1. So voltage across C1, and hence gate voltage of T3, increases linearly with t − t_pre with a slope inversely proportional to value of capacitance C1 (Fig. 5(c)). When post-neuron spikes at t_post (sharp rise in gate voltage of transistor T4 turning on T4 - Fig. 5(b)), since T3 is designed to operate in sub-threshold region,⁴³ drain current through T3 and hence T4 (I_write,1) is proportional to $e−(tpost−tpreτ1)$ for spiking pattern 1 since t_post > t_pre (Fig. 5(d)). The STDP time constant τ₁ is directly proportional to sub-threshold swing of T3 and capacitance C1. Unlike C1, charging/discharging does not happen for C2 for spiking pattern 1 (Fig. 5(c)). This is because the equivalent of T2 for C2 (T6) is connected to post-neuron spike instead of pre-neuron spike and the equivalent of T4 for C2 (T8) is connected to pre-neuron spike instead of post-neuron spike. So I_write,2=0 for spiking pattern 1 since t_post > t_pre, as desired. For spiking pattern 2 (Fig. 6), post-neuron spikes once (Fig. 6(b)) and pre-neuron spikes several times after that (Fig. 6(a)). Hence t_post < t_pre here. In this case, voltage across C2 is reset after the post-neuron spike and then it drops linearly with t − t_post due to discharge through T5, with a slope inversely proportional to value of capacitance C2 (Fig. 6(c)). T7 operates in sub-threshold region here and T8 is pMOS transistor here as opposed to T4 (nMOS). Hence when pre-neuron spikes at t_pre T8 turns on and current through T8 (I_write,2) is proportional to $e−(tpre−tpostτ2)$ and has a negative sign (Fig. 6(d)). Thus, τ₂ is now linearly proportional to sub-threshold swing of T7 and value of capacitance C2. Following similar reasoning as spiking pattern 1, for spiking pattern 2 (t_post < t_pre) I_write,1 =0 (Fig. 6(d)). Thus the STDP rule (equation (3)) is implemented for our DW synapse.

The waveforms in Fig. 5 and Fig. 6 are for specific values of C1 and C2 in the circuit of Fig. 4 which lead to τ₁ = τ₂ = 1.5 μs. However dependence of τ₁ on C1 and τ₂ on C2, as obtained from multiple SPICE simulations of the STDP circuit in Fig. 4 for different values of C1 and C2, are plotted in Section 4 of supplementary material. Variation in values of C1 and C2 due to circuit imperfections leads to variation in values of τ₁ and τ₂. However if τ₁ and τ₂ lie within a certain range the designed SNN can be accurately trained on the given datasets. We discuss this in Section IV where we present the performance metrics for learning/training using our designed SNN.

In the Leaky Integrate Fire (LIF) model of neuron,^18,23 its membrane potential v(t) is governed by the following equation:

where I(t) is input current to the neuron, G_L is membrane conductance, E_L is resting potential of neuron. Once v(t) reaches the threshold potential (V_th), the neuron generates a spike and v(t) drops to E_L. For our designed SNN in next section, we take G_L = 30 pS, E_L = −70 mV, C = 1200 pF and V_th = 20 mV for the LIF model of our neurons. Time between two consecutive spikes generated by the neuron is plotted as a function of input dc current to the neuron after solving equation (6) for the given parameters (Fig. 7(a)). Our choice of LIF parameters is such that the time between two consecutive spikes of the neuron is in the order of the time constant of the STDP synapse (1.5 μs) and hence several orders higher than duration of the “write” current pulse into the DW synapse, which is same as the duration of each spike in our designed SNN (3 ns). This is a requirement for STDP based learning to work (Fig. 5, Fig. 6).

The DW neuron, integrated with a transistor based circuit, we design in Fig. 7(b) satisfies the desired LIF characteristic described above (Fig. 7(a)). Working principle of DW neuron is as follows: input current flowing through HM layer moves the DW in the FM layer from one end of the device to another, much like the DW synapse. However unlike the DW synapse, the MTJ is located only at the other end. When the DW reaches the other end, TMR of the MTJ changes abruptly and spike is generated.²¹ Thus time period between two consecutive spikes is equal to the time taken by the DW to traverse the length of the device (Fig. 7(a)). Since this time period must be several orders higher than the duration of “write” current pulse for DW synapse (3 ns), which is equal to the time taken by the DW to traverse the length of the synapse, the length of the DW neuron (6 μm) is chosen to be much higher than that for the DW synapse (500 nm). Width is also chosen to be higher for the DW neuron (600 nm) compared to the DW synapse (50 nm) because for the same magnitude of input current, higher width corresponds to smaller current density and hence lower velocity of DW. Ta/CoFe/MgO system is chosen for DW neuron instead of Pt/CoFe/MgO since DW velocity is found to be lower in Ta/CoFe/MgO than Pt/CoFe/MgO for the same current density (Fig. 2). The MTJ in the DW neuron has dimensions of 600 nm by 100 nm.

Once the DW reaches the other end, TMR of the MTJ increases due to switching of moment in the free layer. As a result, gate voltage of the transistor T1 in Fig. 7(b) increases, turning it on. This gate voltage is the output of the neuron (V_out) if it is a post-neuron since the spike required for post-neuron is positive (Fig. 5(b), Fig. 6(b)). For pre-neuron another two transistors based standard inverter circuit⁴³ is connected to the gate voltage of this transistor T1 (Section 3 of supplementary material). The output of the inverter circuit shows a negative spike when domain wall in the neuron device reaches the MTJ, as required for a pre-neuron (Fig. 5(a), Fig. 6(a)). The ON current of transistor T1 flows in the reverse direction of input current in the DW neuron and moves DW to its initial position. This is equivalent to v(t) in LIF model of the neuron dropping to E_L after a spike. Additional components are added to the transistor based circuit of Fig. 7(b), as discussed in Section 2 of supplementary material, to incorporate homoeostasis property in the DW neuron if required.

We next design a SNN with a layer of pre-neurons (input stage) connected to a layer of post-neurons (output stage) through synapses (Fig. 8). Based on the spike times of pre-neurons and post-neurons, weights of the synapses are updated by STDP rule (equation (4)) for every input/sample in each epoch. Then the process is repeated for several epochs to achieve learning/training.^18,30,31 Currents proportional to the 16 different features of each input/flower in the Iris training and test sets (after basic feature engineering on the 4 features in the original dataset^29–31) are applied on the pre-neurons. The label/type the input/flower belongs to is determined by the post-neuron that fires most for that input. Since there are 3 types of flowers in the dataset, we use 3 post-neurons. The post-neurons are connected inhibitorily with each other to implement the “Winner Take All” (WTA) mechanism.^18,30,31 We implement the WTA mechanism among the post-neuron circuits of Fig. 7(b) through an additional transistor based circuit.⁴⁴ The circuit schematic of the implementation and corresponding SPICE simulations are shown in Section 5 of supplementary material. Alternatively, WTA can be implemented through dipole coupling among the ferromagnetic layers of the domain wall neuron devices themselves, through which the domain walls move.²⁵ Then the additional transistors we have used to implement WTA mechanism will not be required.

The DW devices designed in previous sections, with the same parameters stated there, are used to model the neurons and synapses in our designed SNN. Learning is achieved through weight update of synapses due to “write” current pulses applied on the DW synapses, following equations (1)–(4). STDP always leads to some degree of unsupervised nature to the learning. However based on how we control the spiking of post-neurons, we have two different learning schemes here- completely unsupervised and partially supervised. Under the completely unsupervised scheme, the post-neurons have an additional homoeostasis property (Section 2 of supplementary material). During training, when input of one type makes a post-neuron spike, its spiking threshold V_th (in the LIF model) goes up by 7 mV, followed by a decay with time constant τ_homeo = 15 μs¹⁸ (Section 2 of supplementary material). Because of the increased threshold, only when another input is of the same type, the incoming current to that post-neuron is large enough for the neuron to spike. For input of other types it does not spike. Thus classification is achieved without a learner.¹⁸ Under the partially supervised scheme, the post-neurons do not have homoeostasis property. Instead, during the training process, for an input of a particular type, inhibitory currents are applied on all post-neurons except the post neuron, which we want to spike most for any input of that type.^30,31 For either scheme, high training accuracy (45 train samples) and test accuracy (105 test samples) are obtained after ≈ 15 epochs (Fig. 9). The net energy dissipated in all the synapses because of Joule heating for “write current” pulses during the learning process is calculated to be in the range of 50-200 fJ.

The classification accuracy for the SNN depends on values of τ₁ and τ₂ since they control the STDP based synaptic weight update rule (equation 3), upon which training/learning of the SNN is based. We observe from our simulations that both train and test accuracy continue to be around 90 percent as desired as long as τ₁ is in between 1.3 μs and 1.9 μs and τ₂ is in between 1 μs and 1.7 μs, given other parameters in SNN are not changed. From Section 4 of supplementary material, this corresponds to variation of capacitance C1 between 0.7 pF and 1.05 pF and C2 between 2.5 pF and 4.8 pF in the STDP exhibiting synapse circuit of Fig. 4. Variation of capacitances within this range due to circuit imperfections will hence not affect the performance of the designed SNN.

We have simulated STDP enabled learning under two different schemes in SNN hardware using DW devices both as synapses ane neurons. We have obtained high classification accuracy on a popular ML dataset- the Iris dataset. Training it on larger and more complicated datasets however involves many more STDP synapse circuits since the number of pre-neurons and post-neurons goes up. If the the STDP rule in equation (3) is simplified by eliminating the exponential characteristic,⁴⁵ number of transistors in the STDP circuit will go down making the overall circuit simpler and more scalable. Training our designed SNN on larger and more complicated datasets will be the subject of our future study.

See supplementary material for circuit implementations of homoeostasis and Winner Take All (WTA) mechanism.

Debanjan Bhowmik thanks Department of Science and Technology (DST), India for INSPIRE Faculty Award and Science and Engineering Research Board (SERB), India for Early Career Research (ECR) Award, which helped fund this research.