The recent success of artificial intelligence (AI) systems has been accompanied by a rapid increase in the computational resources needed to successfully train them. This rate of increase threatens the future development of AI systems as they are presently configured. Unsupervised learning, where systems are trained online instead of through offline computation, offers a possible way forward. Here, we present the design of a synaptic circuit made from superconducting electronics capable of spike-timing dependent plasticity (STDP), a form of unsupervised learning. The synapse is constructed from three sub-circuits, each responsible for a part of the synaptic action. We demonstrate the operation of the synapse through numerical simulation and show that it reproduces the hallmark behaviors of STDP. Combined with existing superconducting neuromorphic components like neurons and axons, this synaptic structure could help form a fast, powerful, and energy-efficient Spiking Neural Network.
The human brain has become an inspiration1 for computing systems. Treated as a computer, the brain is highly parallel, fault-tolerant, adaptable, and extremely energy efficient. It also has an unmatched ability to do complex pattern recognition. The attempt to emulate these properties in a computing system has ignited the fields of neural networks, neuromorphic computing, and artificial intelligence (AI) over the past few decades. Artificial Neural Networks (ANNs) trained with backpropagation2 can now do pattern recognition problems with success rates on par with what the mammalian brain can do. Meanwhile, more cognitive-like systems can now compete3 in games like chess and go at an extremely high level, in some cases beating the top grandmasters. In many ways, the fields of AI and neuromorphic computing have never looked brighter.
Hidden in this recent success, however, is the fact that the time to perform backpropagation, or the time for computers to train long enough to learn winning game strategies, has become extremely long. As noted4 in recent studies, the training times for the most advanced AI systems have increased at a rate that far outpaces Moore's Law and the growth of available computing power. In addition to the long training times, systems trained under backpropagation are not very adaptable, as once they are trained with a given set of data, they are only good to recognize one kind of pattern or to do one kind of task.
Backpropagation and related schemes are known as supervised learning. Looking back to the brain for further inspiration, we note that for the most part, the brain utilizes unsupervised learning, where synaptic weights are varied in situ, through the correlated firings of the attached neurons. The generally accepted paradigm under which unsupervised learning in mammalian brains occurs is known as “Spike-Timing Dependent Plasticity” or STDP. The principles5,6 of STDP can also be applied to realize unsupervised learning in artificial neural networks and related platforms and have in recent years.
STDP establishes a set of rules that relate the firing times of two neurons to a change in the synaptic weight connecting those two neurons. Of course, since the neurons need to spike in time, the type of neural networks in which STDP can be applied is Spiking Neural Networks (SNNs). In SNNs, individual neurons send spikes in time down weighted synaptic connections to other neurons; when the input to any given neuron happens to exceed its threshold, that neuron will spike at that particular time. SNNs are the most biologically realistic class of neural networks with a host7 of applications. Artificial SNNs have been realized both in software and in hardware.8
Superconducting electronics (SCE) offers a platform alternative to semiconductors in which to build an SNN.9 Prior work in our group10–13 and others14–25 have demonstrated spiking neurons, axons and dendrites, and synaptic connections. Possible superconducting components include not only just Josephson junctions but also superconducting nanowires,20,21 phase-slip junctions,16 adiabatic circuits,23,24 magnetic Josephson junctions,14,15 and photonic interconnects.17–19 Recently, it was shown26 how with available technology one could build a neuromorphic system offering a combination of scalability, programmability, biological fidelity, energy efficiency, and speed that would be unmatched in any existing platform. That work considered analog neurons combined with digital synapses in a mixed-signal architecture.
In this work, we propose a type of superconducting synapse circuit capable of exhibiting STDP. The synapse circuit has three distinct sub-circuits, each responsible for a different aspect of synaptic behavior. Through circuit simulation, we demonstrate the behavior of the synapse, showing that it satisfies the requirements of STDP. A large number of these synapses, incorporated with existing superconducting structures, would produce an SNN capable of unsupervised learning.
Our synapse couples two neurons through a post-synaptic current (Ipost). This current is emitted by the synapse in response to a firing event from the pre-synaptic neuron and flows into the post-synaptic neuron; it brings the post-synaptic neuron closer to threshold if the synapse is excitatory, or further from threshold if the synapse is inhibitory. Typically, this means that Ipost is positive for excitatory synapses and negative for inhibitory synapses. The amplitude of the post-synaptic current is determined by the strength or weight of the synapse; the stronger the synapse, the larger the post-synaptic current. Meanwhile, the value of the synaptic weight changes over time. This change in weight is governed by the firing of the pre- and post-synaptic neurons. This process is described by the learning rule, discussed later.
We divide up the behavior of the synapse into three sub-circuits, with each sub-circuit responsible for a different part of the synaptic action. Figure 1(a) shows a block diagram of the synapse and the three sub-circuits. The first sub-circuit is the signal modulation circuit. It has the job of converting the action potential signal (IAP) from the pre-synaptic neuron (N1 in the figure) into the post-synaptic current and sending it to the post-synaptic neuron (N2). The signal modulation circuit must apply the appropriate weighting, so that the amplitude of the post-synaptic current is proportional to the synaptic weight. The memory circuit holds the value of the synaptic weight, keeping track of changes over time. It must interface with the signal modulation circuit, providing the most up-to-date value of the weight. Finally, the learning circuit monitors the timing of the firing of the two neurons, providing updates to the weight. It must interface with the memory circuit, communicating its updates to the value of the weight via the learning rule.
(a) Synapse schematic. N1 and N2 represent the pre-synaptic and post-synaptic neurons, respectively, while the dashed rectangle represents the synapse which is divided into three sub-circuits. (b) Signal modulation circuit. The action potential current is divided between a two-junction loop and the neuron inductance LN. (c) Memory circuit. A large inductor Lm holds a flux that represents the synaptic weight. (d) Learning circuit. A D Flip-flop with a resistor in the loop emits a single-flux quantum pulse if the two pulses arrive within the learning time.
(a) Synapse schematic. N1 and N2 represent the pre-synaptic and post-synaptic neurons, respectively, while the dashed rectangle represents the synapse which is divided into three sub-circuits. (b) Signal modulation circuit. The action potential current is divided between a two-junction loop and the neuron inductance LN. (c) Memory circuit. A large inductor Lm holds a flux that represents the synaptic weight. (d) Learning circuit. A D Flip-flop with a resistor in the loop emits a single-flux quantum pulse if the two pulses arrive within the learning time.
For each of the three sub-circuits, we employ a particular superconducting circuit, shown in Figs. 1(b)–1(d). For the signal modulation circuit, we use an inductive divider with a two-junction loop as27,28 a variable inductor [Fig. 1(b)]. For the memory circuit, we use a Josephson junction in parallel with a large inductor, with a resistor to account for forgetting [Fig. 1(c)]. Finally, for the learning circuit, we use a D Flip-flop29 with a resistor in the superconducting loop [Fig. 1(d)]. The supplementary material goes through each sub-circuit separately and shows how it achieves the desired behavior.
The learning rule, showing how the change in synaptic weight varies with the time difference between the neuron firings.
The learning rule, showing how the change in synaptic weight varies with the time difference between the neuron firings.
In biological systems, there is a large variety of synapses with different learning rules, some of which are not even fully understood. In most synapses, Δw decays exponentially with Δt, also shown in Fig. 2, and often there is an anti-Hebbian term that decreases the synaptic weight by Ω if the pulses arrive in the wrong order (post before pre).
Our goal in this initial design was to approximate typical biological behavior while keeping the circuit relatively simple, rather than trying to match one particular synaptic type in detail. However, some fairly modest enhancements of our design would allow for these biological features. One could implement anti-Hebbian learning with a second learning gate that reverses the inputs from N1 and N2 and is configured to put a negative SFQ pulse into the memory loop. Alternatively, to make a learning rule with a more exponential-like form shown in Fig. 2, one could make the second junction in the learning gate smaller, so that it flips more as the events get closer in time.
Now, we investigate the behavior of the synapse circuit as a whole, to demonstrate STDP behavior. Our proposed experiment is shown in Fig. 3(a), where we show two neurons coupled by a single synapse. Both neurons are biased such that they are firing continuously, by choosing a quiescent, constant input current that is just above threshold. Initially, through appropriate choice of the magnitude of the input current, we configure the pre-synaptic neuron (N1) to fire rapidly but the post-synaptic neuron (N2) to fire slowly. Over time, just by chance, some pulses will arrive within the learning time, causing a learning event. These learning events will increase the synaptic strength and the postsynaptic current. This will cause the post-synaptic neuron to fire a bit more often, making future learning events more common and causing yet a larger value of synaptic strength. This cycle will feedback on itself in a positive way, driving up the synaptic strength toward a maximum value, which is set by the balance of learning and forgetting.
(a) Schematic of the full synapse simulation. The two neurons are set to fire continuously, with N1 firing at a larger rate. The firing of N2 occasionally overlaps with N1 to cause learning events. (b) Full synapse schematic with all three sub-circuits combined.
(a) Schematic of the full synapse simulation. The two neurons are set to fire continuously, with N1 firing at a larger rate. The firing of N2 occasionally overlaps with N1 to cause learning events. (b) Full synapse schematic with all three sub-circuits combined.
Figure 3(b) shows a diagram of the full synapse circuit. In order to test the full synapse in the way described in Fig. 3(a), we used Josephson junction neurons as N1 and N2 whose outputs are coupled to Josephson Transmission Lines (JTLs). The JTL from N1 is split, so that one copy of the pulse goes to the learning gate, and another copy goes to the signal modulation circuit. Meanwhile, the JTL on the output of N2 is sent directly to the learning gate. Other parts of the circuit (not shown) include R-L filters going into the learning gate and a back-biased junction to prevent any current flowing back out of N2.
Figure 4 shows a sample of the dynamics, starting at a time of 3.2 ns after the start of the circuit simulation. The bottom trace (yellow) shows the firing of the pre-synaptic neuron, the middle trace (green) shows the firing of the post-synaptic neuron, and the top trace (red) shows the current through the memory loop, which represents the synaptic weight. The pre-synaptic neuron is firing at a rate of fN1 = 18.9 GHz (period = 53 ps), which is set by our choice of the input bias current and stays constant throughout the simulation. Meanwhile, for the data shown, the post-synaptic neuron is firing at a rate of fN2 = 7.6 GHz (period = 130 ps); this is determined by how often the post-synaptic neuron gets over threshold, which changes throughout the simulation.
Dynamics of the full synapse. The top panel shows action potentials from the two neurons (left axis) along with the memory current (right axis). Learning events are shown with the rectangles. During the second half of the simulation shown, there are no learning events, and forgetting can be observed. The bottom panel shows a blow-up of the region around 3.4 ns, where there is a learning event with Δt = 8 ps and a non-learning event with Δt = 30 ps.
Dynamics of the full synapse. The top panel shows action potentials from the two neurons (left axis) along with the memory current (right axis). Learning events are shown with the rectangles. During the second half of the simulation shown, there are no learning events, and forgetting can be observed. The bottom panel shows a blow-up of the region around 3.4 ns, where there is a learning event with Δt = 8 ps and a non-learning event with Δt = 30 ps.
Several things can be noticed in Fig. 4. First, we see a learning event early on, at a time of about 3.37 ns, where the pre-synaptic neuron happens to fires about 8 ps before the post-synaptic neuron. Shortly after that, the value of the memory current increases, from about 0.92 μA to about 0.95 μA. Meanwhile, a non-learning event occurs at 3.47 ns when the pre-synaptic neuron fires 30 ps before the post-synaptic neuron. Additional learning events can be seen at times equal to 3.65 and 3.9 ns. From 3.9 to 5.1 ns, there is a stretch of time with no memory events; during then, we can notice a small drop in the learning current of about 0.1 μA, due to the forgetting resistor. Finally, the pulsing of the pre-synaptic neuron feeds through to cause small, AC ripples in the memory current, but this does not cause a flux quantum to enter the loop.
If we carefully histogram the difference between the firing times of the pre-synaptic and post-synaptic neurons, we get the data shown in Fig. 5. The solid line shows the histogram of firing time differences that caused a learning event, while the dotted line shows the histogram of firing time differences that did not cause a learning event. We see a nice separation between the two, with learning events clustering between approximately 0 and +18 ps, while the non-learning events occur outside of this range. This graph is a key indication of STDP. The boundary between the learning and non-learning events is not a discontinuous step function like that shown in Fig. 2 but rather exhibits some overlap between learning and non-learning events. This overlap is caused by small transients in the other parts of the circuit (for example, in the neurons), which cause currents to flow in the synapse and slightly blur the sharp theoretical learning function. In real circuits, thermal noise will cause similar effects. This type of behavior occurs in biological synapses as well.
Learning events histogram. Learning events occur when Δt is between 0 and about 18 ps, but not otherwise.
Learning events histogram. Learning events occur when Δt is between 0 and about 18 ps, but not otherwise.
Looking over the whole time of the simulation, from 0 to 60 ns, we see three long-term trends: (1) an increase in the value of the weight, (2) an increase in the size of the post-synaptic current, and (3) an increase in the firing rate of the post-synaptic neuron. These are all shown in Fig. 6.
Long-term behavior of the synapse. The top panel shows the firing frequency of N2, the middle panel shows the amplitude of the post-synaptic current, and the bottom panel shows the memory current. Dots are individual points, and the solid lines are the average.
Long-term behavior of the synapse. The top panel shows the firing frequency of N2, the middle panel shows the amplitude of the post-synaptic current, and the bottom panel shows the memory current. Dots are individual points, and the solid lines are the average.
The rate of learning events, 1/τE, is equal to the firing frequency of N2 times the fraction of N2 events which are learning events. To estimate the fraction of N2 events which are learning events, recall that a learning event occurs when the firing of N2 falls within the learning time of the firing of N1. An estimate of how often this occurs is simply fN1τL, which is about 0.34 in our case. If we plug in an N2 firing frequency of about 8.2 GHz and assume we learn 34% of the time, we do, indeed, end up with the observed steady-state value of Im = 6.3 μA.
At a value of Im = 6.3 μA, the flux coupled into the signal modulation circuit is about 0.4 Φ0. From our analysis of the signal modulation circuit (supplementary material), we predict about an 80% increase in the post-synaptic current, Ipost, at that value of flux; Fig. 6(b) shows close to that, about a 65% increase. This is because we assumed a constant inductance for the post-synaptic neuron, whereas in reality, it is very much a dynamic quantity, as the neuron is constantly pulsing; a more complicated model is needed to predict Ipost with better accuracy.
The parameters with which we chose to simulate are listed in the supplementary material. They were chosen to match with our previous experiments and simulations of JJ neurons and JTLs, which were typically at current densities of 1 to 4.5 kA/cm2. Critical currents are in the 40–100 μA range, with the superconducting quantum interference device (SQUID) junctions a factor of 3 or 4 smaller to get the maximum swing of signal modulation. Going larger in critical current is fine for future designs, provided that the normalized parameters like βL and the damping are kept constant.
Mapping the full parameter space is beyond the scope of this paper but could yield different combinations of speed and dynamic range to better fit a given application. For STDP, the important design parameters are the learning and forgetting time. The learning time is set by the learning inductance divided by the learning resistance (τL = LL/RL), and the forgetting time is set by the memory inductor divided by the forgetting resistance (τF = Lm/Rf). In addition, the amount of learning for each event is . The critical currents of the junctions, the shunt resistances, and the remaining inductors could take on a range of values chosen to impedance match to the neuron circuitry and to optimize signal and speed.
Our synapse circuit was designed to be simple, using only five Josephson junctions. In an SNN, the synapse is the most-repeated element, so keeping the synapse as resource light as possible was an important consideration. If resources are not as much of a constraint and more biological realism is desired, then a number of enhancements to this design are possible, including the aforementioned anti-Hebbian learning and a more biologically realistic learning rule.
With the simple design shown here and today's SCE fabrication technology, however, one could implement an SNN with a large number of neurons that would be fast, energy efficient, and capable of unsupervised learning. Such a machine could be capable of breaking the training bottleneck that we have today and allow neuromorphic computing to reach the next level.
SUPPLEMENTARY MATERIAL
See the supplementary material for a detailed description of each of the sub-circuits and the list of parameters used in the full-synapse simulations.
AUTHOR DECLARATIONS
Conflict of Interest
The authors have no conflicts to disclose.
Author Contributions
Kenneth Segall: Conceptualization (equal); Data curation (equal); Formal analysis (equal); Investigation (equal); Methodology (equal); Supervision (equal); Writing – original draft (equal); Writing – review & editing (equal). Cheeranjeev Purmessur: Data curation (equal); Formal analysis (supporting); Investigation (supporting); Methodology (supporting); Writing – review & editing (equal). Anthony D'Addario: Data curation (equal); Formal analysis (supporting); Investigation (supporting); Methodology (supporting); Writing – review & editing (equal). Daniel A. Schult: Conceptualization (supporting); Data curation (supporting); Formal analysis (supporting); Investigation (supporting); Methodology (supporting); Writing – review & editing (equal).
DATA AVAILABILITY
The data that support the findings of this study are available from the corresponding author upon reasonable request.