Fan-out and fan-in properties of superconducting neuromorphic circuits

In recent years, there has been an increase in the study of neuromorphic circuits made from superconducting electronics.^1–12 The single flux quantum (SFQ) pulse that is released from a Josephson junction when the applied current briefly exceeds its critical current has a lot in common with the action potential in neurons that is released when the membrane potential exceeds its threshold. Superconducting circuits based on Josephson junctions are being developed to generate pulses, send pulses down output lines, transmit variable feed-forward signals, and sum the signals from different sites in a network; these circuits represent the electrical equivalent of biological somas, axons, synapses, and dendrites, respectively. As a result, it is possible to conceive of building spiking neural networks (SNNs) completely from superconducting electronics. SNNs are used for pattern recognition in spatio-temporal datasets and are considered by some to have superior information-processing capabilities to the basic artificial neural networks (ANNs) used in “deep learning.”¹³ They are also good analogs of the human brain and are used to study its computational capabilities.¹⁴ SNNs made from superconducting electronics have the potential to be both faster and more energy-efficient^15,16 than those made from silicon circuits.

Over the past three decades, the field of superconducting electronics¹⁷ has grown and matured, resulting today in a large variety¹⁸ of available circuits that manipulate, control, and store SFQ pulses. These include flip-flops, logic gates, shift registers, transmission lines, splitters, mergers, memory circuits, and comparators, among others. Many of these circuits are already being used in scientific and technological fields such as gate-model quantum computing,¹⁹ adiabatic quantum computing,²⁰ voltage standard technology,²¹ RF filtering,²² analog to digital conversion,²³ low-temperature detector readout,²⁴ and high-frequency waveform synthesis.²⁵ Neuromorphic computing is a new application of SFQ technology, and while the fundamental circuits remain the same at heart, there is a need for some redesign and repurposing of these circuits to perform the necessary functions in a neuromorphic environment.

In particular, topologies with a large fan-in and fan-out are common in neuromorphic computing, following the extraordinary fan-in and fan-out of the human brain (∼10 000). Here, we define fan-in as the number of inputs that a circuit element can accept, whereas fan-out refers to the number of loads that can be connected to a circuit element's output. SFQ circuits typically only have fan-in and fan-out of 2 or 3 at most,²⁶ a number that should be increased for adapting SFQ technology to neuromorphic computing.

In this paper, we explore the problem of fan-out and fan-in in superconducting neuromorphic circuits. Although there are many similarities to the fan-in and fan-out properties of superconducting digital circuits, there are also important differences, most notably the analog nature of the synapse. We make some basic assumptions about a possible system architecture and show both flux- and current-based methods of fan-in and fan-out. We use both circuit simulations and calculations to show how these approaches might scale and what factors might limit them in pushing to larger networks. With modest assumptions about fabrication parameters like inductances and critical currents, we show how a fan-out and fan-in of order 100 is possible.

The paper is organized as follows: In Sec. II, we give an overview of the problem, assuming a basic system architecture. In Sec. III, we show two possible ways of fanning out, including splitter trees and a current fan-out. In Sec. IV, we show two methods of fan-in, one with flux and one with current. In Sec. V, we look at the limiting factors for each of these as one scales to larger networks. Finally, in Sec. VI, we conclude and suggest future work.

In this section, we carefully define the problem of fan-in and fan-out, first in generalized spiking neural networks and then, by assuming a simplified architecture, in a superconducting electronics implementation. Since there is as of yet no agreed-upon architecture for a superconducting neural network, some of our results will change as different architectures are proposed and tested. However, we believe that many of the ideas developed here are general enough that they will translate into future experiments regardless of architecture.

Figure 1 shows a general schematic diagram of a spiking neural network (SNN). Indicated are spiking neurons, synapses, and propagation lines that represent axons and dendrites. Neurons are broken into two parts: the input, where signals from many synapses are summed together, and the output, where a spike is emitted when the sum of the inputs exceeds a threshold. The number of signals summed together on the input constitutes the fan-in, while the number of output lines that a spike is propagated down is the fan-out. The fan-in and fan-out are about the same (on average) for a given network, since most signals start in one neuron and end in another, with the exception of the inputs and outputs to the system.

One important distinction needs to be made about the signals from neurons and synapses. The signals that flow out of neurons (fan-out), called the spikes or “action potentials” in a biological setting, are essentially digital: they all have about the same amplitude and shape, which are fixed by the ion channel dynamics inside the soma. For this reason, the SFQ pulses of superconducting digital logic are ideal to serve as action potentials. Meanwhile, the signals that flow out of synapses into neurons (fan-in), the so-called “post-synaptic potentials,” are analog in nature: although they typically have a similar shape, their amplitude depends on the strength of the synapse, and they can be either positive (excitatory) or negative (inhibitory).²⁷ In many implementations of a spiking neural network, synaptic signals are represented digitally; for example, IBM's True North used a 4-bit digital number to represent the amplitude of the synaptic current.²⁸ However, this represents a quantity that is still inherently analog, and the bit depth that is required to capture this analog behavior is the topic of active investigation.²⁹ 

These ideas make the role of fan-in and fan-out circuitry clearer. For fan-in, we need to sum the analog signals from different synapses coming into the soma. For fan-out, we need to take the output signal of the soma and generate identical copies, each to be sent down a separate line to a synapse.

In our superconducting implementation, we assume a simple two-junction neuron similar to the Josephson junction neuron (JJ neuron) already proposed.¹ Although a single junction will generate a spike when driven above threshold, for larger input currents an unwanted offset will develop; this is avoided in a two-junction neuron. The two junctions also give the neuron more biological realism, as each junction can be likened to an ion channel, with the resulting reproduction of several biologically realistic neuron behaviors.¹ The input to the neuron is either a current or a flux, and the threshold is defined when an SFQ pulse is produced from a given input. This can be tuned by adjusting circuit parameters such as a bias current.

In this section, we will discuss two potential methods, flux based and current based, for fan-out that can be used in superconducting neuromorphic circuits. Because the fan-out in these circuits is effectively a digital operation, one can start with a fan-out circuit that is a series of nested splitters like those used typically in digital SFQ circuits.¹⁸ Figure 2(a) shows the schematic of the binary splitter circuit, which will be nested in layers to achieve a larger fan-out. The input to the first layer is an SFQ pulse coming from a “pre-synaptic” JJ neuron and the output at the final splitter layer will be transmitted to a synapse. In the intermediate layers, the input will be from the previous splitter layer, and the output will go to the subsequent splitter layer. With a network of such splitters, one can fan-out to a given power of 2 with an overhead of 3N_FO-3 junctions, where N_FO is the final fan-out number. It is worth noting that the junction biases can be run in parallel meaning that only one line would be needed for the fan-out circuit of 1-to-128. If one wants to run the bias lines through vias to allow for more flexibility in wiring, then the current in that bias line will most likely be limited by the via critical current, which typically should not exceed 30 mA.

Figure 2(b) shows the current pulse at each layer of a binary splitter, as labeled to the right of the trace, that has been nested seven times to achieve a fan-out of 1-to-128. Note that the output is nearly identical for each trace aside from the slight time delay, as desired. The output was taken at a random location within a given layer and corresponds to the current through the inductor after division (L₂). In this example, we feed an initial DC to SFQ converter with a 25 GHz triangle wave, and its output is fed into the input of the first splitter layer at the node marked “in” in Fig. 2(a). The latency through this fan-out is roughly 6 ps. In this example, we chose the critical current of JJ₁ to be I_c1 = 70 μA with L₁= 15 pH and JJ₂ to have I_c2 = 50 μA with L₂ = 20 pH. We also chose the junctions to be overdamped, quantified by $βc=0.35$ , where $βc=2πIcRn2Cjϕ0$ is the McCumber parameter, and set the bias current to 70% of the respective I_c values. This method provides a straightforward means of copying the signal flowing out of a JJ neuron to an arbitrarily large number of synapses.

In addition to the flux-based fan-out shown in Fig. 2, one can also use a current based fan-out as shown in Fig. 3(a). In this fan-out, a pre-synaptic current spike is resistively split and then reamplified with an amplifying Josephson transmission line (JTL), which can be seen as the increasing I_c values of the junctions in Fig. 3(a). We have run simulations of this scheme at up to a direct division of 1-to-10. This structure is also nestable, meaning that using 1-to-10 splitters one could get to a fan-out of 1-to-1000 in three layers. It should be easier to modify this structure anywhere between a fan-out of 2 and a fan-out of 10 for any single stage, which could reduce the potential overhead if a non-power-of-two fan-out is desired. However, a downside of this design is that when nesting 1-to-10 fan-out splitters, the JTL amplifier required at least four stages meaning that the final junction count overhead is ∼4N_FO, where N_FO is the final fan-out number. More precisely, in the case where reamplification takes an M-junction JTL, the junction count is $NFO=∑i=1kM⋅Ji$⁠, where k is the number of stages and J is the single-stage fan-out (10 in this example). Figure 3(b) shows the output of the last stage junction at each of the three stages in a simulated 1-to-1000 fan-out. We note that the final stage is terminated with a resistor and an inductor, which is why its pulse shape is slightly different from the other two stages as they are driving a 10 Ω resistor divide plus the first stage of the JTL re-amplifier. In this example, we again feed an initial DC to SFQ converter with a 25 GHz triangle wave, and its output is fed into the input node marked in Fig. 3(a).

The results and simulations of this section apply to general fan-out considerations for all SFQ circuits, not just those related to superconducting neuromorphic computing. Although fan-out problems are particularly acute for neuromorphic circuits, they are still a problem for digital SFQ circuits in general. Our results here could be useful for other situations where digital signals need to be fanned out to more than one or two inputs, and our comparison between the current and flux method (see Sec. V) would apply in those cases as well.

In this section, we will discuss two methods for fan-in that can be used in neuromorphic SFQ circuits. Unlike the fan-out, we envision the fan-in to be an analog operation. Figure 4(a) shows the basic schematic for a flux-based fan-in. This circuit is designed for the input on each of the lines (I_{in_1} through $Iin_NFI$⁠) to have already been weighted by a synaptic circuit. With flux-based fan-in, excitatory or inhibitory connections can be made using positive or negative inductive coupling. The summation loop has N_FI small mutual inductors, two JJs, and one more inductor to adjust the behavior of the loop, as well as a bias current, which can adjust the threshold.

While both fan-out circuits were digital and nested to achieve arbitrary levels of fan-out, the fan-in circuit is inherently analog and therefore cannot be nested in the same way. This means that the scaling of these circuits will be under further constraints, the details of which are discussed below. In the simulations presented here, we take a worst-case scenario of trying to drive the summation loop that is the post-synaptic JJ neuron above threshold with a single incoming SFQ pulse. For small fan-in sizes, this is relatively easy, and we show here that even under these “worst-case” conditions, we are able to successfully drive a fan-in of 128-to-1.

In order to drive the summation loop above threshold with a single input, we had to decrease the critical current of the JJs in the post-synaptic JJ neuron. In our simulations, we used JJs with critical current values of 12 μA. These are smaller than junctions that would typically be used for digital SFQ circuits, but it seems reasonable that the extremely low error rates required for, e.g., floating point operations are not required for ANNs, where, e.g., classifiers typically have a few per cent error even when functioning correctly. To be more concrete about the level of noise, we look at the example for thermal noise on a moderately damped 12 μA JJ at 4 K. Assuming a plasma frequency of 1.5 THz and β_c= 12.9 with a shunt resistor used to give an effective β_c= 1, we find that the standard deviation of the change in the critical current is 0.19 μA if the junction was ramped at 1 kHz, and 0.21 μA if the junction was ramped at 10 MHz.³⁰ Assuming that peak deviations are roughly 5 times this value, there would be roughly 1 μA of fluctuation in the critical current from thermal noise. However, we do not use such low critical current throughout the network. For digital operation such as fan-out, it seems more logical to follow the typical low error rate design of digital SFQ circuits. Therefore, if one wants to extend the network further than a single layer, we need to reamplify the signal back to the original level so that it can be fanned-out into the next layer of the network. We include this re-amplification in our simulated circuits as shown in Fig. 4(a), with the amplifying JTL between L_{SQ_1} and the output.

It is worth noting that if multiple incoming neurons are active, then one can adjust the bias (I_{SQ_b}) of the post-synaptic neuron superconducting quantum interference device (SQUID) to change the threshold. This works analogously to modify the biases in a typical software-based ANN. By adjusting this bias, we were able to tune the threshold to activate with between 1 and 10 incoming full-strength neurons and still maintain SFQ type operation. When more than 10 of the incoming neurons were active at full weight, the post-synaptic SQUID loop would emit more than one output pulse. Such over-driving of biological systems can also produce multiple output spikes³¹ and therefore may not be problematic. However, if one wants to maintain a single output spike, then some adjustments of the post-synaptic SQUID loop will be required if the input activity is expected to be very strong. The easiest of these would be to reduce the inductive coupling strength. In our simulations, we used a value of k = 0.5, but physically this can be reduced. The other relatively straight forward way to handle the high-input case would be to eliminate the JTL amplification before the synaptic element and/or increase the critical current of the JJs in the post-synaptic SQUID loop. While this does imply that some knowledge of the network activity will likely be required for the physical design, if more than one pre-synaptic neuron is spiking, then a higher fan-in can be achieved.

Figure 4(b) shows the results of the simulation of a 128-to-1 fan-in circuit operating at 25 GHz under the single active input condition. In this simulation, we started with the fan-out circuit as described in Fig. 2. However, only 1 of the final 128 junctions was connected back to the input pulses coming from the DC to the SFQ input. The rest of the splitter network was terminated to ground on the input to ensure that there were no spikes coming from these JJs while maintaining the impedance of the network to simulate any potential crosstalk effects. In our simulation, one full-weight pulse is then defined by the final stage of a JTL amplifier after the fan-out with I_c = 50 μA.

The traces in Fig. 4(b) show the current pulse through one of the coupling inductors [L_{SQ_1} in Fig. 4(a)] in black, the output current of the summation loop through L_{out_0} is shown in red, and the output current through L_{out_3} after reamplification is shown in blue. Figure 4(c) shows the phase response of JJ_{SQ_2}, exhibiting 2π phase steps for each output pulse of the post-synaptic JJ neuron, which confirms the desired SFQ type operation.

In addition to the flux-based fan-in just described, we also investigated current-based fan-in. Figure 5(a) shows the basic schematic for a current-based fan-in. This circuit is designed for the input on each of the lines (I_{in_1} through $Iin_NFI$⁠) to have already been weighted by a synaptic circuit. We again run simulations for the “worst-case” scenario of trying to drive the post-synaptic JJ neuron with a single input pulse. Figure 5(b) shows the results of simulations of the current-based fan-in of 10-to-1 operating at 25 GHz. The current pulse through L_{SQ_1} in the summation loop is shown in black, output current of the summation loop through L_{out_0} is shown in red, and the output current through L_{out_3} after reamplification is shown in blue. Figure 5(c) shows the phase evolution of JJ_{SQ_2}, which demonstrates both 2π phase steps that accompany the SFQ pulsing of the post-synaptic JJ neuron and the desired SFQ operation of this part of the circuit. Note I_{in_1} was taken as an output spike from a flux-based fan-out of 1-to-10 and was then amplified with a JTL amplifier with a final stage of 200 μA. All other inputs were grounded so as not to cause any additional input current but to properly model the loss of current running back into the inactive inputs, which we will refer to as the crosstalk.

For larger networks, the fan-out methods presented in Sec. III will be limited only by area, junction count, power, and delay. There is no fundamental limit to how many signals can be fanned out with either the splitter method or the current method. The scaling of the junction count with fan-out for each method is given in Fig. 6. The splitter method seems like the better choice, but depending on other design constraints, it is possible that the current method might still be viable.

Depending on the architecture chosen for a superconducting SNN, one may want to maintain a constant pulse timing. Figure 7(a) shows a block diagram schematic of binary splitters connected in a way that maintains a constant time delay at a single output line. In this example, the timing is accomplished using constant wire path lengths, and the junction counts for all outputs. If one were to relax the timing requirement or the single output line requirement, then the circuit size could be reduced. In the example shown in Fig. 7(a), the splitters were nested to three layers giving a fan-out is 1-to-8. One could also extend this structure arbitrarily deep depending on the desired level of fan-out. In this case, the size of the fan-out circuit will be determined primarily by two factors. The size in the vertical direction [as illustrated in Fig. 7(a)] is set by the minimum pitch of the JJs, and the size in the horizontal direction is set by the required inductance.

Using this nominal two wiring layer layout, we estimate that a 1-to-128 fan-out circuit would occupy roughly 175 × 200 μm². To arrive at this estimated area, the following assumptions were used: The minimum JJ diameter is 500 nm, and the separation between JJ edges is 1 μm. This yields a 1.5 μm pitch for the JJs in the final output line, which results in a maximum vertical length of roughly 200 μm for a fan-out of 1-to-128. To estimate the space required in the corresponding horizontal direction, we assumed that the inductance of the strip lines used for wiring can be varied between roughly 0.05 pH/μm and 0.9 pH/μm. Using the structure shown in Fig. 7(a) with the wiring as the source of inductance, we estimate that the horizontal length of a 1-to-128 fan-out circuit would be roughly 175 μm. Reduction in the horizontal length could be achieved with the use of high inductance layers or by adding meanders in the wiring, an example of which is shown in Fig. 7(a) with dashed lines. Reduction in the vertical length could be achieved by reducing the JJ diameter and/or by reducing the space between JJs. We note that this pitch for fan-out matches a reasonable pitch for the fan-in circuit as well. With a 1 pH inductor requiring 1.1 μm, a center-to-center spacing of 1.5 μm should be reasonable. This give us a 128-to-1 fan-in size estimate of roughly 200 × 10 μm², assuming that we run the JTL amplifier down the outside of the fan-in loop.

If space and time delay are permitted, then the binary splitters can be nested much deeper, including to levels similar to the human brain. Figure 7(b) shows example output current spikes from a fan-out circuit using the same basic parameters as outlined in Fig. 2 but extending the circuit to 14 layers deep resulting in a fan-out of 1-to-16 384. This 1-to-16 384 fan-out circuit would occupy an area of roughly 350 μm × 25 mm using the same assumption discussed in Fig. 7(a). This circuit is quite large, and one would hope that further scaling of the JJ technology will help to reduce this size in the future.

There is one additional constraint that must be considered as the fan-out is nested to larger circuits, which is the inclusion of repeater junctions (or JTLs), as shown in circles in Fig. 7(a). These are needed once the impedance of the wire becomes too large for the JJs to drive within the SFQ regime and will minimally increase the junction count. In the 1-to-16 384 fan-out example, the longest wire [following the nominal layout of Fig. 7(a)] is roughly 12 300 μm. This would require roughly 122 additional junctions spread through the first five layers of the fan-out, if we assume wiring with 0.05 pH/μm and a desired inductance of 20 pH. Compared to the total junctions count of roughly 3 N (49 152), this is a negligible design constraint.

In addition to the flux-based splitter fan-out, one can also nest the current-based fan-out to an arbitrarily large size. The footprint of this type of fan-out will be similar in size to the splitter tree if we are constrained to a vertical line output with uniform time delay wiring. The rough size of a fan-out of 1-to-1000 (three layers deep) with this method is 1500 μm in the vertical direction and 250 μm in the horizontal direction. These dimensions were limited by the 1.5 μm pitch in the vertical direction and by the 0.9 pH/μm maximum inductance in the horizontal direction.

Next, we discuss the limits to the two methods of fan-in, current and flux, as shown earlier in Sec. IV. Figure 8 shows a generalized schematic of each method where we have tried to simplify each circuit to contain only the essential parts that affect scaling to large N_FI. Figure 8(a) shows the current method of fan-in, while Fig. 8(b) shows the flux method. In both cases, the input to the fan-in coming down each synapse is an SFQ pulse with current I_in, generated by a junction with critical current I_c1 and a total series inductance (L_s + L_cp). In the current method, this current flows into a summing node and then into an output superconducting quantum interference device (SQUID), which has an inductance of L_sq and a critical current of I_c2. In the flux method, the current I_in flows through the coupling inductance L_cp, which is magnetically coupled to the inductance L_sq. This causes a flux in the output loop that has two junctions with critical currents equal to I_c2.

With these simplified circuits, we can easily derive the value of the signal current I_sig that flows into the output loop in both cases. In the case of the current method, we use the equation for current division. We first calculate the total parallel inductance L_|| from the summing node to ground,

where the term $1/(Lcp+Ls)$ appears (N_FI-1) times, once for each of the remaining fan-in paths aside from the one generating the pulse. For now, we have assumed that L_sq is the inductance going to ground for the signal path, ignoring the higher impedance path through the junctions; we will correct this later. The signal current is then simply $Iin(L||Lsq)$⁠. This leads to

Meanwhile, for the flux method, we use the equations of mutual inductance. With a current of I_in flowing through L_cp, the flux coupled to the output loop is MI_in, where $M=kLcpLsq$ and k is the coupling constant. This then results in a signal current of

where L_tot is the total inductance of the large summing loop. Later, we will relate L_tot to N_FI and L_sq.

Looking at Eqs. (2) and (3), we can make a few assumptions about some of the terms. First, we note that the current I_in that appears in Eqs. (2) and (3) is the result of an SFQ pulse and therefore has a total quantized flux of Φ₀ = 2.07 × 10⁻¹⁵ V s. Since it is coupled through an inductance of (L_cp + L_s), we can write as

In addition, since both the input loop (I_c1 – L_s + L_cp) and the output loop (I_c2 – L_sq) each need to generate an SFQ pulse, there is a constraint on their inductance parameter β_L. Loops capable of generating SFQ pulses need to have a value of β_L in the range β_L ≈ 1–10.³² Loops with β_L < 1 are non-hysteretic and incapable of generating an SFQ pulse; loops with β_L > 10 can store more than one flux quantum and are subject to random phase slips. We define β_L1 and β_L2 as follows:

In our designs, both of these parameters ended up being between 1.5 and 4 and close to each other. [Note that β_L2 in Eq. (6) is defined for the current fan-in case; in the flux case, there would be N_FI SQUID inductors.] Finally, we refer the signal current to the critical current I_c2, which is the junction that must release an SFQ pulse when the sum of the fan-in signals exceed threshold. Ideally, I_sig should be a significant fraction of I_c2 in the neighborhood of at least 20%. Although smaller signal currents (1%–10%) could still potentially cause pulses in the output loop, this would require an external bias current very close to I_c2. In that case, fabrication margins will become a significant concern. We, thus, use 20% as a rough rule of thumb.

We now apply these three assumptions into the equations for the signal and then compare the expression for the two methods. Looking first at the current, we find

where we have assumed that N_FI >> 1 and I_c1 >> I_c2 (chosen to maximize the signal—see below), thus ignoring the second term in the denominator. For the case of flux, we make the simple assumption that L_tot in Eq. (3) is equal to N_FIL_sq, again assuming that N_FI >> 1 and also ignoring the slight modification to the impedance of each of the L_sq inductors due to the coupling back to the input. Then, we obtain

In our case, L_cp = L_sq, so the term in the square root is 1. Note then the similarities between Eqs. (7) and (8). The one major difference is the factor of (I_c1/I_c2) in the flux case. (There is also the factor of k, but this can be of order one.) This allows scaling to a much larger fan-in in the flux case, as one can make this ratio of critical currents (I_c1/I_c2) a factor of 50 or even higher. Thus, the flux method scales to large N_FI better than the current method. This is a major result of our paper. Besides the comparison between Eqs. (7) and (8), we will also show this directly with simulations.

The expressions for the signal current derived in Eqs. (2) and (3) along with their simplified form in Eqs. (7) and (8) can be verified by circuit simulations. We built a WR-SPICE model of the circuits in Fig. 8, choosing parameters that were similar to those in the simulations shown in Sec. IV. In both the current and the flux case, we chose I_c1 = 1000 μA. This allowed us to decrease the size of L_cp and L_s while keeping β_L1 roughly constant, via Eq. (5). Meanwhile, the smaller value of (L_cp + L_s) helped maximize the size of the current I_in, via Eq. (4). The value of I_c2, on the other hand, was kept as small as possible to account for a small signal current. We used both a value of I_c2 = 20 μA, which is safe from the effects of thermal fluctuations at 4 K, and a value of 6 μA, which is very aggressive. Our values of L_s and L_cp were 0.8 pH and 1 pH in the current case and both equal to 1 pH in the flux case. The value of L_sq is different for the flux and the current case and is described below.

After setting these parameters, we varied the fan-in and noted how the signal current changed. Figures 9 and 10 show the cases for current and flux, respectively, where we plot (I_sig/I_c2) as a function of N_FI. The markers show the results from circuit simulations, while the solid and dashed lines show the results from the equations. The dashed line at a value of (I_sig/I_c2) = 0.2 represents our 20% criterion described above.

Looking first at the current, we used Eq. (7) for the fitting, but with the value of L_sq modified to include the extra path to ground in parallel. This path included the additional inductor L_sq and the Josephson inductance of the two junctions with I_c2. This parallel path has an inductance of about 2L_sq, so the total equivalent inductance to ground is then 2/3 L_sq. Thus, we replaced L_sq with 2/3 L_sq to calculate β_L2 in Eq. (6), which is then used in Eq. (7) to calculate the signal current. Figure 9 shows the simulated data and fits for three cases: (i) L_sq = 40 pH and I_c2 = 20 μA; (ii) L_sq = 133 pH and I_c2 = 6 μA; (iii) L_sq = 133 pH and I_c2 = 20 μA. The equations predict the reduction in signal very well. The maximum fan-in, when the signal drops to 20% as denoted by the black dashed line, is about N_FI = 20. Note there is really no way to improve this; going lower in I_c2 means having to go higher in L_sq, which does not increase the signal. Thus, the red and green curves represent about the best performance possible.

Next, we look at fitting the case for flux fan-in. We use L_sq = 1 pH, trying to keep it as small as possible since there are so many of them. Figure 10 shows the simulated data for the two values of I_c2, 20 μA and 6 μA. The dotted lines show the fit from Eq. (8), which assumes that the total inductance in the large loop is N_FIL_sq. These fit the data well at large N_FI, but not at small N_FI. As with the current method, we need to account for the other components in the loop, the two junctions and the remaining L_sq inductor; we call this extra inductance L_ex, which simply adds to N_FIL_sq. In addition, since each of the L_sq inductors couples a small current back to the input loop (the crosstalk current, which will be discussed below), their impedance is reduced below the value of L_sq. The reduced value $Lsq′$ is given by

which can be derived from the circuit model of a transformer. Incorporating both of these corrections, we can then rewrite Eq. (8) as

The corrective term is in square brackets. In Fig. 10, the solid lines show the fits using Eq. (10), showing better agreement at smaller N_FI. At larger values of N_FI, the corrective term gets close to one; thus, the curves become roughly the same. The maximum fan-in for flux is over 100 with the 20 μA case and over 300 in the 6 μA case. Note these are much larger than the fan-in for current, which confirms our conclusion that flux scales better than current.

Finally, we look at the crosstalk current (I_ct) that flows back down synapses, as shown in Fig. 8. For the current method, I_ct is given by

Using our numbers for the case where N_FI = 16, L_sq = 40 pH, and I_c2 = 20 μA, we find a crosstalk current of 33.2 μA using Eq. (11), which compares well to our simulated value of 32.6 μA. This is actually much larger than the signal current for those parameters (6.4 μA). Note that the crosstalk current scales approximately as (1/N_FI), so the inability to increase the fan-in keeps the crosstalk large.

In the flux case, the crosstalk is given by

where L_J1 is the Josephson inductance of the input junction with I_c1, and we have approximated N_FI >> 1. Using our numbers for the case where N_FI = 128 and I_c2 = 20 μA, we find a crosstalk current of 1.3 μA using Eq. (12), the same as what we find in our simulation and much smaller than in the current method. Here, we also find that the crosstalk current scales approximately as (1/N_FI), so the ability to go to large N_FI helps reduce the crosstalk. The size of the crosstalk for flux can thus be made much smaller than for current, which once again confirms our conclusion that flux is indeed the better choice for fan-in.

In thin superconducting neuromorphic circuits, the fan-out is a digital process where the action potential is repeated and copied multiple times. Both flux and current methods give fan-out schemes that accomplish this successfully. Both can be scaled indefinitely to large networks, with a proportional cost in chip area and power dissipation. Since power dissipation scales with the junction count, and each junction dissipating roughly I_cΦ₀ energy per SFQ pulse, a rough estimate of the power dissipated for a 1-to-128 flux-based fan-out circuit using the I_c values given above is 44 aJ. Pulse timing can be preserved throughout subsequent layers of fan-out; if the constraint on timing is lifted, then the cost in area can be reduced. The flux method has a better scaling with the junction number, but current fan-out may have an advantage in latency because of the reduction in serial junctions, depending on the application. In general, although the splitter method seems to be slightly superior at the present time, it is entirely possible that when impedance matching, fabrication tolerances and timing fidelity are all taken into account for a large design, this method may end up being the preferred method of fan-out.

Fan-in is an analog process where the weighted synaptic signals are summed and compared to a threshold. Both a current method and the flux method were proposed. In scaling to large networks, the flux method appears to be superior to the current method, both in preserving a larger signal current and in keeping the crosstalk to a minimum. For small networks, it is possible that the current method could still be used, as it would eliminate the complication of mutual inductance elements.

In our study, we have largely ignored the problem of synaptic weighting, that is covered in other work.⁵ However, the architectural assumptions that we have made in this paper are consistent with typical SFQ weighting schemes that have been proposed in the literature. In addition, in our study of fan-in, we have been careful to consider the worst-case scenario for the signal, insisting that a single synaptic current be capable of invoking an action potential. In real networks, there will most likely be signals from many synapses simultaneously, increasing the total signal beyond the estimates in this paper.

As a final example, we consider a fully connected JJ neuron network on a grid routing scheme, similar to one proposed for optical fully connected networks but with symmetric vertical and horizontal inputs. Figure 11(a) shows a schematic for the basic structure of this routing scheme. The squares contain both the synapses and fan-in circuity and the wires are all digital SFQ. A flux-based fan-out element, such as that in Fig. 2(a), would be placed at each of the locations where a wire branches. Additional junctions may be required to match impedance on the longer runs between splitters depending on the size of the network. This layout would require two wiring layers for crossover and at least an additional layer for wiring biases to the JJs.

We estimate the size of this type of layout by making the following assumptions: The minimum pitch for the synapses is assumed to be 5 μm and is assumed to be the size-limiting circuit element compared to the fan-in inductors. The wiring between neurons is assumed to be on a 3 μm pitch, namely, a 2 μm wire with 1 μm space between wires. The junctions required for splitting and impedance matching are located within the wiring space. Reducing the wire width further is certainly possible but would likely require the use of more repeater junctions in the wiring for impedance matching. With these assumptions, we can estimate the size of the JJ neuron block including synapses and fan-In, as well as the space required for wiring along each side of the neuron.

Assuming that we have an equal number of inputs along each side, the size of the JJ neuron block is roughly 5 μm × N_neurons/4. The additional space required for the 2 JJs and JTL amplifier on the output does not change with the number of neurons. These additional circuit elements are assumed to occupy 10 μm along two sides of the neuron block. The wiring bundle along each side of a JJ neuron block would have roughly $Nneuron/4$ input wires and roughly $Nneuron/4$ output wires. In the example of 16 neurons shown in Fig. 11, this results in five wires along each side of any given neuron block. Figure 2(b) shows the chip length for a given number of neurons assuming the chip is square. Here, a 128 fully connected JJ neuron network would occupy roughly 4.9 × 4.9 mm².

We estimate that 5N² JJs would be needed for the fan-out and impedance matching and that 5 N JJs would be required for the fan in and reamplification. This gives us a total of roughly 83 000 junctions for a fully connected network of 128 JJ neurons. Additional junctions would be required for the synaptic function, for example, 2 N for a SQUID style synapse, but this additional junction count should scale as N rather than N². Both the size and junction count seem within the reach of modern JJ fabrication parameters, making this a very interesting network to investigate further.

Looking forward, there are many possible advantages to a future superconducting neuromorphic processor: (i) the ability to use superconducting multi-chip modules (MCMs)³³ to connect many chips together with no cost in dissipation; (ii) the availability²⁴ of low-power superconducting digital electronics to provide interfacing, multiplexing, and readout between analog units; (iii) the ability of the JJ neuron to attain biological realism¹ with only two junctions; and (iv) possible optical interface to truly expand to brain-sized networks. In this work, we demonstrate that both fan-in and fan-out of order 100 are completely reasonable for existing superconducting fabrication. Thus, we conclude that within the constraints of size, the fan-in and fan-out are not limiting factors for the future of superconducting neuromorphic computing.

M.L.S. and K.S. contributed equally to this work.

The authors wish to thank Manuel Castellanos Beltran and Jeffery Shainline for their helpful input and conversations. Contribution of the National Institute of Standards and Technology, not subject to copyright.

The data that support the findings of this study are available within the article.