Superconducting electronic circuits have much to offer with regard to neuromorphic hardware. Superconducting quantum interference devices (SQUIDs) can serve as an active element to perform the thresholding operation of a neuron's soma. However, a SQUID has a response function that is periodic in the applied signal. We show theoretically that if one restricts the total input to a SQUID to maintain a monotonically increasing response, a large fraction of synapses must be active to drive a neuron to threshold. We then demonstrate that an active dendritic tree (also based on SQUIDs) can significantly reduce the fraction of synapses that must be active to drive the neuron to threshold. In this context, the inclusion of a dendritic tree provides dual benefits of enhancing computational abilities of each neuron and allowing the neuron to spike with sparse input activity.
Motivations for developing artificial spiking neural systems include efficient hardware implementations of brain-inspired algorithms and construction of large-scale systems for studying the mechanisms of cognition. While much effort toward these ends employs semiconductor hardware based on silicon transistors,1–5 superconducting electronics have also received considerable attention. Superconducting circuits based on Josephson junctions (JJs6,7) have strengths that make them appealing for neural systems, including high speed, low energy consumption per operation, and native thresholding/spiking behaviors. In particular, two-junction superconducting quantum intereference devices (SQUIDs) are ubiquitous in superconducting electronics, and several efforts aim to utilize SQUIDs for various neuromorphic operations.8–22
In this work, we will use the following component definitions. A synapse is a circuit that receives a single input from another neuron and produces an electrical current circulating in a storage loop. A dendrite is a circuit that receives an input proportional to the electrical output of one or more synapses and/or dendrites, performs a transfer function on the sum of the inputs, and produces an electrical current circulating in a storage loop as the output. A neuron cell body (also known as a soma) receives input proportional to the electrical output of one or more synapses and/or dendrites, performs a threshold operation on the sum of the inputs, and produces an output pulse if the threshold is exceeded. Outputs from the neuron cell body are routed to many downstream synapses. Fan-in is the collection and localization of multiple synaptic or dendritic signals into a dendrite or neuron cell body.
In SQUID-based neurons, magnetic flux applied to the SQUID loop () serves as synaptic input. Only for greater than some critical value of the flux () will the SQUID produce a train of fluxons as an output signal.23 For , the SQUID will remain in a quiescent state. Additionally, can be tuned with a current bias, Ib. However, SQUID neurons differ significantly from their biological counterparts in that the response to is periodic with a period of the single-flux quantum, . In this work, we consider the ramifications on fan-in if we choose to limit the maximum applied flux of the synapses to ensure a monotonic response and show that a dendritic arbor can significantly improve fan-in properties.
Fan-in was recently analyzed in superconducting neuromorphic circuits, wherein single-flux quanta are used as signals between neurons.24 However, that work was not concerned with the case in which analog synaptic signals integrated and stored over time could drive a SQUID beyond the first half period of its response function. In the present study, we analyze fan-in in the context of leaky-integrator neuronal circuits that were originally designed for use in large-scale superconducting optoelectronic systems.25–28 However, the conclusions of this paper should be applicable to a wide variety of SQUID neurons.
We begin by considering the simple SQUID circuit shown in Fig. 1(a). This is equivalent to a neuron in which synapses directly feed into the neuron cell body, also known as a point neuron. We assume that all net flux applied to the loop () is due to the weighted contributions of synaptic inputs. This assumption may be realized in practice using gradiometer-style pickup coils,29 on-chip magnetic shielding,30,31 and perhaps an additional tuning transformer and bias line.29 Under such conditions, the response of the SQUID would be monotonic for as seen in Fig. 1(b). Unfortunately, for this simple case, the cost of limiting to the monotonic regime is that a large fraction of synapses will need to be active to drive the neuron to threshold. For a neuron with n synapses, each capable of applying a maximum flux of , enforcing monotonicity requires that
To reach the threshold (), a critical number p of synaptic inputs must be active. For simplicity, we assume that each active input supplies the maximum value of the flux. This implies . Combining with the upper-bound in Eq. (1) gives
This value corresponds to the minimum synaptic activity fraction required to generate an output on the SQUID. In other words, p/n gives the minimum fraction of active synapses required for the neuron to fire.
A crucial task is to determine how low can be made in practice. The relevant control parameters are the SQUID bias current Ib, the SQUID loop inductance L, and the critical current of a single Josephson junction Ic. L and Ic are commonly combined in the dimensionless parameter . A value of is a standard choice due to noise considerations.23 In the present work, we have solved the SQUID circuit equations numerically (see Ref. 23 and the supplementary material for model details). We have kept fixed and defined as the minimum value of the applied flux for which at least one fluxon is produced by the SQUID. An empirical fit to these simulations gives p/n as a function of
where A = 0.540 and B = 0.466, and the form is inspired by the analytical solution available in the case.23
Noise prohibits biasing with Ib aribitrarily close to . Single JJs are often biased in superconducting digital electronics with . In a SQUID, this would correspond to , as the SQUID comprises two JJs in parallel, and would require a minimum activity fraction of about 71%. A bias of is an aggressive operating point and corresponds to . Such activity levels are incommensurate with principles of efficient information processing in spiking neural networks. Considerations for sparse coding suggests that only 1%–16% of neurons in the brain may be active at any time due to power considerations.32 Additionally, a recent study posits that only 1% of synapses need be active to generate action potentials in sensory neurons.33 It, thus, appears that biologically realistic activity fractions and monotonic response are incompatible for superconducting point neurons.
However, the point neuron is not an accurate model of biological neurons. Instead, synaptic inputs are processed and filtered by an arbor of active dendrites that perform numerous computations,34–36 including intermediate threshold functions between subsets of synapses and the soma37 and detection of synaptic sequences.38 Active dendrites can be significant for adaptation and plasticity,39,40 can dramatically increase the information storage capacity relative to point neurons,41 and when modulated by inhibitory neurons, the dendritic tree can induce a given neuron to perform distinct computations at different times, enabling a given structural network to dynamically realize myriad functional networks.42 Active dendrites have also been identified for their role in reducing the necessary activity fraction to generate action potentials.33 Discussion of dendritic processing in superconducting neural hardware is found in Ref. 27. We now turn our attention to the effects of active dendrites on the fan-in properties of SQUID neurons with monotonic response.
A schematic of a dendritic tree is shown in Fig. 2(a). The architecture consists of input synapses (shown in blue), multiple levels of dendritic hierarchy (yellow), and the final cell body (green). These components have been defined above, and all three can be implemented with SQUID circuits, a self-similarity that facilitates scalable design and fabrication.
We restrict attention to a homogeneous dendritic tree of the form shown in Fig. 2(a), wherein all dendrites receive the same number of inputs, n, which we refer to as the fan-in factor. The neuron cell body resides at level zero of the dendritic hierarchy, and synapses reside at level H, so the total number of synapses is . In Fig. 2(a), we show a tree with fan-in factor n = 2 and three levels of hierarchy for a total of N = 8 synapses. For a homogeneous dendritic tree, the relationship among the number of synapses, fan-in factor, and hierarchy is plotted in Fig. 2(b). Biological neurons are less uniform and more complex, but homogeneous trees are a good starting point for artificial systems. Figure 2(c) shows how the additional hardware for the dendritic arbor scales a function of the number of synapses.
Equation (3) is applicable to any dendrite or neuron cell body in the dendritic tree, provided the maximum applied flux is limited to . Working backward from the cell body, one can calculate that the minimum number of active synapses required to drive the neuron cell body to threshold is , and the fraction of synaptic activity for threshold is at least
Equation (3) is recovered as the special case of H = 1. The exponential dependence of the threshold activity fraction on H implies that dendritic trees can improve fan-in even with limited depth of the tree. This is illustrated in Fig. 3(a), where the activity fraction as a function of bias is plotted for dendritic trees of varying depths. We see that the point neuron case (H = 1) requires the highest activity fraction, but that the situation improves quickly with depth of the dendritic tree. For instance, with H = 5 and a conventional biasing of , only 17% of synapses need to be active—a significant decrease compared to for a point neuron. Figure 3(b) shows the tree depth required to hit a target activity fraction as a function of the bias current for . Again for , we see that a 1% activity fraction will require at least H = 14, in which case the number of dendrites and synapses are nearly the same. Neuromorphic superconducting systems will almost certainly be more noise tolerant than their digital counterparts,24 so higher bias conditions may be tolerable or even optimal. At these higher “neuromorphic biases,” the required depth of the tree for a given activity fraction is significantly lower. If is taken to be 1.8, a tree depth of only H = 5 is necessary for sub-1% threshold activity fraction. This dendritic tree would require dendrite fan-in of and about 1900 intermediate dendrites. Considering that every synapse requires a SQUID, the additional hardware fraction for this dendritic tree is less than 20%. In general, the number of dendrites, , will scale as for large N [Fig. 2(c)]. The addition of dendritic trees to high fan-in neurons will not be particularly cumbersome (area estimates for synapses can be found in Appendix B of Ref. 43) but can significantly reduce the activity fraction even down to 1%. Such biological values are abjectly impossible for point neurons, whose applied flux is limited to the range of monotonic response, providing a physical motivation for the use of dendritic trees in superconducting neurons to complement the computational motivations discussed above.
The energy consumption of the dendritic arbor itself deserves consideration. For future superconducting systems, dynamic power will dominate static power consumption. The total fraction of all units (synapses, dendrites, and soma) that must be active to reach the threshold is given by
The energy consumption of synapses and dendrites is unlikely to be the same for most technologies. In the optoelectronic case, for example, synaptic events are likely to cost significantly more power than an active dendrite. Additionally, it can be shown that the total number of active units is likely to be higher in the point neuron case for almost all reasonable bias conditions as the number of added dendrites is compensated for by the greatly reduced number of active synapses. This suggests that a dendritic arbor is unlikely to dominate the power budget of high fan-in neurons. Neuronal circuits based on superconducting loops have been proposed in the prior work, particularly with regard to optoelectronic systems.26 We show here an application of these fan-in considerations to the specific case of loop neurons introduced in that work. A circuit diagram is shown in Fig. 4. The dendritic integration (DI) loop integrates signals from activity present at that dendrite (or synapse). The saturation current of the DI loop corresponds to the maximally weighted active synapse discussed previously. A mutual inductor () couples this signal into a second loop, called the dendritic collection (DC) loop. This loop is not strictly necessary but allows for a more standardized design procedure as discussed below. The DC loop applies flux (the weighted contribution of afferent signals) through to the dendritic receiving (DR) loop. The DR loop forms the active component of the dendrite that has been the subject of our discussion thus far. Its output, a train of fluxons, is then coupled into another DI loop, allowing the chain to continue indefinitely. A schematic layout is provided in Fig. 4(b) to provide physical intuition about the circuit.
The question remains: How do we limit the applied flux to enforce monotonicity in practice? For this circuit, a careful choice of inductances will suffice. The mathematical details are given in the supplementary material, but ultimately only a single constraint among all of the inductances is necessary. Additionally, the intermediate DC loop allows the monotonic condition to be met across a wide range of fan-in factors with only being a function of n; the SQUID and its input coil need not to be redesigned for different choices of n. The consequences of the DC loop are further explored in the supplementary material.
We have considered the implications of limiting the maximum flux input to all SQUIDs in a superconducting neural circuit so the response is monotonically increasing. We have found that limiting the applied flux introduces a constraint on the activity fraction of synapses required to reach the threshold, and the addition of a dendritic tree ameliorates the situation. This behavior is independent of most details of the circuit (such as whether or not a collection loop is used). The physical arguments presented here are derived from this decision to limit the applied flux to handle the ostensible “worst-case” scenario in which all synaptic inputs are fully saturated simultaneously. It is fair to question whether it is necessary to design our circuits around this extreme situation. The monotonicity issue could, for instance, be solved by immediately resetting all post-synaptic potentials to zero upon threshold. This is the standard behavior exhibited by most leaky integrate-and-fire models. However, implementing such a mechanism in superconducting hardware without compromising the speed and efficiency of superconducting neurons appears challenging. Additionally, we have argued elsewhere27 that SQUID dendrites provide numerous opportunities for active, analog dendritic processing independent of the fan-in benefits described here. In that context, enforcement of monotonicity appears necessary. For these reasons, we contend that the best course of action is to allow synaptic signals to decay naturally without regard to thresholding events (which also preserves information) while limiting the applied flux in the manner described.
Still, one could argue we are over-preparing for the worst case scenario. Perhaps, we could leave the maximum possible applied flux to each SQUID unrestricted, and instances wherein SQUIDs are driven past a half-period of their response function will be sufficiently rare that we can ignore them in design. For general cognitive activity, we are likely to seek networks balanced at a critical point44–46 between excessive synchronization (order) and insufficient correlation (disorder). When cognitive circuits are poised close to this critical point, neuronal avalanches47 or cell assemblies48,49 are observed to be characterized by a power-law50 or lognormal51 distribution of sizes. A great deal of contemporary research52 indicates that operation near this critical point is advantageous for maximizing dynamic range53,54 and the number of accessible metastable states55 while supporting long-range correlations in network activity.56 With either power-law or lognormal distributions, network activity engaging many neurons is less probable than the activity involving few neurons, but periods of the activity involving large numbers of neurons are not so improbable as to be neglected and may be crucial episodes for information integration across the network. The probability of large events does not decay exponentially and must, therefore, be accommodated in hardware.
We reiterate that the primary assumption entering Eq. (3) is that the maximum applied signal is limited to a certain value. We have considered the ramifications in the specific context of SQUID components, but similar considerations may apply to other hardware. We encourage the reader to consider whether similar arguments may affect their favorite neuromorphic thresholding elements. We also note that limiting the applied flux to may not always be advisable. From the activation function of Fig. 1(b), it is evident that a dendrite with two synapses performs XOR if each synapse couples into the receiving SQUID. When both synapses are active, the device operates outside the monotonic response. We hope this article does not stifle investigation of the full neural utility of engineered SQUID responses.
See the supplementary material for the following three sections: the first gives a summary of the numerical solution to . The second walks through the circuit in Fig. 4, deriving a constraint among the various inductances that will ensure monotonic operation. Additionally, there is a description of likely parameter values for the various components, informed by InductEx simulations. The third section deals with a different circuit variant, in which the collection loop is omitted and single flux operation is considered.
We thank Dr. Ken Segall and Dr. Michael Schneider for helpful discussions. B.A.P. was supported under the financial assistance via Award No. 70NANB18H006 from the U.S. Department of Commerce, National Institute of Standards and Technology.
Conflict of Interest
The authors have no conflicts to disclose.
The data that support the findings of this study are available from the corresponding author upon reasonable request.