An active dendritic tree can mitigate fan-in limitations in superconducting neurons

Motivations for developing artificial spiking neural systems include efficient hardware implementations of brain-inspired algorithms and construction of large-scale systems for studying the mechanisms of cognition. While much effort toward these ends employs semiconductor hardware based on silicon transistors,^1–5 superconducting electronics have also received considerable attention. Superconducting circuits based on Josephson junctions (JJs^6,7) have strengths that make them appealing for neural systems, including high speed, low energy consumption per operation, and native thresholding/spiking behaviors. In particular, two-junction superconducting quantum intereference devices (SQUIDs) are ubiquitous in superconducting electronics, and several efforts aim to utilize SQUIDs for various neuromorphic operations.^8–22 

In this work, we will use the following component definitions. A synapse is a circuit that receives a single input from another neuron and produces an electrical current circulating in a storage loop. A dendrite is a circuit that receives an input proportional to the electrical output of one or more synapses and/or dendrites, performs a transfer function on the sum of the inputs, and produces an electrical current circulating in a storage loop as the output. A neuron cell body (also known as a soma) receives input proportional to the electrical output of one or more synapses and/or dendrites, performs a threshold operation on the sum of the inputs, and produces an output pulse if the threshold is exceeded. Outputs from the neuron cell body are routed to many downstream synapses. Fan-in is the collection and localization of multiple synaptic or dendritic signals into a dendrite or neuron cell body.

In SQUID-based neurons, magnetic flux applied to the SQUID loop (⁠ $Φ a$⁠) serves as synaptic input. Only for $Φ a$ greater than some critical value of the flux (⁠ $Φ a th$⁠) will the SQUID produce a train of fluxons as an output signal.²³ For $Φ a < Φ a th$⁠, the SQUID will remain in a quiescent state. Additionally, $Φ a th$ can be tuned with a current bias, I_b. However, SQUID neurons differ significantly from their biological counterparts in that the response to $Φ a$ is periodic with a period of the single-flux quantum, $Φ 0 = h / 2 e$⁠. In this work, we consider the ramifications on fan-in if we choose to limit the maximum applied flux of the synapses to ensure a monotonic response and show that a dendritic arbor can significantly improve fan-in properties.

Fan-in was recently analyzed in superconducting neuromorphic circuits, wherein single-flux quanta are used as signals between neurons.²⁴ However, that work was not concerned with the case in which analog synaptic signals integrated and stored over time could drive a SQUID beyond the first half period of its response function. In the present study, we analyze fan-in in the context of leaky-integrator neuronal circuits that were originally designed for use in large-scale superconducting optoelectronic systems.^25–28 However, the conclusions of this paper should be applicable to a wide variety of SQUID neurons.

Noise prohibits biasing with I_b aribitrarily close to $2 I c$⁠. Single JJs are often biased in superconducting digital electronics with $I b / I c = 0.7$⁠. In a SQUID, this would correspond to $I b / I c = 1.4$⁠, as the SQUID comprises two JJs in parallel, and would require a minimum activity fraction of about 71%. A bias of $I b / I c = 1.8$ is an aggressive operating point and corresponds to $p / n ≈ 34 %$⁠. Such activity levels are incommensurate with principles of efficient information processing in spiking neural networks. Considerations for sparse coding suggests that only 1%–16% of neurons in the brain may be active at any time due to power considerations.³² Additionally, a recent study posits that only 1% of synapses need be active to generate action potentials in sensory neurons.³³ It, thus, appears that biologically realistic activity fractions and monotonic response are incompatible for superconducting point neurons.

However, the point neuron is not an accurate model of biological neurons. Instead, synaptic inputs are processed and filtered by an arbor of active dendrites that perform numerous computations,^34–36 including intermediate threshold functions between subsets of synapses and the soma³⁷ and detection of synaptic sequences.³⁸ Active dendrites can be significant for adaptation and plasticity,^39,40 can dramatically increase the information storage capacity relative to point neurons,⁴¹ and when modulated by inhibitory neurons, the dendritic tree can induce a given neuron to perform distinct computations at different times, enabling a given structural network to dynamically realize myriad functional networks.⁴² Active dendrites have also been identified for their role in reducing the necessary activity fraction to generate action potentials.³³ Discussion of dendritic processing in superconducting neural hardware is found in Ref. 27. We now turn our attention to the effects of active dendrites on the fan-in properties of SQUID neurons with monotonic response.

A schematic of a dendritic tree is shown in Fig. 2(a). The architecture consists of input synapses (shown in blue), multiple levels of dendritic hierarchy (yellow), and the final cell body (green). These components have been defined above, and all three can be implemented with SQUID circuits, a self-similarity that facilitates scalable design and fabrication.

We restrict attention to a homogeneous dendritic tree of the form shown in Fig. 2(a), wherein all dendrites receive the same number of inputs, n, which we refer to as the fan-in factor. The neuron cell body resides at level zero of the dendritic hierarchy, and synapses reside at level H, so the total number of synapses is $n H ≡ N$⁠. In Fig. 2(a), we show a tree with fan-in factor n = 2 and three levels of hierarchy for a total of N = 8 synapses. For a homogeneous dendritic tree, the relationship among the number of synapses, fan-in factor, and hierarchy is plotted in Fig. 2(b). Biological neurons are less uniform and more complex, but homogeneous trees are a good starting point for artificial systems. Figure 2(c) shows how the additional hardware for the dendritic arbor scales a function of the number of synapses.

The question remains: How do we limit the applied flux $Φ a dr$ to enforce monotonicity in practice? For this circuit, a careful choice of inductances will suffice. The mathematical details are given in the supplementary material, but ultimately only a single constraint among all of the inductances is necessary. Additionally, the intermediate DC loop allows the monotonic condition to be met across a wide range of fan-in factors with only $L di 2$ being a function of n; the SQUID and its input coil need not to be redesigned for different choices of n. The consequences of the DC loop are further explored in the supplementary material.

We have considered the implications of limiting the maximum flux input to all SQUIDs in a superconducting neural circuit so the response is monotonically increasing. We have found that limiting the applied flux introduces a constraint on the activity fraction of synapses required to reach the threshold, and the addition of a dendritic tree ameliorates the situation. This behavior is independent of most details of the circuit (such as whether or not a collection loop is used). The physical arguments presented here are derived from this decision to limit the applied flux to handle the ostensible “worst-case” scenario in which all synaptic inputs are fully saturated simultaneously. It is fair to question whether it is necessary to design our circuits around this extreme situation. The monotonicity issue could, for instance, be solved by immediately resetting all post-synaptic potentials to zero upon threshold. This is the standard behavior exhibited by most leaky integrate-and-fire models. However, implementing such a mechanism in superconducting hardware without compromising the speed and efficiency of superconducting neurons appears challenging. Additionally, we have argued elsewhere²⁷ that SQUID dendrites provide numerous opportunities for active, analog dendritic processing independent of the fan-in benefits described here. In that context, enforcement of monotonicity appears necessary. For these reasons, we contend that the best course of action is to allow synaptic signals to decay naturally without regard to thresholding events (which also preserves information) while limiting the applied flux in the manner described.

Still, one could argue we are over-preparing for the worst case scenario. Perhaps, we could leave the maximum possible applied flux to each SQUID unrestricted, and instances wherein SQUIDs are driven past a half-period of their response function will be sufficiently rare that we can ignore them in design. For general cognitive activity, we are likely to seek networks balanced at a critical point^44–46 between excessive synchronization (order) and insufficient correlation (disorder). When cognitive circuits are poised close to this critical point, neuronal avalanches⁴⁷ or cell assemblies^48,49 are observed to be characterized by a power-law⁵⁰ or lognormal⁵¹ distribution of sizes. A great deal of contemporary research⁵² indicates that operation near this critical point is advantageous for maximizing dynamic range^53,54 and the number of accessible metastable states⁵⁵ while supporting long-range correlations in network activity.⁵⁶ With either power-law or lognormal distributions, network activity engaging many neurons is less probable than the activity involving few neurons, but periods of the activity involving large numbers of neurons are not so improbable as to be neglected and may be crucial episodes for information integration across the network. The probability of large events does not decay exponentially and must, therefore, be accommodated in hardware.

We reiterate that the primary assumption entering Eq. (3) is that the maximum applied signal is limited to a certain value. We have considered the ramifications in the specific context of SQUID components, but similar considerations may apply to other hardware. We encourage the reader to consider whether similar arguments may affect their favorite neuromorphic thresholding elements. We also note that limiting the applied flux to $Φ 0 / 2$ may not always be advisable. From the activation function of Fig. 1(b), it is evident that a dendrite with two synapses performs XOR if each synapse couples $Φ 0 / 2$ into the receiving SQUID. When both synapses are active, the device operates outside the monotonic response. We hope this article does not stifle investigation of the full neural utility of engineered SQUID responses.