Brain-inspired computing with memristors: Challenges in devices, circuits, and systems

Brain-inspired computing aims to improve computing efficiency by simulating how the brain functions (see Fig. 1). This novel computing paradigm is becoming increasingly popular because of its numerous potential applications, such as self-driving, language translation, and even game playing. However, most traditional artificial neural networks (ANNs) are still running on von-Neumann computing architectures^1,2 implemented on transistor-based digital circuits, yielding an energy consumption that is orders of magnitude higher than biological brains.³ 

Novel circuit elements are therefore proposed to reduce this disparity in energy efficiency⁴ (see Fig. 1). One of the most promising candidates is the memristor, a two-terminal electronic element that directly relates electrical charge to flux.⁵ The memristor was first conceptualized by L. O. Chua in 1971⁶ and was linked with physical devices by Williams's team at Hewlett-Packard (HP) Labs in 2008.⁷ Memristors are particularly suitable for serving as synapses in brain-inspired computing architectures thanks to their intrinsic dynamics, analog behavior, nonvolatility, high speed, low power, high density, and great scalability.⁸ 

In this review, we first survey the progress of understanding the working mechanisms of the memristors. Efforts have been made to quantitatively describe the experimentally observed resistive switching. Frameworks have been developed to incorporate multiphysics, including the thermal, chemical, and electrical effects. Such physical modeling could be abstracted to form circuit models, which are of reasonable accuracy and dramatically reduce the complexity for circuit designers.

The main functional units of the brain-inspired computing systems are synapses and neurons that can be directly simulated using memristive synapses and neurons (see Fig. 1). The electrical-bias-history dependent resistance of memristors makes them capable of simulating the synaptic and neural dynamics of the brain, including various short/long-term synaptic plasticities and neural leaky integration-and-fire, which has led to compact and low-energy artificial synapses and neurons. Such devices, particularly artificial synapses, could be conveniently grouped into crossbars to implement weighted sums or vector-matrix multiplications on-chip.⁹ These memristive crossbars implement spiking neural networks (SNNs),¹⁰ fully connected ANNs,¹¹ convolutional neural networks (CNNs),^12,13 and Hopfield recurrent neural networks (RNNs),^14,15 which can be extended to deep neural networks (DNNs)^13,16 for a variety of applications including image recognition,¹⁷ sparse coding,^18,19 temporal sequence classification,²⁰ and reinforcement learning (RL).^21,22

We provide our perspectives on the developments of memristive devices, circuits, systems and algorithms and the advancement of the memristor-based brain-inspired computing (see Fig. 1). The associated challenges, opportunities, and possible solutions will also be discussed.

Memristive switching has been realized in different material systems based on different underlying physical mechanisms (see Fig. 2). In this section, we review the electrochemical metallization, valence change, thermal electrical effect, and phase change mechanisms (PCM) that power the memristive switching and the associated physical/compact modeling.

In 1971, L. O. Chua first theoretically formulated the concept of memristors.⁶ Almost 40 years later, the HP lab experimentally realized the first memristor with a metal/insulator/metal (MIM) structure⁷ and provided empirical evidence to support the resistive switching model in the metal oxide system,³³ which established a link between the theory and the experimental results. There are mainly four types of mechanisms for memristors: electrochemical metallization mechanism (ECM),^34–36 valence change mechanism (VCM),^37–39 thermochemical mechanism (TCM),^40,41 and phase change mechanism (PCM).^42–44 Memristors based on other mechanisms, such as Mott transition,^15,45 photonic-induced switching,^46,47 and ferroelectric transition,^48,49 are also investigated as emerging devices for brain-inspired computing systems.

Electrochemical metallization (ECM) based devices utilize the migration of metal ions to realize the memristive switching. A typical ECM device has a MIM structure. The two metal electrodes are often called the anode and the cathode, according to their electrochemical activities. Active metals, in most cases Ag²⁵ and Cu,⁵⁰ are used as the anodes in ECM devices, while Pt, Au, TiN, ITO, and W are often used as the counter electrode.^7,51,52 A forming process⁵³ is usually needed to incorporate the cations into the solid electrolyte, before stable resistive switching behavior can be obtained. During the forming and switching processes, redox reactions will happen both at metal/electrolyte interfaces^54,55 and inside the electrolyte.^25,56 If a positive voltage is applied to the anode, oxidation reactions will take place at the anode interface, and counter reactions may happen simultaneously at the cathode interface to maintain charge neutrality.³⁵ Following the electric field, these cations will migrate toward the cathode and get reduced inside the electrolyte or at the cathode interface, leading to different filament growth modes.⁵⁶ As long as voltage is applied across the ECM device, the redox and migration would continue, until a conductive filament is formed between the two electrodes, as shown in Fig. 2(a). The formation for the conductive filament switches the device from a high resistance state (HRS) to a low resistance state (LRS)⁵⁷ [see Fig. 2(c)]. When opposite voltage is applied, the redox process and cation migration will take place conversely [see Fig. 2(b)], resulting in the rupture of the conductive filament and a resistance change from LRS to HRS [see Fig. 2(d)]. In ECM, the filament can sometimes be fully dissolved in the solid electrolyte during the RESET process, providing large on/off ratios.^57,58 The growth direction and geometry of the conductive filament strongly depend on the characteristics of the electrolyte layer and the electrode material. Typically, various oxides,^36,59 sulfides,^60,61 selenides,⁶² amorphous silicon,^63,64 and polymers^39,65 are used as the insulating materials. In situ observations via TEM^56,66 and SEM^67,68 have been carried out to unveil the cation migration and filament growth/dissolution processes. In addition to nonvolatile behavior⁶⁹ or the so-called long-term plasticity, volatile or short-term plasticity is also observed in these devices. A typical example is the recently developed diffusive memristor,⁷⁰ where the formed filament breaks into incremental metal nanoclusters spontaneously in order to minimize the interfacial energy, thus functioning as a selector device [see Fig. 2(e)] and mimicking the short-term plasticity⁷¹ in biological synapses [see Fig. 2(f)]. In general, ECM devices have scalability down to atomic scale,⁷² the ability of working under ultralow voltages,^73,74 and a large dynamic range. However, they may suffer from limitations in terms of weight precision and weight update linearity and symmetry.

The valence change mechanism (VCM) is responsible for another major type of memristor. The first memristor fabricated by HP is a VCM device of a Pt/TiO₂/Pt structure. Judging by the internal structure of the device during and after switching, the VCM devices can be classified into the filamentary type and the nonfilamentary type.

VCM devices also possess MIM structures. Similar to ECM cells, the resistive switching behavior in filamentary VCM devices can also be attributed to the formation/dissolution of conductive filaments.⁷⁵ However, the formation of filaments in VCM is often a result of the migration of oxygen vacancies or oxygen anions and the subsequent redox reactions that usually lead to the formation of a suboxide phase with higher conductivity.^28,76 In the as-deposited oxide films, especially those prepared by atomic layer deposition, the films are close to their stoichiometry and the density of oxygen vacancies is usually low, which are not capable of filament-based resistive switching.⁷⁷ Hence, an electroforming process is required to form conductive filaments [see Fig. 3(e)], which is accompanied by the eruption of oxygen gas.⁵⁵ When a reversed voltage is applied, the filaments can be dissolved [see Fig. 3(f)], returning the device back to HRS. Typical materials used for the oxide layer in filamentary type memristors are HfO₂,⁷⁸ TaO_x,⁷⁹ TiO₂,⁸⁰ WO₃,⁸¹ SrTiO₃,⁸² ZnO,²⁷ and SiO_2.⁸³ Metals such as Pt, Au, Ir, W, and Ta are often used as the electrode materials in university labs.^7,52 In situ observations of filament growth/dissolution in VCM have also been performed using TEM⁸⁴ on devices based on ZnO,²⁷ TaO_x,⁸⁴ and HfO_x.⁸⁵ In order to avoid the high-voltage forming process, which is not favorable for array-level neuromorphic applications, optimizations on the materials and device structures of VCM have been carried out, for example, by introducing a bilayer structure with an oxygen deficient layer.⁷⁷ Such a bilayer structure is also found to be able to improve the gradual switching behavior that is desirable for neural network applications.^86,87 Similar to ECM devices, VCM devices can be scaled down to a few nanometers.⁸⁸ Integration with a high-density 3D structure has also been demonstrated.⁸⁹ To date, VCM based memristors are commercially available—memory chips by Panasonic and Fujitsu based on VCM devices entered the market in 2016. However, similar to ECM devices, the fundamental filament formation and dissolution processes in VCM also lead to inevitable device variations, and the weight update linearity/symmetry also requires optimizations for neuromorphic systems with online learning capability.

A potential approach to decrease the device-to-device (D2D) and cycle-to-cycle (C2C) variations might be to avoid the localized, filamentary resistive switching and to explore memristors based on a nonfilamentary mechanism. A typical example is the memristive device based on redox processes occurring at the electrode/oxide interfaces, which lead to varied thicknesses of interfacial layers and thus different resistance states.⁹⁰ Such redox processes taking place uniformly at the interface avoid the stochastic filament formation process and ensure good D2D and C2C uniformity. The varied thickness of the interfacial layer as a function of applied signals has been directly observed,³⁰ where a thicker interfacial layer leads to a switching from LRS to HRS [see Fig. 4(b)], and an inverted process accounts for the SET process, as shown in Fig. 4(a). The typical material system used for nonfilamentary VCM devices are $Ti/Pr0.7Ca0.3MnO3$ (PCMO),⁹¹ ZrO₂:Y (YSZ)/PCMO,⁹² and InGaZnO (IGZO)/α-IGZO.⁹³ In addition to the improved uniformity, such nonfilamentary VCM based artificial synapses also show more incremental weight states in general. However, they still suffer from limitations in symmetry and linearity, which is not advantageous for the online training of ANNs. Different from filamentary type devices where the filament can be quickly heated up by Joule heating to accelerate atom motion for a fast switching and then be quickly cooled down to maintain the positions of moved atoms for a long retention, the nonfilamentary type of device typically suffers from a slow switching speed and a poor retention.⁹⁴ Therefore, one has to resort to a new mechanism for achieving a high switching speed and long retention simultaneously, such as borrowing the concept of battery in a three terminal device.⁹⁵ 

The thermochemical memory (TCM) cell has material systems and device structures akin to those of valence change memory cells. However, herein the main driving force of the stoichiometric changes and redox reactions are thermal effects,⁴¹ which result in a variation of local conductivity in conducting filaments. As a result, the SET (potentiation) and RESET (depression) processes in TCM cells can be realized by voltage signals with the same polarity, that is, a unipolar operation process [see Fig. 3(a)], in contrast to the bipolar operation [see Fig. 3(d)] in VCM and ECM devices. More specifically, the SET and RESET processes are caused by a competition between thermophoresis and diffusion processes, which move the oxygen vacancies into and out of the conduction channel volume, respectively.⁹⁶ The thermophoresis process is dominant when starting from a HRS, which increases the concentration of oxygen vacancies in the filament region and leads to potentiation,⁹⁷ as shown in Fig. 3(b). On the contrary, the diffusion process becomes dominant in on-state devices due to the large concentration gradient of oxygen vacancies, and the filament is dissolved upon the application of voltage signals with the same polarity [see Fig. 3(c)]. It is common that thermochemical and valence change mechanisms coexist in the same metal oxide based memristors, depending on the operation conditions, such as the current compliance.^98,99 NiO is a typical material that demonstrates thermochemical reactions during resistive switching,¹⁰⁰ while thermochemical reactions in TiO₂,¹⁰¹ HfO₂,¹⁰¹ Fe2O₃,¹⁰¹ CoO,¹⁰² Al₂O₃,¹⁰³ and SiO_x¹⁰⁴ have also been investigated.

Phase change memory (PCM) is likely the most mature memristor technology to date,¹⁰⁵ where the SET and RESET processes are based on the crystallization and amorphization of the PCM.¹⁰⁶ During RESET, a high but short voltage pulse is applied, melting part of the phase change materials. Such melted materials cool down very quickly and an amorphous region will be formed, resulting in the resistance change from LRS [see Fig. 4(c)] to HRS [see Fig. 4(d)]. Based on the nucleation rate of the phase change material, the SET process can be classified as nucleation dominated and growth dominated.¹⁰⁷ Typically, Ge₂Sb₂Te₅ (GST)-based devices are nucleation dominated [see Figs. 4(e) and 4(g)], and AgInSbTe (AIST)-based devices are growth dominated [see Figs. 4(f) and 4(h)]. In nucleation dominated PCM devices, incubation first takes place inside the amorphous region, followed by crystal growth. The nanocrystals grow gradually, until the amorphous region is finally transformed into a polycrystalline structure. During such a process, the PCM device gradually switches from HRS to LRS.¹⁰⁸ However, for growth dominated material systems, the crystallization always occurs at the amorphous/crystalline interface. The amorphous material gradually transforms into the crystalline state and finally forms a single crystal once the SET process is complete. For both types of materials, the typical crystallization temperature is about 500–700 K.¹⁰⁹ A voltage pulse higher than the threshold voltage is needed to elevate the temperature of the amorphous region and induce the incubation and crystallization processes, whereas the temperature cannot be too high (∼900 K) to melt the phase change material.

Moreover, the crystallization rate also plays an important role in PCM devices, as it could be the major factor that limits the operation speed of phase change mechanism based memristors. The main reason for the low crystallization speed, usually in ∼nanosecond scale for GST based devices¹¹⁰ and in ∼microsecond scale for AIST based devices,¹¹¹ is still under debate. A dominant explanation³¹ attributes the low crystallization speed to the difference of atomic structures between the amorphous and crystalline states. Preseeding nuclei inside the amorphous region can bypass the incubation and improve the crystallization speed.¹¹² However, such a method requires a prenucleation process, which is not desired in practical applications. Novel phase change materials such as ScSbTe (SST) are also investigated,¹¹³ which has a cubic atomic structure in the amorphous state. This allows it to switch between the amorphous state and the crystalline state with ultrafast speed (<500 fs), making it promising for high-speed applications. In addition, PCM devices also suffer from the resistance shift in HRS, as the amorphous state is metastable due to stress release¹¹⁴ and trap dynamics,¹¹⁵ which could be harmful for device operation. Moreover, the thermal dissipation in PCM and the large driving current required during RESET can be fatal and can prevent further applications in ultralarge scale integration. Hence, structural and material optimization has been performed, for example, by inserting a capping layer at the top interface between the electrode and the phase change material to maintain the material temperate during operations¹¹⁶ or by changing the geometry of the PCM and the bottom electrode (BE) to fully utilize the heat generated during the SET/RESET process.^117,118 As for materials of PCM devices, the combination of tellurium with germanium and antimony has been the focus for material optimizations. New materials, especially tellurides, are still under intensive investigations. In general, PCM-based artificial synapses have demonstrated manufacturing maturity, high scalability, large scale integration, and good stability, making them a promising candidate for neuromorphic systems. However, the amorphization mechanism decides that the weight update in PCM synapses is inherently asymmetric due to the abrupt RESET, which significantly affects network performances in many cases. It should also be noted that the dynamic process of crystallization has also been exploited to build artificial neurons.¹¹⁹ 

The main figure of merit for memristor based artificial synapses includes the weight update linearity, symmetry, resistance level stability, weight precision, energy consumption, scalability, speed, endurance, and retention. The desired performance could depend on detailed applications. For example, high weight update linearity, symmetry, and a large number of states are required to achieve online learning in neuromorphic systems, which will also demand high speed and high endurance because of the frequent update of synaptic weights. To date, most memristor-based artificial synapses suffer from insufficient linearity and symmetry,¹²⁰ although a number of studies have been devoted to optimizing such aspects.^121–124 A modification on the amplitude or width of voltage pulses¹²⁵ can also improve the linearity and symmetry of memristive synapses at the expense of cumbersome periphery circuitry to generate such complicated signals. The operation speed of memristive synapses can reach a few nanoseconds^43,75 with a record speed of 85 ps,¹²⁶ which is comparable to DRAM¹²⁷ and other transistor-based devices or memory circuits¹²⁸ and is much faster than Flash. However, the above requirements can be largely relaxed if only inference is needed, but long retention will be required instead. Furthermore, low energy consumption and high device scalability could be needed for both online learning and inference in order to implement large neural networks on chip. State-of-the-art devices can have an energy consumption as low as few femtojoule²⁶ or even subfemtojoule¹²⁹ per spike, which is already comparable to that of biological synapses. All the memristive synapses mentioned above can take up a cell area of 4F² in actual circuit layout³¹ (F is the footprint of a certain technology node). Compared with DRAM (4F²), static random access memory (SRAM) (140F²), FeRAM (22F²), and magnetic random access memory (MRAM) (20F²),⁴³ such a cell area is competitive and shows the promise of memristors. Looking into array-level requirements, the major challenges lie in the intrinsic device variations, the lack of ideal selectors, and the immature large scale integration technology. The selector issue might be avoided when the array scale is reduced but will be necessary in large scale arrays in order to ensure correct updating and reading. Hence, optimization and exploration on the physics, device, and array level are necessary to ensure future applications of memristors. An interesting trend in device optimizations emerges recently by the introduction of new material systems such as two dimensional (2D) materials,¹³⁰ Mott insulators,^45,131,132 ferroelectric materials,^133,134 magnetic materials,¹³⁵ perovskites,¹³⁶ and organic materials¹³⁷ and/or novel synaptic structures such as synaptic transistors¹³⁰ and ferroelectric transistors.¹³⁴ This has led to a flourish in brain-inspired computing with memristors and opens up new opportunities for high-performance synaptic devices.

The theoretical modeling of memristors could be roughly classified into two categories, i.e., the physical model and the compact model. The physical modeling of a memristor aims to correlate the electrical properties with the underlying physical mechanisms. Guan et al. developed a hybrid model that took into account the electron conduction and ion movement.¹⁵⁵ Onofrio et al. simulated ultrafast resistance switching based on molecular dynamics.¹⁵⁶ 

Recently, the nanoparticle diffusion due to the minimization of interfacial energy has been observed. A direct emulation of both short- and long-term plasticities of synapses is enabled by the diffusive memristor for brain-inspired computing systems. To link the electrical, nanomechanical, and heat degrees of freedom, the resistance of the diffusive memristor model can be approximated as the sum of tunneling resistances between N − 1 nanoparticle islands⁷⁰ 

where R_t is the tunneling resistance amplitude, x₀ and x_N are the spatial coordinates of the input and output terminals (⁠$x0<x1<x2<···<xN−1<xN$⁠), and l is the effective tunneling length. Let L denote the half size of the device. To describe the nanoparticle diffusion and memristive dynamics, an overdamped Langevin equation is employed for metallic nanoparticles trapped by a potential U, responsible for the interfacial energy, and subject to a random force ξ_i, and the magnitude is determined by temperature T⁷⁰ 

where viscosity η is balanced by all other forces acting on nanoparticles. When the voltage V(t) is on, $αV(t)/L$ represents a bias in the electric field $E=V(t)/L$ affecting nanoparticles with induced charge α. The random force $2ηkBTξi$ describes the diffusion of the nanoparticles.

Compact models are abstracted from the physical models with reduced computational complexity preserved ability to capture the essence of memristive dynamics. Such models benefit memristor-based circuit designs, usually calibrated on empirical observations.¹⁵⁷ To represent the link between magnetic flux (⁠$ϕ$⁠) and electric charge (q), the memristance (M(q) in $Ω$⁠) of a memristor can be defined as⁶ 

For example, a compact model for TaO_x-based memristors is based on the following coupled equations:¹⁴⁵ 

where G(x, v) is the conductance of a TaO_x-based memristor, x denotes the state variable for a memristor, v is a varying voltage, and F(x, v) describes the dynamical evolution of state x and varying voltage v. The device conductance is approximated by the parallel combination¹⁴⁵ 

where G_m, a, and b are constants. To better fit the experimental data, the model can be modified as follows:¹⁴⁵ 

where A, $σoff, xoff$⁠, and β are constants and p is the power. This compact model shows success in simulating large memristive arrays for brain-inspired systems with high density. Additionally, the physical insight derived from the model helps understanding the physical mechanism of the memristive devices.

A comparative summary of different memristive models is given in Table I. The initial definition of the memristor is proposed by Chua's model,⁶ which defines the memristor. The HP model⁷ first links the memristor definition with the linear ion drift effects along with nonlinear effects.³³ Simmons tunneling barrier modeling methods^139–141 are developed into SPICE compatible physics-based memristive models, which have been adopted and refined by the Generalized model.¹⁴² Threshold effects^143,144 are also considered in memristive models. In Stanford models, the filament gap is considered as the state variable.^147,148 Zhang's synaptic model¹⁴⁶ features voltage-threshold and can mimic the synaptic behavior of recent memristive devices with different materials. Unipolar devices and temperature effects^149,150 have also been investigated. Other bipolar switching mechanisms,¹⁵¹ e.g., the change in conductive filament (CF) size,¹⁵² and other factors are also taken into account. The diffusive model⁷⁰ enables a direct emulation of both short- and long-term plasticities of brain-inspired synapses.

In this section, the spike-timing-dependent plasticity (STDP) rule will first be introduced. Second, the major units of the brain-inspired computing systems such as synapses and neurons will be explored. In addition, different memristor-based neuronal models and synaptic learning mechanisms will be discussed.

The most intensively studied functional units of the brain-inspired computing systems are synapses and neurons. Synaptic learning rules adopted in DNNs were primarily achieved via the backpropagation (BP) algorithm,¹⁵⁸ which requires a backward pass of the gradient computation in the neural networks. Compared with the BP algorithm, a more biorealistic approach is the STDP learning rule¹⁵⁹ (see Fig. 5). The STDP concept is generalized from the neuroscience field.^160,161 The synaptic weight increases or decreases if the preneuron spikes before or after the postneuron, respectively,

where $A+, A−, τ+$⁠, and $τ−$ are constants. t_post and t_pre are the time-constants of pre- and postsynaptic firings, respectively, which define $Δt=tpost−tpre$⁠.

The early observation of STDP was made on Ag/Si-based memristive devices.⁶³ A similar STDP behavior has also been observed in Cu₂O-based memristive devices.¹⁶² The effort in engineering TaO_x-based first order and second order memristors yields more uniform and controllable STDP behaviors.^121,163 Additionally, 2T1M and 1T1M circuit schemes that simulate the STDP functionality on HfO_x-based memristors for simple online pattern learning and recognition have been reported.¹⁶⁴ To get rid of the waveform engineering that demands the overlapping of pre- and postsynaptic signals, second order memristors and memristors of diffusive dynamics have been developed with intrinsic timing capabilities.^70,163

Memristors are also used as key components of artificial neurons. Until now, Mott-insulators, phase-changes, ferromagnets, and diffusive-ion-based memristive devices have been explored for this purpose.^{10,12,45,165–171} In 2013, Pickett et al. used two NbO₂ Mott-memristors as the counterpart of ion-channels of a biological neuron and capacitors as the cell membrane to emulate the Hodgkin-Huxley (H-H) model.⁴⁵ Besides, Moon et al. developed the NbO₂-based Mott-memristor oscillator for neuron applications.¹⁰ Followed by this, Lin et al.¹⁷⁰ and Zhang et al.¹⁶⁸ demonstrated an Leaky Integrate and Fire (LIF) neuron using a VO₂-based Mott-memristor and an Ag-based diffusive memristor, respectively. Pablo et al. reported an LIF neuron using a single Mott-based memristor, which implemented three basic functions of spiking neurons, i.e., leak, integration, and fire. In 2018, Wang et al.¹² employed diffusive memristor neurons in a fully memristive neural network. In the same year, an integrated capacitive neural network with a memcapacitive leaky LIF neurotransistor has been built.¹⁶⁹ 

Brain-inspired computing utilizes hardware to simulate neurobiological models such as SNNs and ANNs. Neurons in ANNs are characterized by single, static, continuous-valued activation. Biological neurons use discrete spikes to compute and transmit information. SNNs are therefore more biologically realistic than ANNs. However, training DNNs (such as CNNs and RNNs) using SNN models is still challenging. Spiking the neurons' transfer function is usually nondifferentiable, which prevents using the BP algorithm. SNNs still lag behind ANNs in terms of accuracy, but SNNs typically require much fewer operations and are better candidates for processing spatiotemporal data.

In this section, first, the literature on memristive SNNs will be surveyed. Second, the progress of hardware implementation of memristive fully connected ANNs is discussed, with a focus on image recognition. Third, reports on memristive CNNs for DNNs will be summarized, with the same focus on image recognition. Finally, we will introduce memristive Hopfield RNNs to achieve different tasks such as route searching and speech recognition.

Brain-inspired computing models can be broadly divided into ANNs, SNNs, and other extended models.^172,173 Among them, ANNs are mainly used in machine learning, especially deep learning. A more faithful way to mimic how the brain processes information is the SNN. Hardware implementations of SNNs have been demonstrated by IBM TrueNorth, Intel Loihi,¹⁷⁴ and Tsinghua Tianjic.¹⁷⁵ In this subsection, we mainly focus on several unique characteristics of SNNs. First, the outputs of the spiking neurons are time-space-encoded pulses. Second, the timing domain information is encoded by the output of the SNN neuron. Therefore, multiple neurons encode information in a spatiotemporal two dimensional (2D) space.

A learning rule for feedforward SNNs by BP is that the temporal error at the output has been derived.¹⁷⁶ Progress in both unsupervised learning and supervised learning is reviewed.¹⁷⁷ The Winner takes all (WTA) and STDP rules are utilized to build an SNN for position detection in circuit simulation.¹⁷⁸ Recently, brain-inspired SNN learning algorithms have been developed by Wolfgang Maass.^179–181 In addition, memristor-based synapses are combined with CMOS-based LIF output neurons to perform visual pattern recognition tasks.¹⁸² For example, a large SNN with simplified STDP synapses is simulated to recognize mixed National Institute of Standards and Technology (MNIST) digits under different variations.¹⁸³ A CMOS-based brain-inspired hardware system has emulated different spike types observed in cortical pyramidal neurons and coincidence detection.¹⁸⁴ A simple probabilistic WTA network has been reported by employing memristor synapses with weight-dependent STDP behavior.¹⁸⁵ An all-memristive SNN, using phase-change synapses and neurons, has been developed¹⁸⁶ to detect the correlation between input streams. A Hopfield SNN was built using an 11k-bit array of Mo/PCMO memristive synapses for image recognition.¹⁰ In 2016, Serb et al.¹⁸⁵ reported a 4 × 2 SNN trained using the intrinsic STDP of the memristors with WTA neurons, which could cluster simple patterns after the unsupervised training. Pantazi et al.¹⁸⁶ demonstrated a single layer SNN with both phase change memristive synapses and neurons. In 2017, Milo et al. proposed a spiking HNN model with STDP based on 30 excitatory synapses and 30 inhibitory synapses combined with six fully connected digital neurons.¹⁸⁷ In this work, the training, recall, and stability via external stimulation and recurrent co-operation and competition were demonstrated.

Many advances in understanding and modeling SNNs are from the neuroscience research. For instance, an SNN with STDP learning is simulated to differentiate spoken words and find temporal encoding rather than simple rate encoding.²⁰ An SNN architecture called NeuCube¹⁸⁸ has been used for modeling spatio- and spectrotemporal brain data (STBD).

A common learning method for SNNs is the STDP algorithm, which cannot guarantee a high performance in general learning tasks. Therefore, SNNs are more suitable for low accuracy computing applications (see Tables II and III). SNNs are mostly bioplausible, real-time, low power, and spatiotemporal and suitable for brain-inspired computing systems, while ANNs are fast speed, computationally intensive, and suitable for current von-Neumann computing systems. Although BP-based ANNs have achieved many successes in pattern recognition tasks, SNN-based brain-inspired systems are more biorealistic and can be implemented on memristive devices with suitable STDP behaviors (see Table III). The detailed comparison between SNNs and ANNs is also shown in Table II.

Reinforcement learning (RL), which can acquire knowledge and solve problems without human knowledge or supervision, is a promising approach for machine learning. During the RL process, the learning agent makes action and receives rewards from the environment in every time step to achieve the target by the optimal policy. RL can be implemented on both SNNs and ANNs. A hardware-friendly RL algorithm based on memristive SNNs has been proposed²² to update the action-value (Q-value) function and optimize the policy as follows:

where $Q(st,at)$ is the Q value after action $at+1$ in state $st+1$ and $rt,+αQ(st+1,at+1)−Q(st,at)$ represents the time difference (TD). In addition, learning rate η indicates the importance of previous experience. The memristive SNN RL algorithm has been applied to an acrobat system,²² where the agent aims to take less steps and TD to reach the target. RL in ANNs will be discussed in Sec. IV B.

Brain-inspired computing with ANNs based on emerging non-volatile memory (NVM) devices has attracted significant attention.⁹⁴ Initial studies of memristive ANNs mainly focused on fully connected neural networks [see Fig. 6(a)].^190,191 Compared with memristive SNNs, memristive fully connected neural networks for ANNs are much faster and can solve relatively more complex tasks (see Table III). In 1961, crossbars of resistive elements were proposed by Steinbuch¹⁹² to parallelize vector-matrix multiplication utilizing Kirchhoff's current law. However, the proposal was less attractive than digital implementations due to the lack of the tunable resistor until Snider¹⁹³ reinvented this idea with memristors in 2008. Memristors, initially proposed for nonvolatile memory applications, were then engineered for computing application,^63,94,194 which directly use intrinsic physics laws to obviate the large energy and time overhead incurred by frequent data shuttling in von-Neumann systems. Alibart et al.¹⁹⁵ utilized a small array to classify 3 × 3 in 2013. Park et al.¹⁹⁶ used a PCMO memristor array consisting of 192 cells to classify the electroencephalography signals. Also in 2015, Prezioso et al.¹⁹⁰ reported a 12 × 12 passive Al₂O₃/TiO_2-x-based memristive array to classify the 3 × 3 patterns. Followed by this, Burr et al.¹⁹⁷ employed 165 000 phase change memristors that were integrated to implement a three-layer perceptron to classify the MNIST hand written digits using BP on-chip learning methods. Yu et al.¹⁹⁸ reported a three-layer perceptron built on the 16 Mb binary resistive switching memory macrochip for online training to classify a resized MNIST binary network with an accuracy of 96.5%. In 2017, Choi et al.¹⁹⁹ demonstrated an in situ unsupervised training of a 9 × 2 TaO_x-based memristive array to analyze the breast cancer data. In addition, Yao et al.²⁰⁰ used an 8 × 128 1T1M array of HfAl_yO_x-based memristors to classify the Yale face database. In 2018, Hu et al.¹⁹¹ used a 128 × 64 1T1M array of HfO_x memristors to classify the MNIST dataset. The BP algorithm for in situ training [see Fig. 6(b)] was employed by Li et al.¹⁸⁹ Based on the same 128 × 64 1T1M array [see Fig. 6(c)], a two-layer perception [see Fig. 6(d)] was built to recognize reduced size MNIST digits. Ambrogio et al.²⁰¹ demonstrated an artificial synapse by pairing phase change memristors with three-transistor-one-capacitor (3T1C) structures to facilitate linear and symmetrical weight updates, leading to 97.95% accuracy in MNIST digit recognition. Bayat et al.²⁰² demonstrated analog neurons to classify 4 × 4 patterns with Al₂O₃/TiO_2-x-based memristive synapses. Compared with the CMOS approach, analog in situ training offers parallel signal processing with low power consumption.²⁰³ 

As mentioned previously, RL inspired by cognitive neuroscience can make use of the ANNs built on analog memristive arrays.^21,204 The parallel and energy-efficient in situ reinforcement learning with a 3-layer fully connected memristive deep-Q network²¹ has been implemented on a 128 × 64 1T1M array with applications to classic control problems including cart-pole and mountain car games.

DNNs have attracted considerable attention since CNNs show promising performance in pattern recognition. Additionally, CNNs are resilient to image translation and scaling (see Table III). Input data convolve with a set of shared kernels in each convolutional layer [see Fig. 7(a)]. Deep CNNs consist of cascaded layers where each convolutional layer is usually followed by a subsampling layer [see Fig. 7(a)],^18,19 which performs a max pooling or averaging operation to reduce the size of input data. Ultimately, the final layer can be a fully connected neural network. Shared weight kernels are convolved with the entire maps, and the training time is therefore significantly reduced in comparison to fully connected neural networks (see Table III). Compared with memristive SNNs and fully connected ANNs, memristive CNNs can solve a very complex task (see Table III).

To achieve memristive CNNs on-chip, the kernel operation²⁰⁵ and memristive convolutions for feature extraction²⁰⁶ have been proposed.^207–209 Memristor-based 2D convolutional kernels were investigated by Li et al.²⁰³ in 2017. Another important issue for memristive DNNs, the pooling method in memristor-based brain-inspired circuits, was also presented.^18,19 A mini CNN with both memristive synapses and memristive neurons was demonstrated by Wang et al.¹² in 2018. The fully connected layer was trained by a simplified STDP rule. In 2018, a hybrid software-hardware CNN²⁰¹ was presented to classify the MNIST-backrand [see Fig. 7(b)], CIFAR-10 [see Fig. 7(c)], and CIFAR-100 datasets [see Fig. 7(d)].

The Hopfield neural network (HNN) was first proposed by John Hopfield in 1982,²¹⁰ which is a form of recurrent neural network (RNN) [see Fig. 8(a)]. RNNs are becoming increasingly popular for sequence learning tasks such as language modeling,²¹¹ handwriting prediction and generation,²¹² and speech recognition.²¹³ The difference between RNNs and ordinary feedforward neural networks is that the computational neurons in RNNs receive their own output from the previous input. Such an effect in RNNs enables the context learning in sequential inputs (see Table III).

There are two major types of HNNs, i.e., the discrete type and the continuous type. The discrete type always serves as an addressable memory with binary threshold nodes for associative learning, while the continuous type always solves the combinational optimization problem, such as traveling salesman problem (see Table III). In the most ideal situation, they can converge to the global minimum value of energy, but they sometimes converge to a local minimum value in real-world situations. A model for understanding human memory is also provided via Hopfield RNNs. Eryilmaz et al. constructed the first memristive hardware HNN system in a 10 × 10 phase change memristive array.^14,15 The pattern recognition is demonstrated robust against the variations of memristive synapses. The more the training iterations, the larger the variation can be withstood. Subsequently, Hu et al. developed a hardware HNN system for associative memory,²¹⁴ where patterns are stored by the memristive weights. The prestored patterns can be retrieved directly or through associative intermediate states. To prevent the network from being trapped into the local minimum value, Kumar et al. incorporated the chaos signal generated by using a NbO₂ Mott-based memristor into the HNN, and then, an all-memristor-based HNN system was proposed to solve the traveling salesman problem [see Fig. 8(b)].¹³¹ Contributing to the chaos signal (see Fig. 9), the efficiency and accuracy of the system were greatly improved. Combined with convolutional computation, Wang et al. developed recurrent convolutional memristive networks²¹⁵ to classify the MNIST dataset on-chip.

We would like to conclude this section on different memristive brain-inspired architectures such as SNNs, fully connected ANNs, CNNs, and RNNs. A comparative analysis of different memristive brain-inspired architectures is listed in Table III. We anticipate a breakthrough in the demonstration of high efficient training algorithms for brain-inspired computing systems with memristors in the near future. For SNNs, more spike-based data can be generated for exploring new training algorithms. For ANNs, memristive devices with low power, scalable, analog conductance, and long retention are required to be further developed.

In this section, we list major challenges and potential solutions for memristor-based brain-inspired computing systems.²¹⁹ The optimization of such systems is application specific, including but not limited to scaling up the size of the memristive arrays, improving synaptic precision, reducing leakage current, and designing efficient peripheral control circuits. At the algorithm and system level, the research of the brain will constantly inspire the development of memristor-based brain-inspired computing systems.

In principle, brain-inspired computing with memristive synapses requires the memristors to have a wide resistance range to accommodate a large number of resistance states. Low fluctuation and drift in each resistance state and high absolute resistance values are required for the inference, while a large endurance is necessary for repeated programming/training.²²¹ For SNNs, synaptic plasticity of the memritive devices is in demand. For ANNs within one or two layers, analog conductance with a wide resistance range and high endurance is necessary. In addition, a low device variation and symmetric weight updating are required for efficient network training. For DNNs (such as CNNs and RNNs), beyond analog conductance with long retention, low power, and good scalability are also required.

The device stability is critical to the computing accuracy, as the drift of conductance states with time or environmental changes will result in undesired synaptic weight changes. This has been one of the chief challenges for memristive brain-inspired computing systems. Memristors may suffer from different variations (see Table IV). There are two types of device variations: D2D variation and C2C variation. The D2D variation, evident in filamentary type memristors, is associated with a random electroforming process that creates a different filament structure in each device.⁷⁷ Some studies also suggest that reducing the thickness of the bulk of the memristive system can eliminate the effect of breakdown or electrical forming.⁷⁷ The C2C variation comes from the unstable channels or random formation of new channels. The selection of materials can greatly affect the C2C variation. Other origins of variations, such as geometry variation²²² and process variation,²²³ have also been investigated. A method to reduce the impact of the variation is the binarization or quantization of neural network weights. The potential of memristive devices can be unleashed as electronic synapses and neurons in brain-inspired computing systems with appropriate device engineering and algorithm and peripheral circuit designing.

Peripheral circuits control the read/write process in the memristive brain-inspired computing systems. In addition, different peripheral circuits are required for different functions for specific neural networks such as SNNs, ANNs, and CNNs. Take a two-layer memristor-based ANN as an example, for the kth input sample, the system receives an input vector $VI(k)∈ℝM$ and produces an output vector $VO(k)∈ℝN$⁠. The weights of the fully connected layer W is an $N×M$ memristive matrix that transforms the input to the output via the following equation:⁹ 

or

where $i=1,2,…,M$ and $j=1,2,…,N$⁠. CMOS transmission gates (TGs) can be utilized as switches to control the on/off status of the read/write voltage because of their excellent switching characteristics [see Figs. 10(a) and 10(b)]. For memristive synapses in ANNs, a synaptic array combines a memristive crossbar array of M⁻ (G_ji) and a constant resistive circuit of G_s [see Fig. 10(c)]. G_s (⁠$=1/Rs$⁠) is the conductance of R_s, and G_ji (⁠$=1/Rji$⁠) is the conductance of the memristor. $VIi$ is the input voltage. According to Kirchhoff's voltage law, both positive and negative synaptic weights can be represented by a single memristor

Note that if $Gs>Gji$⁠, the synaptic weight W_ji is positive, and on the other hand, if $Gs<Gji$⁠, the synaptic weight W_ji is negative.

For the forward-propagation (FP) process, the comparator A₂ [see Fig. 10(c)] enables a binary step activation function as follows:

where $VH$ and $VL$ (⁠$VL=0$⁠) are the high and low voltages of comparator A₂, respectively.

For the BP process, the error $ΔV$ between the estimator $VO$ and the target output $VT$ is represented by the output of comparator A₃ [see Fig. 10(c)]. During the BP process, the synaptic weights W are updated [see Figs. 10(d) and 10(e)] to minimize the error vector

The final mean square error (MSE) E_e can be expressed as follows:

For the synaptic weight updating process, more specific peripheral circuits are required to control the read/write process and calculate the require time t or voltage amplitude v. The optimization of the peripheral programming circuits is crucial for applications using online learning methods. Recently, possible methods for analog on-chip BP learning for memristive neural networks were proposed.²²⁰ The BP algorithm consists of four major steps, i.e., FP, BP to the output layer, BP to the hidden layer, and synaptic weight updating process. The architecture for the BP algorithm implementation [see Figs. 11(a) and 11(b)] can be divided into different blocks [see Fig. 11(c)]. In the FP step, the dot product of the input matrix and synaptic weight is calculated [see Fig. 11(b)] and passed through the Sigmoid activation function. Detail circuit implementations such as the Sigmoid activation function and analog switch circuits are also shown in Fig. 11(d). To implement the BP algorithm with gradient descent, MAIN BLOCK (MB)1 [see Figs. 11(b) and 11(d)] performs the calculation of the sigmoid and its derivative functions. As the memristive values are read and processed sequentially, MEMORY UNIT(MU) 4 and 5 [see Fig. 11(b)] can be utilized to store the updated values. The weight updating process is implemented by applying the voltage pulses of a particular duration t and amplitude v across each memristor. The update voltage pulse depends on the desired change in weights or error. The amplitude of the update pulse is calculated by weight updating circuits MB2 and MB4 [see Figs. 11(b) and 11(d)]. The final stage in the BP algorithm is the weight updating to minimize the error. Brain-inspired computing systems with memristors are expected to further improve the performance of online learning and reduce the complexity of peripheral programming circuits in the future (see Table IV).

Training and testing are the standard processes for ANNs especially DNNs. During the training process, the training data are utilized to minimize the error by optimizing the parameters of the learning system. The testing process, on the other hand, is usually a straightforward evaluation of the network output with testing data (which have no overlap with the training data). The frequent read/write of the parameters of memristive synaptic weights demands fast and endurable programming of memristors. In addition, there is a traded-off between hardware cost and testing accuracy.

For ANNs within one or two layers, the testing accuracy is lower compared with DNNs (such as CNNs and RNNs). The performance can be improved with large scale systems to solve more complex tasks.

While CNNs suffer from the problem of excessive power dissipation and poor scalability, the implementation of a fully on-chip system without software part needs to be further investigated. Recently, research efforts on network pruning and precision reduction demonstrate that low precision neural networks such as binary neural networks (BNNs)¹⁹⁸ and quantized neural networks (QNNs)¹³ based on CNNs are capable of carrying relatively complex pattern recognition tasks such as MNIST, CIFAR-10, and CIFAR-100. In 2018, Ambrogio et al.²⁰¹ presented a hybrid software-hardware convolutional neural network to classify the widely used CIFAR-10 and CIFAR-100 datasets with accuracies of 88.29% and 67.96% [see Figs. 12(b) and 12(c)]. Training and testing accuracies of MNIST, MNIST-backrand, CIFAR-10, and CIFAR-100 datasets are listed in Fig. 13(a). The simulation results of the training on the MNIST dataset with different techniques are also shown in Fig. 13(b). Efforts are needed to develop new algorithms that can exploit the unique properties of the memristive devices to realize a compact and energy-efficient memristive brain-inspired computing system (see Table IV).

RNNs are more suitable for other tasks such as associative learning²¹⁴ and traveling salesman problem.¹³¹ However, the refinement of the circuits and architecture, improved scalability, and different applications are still required.

SNNs still lag behind ANNs in terms of accuracy. Taking the handwritten digit recognition task as an example, the accuracy of memristive SNNs is only 82%,¹²⁵ while the accuracy of memristive ANNs achieves 94%.²¹⁸ The current challenge is lacking highly efficient training algorithms and spike-based data. Scalable on-chip neurons that produce spikes with a small chip area and power dissipation shall be developed to work with memristive synapses.

Memristor-based brain-inspired computing is still in an infancy stage with abundant opportunities and challenges. Innovations are critical not only to the materials and devices but also to the circuits and systems. Research studies should better exploit the underlying device physics and brain-inspired algorithms. In addition, novel materials and devices may further advance memristive brain-inspired computing systems. To make further progress on efficient implementation of memristor-based brain-inspired computing systems, the key challenges and potential strategies from device, circuit, and system levels are provided, as listed in Table IV.

At the device level, more reliable and practical memristive devices and crossbar arrays are required. The memristive mechanism needs to be more thoroughly comprehended to further improve the performance. Furthermore, more theoretical efforts are needed to design the material stack based on the new understanding of mechanisms (see Table IV).

At the circuit level, memristive circuits are desired to be effectively enlarged with efficient read/write schemes. Equipping memristors with transistor-based selectors or rectifying capabilities can reduce the sneak path currents. To overcome the device variations, low precision algorithms and systems such as fuzzy learning methods and quantized neural networks can be beneficial. The fuzzy modeling method can be used to define the memristive states in a dynamic way. QNNs (including BNNs) can simultaneously accelerate and compress the learning process while suffering negligible loss in training accuracy. Large scale memristor-based circuits for brain-inspired computing systems are also required to further increase the computation speed (see Table IV).

Finally, at the system level, the learning algorithms of memristive ANNs and SNNs are still under development. Practical network topology and the learning algorithm for both ANNs and SNNs are required to develop a general computing system for ANNs and SNNs, including data mapping, dot product, and STDP. In addition, ANNs and SNNs using RL can be further explored to solve different practical problems beyond the existing demonstrations (see Table IV).

In summary, this article provides a comprehensive review of various switching mechanisms of memristors and neural and synaptic functionalities that can be potentially implemented in memristive devices, circuits, and systems. The challenges and potential solutions have been discussed at the device, circuit, and system levels. We believe that this article will stimulate more research efforts in this exciting field.

This study was partially supported by the National Natural Science Foundation of China (No. 61806129) and China Post-Doctoral Science Foundation (Nos. 2018M640820 and 2019T120751) and partially supported by the U.S. Air Force Research Laboratory (AFRL) (Grant No. FA8750-15-2-0044). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of AFRL.