The development of memristive device technologies has reached a level of maturity to enable the design and fabrication of complex and large-scale hybrid memristive-Complementary Metal-Oxide Semiconductor (CMOS) neural processing systems. These systems offer promising solutions for implementing novel in-memory computing architectures for machine learning and data analysis problems. We argue that they are also ideal building blocks for integration in neuromorphic electronic circuits suitable for ultra-low power brain-inspired sensory processing systems, therefore leading to innovative solutions for always-on edge-computing and Internet-of-Things applications. Here, we present a recipe for creating such systems based on design strategies and computing principles inspired by those used in mammalian brains. We enumerate the specifications and properties of memristive devices required to support always-on learning in neuromorphic computing systems and to minimize their power consumption. Finally, we discuss in what cases such neuromorphic systems can complement conventional processing ones and highlight the importance of exploiting the physics of both the memristive devices and the CMOS circuits interfaced to them.

1.
E. O.
Neftci
, “
Data and power efficient intelligence with neuromorphic learning machines
,”
iScience
5
,
52
68
(
2018
).
2.
C. S.
Thakur
,
J. L.
Molin
,
G.
Cauwenberghs
,
G.
Indiveri
,
K.
Kumar
,
N.
Qiao
,
J.
Schemmel
,
R.
Wang
,
E.
Chicca
,
J.
Olson Hasler
,
J.-S.
Seo
,
S.
Yu
,
Y.
Cao
,
A.
van Schaik
, and
R.
Etienne-Cummings
, “
Large-scale neuromorphic spiking array processors: A quest to mimic the brain
,”
Front. Neurosci.
12
,
891
(
2018
).
3.
Y.
Li
,
Z.
Wang
,
R.
Midya
,
Q.
Xia
, and
J. J.
Yang
, “
Review of memristor devices in neuromorphic computing: Materials sciences and device challenges
,”
J. Phys. D
51
,
503002
(
2018
).
4.
Y.
van de Burgt
,
A.
Melianas
,
S. T.
Keene
,
G.
Malliaras
, and
A.
Salleo
, “
Organic electronics for neuromorphic computing
,”
Nat. Electron.
1
,
386
397
(
2018
).
5.
G. W.
Burr
,
R. M.
Shelby
,
A.
Sebastian
,
S.
Kim
,
S.
Kim
,
S.
Sidler
,
K.
Virwani
,
M.
Ishii
,
P.
Narayanan
,
A.
Fumarola
 et al, “
Neuromorphic computing using non-volatile memory
,”
Adv. Phys.: X
2
,
89
124
(
2017
).
6.
C.
Mead
, “
Neuromorphic electronic systems
,”
Proc. IEEE
78
,
1629
1636
(
1990
).
7.
E.
Chicca
,
F.
Stefanini
,
C.
Bartolozzi
, and
G.
Indiveri
, “
Neuromorphic electronic circuits for building autonomous cognitive systems
,”
Proc. IEEE
102
,
1367
1388
(
2014
).
8.
S.
Furber
,
F.
Galluppi
,
S.
Temple
, and
L.
Plana
, “
The SpiNNaker project
,”
Proc. IEEE
102
,
652
665
(
2014
).
9.
F.
Akopyan
,
J.
Sawada
,
A.
Cassidy
,
R.
Alvarez-Icaza
,
J.
Arthur
,
P.
Merolla
,
N.
Imam
,
Y.
Nakamura
,
P.
Datta
,
G.-J.
Nam
 et al, “
Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip
,”
IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.
34
,
1537
1557
(
2015
).
10.
M.
Davies
,
N.
Srinivasa
,
T.-H.
Lin
,
G.
Chinya
,
Y.
Cao
,
S. H.
Choday
,
G.
Dimou
,
P.
Joshi
,
N.
Imam
,
S.
Jain
 et al, “
Loihi: A neuromorphic manycore processor with on-chip learning
,”
IEEE Micro
38
,
82
99
(
2018
).
11.
D.
Ielmini
and
R.
Waser
,
Resistive Switching: From Fundamentals of Nanoionic Redox Processes to Memristive Device Applications
(
John Wiley & Sons
,
2015
).
12.
I.
Boybat
,
M. L.
Gallo
,
T.
Moraitis
,
T.
Parnell
,
T.
Tuma
,
B.
Rajendran
,
Y.
Leblebici
,
A.
Sebastian
,
E.
Eleftheriou
 et al, “
Neuromorphic computing with multi-memristive synapses
,”
Nat. Commun.
9
,
2514
(
2018
).
13.
Y.
LeCun
,
Y.
Bengio
, and
G.
Hinton
, “
Deep learning
,”
Nature
521
,
436
444
(
2015
).
14.
J.
Schmidhuber
, “
Deep learning in neural networks: An overview
,”
Neural Networks
61
,
85
117
(
2015
).
15.
A.
Sebastian
,
M. L.
Gallo
, and
E.
Eleftheriou
, “
Computational phase-change memory: Beyond von Neumann computing
,”
J. Phys. D
52
,
443002
(
2019
).
16.
S.
Ambrogio
,
P.
Narayanan
,
H.
Tsai
,
R. M.
Shelby
,
I.
Boybat
,
C.
di Nolfo
,
S.
Sidler
,
M.
Giordano
,
M.
Bodini
,
N. C. P.
Farinha
,
B.
Killeen
,
C.
Cheng
,
Y.
Jaoudi
, and
G. W.
Burr
, “
Equivalent-accuracy accelerated neural-network training using analogue memory
,”
Nature
558
,
60
67
(
2018
).
17.
S.
Dai
,
Y.
Zhao
,
Y.
Wang
,
J.
Zhang
,
L.
Fang
,
S.
Jin
,
Y.
Shao
, and
J.
Huang
, “
Recent advances in transistor-based artificial synapses
,”
Adv. Funct. Mater.
29
,
1903700
(
2019
).
18.
E.
Covi
,
S.
Brivio
,
A.
Serb
,
T.
Prodromakis
,
M.
Fanciulli
, and
S.
Spiga
, “
Analog memristive synapse in spiking networks implementing unsupervised learning
,”
Front. Neurosci.
10
,
1
13
(
2016
).
19.
S. H.
Jo
,
T.
Chang
,
I.
Ebong
,
B. B.
Bhadviya
,
P.
Mazumder
, and
W.
Lu
, “
Nanoscale memristor device as synapse in neuromorphic systems
,”
Nano Lett.
10
,
1297
1301
(
2010
).
20.
J. J.
Yang
and
Q.
Xia
, “
Organic electronics: Battery-like artificial synapses
,”
Nat. Mater.
16
,
396
(
2017
).
21.
R.
Berdan
,
E.
Vasilaki
,
A.
Khiat
,
G.
Indiveri
,
A.
Serb
, and
T.
Prodromakis
, “
Emulating short-term synaptic dynamics with memristive devices
,”
Sci. Rep.
6
,
18639
(
2016
).
22.
G.
Indiveri
and
S.-C.
Liu
, “
Memory and information processing in neuromorphic systems
,”
Proc. IEEE
103
,
1379
1397
(
2015
).
23.
G.
Indiveri
and
Y.
Sandamirskaya
, “
The importance of space and time for signal processing in neuromorphic agents
,”
IEEE Signal Process. Mag.
36
,
16
28
(
2019
).
24.
P. A.
Merolla
,
J. V.
Arthur
,
R.
Alvarez-Icaza
,
A. S.
Cassidy
,
J.
Sawada
,
F.
Akopyan
,
B. L.
Jackson
,
N.
Imam
,
C.
Guo
,
Y.
Nakamura
,
B.
Brezzo
,
I.
Vo
,
S. K.
Esser
,
R.
Appuswamy
,
B.
Taba
,
A.
Amir
,
M. D.
Flickner
,
W. P.
Risk
,
R.
Manohar
, and
D. S.
Modha
, “
A million spiking-neuron integrated circuit with a scalable communication network and interface
,”
Science
345
,
668
673
(
2014
).
25.
J.
Backus
, “
Can programming be liberated from the von Neumann style?: A functional style and its algebra of programs
,”
Commun. ACM
21
,
613
641
(
1978
).
26.
G.
Indiveri
,
B.
Linares-Barranco
,
R.
Legenstein
,
G.
Deligeorgis
, and
T.
Prodromakis
, “
Integration of nanoscale memristor synapses in neuromorphic computing architectures
,”
Nanotechnology
24
,
384010
(
2013
).
27.
J.
von Neumann
, “
First draft of a report on the EDVAC
,”
IEEE Ann. Hist. Comput.
15
,
27
75
(
1993
).
28.
W.
McCulloch
and
W.
Pitts
, “
A logical calculus of the ideas immanent in nervous activity
,”
Bull. Math. Biophys.
5
,
115
133
(
1943
).
29.
M.
Shadlen
and
J.
Movshon
, “
Synchrony unbound: A critical evaluation of the temporal binding hypothesis
,”
Neuron
24
,
67
77
(
1999
).
30.
K.
Boahen
, “
Communicating neuronal ensembles between neuromorphic chips
,” in
Neuromorphic Systems Engineering
, edited by
T.
Lande
(
Kluwer Academic
,
Norwell, MA
,
1998
), pp.
229
259
.
31.
S.
Moradi
,
N.
Qiao
,
F.
Stefanini
, and
G.
Indiveri
, “
A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (DYNAPs)
,”
IEEE Trans. Biomed. Circuits Syst.
12
,
106
122
(
2018
).
32.
N.
Qiao
,
C.
Bartolozzi
, and
G.
Indiveri
, “
An ultralow leakage synaptic scaling homeostatic plasticity circuit with configurable time scales up to 100 ks
,”
IEEE Trans. Biomed. Circuits Syst.
11
,
1271
(
2017
).
33.
J.
Yoon
,
H.
Jung
,
Z.
Wang
,
K. M.
Kim
,
H.
Wu
,
V.
Ravichandran
,
Q.
Xia
,
C. S.
Hwang
, and
J. J.
Yang
, “
An artificial nociceptor based on a diffusive memristor
,”
Nat. Commun.
9
,
417
(
2018
).
34.
Z.
Wang
,
S.
Joshi
,
S. E.
Savel'ev
,
H.
Jiang
,
R.
Midya
,
P.
Lin
,
M.
Hu
,
N.
Ge
,
J. P.
Strachan
,
Z.
Li
,
Q.
Wu
,
M.
Barnell
,
G.-L.
Li
,
H. L.
Xin
,
R. S.
Williams
,
Q.
Xia
, and
J. J.
Yang
, “
Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing
,”
Nat. Mater.
16
,
101
(
2017
).
35.
X.
Zhang
,
S.
Liu
,
X.
Zhao
,
F.
Wu
,
Q.
Wu
,
W.
Wang
,
R.
Cao
,
Y.
Fang
,
H.
Lv
,
S.
Long
,
Q.
Liu
, and
M.
Liu
, “
Emulating short-term and long-term plasticity of bio-synapse based on Cu/a-Si/Pt memristor
,”
IEEE Electron Device Lett.
38
,
1208
1211
(
2017
).
36.
T.
Ohno
,
T.
Hasegawa
,
T.
Tsuruoka
,
K.
Terabe
,
J.
Gimzewski
, and
M.
Aono
, “
Short-term plasticity and long-term potentiation mimicked in single inorganic synapses
,”
Nat. Mater.
10
,
591
595
(
2011
).
37.
J.
Xiong
,
R.
Yang
,
J.
Shaibo
,
H. M.
Huang
,
H. K.
He
,
W.
Zhou
, and
X.
Guo
, “
Bienenstock, Cooper, and Munro learning rules realized in second-order memristors with tunable forgetting rate
,”
Adv. Funct. Mater.
29
,
1807316
(
2019
).
38.
M.
Payvand
,
M. V.
Nair
,
L. K.
Müller
, and
G.
Indiveri
, “
A neuromorphic systems approach to in-memory computing with non-ideal memristive devices: From mitigation to exploitation
,”
Faraday Discuss.
213
,
487
510
(
2019
).
39.
S.
Pi
,
C.
Li
,
H.
Jiang
,
W.
Xia
,
H.
Xin
,
J. J.
Yang
, and
Q.
Xia
, “
Memristor crossbar arrays with 6-nm half-pitch and 2-nm critical dimension
,”
Nat. Nanotechnol.
14
,
35
(
2019
).
40.
Q.
Xia
and
J. J.
Yang
, “
Memristive crossbar arrays for brain-inspired computing
,”
Nat. Mater.
18
,
309
(
2019
).
41.
M. V.
Nair
and
G.
Indiveri
, “
A differential memristive current-mode circuit
,” European patent application EP 17183461.7 (27 July
2017
).
42.
M.
Payvand
,
L. K.
Muller
, and
G.
Indiveri
, “
Event-based circuits for controlling stochastic learning with memristive devices in neuromorphic architectures
,” in
IEEE International Symposium on Circuits and Systems (ISCAS)
(
IEEE
,
2018
), pp.
1
5
.
43.
E. O.
Neftci
,
B. U.
Pedroni
,
S.
Joshi
,
M.
Al-Shedivat
, and
G.
Cauwenberghs
, “
Stochastic synapses enable efficient brain-inspired learning machines
,”
Front. Neurosci.
10
,
241
(
2016
).
44.
S.
Brivio
,
D.
Conti
,
M. V.
Nair
,
J.
Frascaroli
,
E.
Covi
,
C.
Ricciardi
,
G.
Indiveri
, and
S.
Spiga
, “
Extended memory lifetime in spiking neural networks employing memristive synapses with nonlinear conductance dynamics
,”
Nanotechnology
30
,
015102
(
2019
).
45.
N.
Diederich
,
T.
Bartsch
,
H.
Kohlstedt
, and
M.
Ziegler
, “
A memristive plasticity model of voltage-based STDP suitable for recurrent bidirectional neural networks in the hippocampus
,”
Sci. Rep.
8
,
9367
(
2018
).
46.
A.
Fantini
,
L.
Goux
,
R.
Degraeve
,
D. J.
Wouters
,
N.
Raghavan
,
G.
Kar
,
A.
Belmonte
,
Y.
Chen
,
B.
Govoreanu
, and
M.
Jurczak
, “
Intrinsic switching variability in Hfo2 RRAM
,” in
2013 5th IEEE International Memory Workshop
(
IEEE
,
2013
), pp.
30
33
.
47.
M.
Suri
and
V.
Parmar
, “
Exploiting intrinsic variability of filamentary resistive memory for extreme learning machine architectures
,”
IEEE Trans. Nanotechnol.
14
,
963
968
(
2015
).
48.
A.
Schönhals
,
R.
Waser
, and
D. J.
Wouters
, “
Improvement of SET variability in TaOx based resistive RAM devices
,”
Nanotechnology
28
,
465203
(
2017
).
49.
A.
Prakash
,
D.
Deleruyelle
,
J.
Song
,
M.
Bocquet
, and
H.
Hwang
, “
Resistance controllability and variability improvement in a Taox-based resistive memory for multilevel storage application
,”
Appl. Phys. Lett.
106
,
233104
(
2015
).
50.
B.
Govoreanu
,
D.
Crotti
,
S.
Subhechha
,
L.
Zhang
,
Y. Y.
Chen
,
S.
Clima
,
V.
Paraschiv
,
H.
Hody
,
C.
Adelmann
,
M.
Popovici
,
O.
Richard
, and
M.
Jurczak
, “
A-VMCO: A novel forming-free, self-rectifying, analog memory cell with low-current operation, nonfilamentary switching and excellent variability
,” in
2015 Symposium on VLSI Technology
(VLSI Technology) (
IEEE
,
2015
), pp.
T132
T133
.
51.
X.
Sheng
,
C. E.
Graves
,
S.
Kumar
,
X.
Li
,
B.
Buchanan
,
L.
Zheng
,
S.
Lam
,
C.
Li
, and
J. P.
Strachan
, “
Low-conductance and multilevel CMOS-integrated nanoscale oxide memristors
,”
Adv. Electron. Mater.
5
,
1800876
(
2019
).
52.
Y.
Freund
and
R. E.
Schapire
, “
A decision-theoretic generalization of on-line learning and an application to boosting
,”
J. Comput. Syst. Sci.
55
,
119
139
(
1997
).
53.
H.
Jaeger
and
H.
Haas
, “
Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication
,”
Science
304
,
78
80
(
2004
).
54.
W.
Maass
,
T.
Natschläger
, and
H.
Markram
, “
Real-time computing without stable states: A new framework for neural computation based on perturbations
,”
Neural Comput.
14
,
2531
2560
(
2002
).
55.
S.
Sheik
,
M.
Coath
,
G.
Indiveri
,
S.
Denham
,
T.
Wennekers
, and
E.
Chicca
, “
Emergent auditory feature tuning in a real-time neuromorphic VLSI system
,”
Front. Neurosci.
6
,
17
(
2012
).
56.
O.
Richter
,
R. F.
Reinhard
,
S.
Nease
,
J.
Steil
, and
E.
Chicca
, “
Device mismatch in a neuromorphic system implements random features for regression
,” in
2015 IEEE Biomedical Circuits and Systems Conference (BioCAS)
(
IEEE
,
2015
), pp.
1
4
.
57.
A.
Das
,
P.
Pradhapan
,
W.
Groenendaal
,
P.
Adiraju
,
R. T.
Rajan
,
F.
Catthoor
,
S.
Schaafsma
,
J. L.
Krichmar
,
N. D.
Dutt
, and
C. V.
Hoof
, “
Unsupervised heart-rate estimation in wearables with liquid states and a probabilistic readout
,”
Neural Networks
99
,
134
147
(
2018
).
58.
E.
Donati
,
M.
Payvand
,
N.
Risi
,
R.
Krause
,
K.
Burelo
,
T.
Dalgaty
,
E.
Vianello
, and
G.
Indiveri
, “
Processing EMG signals using reservoir computing on an event-based neuromorphic system
,” in
Biomedical Circuits and Systems Conference (BioCAS)
(
IEEE
,
2018
), pp.
1
4
.
59.
F.
Bauer
,
D.
Muir
, and
G.
Indiveri
, “
Real-time ultra-low power ECG anomaly detection using an event-driven neuromorphic processor
,”
IEEE Trans. Biomed. Circuits Syst.
13
(
6
),
1575
1582
(
2019
).
60.
S.
Gaba
,
P.
Sheridan
,
J.
Zhou
,
S.
Choi
, and
W.
Lu
, “
Stochastic memristive devices for computing and neuromorphic applications
,”
Nanoscale
5
,
5872
5878
(
2013
).
61.
S.
Ambrogio
,
S.
Balatti
,
V.
Milo
,
R.
Carboni
,
Z.-Q.
Wang
,
A.
Calderoni
,
N.
Ramaswamy
, and
D.
Ielmini
, “
Neuromorphic learning and recognition with one-transistor-one-resistor synapses and bistable metal oxide RRAM
,”
IEEE Trans. Electron Devices
63
,
1508
1515
(
2016
).
62.
J. J.
Yang
,
D. B.
Strukov
, and
D. R.
Stewart
, “
Memristive devices for computing
,”
Nat. Nanotechnol.
8
,
13
24
(
2013
).
63.
M.
Al-Shedivat
,
R.
Naous
,
G.
Cauwenberghs
, and
K. N.
Salama
, “
Memristors empower spiking neurons with stochasticity
,”
IEEE J. Emerging Sel. Top. Circuits Syst.
5
,
242
253
(
2015
).
64.
M.
Suri
,
D.
Querlioz
,
O.
Bichler
,
G.
Palma
,
E.
Vianello
,
D.
Vuillaume
,
C.
Gamrat
, and
B.
DeSalvo
, “
Bio-inspired stochastic computing using binary CBRAM synapses
,”
IEEE Trans. Electron Devices
60
,
2402
2409
(
2013
).
65.
S.
Balatti
,
S.
Ambrogio
,
R.
Carboni
,
V.
Milo
,
Z.
Wang
,
A.
Calderoni
,
N.
Ramaswamy
, and
D.
Ielmini
, “
Physical unbiased generation of random numbers with coupled resistive switching devices
,”
IEEE Trans. Electron Devices
63
,
2029
2035
(
2016
).
66.
S.
Habenschuss
,
Z.
Jonke
, and
W.
Maass
, “
Stochastic computations in cortical microcircuit models
,”
PLoS Comput. Biol.
9
,
e1003311
(
2013
).
67.
A.
Destexhe
and
D.
Contreras
, “
Neuronal computations with stochastic network states
,”
Science
314
,
85
90
(
2006
).
68.
S.
Fusi
and
W.
Senn
, “
Eluding oblivion with smart stochastic selection of synaptic updates
,”
Chaos
16
,
026112
(
2006
).
69.
I.
Ginzburg
and
H.
Sompolinsky
, “
Theory of correlations in stochastic neural networks
,”
Phys. Rev. E
50
,
3171
3191
(
1994
).
70.
T.
Dalgaty
,
M.
Payvand
,
B.
De Salvo
,
J.
Casaz
,
G.
Lama
,
E.
Nowak
,
G.
Indiveri
, and
E.
Vianello
, “
Hybrid CMOS-RRAM neurons with intrinsic plasticity
,” in
International Symposium on Circuits and Systems (ISCAS)
(
IEEE
,
2019
).
71.
A.
Yousefzadeh
,
E.
Stromatias
,
M.
Soto
,
T.
Serrano-Gotarredona
, and
B.
Linares-Barranco
, “
On practical issues for stochastic STDP hardware with 1-bit synaptic weights
,”
Front. Neurosci.
12
,
665
(
2018
).
72.
J.
Leugering
and
G.
Pipa
, “
A unifying framework of synaptic and intrinsic plasticity in neural populations
,”
Neural Comput.
30
,
945
986
(
2018
).
73.
S.
Fusi
and
L.
Abbott
, “
Limits on the memory storage capacity of bounded synapses
,”
Nat. Neurosci.
10
,
485
493
(
2007
).
74.
S.
Ganguli
,
D.
Huh
, and
H.
Sompolinsky
, “
Memory traces in dynamical systems
,”
Proc. Natl. Acad. Sci.
105
,
18970
18975
(
2008
).
75.
J.
Frascaroli
,
S.
Brivio
,
E.
Covi
, and
S.
Spiga
, “
Evidence of soft bound behaviour in analogue memristive devices for neuromorphic computing
,”
Sci. Rep.
8
,
7178
(
2018
).
76.
P.
Del Giudice
,
S.
Fusi
, and
M.
Mattia
, “
Modeling the formation of working memory with networks of integrate-and-fire neurons connected by plastic synapses
,”
J. Physiol.
97
,
659
681
(
2003
).
77.
F.
Zenke
,
E. J.
Agnes
, and
W.
Gerstner
, “
Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks
,”
Nat. Commun.
6
,
6922
(
2015
).
78.
J.
Bill
and
R.
Legenstein
, “
A compound memristive synapse model for statistical learning through STDP in spiking neural networks
,”
Front. Neurosci.
8
,
412
(
2014
).
79.
G.
Plastiras
,
M.
Terzi
,
C.
Kyrkou
, and
T.
Theocharidcs
, “
Edge intelligence: Challenges and opportunities of near-sensor machine learning applications
,” in
2018 IEEE 29th International Conference on Application-Specific Systems, Architectures and Processors (ASAP)
(
IEEE
,
2018
), pp.
1
7
.
You do not currently have access to this content.