The basic building blocks of every neural network are neurons and their inter-cellular connections, called synapses. In nature, synapses play a crucial role in learning and memory, since they are plastic, which means that they change their state depending on the neural activity of the respectively coupled neurons. In neuromorphic systems, the functionality of neurons and synapses is emulated in hardware systems by employing very-large-scale integration technology. In this context, it seems rather natural to use non-volatile memory technology to mimic synaptic functionality. In particular, memristive devices are promising candidates for neuromorphic computing, since they allow one to emulate synaptic functionalities in a detailed way with a significantly reduced power usage and a high packing density. This tutorial aims to provide insight on current investigations in the field to address the following fundamental questions: How can functionalities of synapses be emulated with memristive devices? What are the basic requirements to realize artificial inorganic neurons and synapses? Which material systems and device structures can be used for this purpose? And how can cellular synaptic functionality be used in networks for neuromorphic computing? Even if those questions are part of current research and not yet answered in detail, our aim is to present concepts that address those questions. Furthermore, this tutorial focuses on spiking neural models, which enables mimicking biological computing as realistically as possible.
REFERENCES
1.
See https://www.wiley.com/en-us/Emerging+Nanoelectronic+Devices-p-9781118447741 for “Emerging Nanoelectronic Devices” (accessed February 8, 2018).
2.
3.
4.
R.
Waser
et al, “Introduction to nanoionic elements for information technology
,” in Resistive Switching
, edited by D.
Ielmini
and R.
Waser
(Wiley-VCH Verlag GmbH & Co. KGaA
, 2016
), pp. 1
–30
.5.
6.
E.
Kandel
and J.
Schwartz
, Principles of Neural Science
, 5th ed. (McGraw Hill Professional
, 2013
).7.
W.
Gerstner
and W.
Kistler
, Spiking Neuron Models
(Cambridge University Press
, 2002
). See http://www.cambridge.org/de/academic/subjects/computer-science/computational-biology-and-bioinformatics/spiking-neuron-models-single-neurons-populations-plasticity?format=PB&isbn=9780521890793; accessed 3 August 2016.8.
A. K.
Engel
, P. R.
Roelfsema
, P.
Fries
, M.
Brecht
, and W.
Singer
, “Role of the temporal domain for response selection and perceptual binding
,” Cereb. Cortex
7
(6
), 571
–582
(1997
). 9.
P.
Uhlhaas
et al, “Neural synchrony in cortical networks: History, concept and current status
,” Front. Integr. Neurosci.
3
17 (2009
). 10.
W.
Gerstner
, H.
Sprekeler
, and G.
Deco
, “Theory and simulation in neuroscience
,” Science
338
(6103
), 60
–65
(2012
). 11.
J.
Schmidhuber
, “Deep learning in neural networks: An overview
,” Neural Netw.
61
, 85
–117
(2015
). 12.
W.
Gerstner
, W. M.
Kistler
, R.
Naud
, and L.
Paninski
, “Neuronal dynamics: From single neurons to networks and models of cognition,” Cambridge Core, July 2014. See https://www.cambridge.org/core/books/neuronal-dynamics/75375090046733765596191E23B2959D; accessed 30 April 2018.13.
W.
Maass
, “Networks of spiking neurons: The third generation of neural network models
,” Neural Netw.
10
(9
), 1659
–1671
(1997
). 14.
H.
Markram
, W.
Gerstner
, and P. J.
Sjöström
, “Spike-timing dependent plasticity: A comprehensive overview
,” Front. Synaptic Neurosci.
4
, 2 (2012
). 15.
C. G.
Mayr
, S.
Sheik
, C.
Bartolozzi
, and E.
Chicca
, “Editorial: Synaptic plasticity for neuromorphic systems
,” Front. Neurosci.
10
214 (2016
).16.
R. C.
Malenka
and M. F.
Bear
, “LTP and LTD: An embarrassment of riches
,” Neuron
44
(1
), 5
–21
(2004)
. 17.
L.
Chua
, “Memristor-the missing circuit element
,” IEEE Trans. Circuit Theory
18
(5
), 507
–519
(1971
). 18.
D. B.
Strukov
, G. S.
Snider
, D. R.
Stewart
, and R. S.
Williams
, “The missing memristor found
,” Nature
453
, 7191, 80
–83
(2008
). 19.
S. H.
Jo
, T.
Chang
, I.
Ebong
, B. B.
Bhadviya
, P.
Mazumder
, and W.
Lu
, “Nanoscale memristor device as synapse in neuromorphic systems
,” Nano Lett.
10
(4
), 1297
–1301
(2010
). 20.
T.
Ohno
, T.
Hasegawa
, T.
Tsuruoka
, K.
Terabe
, J. K.
Gimzewski
, and M.
Aono
, “Short-term plasticity and long-term potentiation mimicked in single inorganic synapses
,” Nat. Mater.
10
(8
), 591
–595
(2011
). 21.
B.
Linares-Barranco
, T.
Serrano-Gotarredona
, L. A.
Camuñas-Mesa
, J. A.
Perez-Carrasco
, C.
Zamarreño-Ramos
, and T.
Masquelier
, “On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex
,” Neuromorphic Eng.
5
, 26
(2011
). 22.
I.
Kataeva
, F.
Merrikh-Bayat
, E.
Zamanidoost
, and D.
Strukov
, “Efficient training algorithms for neural networks based on memristive crossbar circuits
,” in 2015 International Joint Conference on Neural Networks (IJCNN)
(IEEE, 2015
), pp. 1
–8
.23.
H. A. D.
Nguyen
, J.
Yu
, L.
Xie
, M.
Taouil
, S.
Hamdioui
, and D.
Fey
, “Memristive devices for computing: Beyond CMOS and beyond von Neumann
,” in 2017 IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SoC)
(IEEE, 2017
), pp. 1
–10
.24.
F. M.
Bayat
, M.
Prezioso
, B.
Chakrabarti
, H.
Nili
, I.
Kataeva
, and D.
Strukov
, “Implementation of multilayer perceptron network with highly uniform passive memristive crossbar circuits
,” Nat. Commun.
9
(1
), 2331
(2018
). 25.
D. S.
Jeong
, I.
Kim
, M.
Ziegler
, and H.
Kohlstedt
, “Towards artificial neurons and synapses: A materials point of view
,” RSC Adv.
3
(10
), 3169
–3183
(2013
). 26.
R.
Waser
, R.
Dittmann
, G.
Staikov
, and K.
Szot
, “Redox-based resistive switching memories - nanoionic mechanisms, prospects, and challenges
,” Adv. Mater.
21
(25–26
), 2632
–2663
(2009
). 27.
J. J.
Yang
, D. B.
Strukov
, and D. R.
Stewart
, “Memristive devices for computing
,” Nat. Nanotechnol.
8
(1
), 13
–24
(2013
). 28.
A.
Chanthbouala
et al, “A ferroelectric memristor
,” Nat. Mater.
11
(10
), 860
–864
(2012
). 29.
E. Y.
Tsymbal
and H.
Kohlstedt
, “Tunneling across a ferroelectric
,” Science
313
(5784
), 181
–183
(2006
). 30.
M. A.
Zidan
, A.
Chen
, G.
Indiveri
, and W. D.
Lu
, “Memristive computing devices and applications
,” J. Electroceramics
39
(1–4
), 4
–20
(2017
). 31.
G. W.
Burr
et al, “Neuromorphic computing using non-volatile memory
,” Adv. Phys. X
2
(1
), 89
–124
(2017
). 32.
Memristors and Memristive Systems, edited by Ronald Tetzlaff (Springer, 2013).
33.
M.
Hu
et al, “Memristor-based analog computation and neural network classification with a dot product engine
,” Adv. Mater.
30
(9
), 1705914
(2018
). 34.
L.
Yavits
and R.
Ginosar
, “Sparse matrix multiplication on CAM based accelerator,” e-print arXiv:170509937Cs (May 2017).35.
G.
Dearnaley
, A. M.
Stoneham
, and D. V.
Morgan
, “Electrical phenomena in amorphous oxide films
,” Rep. Prog. Phys
33
(3
), 1129
(1970
). 36.
T. W.
Hickmott
, “Low-frequency negative resistance in thin anodic oxide films
,” J. Appl. Phys.
33
(9
), 2669
–2682
(1962
). 37.
A.
Beck
, J. G.
Bednorz
, C.
Gerber
, C.
Rossel
, and D.
Widmer
, “Reproducible switching effect in thin oxide films for memory applications
,” Appl. Phys. Lett.
77
(1
), 139
–141
(2000
). 38.
J.
Backus
, “Can programming be liberated from the von Neumann style? A functional style and its algebra of programs
,” presented at the ACM Turing Award Lectures, 2007
(1977
).39.
M. D.
Ventra
, Y. V.
Pershin
, and L. O.
Chua
, “Circuit elements with memory: Memristors, memcapacitors, and meminductors
,” Proc. IEEE
97
(10
), 1717
–1724
(2009
). 40.
J.-X.
Shen
, D.-S.
Shang
, Y.-S.
Chai
, S.-G.
Wang
, B.-G.
Shen
, and Y.
Sun
, “Mimicking synaptic plasticity and neural network using memtransistors
,” Adv. Mater.
30
(12
), 1706717
(2018
). 41.
See http://onlinelibrary.wiley.com/doi/10.1002/adma.201103723/full for “The Memristive Magnetic Tunnel Junction as a Nanoscopic Synapse-Neuron System – Krzysteczko – 2012 – Advanced Materials – Wiley Online Library” (accessed February 22, 2018).
42.
D.
Pantel
, S.
Goetze
, D.
Hesse
, and M.
Alexe
, “Reversible electrical switching of spin polarization in multiferroic tunnel junctions
,” Nat. Mater.
11
(4
), 289
–293
(2012
). 43.
H. Y.
Jeong
et al, “Graphene oxide thin films for flexible nonvolatile memory applications
,” Nano Lett.
10
(11
), 4381
–4386
(2010
). 44.
M.
Ziegler
, M.
Oberländer
, D.
Schroeder
, W. H.
Krautschneider
, and H.
Kohlstedt
, “Memristive operation mode of floating gate transistors: A two-terminal memflash-cell
,” Appl. Phys. Lett.
101
(26
), 263504
(2012
). 45.
C.
Riggert
, M.
Ziegler
, D.
Schroeder
, W. H.
Krautschneider
, and H.
Kohlstedt
, “MemFlash device: Floating gate transistors as memristive devices for neuromorphic computing
,” Semicond. Sci. Technol.
29
(10
), 104011
(2014
). 46.
A.
Sawa
, “Resistive switching in transition metal oxides
,” Mater. Today
11
(6
), 28
–36
(2008
). 47.
M.
Ziegler
, C.
Riggert
, M.
Hansen
, T.
Bartsch
, and H.
Kohlstedt
, “MemristiveHebbian plasticity model: Device requirements for the emulation of Hebbian plasticity based on memristive devices
,” IEEE Trans. Biomed. Circuits Syst.
9
(2
), 197
–206
(2015
). 48.
See http://onlinelibrary.wiley.com/doi/10.1113/jphysiol.1926.sp002273/full for “The impulses produced by sensory nerve endings – Adrian – 1926 – The Journal of Physiology – Wiley Online Library” (accessed February 6, 2017).
49.
S.
Thorpe
, D.
Fize
, and C.
Marlot
, “Speed of processing in the human visual system
,” Nature
381
(6582
), 520
–522
(1996
). 50.
51.
D.
Silver
et al, “Mastering the game of Go with deep neural networks and tree search
,” Nature
529
(7587
), 484
–489
(2016
). 52.
See https://journals.lww.com/neuroreport/Abstract/1998/01260/Rapid_categorization_of_natural_images_by_rhesus.23.aspx for “Rapid categorization of natural images by rhesus monkeys: NeuroReport” (accessed May 29, 2018).
53.
S.
Thorpe
, A.
Delorme
, and R.
Van Rullen
, “Spike-based strategies for rapid processing
,” Neural Netw.
14
(6
), 715
–725
( 2001
). 54.
See http://www.cambridge.org/de/academic/subjects/computer-science/computational-biology-and-bioinformatics/neuronal-dynamics-single-neurons-networks-and-models-cognition for “Neuronal dynamics single neurons networks and models cognition|Computational biology and bioinformatics,” Cambridge University Press (accessed February 20, 2018).
55.
E.
Chicca
, F.
Stefanini
, C.
Bartolozzi
, and G.
Indiveri
, “Neuromorphic electronic circuits for building autonomous cognitive systems
,” Proc. IEEE
102
(9
), 1367
–1388
(2014
). 56.
Shih-Chii
Liu
, Jörg
Kramer
, Giacomo
Indiveri
, Tobias
Delbrück
, Rodney
Douglas
, and Carver A.
Mead
, Analog VLSI
(MIT Press
, 2002
). See https://mitpress.mit.edu/books/analog-vlsi; accessed 3 August 2016.57.
G.
Indiveri
et al, “Neuromorphic silicon neuron circuits
,” Front. Neurosci.
5
, 73 (2011
).58.
M.
Mahowald
and R.
Douglas
, “A silicon neuron
,” Nature
354
(6354
), 515
–518
(1991
). 59.
M.
Ignatov
, M.
Hansen
, M.
Ziegler
, and H.
Kohlstedt
, “Synchronization of two memristively coupled van der Pol oscillators
,” Appl. Phys. Lett.
108
(8
), 084105
(2016
). 60.
D. S.
Jeong
, H.
Lim
, G.-H.
Park
, C. S.
Hwang
, S.
Lee
, and B.
Cheong
, “Threshold resistive and capacitive switching behavior in binary amorphous GeSe
,” J. Appl. Phys.
111
(10
), 102807
(2012
). 61.
M.
Ignatov
, M.
Ziegler
, M.
Hansen
, A.
Petraru
, and H.
Kohlstedt
, “A memristive spiking neuron with firing rate coding
,” Front. Neurosci.
9
376 (2015
). 62.
M. D.
Pickett
, G.
Medeiros-Ribeiro
, and R. S.
Williams
, “A scalable neuristor built with Mott memristors
,” Nat. Mater.
12
(2
), 114
–117
(2013
). 63.
L.
Worschech
et al, “Universal and reconfigurable logic gates in a compact three-terminal resonant tunneling diode
,” Appl. Phys. Lett.
96
(4
), 042112
(2010
). 64.
T. V. P.
Bliss
and T.
Lømo
, “Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path
,” J. Physiol.
232
(2
), 331
–356
(Jul. 1973
). 65.
E. L.
Bienenstock
, L. N.
Cooper
, and P. W.
Munro
, “Theory for the development of neuron selectivity: Orientation specificity and binocular interaction in visual cortex
,” J. Neurosci.
2
(1
), 32
–48
(1982
). 66.
E.
Oja
, “Simplified neuron model as a principal component analyzer
,” J. Math. Biol.
15
(3
), 267
–273
(1982
). 67.
E. R.
Kandel
, In Search of Memory: The Emergence of a New Science of Mind
(W.W. Norton & Co.
, New York
, 2006
).68.
M.
Ziegler
et al, “An electronic version of Pavlov’s dog
,” Adv. Funct. Mater.
22
(13
), 2744
–2749
(2012
). 69.
O.
Bichler
et al, “Pavlov’s dog associative learning demonstrated on synaptic-like organic transistors
,” Neural Comput.
25
(2
), 549
–566
(2012
). 70.
G.
Bi
and M.
Poo
, “Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type
,” J. Neurosci.
18
(24
), 10464
–10472
(1998
). 71.
G. W.
Burr
et al., “Access devices for 3D crosspoint memory
,” J. Vac. Sci. Technol. B
. 32
, 4
, 040802
(2014
). 72.
J. S.
Seo
et al., “A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons
,” in 2011 IEEE Custom Integrated Circuits Conference (CICC)
(IEEE, 2011
), pp. 1
–4
.73.
H. Y.
Chen
, S.
Yu
, B.
Gao
, P.
Huang
, J.
Kang
, and H. S. P.
Wong
, “HfOx based vertical resistive random access memory for cost-effective 3D cross-point architecture without cell selector
,” in 2012 International Electron Devices Meeting
(IEEE, 2012
), pp. 20.7.1-20.7.4
.74.
S.
Ambrogio
, S.
Balatti
, F.
Nardi
, S.
Facchinetti
, and D.
Ielmini
, “Spike-timing dependent plasticity in a transistor-selected resistive switching memory
,” Nanotechnology
24
(38
), 384012
(2013
). 75.
See https://pubs.acs.org/doi/abs/10.1021/nl203687n for “A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications – nano letters (ACS Publications)” (accessed May 15, 2018).
76.
P.
Yao
et al, “Face classification using electronic synapses
,” Nat. Commun.
8
, 15199
(May 2017
). ©2018 Authors. Published by AIP Publishing.
2018
Author(s)
You do not currently have access to this content.
