We describe the continuous-time dynamics of networks implemented on Field Programable Gate Arrays (FPGAs). The networks can perform Boolean operations when the FPGA is in the clocked (digital) mode; however, we run the programed FPGA in the unclocked (analog) mode. Our motivation is to use these FPGA networks as ultrafast machine-learning processors, using the technique of reservoir computing. We study both the undriven dynamics and the input response of these networks as we vary network design parameters, and we relate the dynamics to accuracy on two machine-learning tasks.
REFERENCES
1.
H.
Jaeger
, “The ‘echo state’ approach to analysing and training recurrent neural networks—With an erratum note,” GMD Report No. 148, German National Research Center for Information Technology, 2001.2.
W.
Maass
, T.
Natschlaeger
, and H.
Markram
, “Real-time computing without stable states: A new framework for neural computation based on perturbations
,” Neural Comput.
14
, 2531
–2560
(2002
). 3.
R.
Girshick
, “Fast R-CNN,” in 2015 IEEE International Conference on Computer Vision (IEEE, 2015).4.
G.
Hinton
, “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups
,” IEEE Signal Process.
29
, 82
–97
(2012
). 5.
D.
Silver
et al., “Mastering the game of go with deep neural networks and tree search
,” Nature
529
, 484
–489
(2016
). 6.
P.
Pattison
, Y.
Kalish
, and D.
Lusher
, “An introduction to exponential random graph (p*) models for social networks
,” Soc. Netw.
29
, 173
–191
(2007
). 7.
R.
Olfati-Saber
, “Flocking for multi-agent dynamic systems: Algorithms and theory
,” IEEE Trans. Automat. Contr.
51
, 401
–420
(2006
). 8.
S. A.
Kauffman
, “Metabolic stability and epigenesis in randomly constructed genetic nets
,” J. Theor. Biol.
22
, 437
–467
(1969
). 9.
B.
Derrida
and Y.
Pomeau
, “Random networks of automata: A simple annealed approximation
,” Europhys. Lett.
1
, 45
–49
(1986
). 10.
B.
Mesot
and C.
Teuscher
, “Critical values in asynchronous random Boolean networks,” in Advances in Artificial Life, ECAL 2003, edited by W. Banzhaf, J. Ziegler, T. Christaller, P. Dittrich, and J. T. Kim (Springer, Berlin, 2003).11.
B.
Schrauwen
, D.
Verstraeten
, and J.
Van Campenhout
, “An overview of reservoir computing: Theory, applications and implementations,” in Proceedings of the 15th European Symposium on Artificial Neural Network (Schloss Dagstuhl, 2007), pp. 471–482; available at https://dblp.uni-trier.de/db/conf/esann/esann2007.html.12.
M.
Lukoševičius
, H.
Jaeger
, and B.
Schrauwen
, “Resrvoir computing trends
,” K ünsliche Intelligenz
26
, 365
–371
(2012
). 13.
M.
Lukoševičius
, “A practical guide to applying echo state networks,” in Neural Networks: Tricks of the Trade, edited by G. Montavon, G. B. Orr, and K. R. Müller (Springer, Berlin, 2012), pp. 659–686.14.
N.
Schaetti
, M.
Salomon
, and R.
Couturier
, “Echo state networks-based reservoir computing for MNIST handwritten digits recognition,” in 2016 IEEE International Conference on Computational Science and Engineering (CSE) (IEEE, 2016), pp. 484–491.15.
I.
Ilies
et al., “Stepping forward through echoes of the past: Forecasting with echo state networks,” in 2006/07 Forecasting Competition for Neural Networks & Computational Intelligence (NN3) (2007); available at http://www.neural-forecasting-competition.com/NN3/results.htm.16.
E. A.
Antonelo
, B.
Schrauwen
, and D.
Stroobandt
, “Event detection and localization for small mobile robots using reservoir computing
,” Neural Netw.
21
, 862
–871
(2008
). 17.
M.
Salmen
and P. G.
Ploger
, “Echo state networks used for motor control,” in Proceedings of the 2005 IEEE International Conference on Robotics and Automation (IEEE, 2005), pp. 1953–1958.18.
H.
Jaeger
and H.
Haas
, “Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication
,” Science
304
, 80
(2004
). 19.
P.
Antonik
, A.
Smerieri
, F.
Duport
, M.
Haelterman
, and S.
Massar
, “FPGA implementation of reservoir computing with online learning,” in 24th Belgian-Dutch Conference on Machine Learning (2015); available at https://www.semanticscholar.org/paper/FPGA-Implementation-of-Reservoir-Computing-with-Antonik-Smerieri/1fdf965c77a08802276706c872f19073b9d3a0b1.20.
F.
Triefenbach
, A.
Jalalvand
, B.
Schrauwen
, and J.
Martens
, “Phoneme recognition with large hierarchical reservoirs,” in Advances in Neural Information Processing Systems 23 (NIPS 2010) (2010), pp. 2307–2315; available at https://papers.nips.cc/book/advances-in-neural-information-processing-systems-23-2010.21.
P.
Buteneers
et al., “Automatic detection of epileptic seizures on the intra-cranial electroencephalogram of rats using reservoir computing
,” Artif. Intell. Med.
53
, 215
–223
(2010
). 22.
P.-J.
Kindermans
, P.
Buteneers
, D.
Verstraeten
, and B.
Schrauwen
, “An uncued brain-computer interface using reservoir computing,” in Proceedings of the Workshop on Machine Learning for Assistive Technologies (2010); available at https://biblio.ugent.be/publication/1447714.23.
M.
Escalona-Morán
, M. C.
Soriano
, J.
García-Prieto
, I.
Fischer
, and C. R.
Mirasso
, “Multivariate nonlinear time-series estimation using delay-based reservoir computing
,” Eur. Phys. J. Spec. Top.
223
, 2903
–1912
(2014
). 24.
J.
Pathak
et al., “Using machine learning in conjunction with a knowledge-based model
,” Chaos
28
, 041101
(2018
). 25.
J.
Pathak
, B.
Hunt
, M.
Girvan
, Z.
Lu
, and E.
Ott
, “Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach
,” Phys. Rev. Lett.
120
, 024102
(2018
). 26.
L.
Appelant
et al., “Information processing using a single dynamical node as complex system
,” Nat. Commun.
2
, 468
(2011
). 27.
L.
Larger
et al., “High-speed photonic reservoir computing using a time-delay-based architecture: Million words per second classification
,” Phys. Rev. X
7
, 011015
(2017
). 28.
Y.
Paquot
et al., “Optoelectronic reservoir computing
,” Sci. Rep.
2
, 287
(2012
). 29.
R.
Zhang
et al., “Boolean chaos
,” Phys. Rev. E
80
, 045202
(2009
). 30.
D. P.
Rosin
, D.
Rontani
, and D. J.
Gauthier
, “Ultrafast physical generation of random numbers using hybrid Boolean networks
,” Phys. Rev. E
87
, 040902
(2013
). 31.
D. P.
Rosin
, D.
Rontani
, D. J.
Gauthier
, and E.
Schöll
, “Experiments on autonomous Boolean networks
,” Chaos
23
, 025102
(2013
). 32.
D. P.
Rosin
, D.
Rontani
, and D. J.
Gauthier
, “Synchronization of coupled Boolean phase oscillators
,” Phys. Rev. E
89
, 042907
(2014
). 33.
D. P.
Rosin
, D.
Rontani
, N. D.
Haynes
, E.
Schöll
, and D. J.
Gauthier
, “Transient scaling and resurgence of chimera states in networks of Boolean phase oscillators
,” Phys. Rev. E
90
, 030902
(2014
). 34.
D. P.
Rosin
, D.
Rontani
, N. D.
Haynes
, E.
Schöll
, and D. J.
Gauthier
, “Computation at the edge of chaos: Phase transitions and emergent computation
,” Physica D
42
, 12
–37
(1990
). 35.
Y.
LeCun
, C.
Cortes
, and C. J. C.
Burges
, see http://yann.lecun.com/exdb/mnist/ for “MNIST handwritten digit database.”36.
S.
Kokalj-Filipović
, R.
Miller
, and J.
Morman
, “Autoencoders for training compact deep learning rf classifiers for wireless protocols
,” in 20th IEEE International Workshop on Signal Processing Advances in Wireless Communications
(IEEE
, 2019
). 37.
N.
Srivastava
, G.
Hinton
, A.
Krizhevsky
, I.
Sutskever
, and R.
Salakhutdinov
, “Dropout: A simple way to prevent neural networks from overfitting
,” J. Mach. Learn. Res.
15
, 1929
–1958
(2014
); available at http://jmlr.org/papers/v15/srivastava14a.html.38.
D.
Lathrop
, I.
Shani
, P.
Megson
, A.
Restelli
, and A. R.
Mautino
, “Integrated circuit designs for reservoir computing and machine learning,” International Patent Application PCT/US2018/032902 (2018).© 2019 Author(s).
2019
Author(s)
You do not currently have access to this content.