Inspired by chaotic firing of neurons in the brain, we propose ChaosNet—a novel chaos based artificial neural network architecture for classification tasks. ChaosNet is built using layers of neurons, each of which is a 1D chaotic map known as the Generalized Luröth Series (GLS) that has been shown in earlier works to possess very useful properties for compression, cryptography, and for computing XOR and other logical operations. In this work, we design a novel learning algorithm on ChaosNet that exploits the topological transitivity property of the chaotic GLS neurons. The proposed learning algorithm gives consistently good performance accuracy in a number of classification tasks on well known publicly available datasets with very limited training samples. Even with as low as seven (or fewer) training samples/class (which accounts for less than 0.05% of the total available data), ChaosNet yields performance accuracies in the range of . We demonstrate the robustness of ChaosNet to additive parameter noise and also provide an example implementation of a two layer ChaosNet for enhancing classification accuracy. We envisage the development of several other novel learning algorithms on ChaosNet in the near future.
REFERENCES
1.
P.
Faure
and H.
Korn
, “Is there chaos in the brain? I. Concepts of nonlinear dynamics and methods of investigation
,” C. R. de l’Académie des Sci. Ser. III Sci. de la Vie
324
, 773
–793
(2001
). 2.
H.
Korn
and P.
Faure
, “Is there chaos in the brain? II. Experimental evidence and related models
,” C. R. Biol.
326
, 787
–840
(2003
). 3.
Y.
Fan
and A. V.
Holden
, “Bifurcations, burstings, chaos and crises in the Rose-Hindmarsh model for neuronal activity
,” Chaos, Solitons Fractals
3
, 439
–449
(1993
). 4.
Y.
Ding
, J. H.
Sohn
, M. G.
Kawczynski
, H.
Trivedi
, R.
Harnish
, N. W.
Jenkins
, D.
Lituiev
, T. P.
Copeland
, M. S.
Aboian
, C.
Mari Aparici
et al., “A deep learning model to predict a diagnosis of Alzheimer disease by using 18f-FDG PET of the brain
,” Radiology
290
, 456
–464
(2018
). 5.
N.
Harikrishnan
, R.
Vinayakumar
, and K.
Soman
, “A machine learning approach towards phishing email detection,” in Proceedings of the Anti-Phishing Pilot at ACM International Workshop on Security and Privacy Analytics (IWSPA AP) (2018), Vol. 2013, pp. 455–468.6.
A.
Graves
, A.-R.
Mohamed
, and G.
Hinton
, “Speech recognition with deep recurrent neural networks,” in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, 2013), pp. 6645–6649.7.
A. M.
Saxe
, Y.
Bansal
, J.
Dapello
, M.
Advani
, A.
Kolchinsky
, B. D.
Tracey
, and D. D.
Cox
, “On the information bottleneck theory of deep learning,” in International Conference on Learning Representations (2018).8.
N.
Tishby
and N.
Zaslavsky
, “Deep learning and the information bottleneck principle,” in 2015 IEEE Information Theory Workshop (ITW) (IEEE, 2015), pp. 1–5.9.
C. B.
Delahunt
and J. N.
Kutz
, “Putting a bug in ML: The moth olfactory network learns to read MNIST,” preprint arXiv:1802.05405 (2018).10.
K.
Aihara
, T.
Takabe
, and M.
Toyoda
, “Chaotic neural networks
,” Phys. Lett. A
144
, 333
–340
(1990
). 11.
N.
Crook
and T. O.
Scheper
, “A novel chaotic neural network architecture,” in ESANN'2001 Proceedings—European Symposium on Artificial Neural Networks Bruges (Belgium), 25–27 April 2001 (D-Facto, 2001), pp. 295–300.12.
W. J.
Freeman
et al., Mass Action in the Nervous System
(Academic Press
, 1975
), Vol. 2004.13.
H.-J.
Chang
and W. J.
Freeman
, “Parameter optimization in models of the olfactory neural system
,” Neural Netw.
9
, 1
–14
(1996
). 14.
R.
Kozma
and W. J.
Freeman
, “A possible mechanism for intermittent oscillations in the kiii model of dynamic memories-the case study of olfaction,” in IJCNN’99 International Joint Conference on Neural Networks (Cat. No. 99CH36339) (IEEE, 1999), Vol. 1, pp. 52–57.15.
I.
Tsuda
, “Dynamic link of memory–chaotic memory map in nonequilibrium neural networks
,” Neural Netw.
5
, 313
–326
(1992
). 16.
J. S.
Nicolis
and I.
Tsuda
, “Chaotic dynamics of information processing: The “magic number seven plus-minus two” revisited
,” Bull. Math. Biol.
47
, 343
–365
(1985
). 17.
K.
Kaneko
, “Lyapunov analysis and information flow in coupled map lattices
,” Phys. D Nonlinear Phenom.
23
, 436
–447
(1986
). 18.
K.
Kaneko
, “Clustering, coding, switching, hierarchical ordering, and control in a network of chaotic elements
,” Phys. D Nonlinear Phenom.
41
, 137
–172
(1990
). 19.
A.
Kathpalia
and N.
Nagaraj
, “A novel compression based neuronal architecture for memory encoding,” in Proceedings of the 20th International Conference on Distributed Computing and Networking (ACM, 2019), pp. 365–370.20.
Z.
Aram
, S.
Jafari
, J.
Ma
, J. C.
Sprott
, S.
Zendehrouh
, and V.-T.
Pham
, “Using chaotic artificial neural networks to model memory in the brain
,” Commun. Nonlinear Sci. Numerical Simul.
44
, 449
–459
(2017
). 21.
K. T.
Alligood
, T. D.
Sauer
, and J. A.
Yorke
, Chaos
(Springer
, 1996
).22.
Deterministic chaos is characterized by the “ Butterfly Effect”—sensitive dependence of behavior to minute changes in initial conditions.
23.
A.
Babloyantz
and C.
Lourenço
, “Brain chaos and computation
,” Int. J. Neural Syst.
7
, 461
–471
(1996
). 24.
C.
Barras
, “Mind maths: Brainquakes on the edge of chaos
,” New Scientist
217
, 36
(2013
). 25.
T.
Elbert
, B.
Rockstroh
, Z. J.
Kowalik
, M.
Hoke
, M.
Molnar
, J. E.
Skinner
, and N.
Birbaumer
, “Chaotic brain activity
,” Electroencephalogr. Clin. Neurophysiol./Suppl.
44
, 441
–449
(1995
).26.
J.
Sprott
, “Is chaos good for learning?
,” Nonlinear Dyn. Psychol. Life Sci.
17
, 223
–232
(2013
).27.
G.
Baghdadi
, S.
Jafari
, J.
Sprott
, F.
Towhidkhah
, and M. H.
Golpayegani
, “A chaotic model of sustaining attention problem in attention deficit disorder
,” Commun. Nonlinear Sci. Numer. Simul.
20
, 174
–185
(2015
). 28.
A. L.
Hodgkin
and A. F.
Huxley
, “A quantitative description of membrane current and its application to conduction and excitation in nerve
,” J. Phys.
117
, 500
–544
(1952
). 29.
J. L.
Hindmarsh
and R.
Rose
, “A model of neuronal bursting using three coupled first order differential equations
,” Proc. R. Soc. Lond. B Biol. Sci.
221
, 87
–102
(1984
). 30.
R.
FitzHugh
, “Impulses and physiological states in theoretical models of nerve membrane
,” Biophys. J.
1
, 445
–466
(1961
). 31.
J.
Nagumo
, S.
Arimoto
, and S.
Yoshizawa
, “An active pulse transmission line simulating nerve axon
,” Proc. IRE
50
, 2061
–2070
(1962
). 32.
A.
Zerroug
, L.
Terrissa
, and A.
Faure
, “Chaotic dynamical behavior of recurrent neural network
,” Annu. Rev. Chaos Theory Bifurc. Dyn. Syst.
4
, 55
–66
(2013
).33.
N. B.
Harikrishnan
and N.
Nagaraj
, “A novel chaos theory inspired neuronal architecture,” preprint arXiv:1905.12601 (2019).34.
R. C.
Staudemeyer
and C. W.
Omlin
, “Extracting salient features for network intrusion detection using machine learning methods
,” S. Afr. Comput. J.
52
, 82
–96
(2014
). 35.
K.
Dajani
and C.
Kraaikamp
, Ergodic Theory of Numbers
(Cambridge University Press
, 2002
), Vol. 29.36.
Let be the trajectory of a chaotic map with initial condition , where . The interval is partitioned into sub intervals denoted as . If , then we denote by the symbol The new sequence of symbol is the symbolic sequence of the trajectory of .
37.
Generating Markov Partition or GMP is based on splitting the state space into a complete set of disjoint regions, namely, it covers all state space and enables associating a one-to-one correspondence between trajectories and itinerary sequences of symbols (L and R) without losing any information?.
38.
N.
Nagaraj
, “Novel applications of chaos theory to coding and cryptography,” Ph.D. thesis (NIAS, 2008).39.
N.
Nagaraj
, P. G.
Vaidya
, and K. G.
Bhat
, “Arithmetic coding as a non-linear dynamical system
,” Commun. Nonlinear Sci. Numer. Simul.
14
, 1013
–1020
(2009
). 40.
N.
Nagaraj
, “Using cantor sets for error detection
,” PeerJ Comput. Sci.
5
, e171
(2019
). 41.
K.-W.
Wong
, Q.
Lin
, and J.
Chen
, “Simultaneous arithmetic coding and encryption using chaotic maps
,” IEEE Trans. Circuits Syst. II Express Briefs
57
, 146
–150
(2010
). 42.
W.
Gerstner
, W. M.
Kistler
, R.
Naud
, and L.
Paninski
, Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition
(Cambridge University Press
, 2014
).43.
For a nonconstant matrix , normalization is achieved by performing . A constant matrix is normalized to all ones.
44.
G.
Cybenko
, “Approximation by superpositions of a sigmoidal function
,” Math. Control Signals Syst.
2
, 303
–314
(1989
). 45.
Y.
LeCun
and C.
Cortes
, See http://yann.lecun.com/exdb/mnist/ for “MNIST Handwritten Digit Database” (2010).46.
K.
Cup
, See http://www.kdd.org/kdd-cup/view/kdd-cup-1999/Data for “Data (1999)” (1999).47.
R. P.
Lippmann
, D. J.
Fried
, I.
Graf
, J. W.
Haines
, K. R.
Kendall
, D.
McClung
, D.
Weber
, S. E.
Webster
, D.
Wyschogrod
, R. K.
Cunningham
et al., “Evaluating intrusion detection systems: The 1998 darpa off-line intrusion detection evaluation,” in Proceedings DARPA Information Survivability Conference and Exposition. DISCEX’00 (IEEE, 2000), Vol. 2, pp. 12–26.48.
See http://archive.ics.uci.edu/ml/datasets/iris for database for Iris data set.
49.
C. L.
Blake
and C. J.
Merz
, See http://www.ics.uci.edu/∼mlearn/MLRepository.html for “UCI Repository of Machine Learning Databases” (1998).50.
See http://phl.upr.edu/hec for “The habitable Exoplanet Catalog.”
51.
A.
M’endez
, “The night sky of exo-planets,” Hipparcos catalog (2011).52.
S.
Saha
, N.
Nagaraj
, A.
Mathur
, and R.
Yedida
, “Evolution of novel activation functions in neural network training with applications to classification of exoplanets,” preprint arXiv:1906.01975 (2019).53.
S.
Saha
, S.
Basak
, M.
Safonova
, K.
Bora
, S.
Agrawal
, P.
Sarkar
, and J.
Murthy
, “Theoretical validation of potential habitability via analytical and boosted tree methods: An optimistic study on recently discovered exoplanets
,” Astron. Comput.
23
, 141
–150
(2018
). 54.
55.
T. M.
Cover
, P.
Hart
et al., “Nearest neighbor pattern classification
,” IEEE Trans. Inf. Theory
13
, 21
–27
(1967
). 56.
M. A.
Hearst
, S. T.
Dumais
, E.
Osuna
, J.
Platt
, and B.
Scholkopf
, “Support vector machines
,” IEEE Intell. Syst. Appl.
13
, 18
–28
(1998
). 57.
Y.
LeCun
, Y.
Bengio
, and G.
Hinton
, “Deep learning
,” Nature
521
, 436
(2015
). 58.
F.
Pedregosa
, G.
Varoquaux
, A.
Gramfort
, V.
Michel
, B.
Thirion
, O.
Grisel
, M.
Blondel
, P.
Prettenhofer
, R.
Weiss
, V.
Dubourg
, J.
Vanderplas
, A.
Passos
, D.
Cournapeau
, M.
Brucher
, M.
Perrot
, and E.
Duchesnay
, “Scikit-learn: Machine learning in python
,” J. Mach. Learn. Res.
12
, 2825
–2830
(2011
).59.
F.
Chollet
et al., See https://keras.io for “Keras” (2015).60.
K.
Bora
, S.
Saha
, S.
Agrawal
, M.
Safonova
, S.
Routh
, and A.
Narasimhamurthy
, “Cd-hpf: New habitability score via data analytic modeling
,” Astron. Computing
17
, 129
–143
(2016
). 61.
Hyperparameters are rarely subjected to noise and hence we ignore this scenario. It is always possible to protect the hyperparameters by using strong error correction codes.
© 2019 Author(s).
2019
Author(s)
You do not currently have access to this content.
