Computational modeling and experimental/clinical prediction of the complex signals during cardiac arrhythmias have the potential to lead to new approaches for prevention and treatment. Machine-learning (ML) and deep-learning approaches can be used for time-series forecasting and have recently been applied to cardiac electrophysiology. While the high spatiotemporal nonlinearity of cardiac electrical dynamics has hindered application of these approaches, the fact that cardiac voltage time series are not random suggests that reliable and efficient ML methods have the potential to predict future action potentials. This work introduces and evaluates an integrated architecture in which a long short-term memory autoencoder (AE) is integrated into the echo state network (ESN) framework. In this approach, the AE learns a compressed representation of the input nonlinear time series. Then, the trained encoder serves as a feature-extraction component, feeding the learned features into the recurrent ESN reservoir. The proposed AE-ESN approach is evaluated using synthetic and experimental voltage time series from cardiac cells, which exhibit nonlinear and chaotic behavior. Compared to the baseline and physics-informed ESN approaches, the AE-ESN yields mean absolute errors in predicted voltage 6–14 times smaller when forecasting approximately 20 future action potentials for the datasets considered. The AE-ESN also demonstrates less sensitivity to algorithmic parameter settings. Furthermore, the representation provided by the feature-extraction component removes the requirement in previous work for explicitly introducing external stimulus currents, which may not be easily extracted from real-world datasets, as additional time series, thereby making the AE-ESN easier to apply to clinical data.
REFERENCES
1.
A.
Karma
and R. F.
Gilmour
, Jr., “Nonlinear dynamics of heart rhythm disorders
,” Phys. Today
60
(3
), 51
–57
(2007
). 2.
E. M.
Cherry
, F. H.
Fenton
, and R. F.
Gilmour
, Jr., “Mechanisms of ventricular arrhythmias: A dynamical systems-based perspective
,” Am. J. Physiol. Heart Circ. Physiol.
302
, H2451
–H2463
(2012
). 3.
V.
Kappadan
, S.
Telele
, I.
Uzelac
, F.
Fenton
, U.
Parlitz
, S.
Luther
, and J.
Christoph
, “High-resolution optical measurement of cardiac restitution, contraction, and fibrillation dynamics in beating vs blebbistatin-uncoupled isolated rabbit hearts
,” Front. Physiol.
11
, 464
(2020
). 4.
J. B.
Nolasco
and R. W.
Dahlen
, “A graphic method for the study of alternation in cardiac action potentials
,” J. Appl. Physiol.
25
, 191
–196
(1968
). 5.
M. R.
Guevara
, G.
Ward
, A.
Shrier
, and L.
Glass
, “Electrical alternans and period-doubling bifurcations
,” Comput. Cardiol.
11
, 167
–170
(1984
).6.
A.
Karma
, “Electrical alternans and spiral wave breakup in cardiac tissue
,” Chaos
4
, 461
–472
(1994
). 7.
D. D.
Chen
, R. A.
Gray
, I.
Uzelac
, C.
Herndon
, and F. H.
Fenton
, “Mechanism for amplitude alternans in electrocardiograms and the initiation of spatiotemporal chaos
,” Phys. Rev. Lett.
118
, 168101
(2017
). 8.
A.
Gizzi
, E. M.
Cherry
, R. F.
Gilmour
, Jr., S.
Luther
, S.
Filippi
, and F. H.
Fenton
, “Effects of pacing site and stimulation history on alternans dynamics and the development of complex spatiotemporal patterns in cardiac tissue
,” Front. Card. Electrophysiol.
4
, 71
(2013
).9.
J. M.
Pastore
, S. D.
Girouard
, K. R.
Laurita
, F. G.
Akar
, and D. S.
Rosenbaum
, “Mechanism linking T-wave alternans to the genesis of cardiac fibrillation
,” Circulation
99
, 1385
–1394
(1999
). 10.
M. A.
Watanabe
, F. H.
Fenton
, S. J.
Evans
, H. M.
Hastings
, and A.
Karma
, “Mechanisms for discordant alternans
,” J. Cardiovasc. Electrophysiol.
12
, 196
–206
(2001
). 11.
E. M.
Cherry
and F. H.
Fenton
, “Suppression of alternans and conduction blocks despite steep APD restitution: Electrotonic, memory, and conduction velocity restitution effects
,” Am. J. Physiol. Heart Circ. Physiol.
286
, H2332
–H2341
(2004
). 12.
I.
Uzelac
, Y. C.
Ji
, D.
Hornung
, J.
Schröder-Scheteling
, S.
Luther
, R. A.
Gray
, E. M.
Cherry
, and F. H.
Fenton
, “Simultaneous quantification of spatially discordant alternans in voltage and intracellular calcium in Langendorff-perfused rabbit hearts and inconsistencies with models of cardiac action potentials and Ca transients
,” Front. Physiol.
8
, 819
(2017
). 13.
F. X.
Witkowski
, L. J.
Leon
, P. A.
Penkoske
, W. R.
Giles
, M. L.
Spano
, W. L.
Ditto
, and A. T.
Winfree
, “Spatiotemporal evolution of ventricular fibrillation
,” Nature
392
, 78
–82
(1998
). 14.
R. A.
Gray
, A. M.
Pertsov
, and J.
Jalife
, “Spatial and temporal organization during cardiac fibrillation
,” Nature
392
, 75
–78
(1998
). 15.
J.
Christoph
, M.
Chebbok
, C.
Richter
, J.
Schröder-Schetelig
, P.
Bittihn
, S.
Stein
, I.
Uzelac
, F. H.
Fenton
, G.
Hasenfuß
, R.
Gilmour
, Jr., and S.
Luther
, “Electromechanical vortex filaments during cardiac fibrillation
,” Nature
555
, 667
–672
(2018
). 16.
W. J.
Rappel
, F.
Fenton
, and A.
Karma
, “Spatiotemporal control of wave instabilities in cardiac tissue
,” Phys. Rev. Lett.
83
, 456
–459
(1999
). 17.
D. J.
Christini
, M. L.
Riccio
, C. A.
Culianu
, J. J.
Fox
, A.
Karma
, and R. F.
Gilmour
, Jr., “Control of electrical alternans in canine cardiac Purkinje fibers
,” Phys. Rev. Lett.
96
, 104101
(2006
). 18.
C. M.
Berger
, J. W.
Cain
, J. E. S.
Socolar
, and D. J.
Gauthier
, “Control of electrical alternans in simulations of paced myocardium using extended time-delay autosynchronization
,” Phys. Rev. E
76
, 041917
(2007
). 19.
A.
Garzón
, R. O.
Grigoriev
, and F. H.
Fenton
, “Continuous-time control of alternans in long Purkinje fibers
,” Chaos
24
, 033124
(2014
). 20.
K.
Kulkarni
, S. W.
Lee
, R.
Kluck
, and E. G.
Tolkacheva
, “Real-time closed loop diastolic interval control prevents cardiac alternans in isolated whole rabbit hearts
,” Ann. Biomed. Eng.
46
, 555
–566
(2018
). 21.
B.-R.
Choi
, W.
Jang
, and G.
Salama
, “Spatially discordant voltage alternans cause wavebreaks in ventricular fibrillation
,” Heart Rhythm
4
, 1057
–1068
(2007
). 22.
S.
Shahi
, C. D.
Marcotte
, C. J.
Herndon
, F. H.
Fenton
, Y.
Shiferaw
, and E.
Cherry
, “Long-time prediction of arrhythmic cardiac action potentials using recurrent neural networks and reservoir computing
,” Front. Physiol.
12
, 1585
(2021
). 23.
N.
Ibtehaz
, M. S.
Rahman
, and M. S.
Rahman
, “VFPred: A fusion of signal processing and machine learning techniques in detecting ventricular fibrillation from ECG signals
,” Biomed. Signal Process. Control
49
, 349
–359
(2019
). 24.
G. T.
Taye
, E. B.
Shim
, H.-J.
Hwang
, and K. M.
Lim
, “Machine learning approach to predict ventricular fibrillation based on QRS complex shape
,” Front. Physiol.
10
, 1193
(2019
). 25.
M.
Murugappan
, L.
Murugesan
, S.
Jerritta
, and H.
Adeli
, “Sudden cardiac arrest (SCA) prediction using ECG morphological features
,” Arab. J. Sci. Eng.
46
, 947
–961
(2021
). 26.
Z.
Bar-Joseph
, G. K.
Gerber
, D. K.
Gifford
, T. S.
Jaakkola
, and I.
Simon
, “Continuous representations of time-series gene expression data
,” J. Comput. Biol.
10
, 341
–356
(2003
). 27.
M.
Ghil
and R.
Vautard
, “Interdecadal oscillations and the warming trend in global temperature time series
,” Nature
350
, 324
–327
(1991
). 28.
K.
Limthong
, “Real-time computer network anomaly detection using machine learning techniques
,” J. Adv. Comput. Netw.
1
, 126
–133
(2013
).29.
Q.
Gong
, Y.
Chen
, X.
He
, Z.
Zhuang
, T.
Wang
, H.
Huang
, X.
Wang
, and X.
Fu
, “DeepScan: Exploiting deep learning for malicious account detection in location-based social networks
,” IEEE Commun. Mag.
56
, 21
–27
(2018
). 30.
V.
Plagianakos
and E.
Tzanaki
, “Chaotic analysis of seismic time series and short term forecasting using neural networks,” in IJCNN’01. International Joint Conference on Neural Networks. Proceedings (Cat. No. 01CH37222) (IEEE, 2001), Vol. 3, pp. 1598–1602.31.
R. S.
Tsay
, Analysis of Financial Time Series
(John Wiley & Sons
, 2005
), Vol. 543.32.
A.
Dingli
and K. S.
Fournier
, “Financial time series forecasting—A deep learning approach
,” Int. J. Mach. Learn. Comput.
7
, 118
–122
(2017
). 33.
H.
Zhao
, “A chaotic time series prediction based on neural network: Evidence from the Shanghai Composite Index in China,” in 2009 International Conference on Test and Measurement (IEEE, 2009), Vol. 2, pp. 382–385.34.
S.
Takahashi
, Y.
Chen
, and K.
Tanaka-Ishii
, “Modeling financial time-series with generative adversarial networks
,” Phys. A
527
, 121261
(2019
). 35.
R.
Billinton
, H.
Chen
, and R.
Ghajar
, “Time-series models for reliability evaluation of power systems including wind energy
,” Microelectron. Reliab.
36
, 1253
–1261
(1996
). 36.
D. W.
Bunn
, “Forecasting loads and prices in competitive power markets
,” Proc. IEEE
88
, 163
–169
(2000
). 37.
A.
Deihimi
and H.
Showkati
, “Application of echo state networks in short-term electric load forecasting
,” Energy
39
, 327
–340
(2012
). 38.
R.
Zhou
, F.
Liu
, and C. W.
Gravelle
, “Deep learning for modulation recognition: A survey with a demonstration
,” IEEE Access
8
, 67366
–67376
(2020
). 39.
Y.
Zhang
, P.
Tiwari
, D.
Song
, X.
Mao
, P.
Wang
, X.
Li
, and H. M.
Pandey
, “Learning interaction dynamics with an interactive LSTM for conversational sentiment analysis
,” Neural Netw.
133
, 40
–56
(2021
). 40.
Y.
Chen
, J.
Wu
, and H.
Bu
, “Stock market embedding and prediction: A deep learning method,” in 2018 15th International Conference on Service Systems and Service Management (ICSSSM) (IEEE, 2018), pp. 1–6.41.
H.
Jaeger
, Tutorial on Training Recurrent Neural Networks, Covering BPPT, RTRL, EKF and the “Echo State Network” Approach
(GMD-Forschungszentrum Informationstechnik Bonn
, 2002
), Vol. 5.42.
M.
Lukoševičius
and H.
Jaeger
, “Reservoir computing approaches to recurrent neural network training
,” Comput. Sci. Rev.
3
, 127
–149
(2009
). 43.
F. M.
Bianchi
, E.
Maiorino
, M. C.
Kampffmeyer
, A.
Rizzi
, and R.
Jenssen
, “Other recurrent neural networks models,” in Recurrent Neural Networks for Short-Term Load Forecasting: An Overview and Comparative Analysis (Springer International Publishing, Cham, 2017), pp. 31–39.44.
Z.
Han
, J.
Zhao
, H.
Leung
, K. F.
Ma
, and W.
Wang
, “A review of deep learning models for time series prediction
,” IEEE Sens. J.
21
, 7833
–7848
(2021
). 45.
C.
Sun
, M.
Song
, S.
Hong
, and H.
Li
, “A review of designs and applications of echo state networks,” arXiv:2012.02974 (2020).46.
E.
Bollt
, “On explaining the surprising success of reservoir computing forecaster of chaos? The universal machine learning dynamical system with contrast to VAR and DMD
,” Chaos
31
, 013108
(2021
). 47.
D. J.
Gauthier
, E.
Bollt
, A.
Griffith
, and W. A. S.
Barbosa
, “Next generation reservoir computing
,” Nat. Commun.
12
, 5564
(2021
). 48.
H.
Jaeger
, M.
Lukoševičius
, D.
Popovici
, and U.
Siewert
, “Optimization and applications of echo state networks with leaky-integrator neurons
,” Neural Netw.
20
, 335
–352
(2007
). 49.
J.
Pathak
, A.
Wikner
, R.
Fussell
, S.
Chandra
, B. R.
Hunt
, M.
Girvan
, and E.
Ott
, “Hybrid forecasting of chaotic processes: Using machine learning in conjunction with a knowledge-based model
,” Chaos
28
, 041101
(2018
). 50.
N. A. K.
Doan
, W.
Polifke
, and L.
Magri
, “Physics-informed echo state networks
,” J. Comput. Sci.
47
, 101237
(2020
). 51.
D. K.
Oh
, “Toward the fully physics-informed echo state network—An ODE approximator based on recurrent artificial neurons,” arXiv:2011.06769 (2020).52.
I.
Goodfellow
, Y.
Bengio
, A.
Courville
, and Y.
Bengio
, Deep Learning
(MIT Press
, 2016
), Vol. 1.53.
D. E.
Rumelhart
, G. E.
Hinton
, and R. J.
Williams
, “Learning internal representations by error propagation,” Technical Report, Institute for Cognitive Science, University of California, San Diego, La Jolla, CA, 1985.54.
N.
Japkowicz
, C.
Myers
, M.
Gluck
et al., “A novelty detection approach to classification,” in IJCAI’95: Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI, 1995), Vol. 1, pp. 518–523.55.
B.
Lindemann
, B.
Maschler
, N.
Sahlab
, and M.
Weyrich
, “A survey on anomaly detection for technical systems using LSTM networks
,” Comput. Ind.
131
, 103498
(2021
). 56.
G. E.
Hinton
and R. R.
Salakhutdinov
, “Reducing the dimensionality of data with neural networks
,” Science
313
, 504
–507
(2006
). 57.
X. J.
Guijuan Zhang
and Y.
Liu
, “A survey of autoencoder-based recommender systems
,” Front. Comput. Sci.
14
, 430
(2020
). 58.
M.
Chen
, Z.
Xu
, K.
Weinberger
, and F.
Sha
, “Marginalized denoising autoencoders for domain adaptation,” arXiv:1206.4683 (2012).59.
G.
Hinton
, S.
Osindero
, and Y.-W.
Teh
, “A fast learning algorithm for deep belief nets
,” Neural Comput.
18
, 1527
–1554
(2006
). 60.
G.
Hinton
, L.
Deng
, D.
Yu
, G.
Dahl
, A.-R.
Mohamed
, N.
Jaitly
, A.
Senior
, V.
Vanhoucke
, P.
Nguyen
, T.
Sainath
, and B.
Kingsbury
, “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups
,” IEEE Signal Process. Mag.
29
, 82
–97
(2012
). 61.
I.
Sutskever
, J.
Martens
, G.
Dahl
, and G.
Hinton
, “On the importance of initialization and momentum in deep learning,” in International Conference on Machine Learning (PMLR, 2013), pp. 1139–1147.62.
P.
Vincent
, H.
Larochelle
, I.
Lajoie
, Y.
Bengio
, P.-A.
Manzagol
, and L.
Bottou
, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion
,” J. Mach. Learn. Res.
11
, 3371–3408
(2010
).63.
C.
Zhou
and R. C.
Paffenroth
, “Anomaly detection with robust deep autoencoders,” in Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Association for Computing Machinery, 2017), pp. 665–674.64.
J.
Ashraf
, A. D.
Bakhshi
, N.
Moustafa
, H.
Khurshid
, A.
Javed
, and A.
Beheshti
, “Novel deep learning-enabled LSTM autoencoder architecture for discovering anomalous events from intelligent transportation systems
,” IEEE Trans. Intell. Transp. Syst.
2
, 4507–4518
(2020
).65.
S.
Maleki
, S.
Maleki
, and N. R.
Jennings
, “Unsupervised anomaly detection with LSTM autoencoders using statistical data-filtering
,” Appl. Soft Comput.
108
, 107443
(2021
). 66.
P.
Liu
, X.
Sun
, Y.
Han
, Z.
He
, W.
Zhang
, and C.
Wu
, “Arrhythmia classification of LSTM autoencoder based on time series anomaly detection
,” Biomed. Signal Process. Control
71
, 103228
(2022
). 67.
R.
Chandra
, S.
Goyal
, and R.
Gupta
, “Evaluation of deep learning models for multi-step ahead time series prediction
,” IEEE Access
9
, 83105
–83123
(2021
). 68.
A. H.
Mirza
and S.
Cosan
, “Computer network intrusion detection using sequential LSTM neural networks autoencoders,” in 2018 26th Signal Processing and Communications Applications Conference (SIU) (IEEE, 2018), pp. 1–4.69.
T.
Tang
, J.
Jia
, and H.
Mao
, “Dance with melody: An LSTM-autoencoder approach to music-oriented dance synthesis,” in Proceedings of the 26th ACM International Conference on Multimedia (Association for Computing Machinery, 2018), pp. 1598–1606.70.
B.
Hou
, J.
Yang
, P.
Wang
, and R.
Yan
, “LSTM-based auto-encoder model for ECG arrhythmias classification
,” IEEE Trans. Instrum. Meas.
69
, 1232
–1240
(2019
). 71.
K.
Jun
, D.-W.
Lee
, K.
Lee
, S.
Lee
, and M. S.
Kim
, “Feature extraction using an RNN autoencoder for skeleton-based abnormal gait recognition
,” IEEE Access
8
, 19196
–19207
(2020
). 72.
M.
Lukoševičius
, “A practical guide to applying echo state networks,” in Neural Networks: Tricks of the Trade (Springer, 2012), pp. 659–686.73.
P.
Lara-Benítez
, M.
Carranza-García
, and J. C.
Riquelme
, “An experimental review on deep learning architectures for time series forecasting,” arXiv:2103.12057 (2021).74.
M.
Yousefi-Azar
, V.
Varadharajan
, L.
Hamey
, and U.
Tupakula
, “Autoencoder-based feature learning for cyber security applications,” in 2017 International Joint Conference on Neural Networks (IJCNN) (IEEE, 2017), pp. 3854–3861.75.
F.
Fenton
and A.
Karma
, “Vortex dynamics in three-dimensional continuous myocardium with fiber rotation: Filament instability and fibrillation
,” Chaos
8
, 20
–47
(1998
). 76.
F. H.
Fenton
, E. M.
Cherry
, H. M.
Hastings
, and S. J.
Evans
, “Multiple mechanisms of spiral wave breakup in a model of cardiac electrical activity
,” Chaos
12
, 852
–892
(2002
). 77.
G. W.
Beeler
and H.
Reuter
, “Reconstruction of the action potential of ventricular myocardial fibres
,” J. Physiol.
268
, 177
–210
(1977
). 78.
Z.
Qu
, J. N.
Weiss
, and A.
Garfinkel
, “Spatiotemporal chaos in a simulated ring of cardiac cells
,” Phys. Rev. Lett.
78
, 1387
(1997
). 79.
S.
Shahi
, F. H.
Fenton
, and E. M.
Cherry
, “Prediction of chaotic time series using recurrent neural networks and reservoir computing techniques: A comparative study
,” Mach. Learn. Appl.
8
, 100300
(2022
). 80.
S. J.
Pan
and Q.
Yang
, “A survey on transfer learning
,” IEEE Trans. Knowl. Data Eng.
22
, 1345
–1359
(2009
). 81.
K.
Weiss
, T. M.
Khoshgoftaar
, and D.
Wang
, “A survey of transfer learning
,” J. Big Data
3
, 9
(2016
). © 2022 Author(s). Published under an exclusive license by AIP Publishing.
2022
Author(s)
You do not currently have access to this content.
