Koopman mode decomposition and tensor component analysis [also known as CANDECOMP (canonical decomposition)/PARAFAC (parallel factorization)] are two popular approaches of decomposing high dimensional datasets into modes that capture the most relevant features and/or dynamics. Despite their similar goal, the two methods are largely used by different scientific communities and are formulated in distinct mathematical languages. We examine the two together and show that, under certain conditions on the data, the theoretical decomposition given by the tensor component analysis is the same as that given by Koopman mode decomposition. This provides a “bridge” with which the two communities should be able to more effectively communicate. Our work provides new possibilities for algorithmic approaches to Koopman mode decomposition and tensor component analysis and offers a principled way in which to compare the two methods. Additionally, it builds upon a growing body of work showing that dynamical systems theory and Koopman operator theory, in particular, can be useful for problems that have historically made use of optimization theory.
REFERENCES
1.
B. O.
Koopman
, “Hamiltonian systems and transformation in Hilbert space
,” Proc. Natl. Acad. Sci.
17
, 315
(1931
). 2.
B. O.
Koopman
and J. V.
Neumann
, “Dynamical systems of continuous spectra
,” Proc. Natl. Acad. Sci.
18
, 255
(1932
). 3.
I.
Mezic
, “Spectral properties of dynamical systems, model reduction and decompositions
,” Nonlinear Dyn.
41
, 309
(2005
). 4.
M.
Budisíc
, R.
Mohr
, and I.
Mezic
, “Applied Koopmanism
,” Chaos
22
, 047510
(2012
). 5.
I.
Mezic
, “Spectrum of the Koopman operator, spectral expansions in functional spaces, and state-space geometry
,” J. Nonlinear Sci.
30
, 2091
(2019
). 6.
S. L.
Brunton
, M.
Budišić
, E.
Kaiser
, and J. N.
Kutz
, “Modern Koopman theory for dynamical systems,” arXiv:2102.12086 (2021).7.
I.
Mezic
, “On numerical approximations of the Koopman operator,” arXiv:2009.05883 (2020).8.
C. W.
Rowley
, I.
Mezic
, S.
Bagheri
, P.
Schlatter
, and S. S.
Henningson
, “Spectral analysis of nonlinear flows
,” J. Fluid Mech.
641
, 115
(2009
). 9.
I.
Mezic
, “Analysis of fluid flows via spectral properties of the Koopman operator
,” Annu. Rev. Fluid Mech.
45
, 357
(2013
). 10.
H.
Arbabi
and I.
Mezic
, “Study of dynamics in post transient flows using Koopman mode decomposition
,” Phys. Rev. Fluids
2
, 124402
(2017
). 11.
Y.
Susuki
, I.
Mezíc
, F.
Raak
, and T.
Hikihara
, “Applied Koopman operator theory for power systems technology
,” Nonlinear Theor. Appl. IEICE
7
, 430
(2016
). 12.
M.
Korda
, Y.
Susuki
, and I.
Mezic
, “Power grid transient stabilization using Koopman model predictive control
,” IFAC-PapersOnLine
51
, 297
(2018
). 13.
A. S.
Dogra
and W. T.
Redman
, “Optimizing neural networks via Koopman operator theory,” in Advances in Neural Information Processing Systems 33, NeurIPS 2020 (Neural Information Processing Systems Foundation, 2020).14.
I.
Manojlović
, M.
Fonoberova
, R.
Mohr
, A.
Andrejčuk
, Z.
Drmač
, Y.
Kevrekidis
, and I.
Mezić
, “Applications of Koopman mode analysis to neural networks,” arXiv:2006.11765 (2020).15.
M. E.
Tano
, G. D.
Portwood
, and J. C.
Ragusa
, “Accelerating training in artificial neural networks with dynamic mode decomposition,” arXiv:2006.14371 (2020).16.
I.
Naiman
and O.
Azencot
, “A Koopman approach to understanding sequence neural models,” arXiv:2102.07824 (2021).17.
B. W.
Brunton
, L. A.
Johnson
, J. G.
Ojemann
, and J. N.
Kutz
, “Extracting spatial-temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition
,” J. Neurosci. Methods
258
, 1
(2016
). 18.
S.
Klus
, F.
Nüske
, P.
Koltai
, H.
Wu
, I.
Kevrekidis
, C.
Schutte
, and F.
Noe
, “Data-driven model reduction and transfer operator approximation
,” J. Nonlinear Sci.
28
, 985
(2018
). 19.
H.
Lu
and D. M.
Tartakovsky
, “Prediction accuracy of dynamic mode decomposition
,” SIAM J. Sci. Comput.
42
, A1639
–A1662
(2020
). 20.
S.
Bradde
and W.
Bialek
, “PCA meets RG
,” J. Stat. Phys.
167
, 462
–475
(2017
). 21.
W. T.
Redman
, “Renormalization group as a Koopman operator
,” Phys. Rev. E
101
, 060104
(2020
). 22.
N.
Marrouch
, J.
Slawinska
, D.
Giannakis
, and H.
Read
, “Data-driven Koopman operator approach for computational neuroscience
,” Ann. Math. Artif. Intell.
88
, 1155
–1173
(2020
). 23.
S.
Balakhrishnan
, A.
Hasnain
, N.
Boddupalli
, D. M.
Joshy
, R. G.
Egbet
, and E.
Yeung
, “Prediction of fitness in bacteria with causal jump dynamic mode decomposition,” in 2020 American Control Conference (IEEE, 2020), pp. 3749–3756.24.
J. D.
Carroll
and J.-J.
Chang
, “Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition
,” Psychometrika
35
, 283
–319
(1970
). 25.
R.
Harshman
, “Foundations of the PARAFAC procedure: Models and conditions for an “explanatory” multi-model factor analysis,” UCLA Working Papers in Phonetics, 1970.26.
N.
Vervliet
, O.
Debals
, L.
Sorber
, M.
Van Barel
, and L.
De Lathauwer
, Tensorlab 3.0.27.
T. G.
Kolda
and B. W.
Bader
, “Tensor decompositions and applications
,” SIAM Rev.
51
, 455
–500
(2009
). 28.
D.
Hong
, T. G.
Kolda
, and J. D.
Duersch
, “Generalized canonical polyadic tensor decomposition
,” SIAM Rev.
62
, 133
–163
(2020
). 29.
S.
Rambhatla
, X.
Li
, and J.
Haupt
, “Provable online CP/PARAFAC decomposition of a structured tensor via dictionary learning,” in Advances in Neural Information Processing Systems 33, NeurIPS 2020 (, 2020).30.
J. B.
Kruskal
, “Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics
,” Linear Algebra Appl.
18
, 95
–138
(1977
). 31.
E.
Martınez-Montes
, P. A.
Valdés-Sosa
, F.
Miwakeichi
, R. I.
Goldman
, and M. S.
Cohen
, “Concurrent EEG/fMRI analysis by multiway partial least squares
,” NeuroImage
22
, 1023
–1034
(2004
). 32.
F.
Miwakeichi
, E.
Martınez-Montes
, P. A.
Valdés-Sosa
, N.
Nishiyama
, H.
Mizuhara
, and Y.
Yamaguchi
, “Decomposing EEG data into space–time–frequency components using parallel factor analysis
,” NeuroImage
22
, 1035
–1045
(2004
). 33.
C.
Beckmann
and S.
Smith
, “Tensorial extensions of independent component analysis for multisubject FMRI analysis
,” NeuroImage
25
, 294
–311
(2005
). 34.
E.
Acar. C. Aykut-Bingol
, H.
Bingol
, R.
Bro
, and B.
Yener
, “Multiway analysis of epilepsy tensors
,” Bioinformatics
23
, i10
–i18
(2007
). 35.
A.
Cichocki
, M.
De Vos
, L.
De Lathauwer
, B.
Vanrumste
, S.
Van Huffel
, and W.
Van Paesschen
, “Canonical decomposition of ictal scalp EEG and accurate source localisation: Principles and simulation study
,” Comput. Intell. Neurosci.
2007
, 058253
(2007
).36.
M.
De Vos
, A.
Vergult
, L.
De Lathauwer
, W.
De Clercq
, S.
Van Huffel
, P.
Dupont
, A.
Palmini
, and W.
Van Paesschen
, “Canonical decomposition of ictal scalp EEG reliably detects the seizure onset zone
,” NeuroImage
37
, 844
–854
(2007
). 37.
M.
Mørup
, L. K.
Hansen
, C. S.
Herrmann
, J.
Parnas
, and S. M.
Arnfred
, “Parallel factor analysis as an exploratory tool for wavelet transformed event-related EEG
,” NeuroImage
29
, 938
–947
(2006
). 38.
M.
Mørup
, L. K.
Hansen
, and S. M.
Arnfred
, “ERPWAVELAB: A toolbox for multi-channel analysis of time–frequency transformed event related potentials
,” J. Neurosci. Methods
161
, 361
–368
(2007
). 39.
A. H.
Williams
, T. H.
Kim
, F.
Wang
, S.
Vyas
, S. I.
Ryu
, K. V.
Shenoy
, M.
Schnitzer
, T. G.
Kolda
, and S.
Ganguli
, “Unsupervised discovery of demixed, low-dimensional neural dynamics across multiple timescales through tensor component analysis
,” Neuron
98
, 1099
–1115
(2018
). 40.
C. M.
Constantinople
, A. T.
Piet
, P.
Bibawi
, A.
Akrami
, C.
Kopec
, and C. D.
Brody
, “Lateral orbitofrontal cortex promotes trial-by-trial learning of risky, but not spatial, biases
,” eLife
8
, e49744
(2019
). 41.
J.
Xia
, T. D.
Marks
, M. J.
Goard
, and R.
Wessel
, “Diverse co-active neurons encode stimulus-driven and stimulus-independent variables
,” J. Neurophysiol.
124
(5), 1505–1517
(2020
).42.
Y.
Zhu
, J.
Liu
, K.
Mathiak
, T.
Ristaniemi
, and F.
Cong
, “Deriving electrophysiological brain network connectivity via tensor component analysis during freely listening to music
,” IEEE Trans. Neural Syst. Rehabil. Eng.
28
409
–418
(2020
). 43.
Y.
Zhu
, J.
Liu
, C.
Ye
, K.
Mathiak
, P.
Astikainen
, T.
Ristaniemi
, and F.
Cong
, “Discovering dynamic task-modulated functional networks with specific spectral modes using MEG
,” NeuroImage
218
, 116924
(2020
). 44.
J.
Xia
, T. D.
Marks
, M. J.
Goard
, and R.
Wessel
, “Stable representation of a naturalistic movie emerges from episodic activity with gain variability,” Version 1, Research Square; see (2021).45.
S.
Klus
, P.
Gelß
, S.
Peitz
, and C.
Schütte
, “Tensor-based dynamic mode decomposition
,” Nonlinearity
31
(7
), 3359
(2018
). 46.
P.
Gelß
, S.
Klus
, J.
Eiset
, and C.
Schütte
, “Multidimensional approximation of nonlinear dynamical systems
,” J. Comput. Nonlinear Dyn.
14
(6
), 061006
(2019
). 47.
B.
Lusch
, E. C.
Chi
, and J. N.
Kutz
, “Shape constrained tensor decompositions,” in 2019 IEEE International Conference on Data Science and Advanced Analytics (DSAA) (IEEE, 2019), pp. 287–297.48.
P. J.
Schmid
, “Dynamic mode decomposition of numerical and experimental data
,” J. Fluid Mech.
656
, 5
–28
(2010
). 49.
J. H.
Tu
, C. W.
Rowley
, D. M.
Luchtenburg
, S. L.
Brunton
, and J. N.
Kutz
, “On dynamic mode decomposition: Theory and applications
,” J. Comput. Dyn.
1
(2
), 391
–421
(2014
). 50.
K. K.
Chen
, J. H.
Tu
, and C. W.
Rowley
, “Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses
,” J. Nonlinear Sci.
22
(6
), 887
–915
(2012
). 51.
M. O.
Williams
, I. G.
Kevrekidis
, and C. W.
Rowley
, “A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition
,” J. Nonlinear Sci.
25
, 1307
(2015
). 52.
H.
Arbabi
and I.
Mezić
, “Ergodic theory, dynamic mode decomposition, and computation of spectral properties of the Koopman operator
,” SIAM J. Appl. Dyn. Syst.
16
(4
), 2096
–2126
(2017
). 53.
F.
Takens
, “Detecting strange attractors in turbulence,” in Dynamical Systems and Turbulence, Warwick 1980, Lecture Notes in Mathematics Vol. 898, edited by D. Rand and L. S. Young (Springer, Berlin, 1981).54.
M.
Kamb
, E.
Kaiser
, S. L.
Brunton
, and J. N.
Kutz
, “Time-delay observables for Koopman: Theory and applications
,” SIAM J. Appl. Dyn. Syst.
19
(2
), 886
–917
(2020
). 55.
F. L.
Hitchcock
, “The expression of a tensor or a polyadic as a sum of products
,” J. Math. Phys.
6
, 164
–189
(1927
). 56.
F. L.
Hitchcock
, “Multiple invariants and generalized rank of a p-way matrix or tensor
,” J. Math. Phys.
7
, 39
–79
(1928
). 57.
D.
Muti
and S.
Bourennane
, “Multidimensional filtering based on a tensor approach
,” Signal Process.
85
, 2338
–2353
(2005
). 58.
L.
De Lathauwer
and J.
Castaing
, “Tensor-based techniques for the blind separation of DS-CDMA signal
,” Signal Process.
87
, 322
–336
(2007
). 59.
L.
De Lathauwer
and A.
De Baynast
, “Blind deconvolution of DS-CDMA signals by means of decomposition in rank-(1, L, L terms
,” IEEE Trans. Signal Process.
56
, 1562
–1571
(2008
). 60.
L. R.
Tucker
, “Some mathematical notes on three-mode factor analysis
,” Psychometrika
31
, 279
–311
(1966
). 61.
C. J.
Appellof
and E. R.
Davidson
, “Strategies for analyzing data from video fluorometric monitoring of liquid chromatographic effluents
,” Anal. Chem.
53
, 2053
–2056
(1981
). 62.
S.
Leurgans
and R. T.
Ross
, “Multilinear models: Applications in spectroscopy
,” Stat. Sci.
7
, 289
–310
(1992
). 63.
R.
Henrion
, “Body diagonalization of core matrices in three-way principal components analysis: Theoretical bounds and simulation
,” J. Chemometrics
7
, 477
–494
(1993
). 64.
A. K.
Smilde
, Y.
Wang
, and B. R.
Kowalski
, “Theory of medium-rank second-order calibration with restricted-Tucker models
,” J. Chemometrics
8
, 21
–36
(1994
). 65.
H. A. L.
Kiers
, “A three-step algorithm for CANDECOMP/PARAFAC analysis of large data sets with multicollinearity
,” J. Chemometrics
12
, 155
–171
(1998
). 66.
B. M.
Wise
, N. B.
Gallagher
, and E. B.
Martin
, “Application of PARAFAC2 to fault detection and diagnosis in semiconductor etch
,” J. Chemometrics
15
, 285
–298
(2001
). 67.
R.
Bro
, C. A.
Andersson
, and H. A. L.
Kiers
, “PARAFAC2—Part II. Modeling chromotographic data with retention time shifts
,” J. Chemometrics
13
, 295
–309
(1999
). 68.
C. M.
Andersson
and R.
Bro
, “Practical aspects of PARAFAC modeling fluorescence excitation-emission data
,” J. Chemometrics
17
, 200
–215
(2003
). 69.
A.
Smilde
, R.
Bro
, and P.
Geladi
, Multi-Way Analysis: Applications in the Chemical Sciences
(Wiley
, Chicester, England
, 2004
).70.
J. B.
Kruskal
, “Rank, decomposition, and uniqueness for 3-way and N-way arrays,” in Multiway Data Analysis, edited by R. Coppi and S. Bolasco (North-Holland, Amsterdamn, 1989), pp. 7–18.71.
J. M. F.
Ten Berge
and H. A. L.
Kiers
, “Simplicity of core arrays in three-way principal component analysis and the typical rank of pq2 arrays
,” Linear Algebra Appl.
294
, 169
–179
(1999
). 72.
J. M. F.
Ten Berge
, “The typical rank of tall three-way arrays
,” Psychometrika
65
, 525
–532
(2000
). 74.
S.
Bagheri
, “Koopman-mode decomposition of the cylinder wake
,” J. Fluid Mech.
726
, 596
–623
(2013
). 75.
N.
Takeishi
, Y.
Kawahara
, and T.
Yairi
, “Learning Koopman invariant subspace for dynamic mode decomposition,” in Advances in Neural Information Processing Systems 30, NIPS 17 (Neural Information Processing Systems Foundation, 2017), pp. 1130–1140.76.
Q.
Li
, F.
Dietrich
, E. M.
Bollt
, and I. G.
Kevrekidis
, Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator
,” Chaos
, 27
, 103111
(2017
). 77.
B.
Lusch
, J. N.
Kutz
, and S. L.
Brunton
, “Deep learning for universal linear embeddings of nonlinear dynamics
,” Nat. Commun.
9
, 4950
(2018
). 78.
S. E.
Otto
and C. W.
Rowley
, “Linearly recurrent autoencoder networks for learning dynamics
,” SIAM J. Appl. Dyn. Syst.
18
(1
), 558
–593
(2019
). 79.
E.
Yeung
, S.
Kundu
, and N.
Hodas
, “Learning deep neural network representations for Koopman operators of nonlinear dynamical systems,” in American Control Conference 2019 (IEEE, 2019), pp. 4832–4839.80.
C.
Battaglino
, G.
Ballard
, and T. G.
Kolda
, “A practical randomized CP tensor decomposition
,” SIAM J. Matrix Anal. Appl.
39
(2
), 876
–901
(2018
). 81.
N. B.
Erichson
, K.
Manohar
, S. L.
Brunton
, and J. N.
Kutz
, “Randomized CP tensor decomposition
,” Mach. Learn.: Sci. Technol.
1
, 025012
(2020
). 82.
P.
Paatero
and U.
Tapper
, “Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values
,” Environmetrics
5
(2
), 111
–126
(1994
). 83.
D. D.
Lee
and H. S.
Seung
, “Learning the parts of objects by non-negative matrix factorization
,” Nature
401
, 788
–791
(1999
). 84.
Y.
Qi
, P.
Comon
, and L. H.
Lim
, “Uniqueness of nonnegative tensor approximations
,” IEEE Trans. Inf. Theor.
62
, 2170
–2183
(2016
). 85.
S.
Maćešić
, N.
Črnjarić-Žic
, and I.
Mezić
, “Koopman operator family spectrum for nonautonomous systems
,” SIAM J. Appl. Dyn. Syst.
17
(4
), 2478
–2515
(2018
). 86.
F.
Dietrich
, T. N.
Thiem
, and I. G.
Kevrekidis
, “On the Koopman operator of algorithms
,” SIAM J. Appl. Dyn. Syst.
19
(2
), 860
–885
(2020
). 87.
T.
Sahai
, “Dynamical systems theory and algorithms for NP-hard problems,” in Advances in Dynamics, Optimization and Computation. SON 2020. Studies in Systems, Decision and Control, edited by O. Junge, O. Schütze, G. Froyland, S. Ober-Blöbaum, and K. Padberg-Gehle (Springer, Cham, 2020), Vol. 304.© 2021 Author(s). Published under an exclusive license by AIP Publishing.
2021
Author(s)
You do not currently have access to this content.
