In this Communication, we demonstrate that a deep artificial neural network based on a transformer architecture with self-attention layers can predict the long-time population dynamics of a quantum system coupled to a dissipative environment provided that the short-time population dynamics of the system is known. The transformer neural network model developed in this work predicts the long-time dynamics of spin-boson model efficiently and very accurately across different regimes, from weak system–bath coupling to strong coupling non-Markovian regimes. Our model is more accurate than classical forecasting models, such as recurrent neural networks, and is comparable to the state-of-the-art models for simulating the dynamics of quantum dissipative systems based on kernel ridge regression.
REFERENCES
1.
2.
H.-P.
Breuer
and F.
Petruccione
, The Theory of Open Quantum Systems
(Oxford University Press
, Oxford
, 2002
).3.
Y.
Tanimura
and R.
Kubo
, “Two-time correlation functions of a system coupled to a heat bath with a Gaussian-Markoffian interaction
,” J. Phys. Soc. Jpn.
58
, 1199
–1206
(1989
).4.
Y.
Tanimura
, “Numerically ‘exact’ approach to open quantum dynamics: The hierarchical equations of motion (HEOM)
,” J. Chem. Phys.
153
, 020901
(2020
).5.
H.
Wang
and M.
Thoss
, “Multilayer formulation of the multiconfiguration time-dependent Hartree theory
,” J. Chem. Phys.
119
, 1289
–1299
(2003
).6.
H.-D.
Meyer
, U.
Manthe
, and L. S.
Cederbaum
, “The multi-configurational time-dependent Hartree approach
,” Chem. Phys. Lett.
165
, 73
–78
(1990
).7.
S.
Kundu
and N.
Makri
, “PathSum: A C++ and Fortran suite of fully quantum mechanical real-time path integral methods for (multi-)system + bath dynamics
,” J. Chem. Phys.
158
, 224801
(2023
).8.
D. E.
Makarov
and N.
Makri
, “Path integrals for dissipative systems by tensor multiplication. Condensed phase quantum dynamics for arbitrarily long time
,” Chem. Phys. Lett.
221
, 482
–491
(1994
).9.
B.
Luo
, J.
Ye
, C.
Guan
, and Y.
Zhao
, “Validity of time-dependent trial states for the Holstein polaron
,” Phys. Chem. Chem. Phys.
12
, 15073
–15084
(2010
).10.
J.
Ren
, Z.
Shuai
, and G.
Kin-Lic Chan
, “Time-dependent density matrix renormalization group algorithms for nearly exact absorption and fluorescence spectra of molecular aggregates at both zero and finite temperature
,” J. Chem. Theory Comput.
14
, 5027
–5039
(2018
).11.
S. M.
Greene
and V. S.
Batista
, “Tensor-train split-operator Fourier transform (TT-soft) method: Multidimensional nonadiabatic quantum dynamics
,” J. Chem. Theory Comput.
13
, 4034
–4042
(2017
).12.
Y.-A.
Yan
and J.
Shao
, “Stochastic description of quantum Brownian dynamics
,” Front. Phys.
11
, 110309
(2016
).13.
C.-Y.
Hsieh
and J.
Cao
, “A unified stochastic formulation of dissipative quantum dynamics. I. Generalized hierarchical equations
,” J. Chem. Phys.
148
, 014103
(2018
).14.
L.
Han
, A.
Ullah
, Y.-A.
Yan
, X.
Zheng
, Y.
Yan
, and V.
Chernyak
, “Stochastic equation of motion approach to fermionic dissipative dynamics. I. Formalism
,” J. Chem. Phys.
152
, 204105
(2020
).15.
A.
Ullah
, L.
Han
, Y.-A.
Yan
, X.
Zheng
, Y.
Yan
, and V.
Chernyak
, “Stochastic equation of motion approach to fermionic dissipative dynamics. II. Numerical implementation
,” J. Chem. Phys.
152
, 204106
(2020
).16.
D.
Brian
and X.
Sun
, “Generalized quantum master equation: A tutorial review and recent advances
,” Chin. J. Chem. Phys.
34
, 497
–524
(2021
).17.
S.
Nakajima
, “On quantum theory of transport phenomena: Steady diffusion
,” Prog. Theor. Phys.
20
, 948
–959
(1958
).18.
R.
Zwanzig
, “Ensemble method in the theory of irreversibility
,” J. Chem. Phys.
33
, 1338
–1341
(1960
).19.
Q.
Shi
and E.
Geva
, “A new approach to calculating the memory kernel of the generalized quantum master equation for an arbitrary system–bath coupling
,” J. Chem. Phys.
119
, 12063
–12076
(2003
).20.
E.
Mulvihill
and E.
Geva
, “A road map to various pathways for calculating the memory kernel of the generalized quantum master equation
,” J. Phys. Chem. B
125
, 9834
–9852
(2021
).21.
A.
Kelly
and T. E.
Markland
, “Efficient and accurate surface hopping for long time nonadiabatic quantum dynamics
,” J. Chem. Phys.
139
, 014104
(2013
).22.
J.
Cerrillo
and J.
Cao
, “Non-Markovian dynamical maps: Numerical processing of open quantum trajectories
,” Phys. Rev. Lett.
112
, 110401
(2014
).23.
A. A.
Kananenka
, C.-Y.
Hsieh
, J.
Cao
, and E.
Geva
, “Accurate long-time mixed quantum-classical Liouville dynamics via the transfer tensor method
,” J. Phys. Chem. Lett.
7
, 4809
–4814
(2016
).24.
M.
Buser
, J.
Cerrillo
, G.
Schaller
, and J.
Cao
, “Initial system-environment correlations via the transfer-tensor method
,” Phys. Rev. A
96
, 062122
(2017
).25.
A.
Gelzinis
, E.
Rybakovas
, and L.
Valkunas
, “Applicability of transfer tensor method for open quantum system dynamics
,” J. Chem. Phys.
147
, 234108
(2017
).26.
Y.-Q.
Chen
, K.-L.
Ma
, Y.-C.
Zheng
, J.
Allcock
, S.
Zhang
, and C.-Y.
Hsieh
, “Non-Markovian noise characterization with the transfer tensor method
,” Phys. Rev. Appl.
13
, 034045
(2020
).27.
L. E.
Herrera Rodríguez
and A. A.
Kananenka
, “Convolutional neural networks for long time dissipative quantum dynamics
,” J. Phys. Chem. Lett.
12
, 2476
–2483
(2021
).28.
A.
Ullah
and P. O.
Dral
, “Speeding up quantum dissipative dynamics of open systems with kernel methods
,” New J. Phys.
23
, 113019
(2021
).29.
A.
Ullah
and P. O.
Dral
, “Predicting the future of excitation energy transfer in light-harvesting complex with artificial intelligence-based quantum dynamics
,” Nat. Commun.
13
, 1930
(2022
).30.
A.
Ullah
and P. O.
Dral
, “One-shot trajectory learning of open quantum systems dynamics
,” J. Phys. Chem. Lett.
13
, 6037
–6041
(2022
).31.
A.
Ullah
, L. E.
Herrera Rodríguez
, P. O.
Dral
, and A. A.
Kananenka
, “QD3SET-1: A database with quantum dissipative dynamics datasets
,” Front. Phys.
11
, 1223973
(2023
).32.
A.
Ullah
and P. O.
Dral
, “MLQD: A package for machine learning-based quantum dissipative dynamics
,” Comput. Phys. Commun.
294
, 108940
(2024
).33.
K.
Lin
, J.
Peng
, F. L.
Gu
, and Z.
Lan
, “Simulation of open quantum dynamics with bootstrap-based long short-term memory recurrent neural network
,” J. Phys. Chem. Lett.
12
, 10225
–10234
(2021
).34.
D.
Wu
, Z.
Hu
, J.
Li
, and X.
Sun
, “Forecasting nonadiabatic dynamics using hybrid convolutional neural network/long short-term memory network
,” J. Chem. Phys.
155
, 224104
(2021
).35.
A. V.
Akimov
, “Extending the time scales of nonadiabatic molecular dynamics via machine learning in the time domain
,” J. Phys. Chem. Lett.
12
, 12119
–12128
(2021
).36.
L.
Zhang
, S. V.
Pios
, M.
Martyka
, F.
Ge
, Y.-F.
Hou
, Y.
Chen
, L.
Chen
, J.
Jankowska
, M.
Barbatti
, and P. O.
Dral
, “MLatom software ecosystem for surface hopping dynamics in Python with quantum mechanical and machine learning methods
,” J. Chem. Theory Comput.
20
, 5043
–5057
(2024
).37.
K.
Lin
, J.
Peng
, C.
Xu
, F. L.
Gu
, and Z.
Lan
, “Automatic evolution of machine-learning-based quantum dynamics with uncertainty analysis
,” J. Chem. Theory Comput.
18
, 5837
–5855
(2022
).38.
D.
Luo
, Z.
Chen
, J.
Carrasquilla
, and B. K.
Clark
, “Autoregressive neural network for simulating open quantum systems via a probabilistic formulation
,” Phys. Rev. Lett.
128
, 090501
(2022
).39.
L. E. H.
Rodríguez
, A.
Ullah
, K. J. R.
Espinosa
, P. O.
Dral
, and A. A.
Kananenka
, “A comparative study of different machine learning methods for dissipative quantum dynamics
,” Mach. Learn.: Sci. Technol.
3
, 045016
(2022
).40.
A.
Ullah
, Y.
Huang
, M.
Yang
, and P. O.
Dral
, “Physics-informed neural networks and beyond: Enforcing physical constraints in quantum dissipative dynamics
,” Digital Discovery
3
, 2052
(2024
).41.
A.
Vaswani
, N.
Shazeer
, N.
Parmar
, J.
Uszkoreit
, L.
Jones
, A. N.
Gomez
, Ł.
Kaiser
, and I.
Polosukhin
, “Attention is all you need
,” in Advances in Neural Information Processing Systems
(Curran Associates, Inc.
, 2017
), Vol. 30
.42.
A.
Dosovitskiy
, L.
Beyer
, A.
Kolesnikov
, D.
Weissenborn
, X.
Zhai
, T.
Unterthiner
, M.
Dehghani
, M.
Minderer
, G.
Heigold
, S.
Gelly
et al, “An image is worth 16 × 16 words: Transformers for image recognition at scale
,” arXiv:2010.11929 (2020
).43.
A.
Radford
, J. W.
Kim
, T.
Xu
, G.
Brockman
, C.
McLeavey
, and I.
Sutskever
, “Robust speech recognition via large-scale weak supervision
,” in International Conference on Machine Learning
(PMLR
, 2023
), pp. 28492
–28518
.44.
T.
Wolf
, L.
Debut
, V.
Sanh
, J.
Chaumond
, C.
Delangue
, A.
Moi
, P.
Cistac
, T.
Rault
, R.
Louf
, M.
Funtowicz
et al, “Transformers: State-of-the-art natural language processing
,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
(Association for Computational Linguistics
, 2020
), pp. 38
–45
.45.
C.
Donoso-Oliva
, I.
Becker
, P.
Protopapas
, G.
Cabrera-Vives
, M.
Vishnu
, and H.
Vardhan
, “ASTROMER: A transformer-based embedding for the representation of light curves
,” Astron. Astrophys.
670
, A54
(2023
).46.
A. J.
Leggett
, S.
Chakravarty
, A. T.
Dorsey
, M. P.
Fisher
, A.
Garg
, and W.
Zwerger
, “Dynamics of the dissipative two-state system
,” Rev. Mod. Phys.
59
, 1
(1987
).47.
Y.
Makhlin
, G.
Schön
, and A.
Shnirman
, “Quantum-state engineering with Josephson-junction devices
,” Rev. Mod. Phys.
73
, 357
(2001
).48.
A.
Winter
, H.
Rieger
, M.
Vojta
, and R.
Bulla
, “Quantum phase transition in the sub-ohmic spin-boson model: Quantum Monte Carlo study with a continuous imaginary time cluster algorithm
,” Phys. Rev. Lett.
102
, 030601
(2009
).49.
A.
Alvermann
and H.
Fehske
, “Sparse polynomial space approach to dissipative quantum systems: Application to the sub-ohmic spin-boson model
,” Phys. Rev. Lett.
102
, 150601
(2009
).50.
A.
Garg
, J. N.
Onuchic
, and V.
Ambegaokar
, “Effect of friction on electron transfer in biomolecules
,” J. Chem. Phys.
83
, 4491
–4503
(1985
).51.
H.
Wang
, X.
Song
, D.
Chandler
, and W. H.
Miller
, “Semiclassical study of electronically nonadiabatic dynamics in the condensed-phase: Spin-boson problem with Debye spectral density
,” J. Chem. Phys.
110
, 4828
–4840
(1999
).52.
Z.
Niu
, G.
Zhong
, and H.
Yu
, “A review on the attention mechanism of deep learning
,” Neurocomputing
452
, 48
–62
(2021
).53.
M.
Corbetta
and G. L.
Shulman
, “Control of goal-directed and stimulus-driven attention in the brain
,” Nat. Rev. Neurosci.
3
, 201
–215
(2002
).54.
D.
Bahdanau
, K.
Cho
, and Y.
Bengio
, “Neural machine translation by jointly learning to align and translate
,” arXiv:1409.0473 (2014
).55.
M. M.
Aliabadi
, H.
Emami
, M.
Dong
, and Y.
Huang
, “Attention-based recurrent neural network for multistep-ahead prediction of process performance
,” Comput. Chem. Eng.
140
, 106931
(2020
).56.
D.
Moreno-Cartagena
, G.
Cabrera-Vives
, P.
Protopapas
, C.
Donoso-Oliva
, M.
Pérez-Carrasco
, and M.
Cádiz-Leyton
, “Positional encodings for light curve transformers: Playing with positions and attention
,” arXiv:2308.06404 (2023
).57.
J. R.
Johansson
, P. D.
Nation
, and F.
Nori
, “QuTiP: An open-source Python framework for the dynamics of open quantum systems
,” Comput. Phys. Commun.
183
, 1760
–1772
(2012
).58.
59.
M.
Abadi
, A.
Agarwal
, P.
Barham
, E.
Brevdo
, Z.
Chen
, C.
Citro
, G. S.
Corrado
, A.
Davis
, J.
Dean
, M.
Devin
, S.
Ghemawat
, I.
Goodfellow
, A.
Harp
, G.
Irving
, M.
Isard
, Y.
Jia
, R.
Jozefowicz
, L.
Kaiser
, M.
Kudlur
, J.
Levenberg
, D.
Mané
, R.
Monga
, S.
Moore
, D.
Murray
, C.
Olah
, M.
Schuster
, J.
Shlens
, B.
Steiner
, I.
Sutskever
, K.
Talwar
, P.
Tucker
, V.
Vanhoucke
, V.
Vasudevan
, F.
Viégas
, O.
Vinyals
, P.
Warden
, M.
Wattenberg
, M.
Wicke
, Y.
Yu
, and X.
Zheng
, TensorFlow: Large-scale machine learning on heterogeneous systems
, 2015
, http://tensorflow.org.60.
61.
T.
O’Malley
, E.
Bursztein
, J.
Long
, F.
Chollet
, H.
Jin
, L.
Invernizzi
et al, Kerastuner
, 2019
, https://github.com/keras-team/keras-tuner.© 2024 Author(s). Published under an exclusive license by AIP Publishing.
2024
Author(s)
You do not currently have access to this content.
