The effect of hyperparameter selection in deep learning (DL) models for fluid dynamics remains an open question in the current scientific literature. Many authors report results using deep learning models. However, better insight is required to assess deep learning models' behavior, particularly for complex datasets such as turbulent signals. This study presents a meticulous investigation of the long short-term memory (LSTM) hyperparameters, focusing specifically on applications involving predicting signals in shock turbulent boundary layer interaction. Unlike conventional methodologies that utilize automated optimization techniques, this research explores the intricacies and impact of manual adjustments to the deep learning model. The investigation includes the number of layers, neurons per layer, learning rate, dropout rate, and batch size to investigate their impact on the model's predictive accuracy and computational efficiency. The paper details the iterative tuning process through a series of experimental setups, highlighting how each parameter adjustment contributes to a deeper understanding of complex, time-series data. The findings emphasize the effectiveness of precise manual tuning in achieving superior model performance, providing valuable insights to researchers and practitioners who seek to leverage long short-term memory networks for intricate temporal data analysis. The optimization not only refines the predictability of the long short-term memory in specific contexts but also serves as a guide for similar manual tuning in other specialized domains, thereby informing the development of more effective deep learning models.

1.
N.
Pavitha
and
S.
Sugave
, “
Optimizing machine learning models: An adaptive hyperparameter tuning approach
,”
Int. J. Intell. Syst. Appl. Eng.
11
,
344
354
(
2023
).
2.
M.
Zuccotto
,
A.
Castellini
,
D. L.
Torre
,
L.
Mola
, and
A.
Farinelli
, “
Reinforcement learning applications in environmental sustainability: A review
,”
Artif. Intell. Rev.
57
,
88
(
2024
).
3.
M.
Yang
,
M. K.
Lim
,
Y.
Qu
,
X.
Li
, and
D.
Ni
, “
Deep neural networks with l1 and l2 regularization for high dimensional corporate credit risk prediction
,”
Expert Syst. Appl.
213
,
118873
(
2023
).
4.
G.
Van Houdt
,
C.
Mosquera
, and
G.
Nápoles
, “
A review on the long short-term memory model
,”
Artif. Intell. Rev.
53
,
5929
5955
(
2020
).
5.
T.
Makarovskikh
and
M.
Abotaleb
, “
Hyper-parameter tuning for long short-term memory (lstm) algorithm to forecast a disease spreading
,” in
2022 VIII International Conference on Information Technology and Nanotechnology (ITNT)
(IEEE,
2022
), pp.
1
6
.
6.
D.
Drikakis
,
I. W.
Kokkinakis
,
P.
Tirchas
, and
S. M.
Spottswood
, “
Physical consistency and invariance in machine learning of turbulent signals
,”
Phys. Fluids
36
,
016130
(
2024a
).
7.
P.
Shetty
,
R.
Udhayakumar
,
A.
Patil
,
M.
Manwal
,
P. S.
Vadar
, and
N.
Pavitha
, “
Application of natural language processing (NLP) in machine learning
,” in
2023 3rd International Conference on Advancement in Electronics & Communication Engineering (AECE)
(IEEE,
2023
), pp.
949
957
.
8.
N.
Pavitha
and
S.
Sugave
, “
Ensemble approach with hyperparameter tuning for credit worthiness prediction
,” in
2022 IEEE 3rd Global Conference for Advancement in Technology (GCAT)
(IEEE,
2022
), pp.
1
5
.
9.
D.
Drikakis
,
I. W.
Kokkinakis
, and
P.
Tirchas
, “
Sparsity and mixing effects in deep learning predictions of temperature and humidity
,”
Phys. Fluids
36
,
085191
(
2024
).
10.
K.
Poulinakis
,
D.
Drikakis
,
I. W.
Kokkinakis
,
S. M.
Spottswood
, and
T.
Dbouk
, “
Lstm reconstruction of turbulent pressure fluctuation signals
,”
Computation
12
,
4
(
2024
).
11.
K.
Poulinakis
,
D.
Drikakis
,
I. W.
Kokkinakis
, and
S. M.
Spottswood
, “
Deep learning reconstruction of pressure fluctuations in supersonic shock–boundary layer interaction
,”
Phys. Fluids
35
,
076117
(
2023a
).
12.
N.
Chen
, “
Exploring the development and application of LSTM variants
,”
ACE
53
,
103
107
(
2024
).
13.
H.
Yadav
and
A.
Thakkar
, “
NOA-LSTM: An efficient LSTM cell architecture for time series forecasting
,”
Expert Syst. Appl.
238
,
122333
(
2024
).
14.
L.
Liao
,
H.
Li
,
W.
Shang
, and
L.
Ma
, “
An empirical study of the impact of hyperparameter tuning and model optimization on the performance properties of deep neural networks
,”
ACM Trans. Software Eng. Methodol.
31
,
1
(
2022
).
15.
S. J.
Quan
, “
Comparing hyperparameter tuning methods in machine learning based urban building energy modeling: A study in Chicago
,”
Energy Build.
317
,
114353
(
2024
).
16.
S.
Wang
,
C.
Ma
,
Y.
Xu
,
J.
Wang
, and
W.
Wu
, “
A hyperparameter optimization algorithm for the lstm temperature prediction model in data center
,”
Sci. Program.
2022
(
1
),
6519909
.
17.
D.
Wen
,
S.
Zheng
,
J.
Chen
,
Z.
Zheng
,
C.
Ding
, and
L.
Zhang
, “
Hyperparameter-optimization-inspired long short-term memory network for air quality grade prediction
,”
Information
14
,
243
(
2023
).
18.
J. O.
Afape
,
A. A.
Willoughby
,
M. E.
Sanyaolu
,
O. O.
Obiyemi
,
K.
Moloi
,
J. O.
Jooda
, and
O. F.
Dairo
, “
Improving millimetre-wave path loss estimation using automated hyperparameter-tuned stacking ensemble regression machine learning
,”
Results Eng.
22
,
102289
(
2024
).
19.
D.
Belete
and
M. D.
Huchaiah
, “
Grid search in hyperparameter optimization of machine learning models for prediction of HIV/AIDS test results
,”
Int. J. Comput. Appl.
44
,
875
812
(
2022
).
20.
J.
Bergstra
and
Y.
Bengio
, “
Random search for hyper-parameter optimization
,”
J. Mach. Learn. Res.
13
,
281
305
(
2012a
).
21.
P.
Eleftheriadis
,
S.
Leva
, and
E.
Ogliari
, “
Bayesian hyperparameter optimization of stacked bidirectional long short-term memory neural network for the state of charge estimation
,”
Sustainable Energy Grids Networks
36
,
101160
(
2023
).
22.
X.
Du
,
H.
Xu
, and
F.
Zhu
, “
Understanding the effect of hyperparameter optimization on machine learning models for structure design problems
,”
Comput-Aided Des.
135
,
103013
(
2021
).
23.
L.
Yang
and
A.
Shami
, “
On hyperparameter optimization of machine learning algorithms: Theory and practice
,”
Neurocomputing
415
,
295
316
(
2020
).
24.
R.
Semola
,
J.
Hurtado
,
V.
Lomonaco
, and
D.
Bacciu
, “
Adaptive hyperparameter optimization for continual learning scenarios
,” arXiv:2403.07015 (
2024
).
25.
I. W.
Kokkinakis
,
D.
Drikakis
,
S. M.
Spottswood
,
K. R.
Brouwer
, and
Z. B.
Riley
, “
High-speed shock–boundary-layer interaction over deformed surfaces
,”
Phys. Fluids
35
,
106109
(
2023
).
26.
I. W.
Kokkinakis
,
D.
Drikakis
,
S.
Michael Spottswood
,
K. R.
Brouwer
, and
Z. B.
Riley
, “
Aeroacoustic loading of impinging supersonic boundary-layer interaction on statically deformed surfaces
,”
AIAA J.
62
,
2520
2537
(
2024
).
27.
F.
Sofos
,
D.
Drikakis
,
I. W.
Kokkinakis
, and
S. M.
Spottswood
, “
Convolutional neural networks for compressible turbulent flow reconstruction
,”
Phys. Fluids
35
,
116120
(
2023
).
28.
D.
Drikakis
,
I. W.
Kokkinakis
,
D.
Fung
, and
S. M.
Spottswood
, “
Generalizability of transformer-based deep learning for multidimensional turbulent flow data
,”
Phys. Fluids
36
,
026102
(
2024c
).
29.
F.
Sofos
,
D.
Drikakis
,
I. W.
Kokkinakis
, and
S. M.
Spottswood
, “
A deep learning super-resolution model for turbulent image upscaling and its application to shock wave–boundary layer interaction
,”
Phys. Fluids
36
,
025117
(
2024
).
30.
D.
Drikakis
,
I. W.
Kokkinakis
,
D.
Fung
, and
S. M.
Spottswood
, “
Self-supervised transformers for turbulent flow time series
,”
Phys. Fluids
36
,
065113
(
2024d
).
31.
M.
Klein
,
A.
Sadiki
, and
J.
Janicka
, “
A digital filter based generation of inflow data for spatially developing direct numerical or large eddy simulations
,”
J. Comput. Phys.
186
,
652
665
(
2003
).
32.
E.
Touber
and
N. D.
Sandham
, “
Large-eddy simulation of low-frequency unsteadiness in a turbulent shock-induced separation bubble
,”
Theor. Comput. Fluid Dyn.
23
,
79
107
(
2009
).
33.
I.
Kokkinakis
,
D.
Drikakis
,
K.
Ritos
, and
S. M.
Spottswood
, “
Direct numerical simulation of supersonic flow and acoustics over a compression ramp
,”
Phys. Fluids
32
,
066107
(
2020
).
34.
B. J. R.
Thornber
and
D.
Drikakis
, “
Numerical dissipation of upwind schemes in low Mach flow
,”
Int. J. Numer. Methods Fluids
56
,
1535
1541
(
2008
).
35.
A.
Mosedale
and
D.
Drikakis
, “
Assessment of very high order of accuracy in implicit les models
,”
J. Fluids Eng.
129
,
1497
1503
(
2007
).
36.
F.
Grinstein
,
L.
Margolin
, and
W.
Rider
, “
Implicit large eddy simulation: Computing turbulent fluid dynamics
,” in
Implicit Large Eddy Simulation: Computing Turbulent Fluid Dynamics
(Cambridge University Press,
2007
).
37.
D.
Drikakis
,
C.
Fureby
,
F.
Grinstein
, and
D.
Youngs
, “
Simulation of transition and turbulence decay in the Taylor–Green vortex
,”
J. Turbul.
8
,
N20
(
2007
).
38.
D.
Drikakis
,
K.
Ritos
,
S. M.
Spottswood
, and
Z. B.
Riley
, “
Flow transition to turbulence and induced acoustics at Mach 6
,”
Phys. Fluids
33
,
076112
(
2021
).
39.
D. S.
Balsara
and
C.-W.
Shu
, “
Monotonicity preserving weighted essentially non-oscillatory schemes with increasingly high order of accuracy
,”
J. Comput. Phys.
160
,
405
452
(
2000
).
40.
E. F.
Toro
,
M.
Spruce
, and
W.
Speares
, “
Restoration of the contact surface in the HLL-Riemann solver
,”
Shock Waves
4
,
25
34
(
1994
).
41.
R.
Spiteri
and
S.
Ruuth
, “
A new class of optimal high-order strong-stability-preserving time discretization methods
,”
SIAM J. Numer. Anal.
40
,
469
491
(
2002
).
42.
I.
Kokkinakis
and
D.
Drikakis
, “
Implicit large eddy simulation of weakly-compressible turbulent channel flow
,”
Comput. Methods Appl. Mech. Eng.
287
,
229
261
(
2015
).
43.
K.
Poulinakis
,
D.
Drikakis
,
I. W.
Kokkinakis
, and
S. M.
Spottswood
, “
Machine-learning methods on noisy and sparse data
,”
Mathematics
11
,
236
(
2023b
).
44.
N.
Reimers
and
I.
Gurevych
, “
Optimal hyperparameters for deep LSTM-networks for sequence labeling tasks
,” arXiv:1707.06799 (
2017
).
45.
P.
Liashchynskyi
and
P.
Liashchynskyi
, “
Grid search, random search, genetic algorithm: A big comparison for NAS
,” arXiv:1912.06059 (
2019
).
46.
J.
Wu
,
X.-Y.
Chen
,
H.
Zhang
,
L.-D.
Xiong
,
H.
Lei
, and
S.-H.
Deng
, “
Hyperparameter optimization for machine learning models based on bayesian optimization
,”
J. Electron. Sci. Technol.
17
,
26
40
(
2019
).
You do not currently have access to this content.