Achieving precise control over self-propelled undulatory swimmers requires a deep understanding of their intricate dynamics. This paper presents a method for addressing optimal control problems in this context by leveraging surrogate models. We develop a Navier–Stokes solver using a volume penalization method to simulate the fluid–structure interaction inherent in swimming dynamics. An offline phase generates training data through open-loop simulations across a defined range of control inputs, enabling the training of a surrogate model. This model significantly reduces computational costs, particularly in optimization and control contexts. Utilizing these surrogate models, we compute control strategies to address two key challenges: precise velocity tracking and optimizing swimmer efficiency. First, we employ model predictive control to enable velocity tracking against a reference signal, allowing swift adjustments of the swimmer's frequency and amplitude. Second, we tackle the minimization of the swimmer's cost of transport, resulting in a solution akin to a burst-and-coast strategy. Despite achieving energy performance comparable to continuous swimming cases, mismatches between the surrogate model and the high fidelity simulation significantly impact the quality of the obtained solution. This work sheds light on the potential of surrogate models in optimizing self-propelled swimming behavior and underscores the importance of addressing model mismatches for more accurate control strategies in the future.
REFERENCES
1.
F.
Fish
and
G.
Lauder
, “
Passive and active flow control by swimming fishes and mammals
,” Annu. Rev. Fluid Mech.
38
, 193
–224
(2006
).2.
S.
Scott Collis
,
R. D.
Joslin
,
A.
Seifert
, and
V.
Theofilis
, “
Issues in active flow control: Theory, control, simulation, and experiment
,” Prog. Aerosp. Sci.
40
, 237
–289
(2004
).3.
L. N.
Cattafesta
and
M.
Sheplak
, “
Actuators for active flow control
,” Annu. Rev. Fluid Mech.
43
, 247
–272
(2011
).4.
S.
Gu
,
T.
Lillicrap
,
I.
Sutskever
, and
S.
Levine
, “
Continuous deep q-learning with model-based acceleration
,” in Proceedings of The 33rd International Conference on Machine Learning
, Proceedings of Machine Learning Research, Vol.
48
, edited by
M. F.
Balcan
and
K. Q.
Weinberger
(
PMLR
,
New York, NY
, 2016
), pp. 2829
–2838
.5.
Y.
Chebotar
,
K.
Hausman
,
M.
Zhang
,
G.
Sukhatme
,
S.
Schaal
, and
S.
Levine
, “
Combining model-based and model-free updates for trajectory-centric reinforcement learning
,” in Proceedings of the 34th International Conference on Machine Learning
, Proceedings of Machine Learning Research, Vol.
70
, edited by
D.
Precup
and
Y. W.
Teh
(
PMLR
, 2017
), pp. 703
–711
.6.
A.
Nagabandi
,
G.
Kahn
,
R. S.
Fearing
, and
S.
Levine
, “
Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning
,” in 2018 IEEE International Conference on Robotics and Automation (ICRA)
(
IEEE
, 2018
).7.
N.
Gautier
,
J.-L.
Aider
,
T.
Duriez
,
B. R.
Noack
,
M.
Segond
, and
M.
Abel
, “
Closed-loop separation control using machine learning
,” J. Fluid Mech.
770
, 442
–457
(2015
).8.
G.
Novati
,
S.
Verma
,
D.
Alexeev
,
D.
Rossinelli
,
W. M.
van Rees
, and
P.
Koumoutsakos
, “
Synchronisation through learning for two self-propelled swimmers
,” Bioinspiration Biomimetics
12
, 036001
(2017
).9.
J.
Rabault
,
F.
Ren
,
W.
Zhang
,
H.
Tang
, and
H.
Xu
, “
Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization
,” J. Hydrodyn.
32
, 234
–246
(2020
).10.
F.
Ren
,
J.
Rabault
, and
H.
Tang
, “
Applying deep reinforcement learning to active flow control in weakly turbulent conditions
,” Phys. Fluids
33
, 037121
(2021
).11.
A. G.
Nair
,
C.-A.
Yeh
,
E.
Kaiser
,
B. R.
Noack
,
S. L.
Brunton
, and
K.
Taira
, “
Cluster-based feedback control of turbulent post-stall separated flows
,” J. Fluid Mech.
875
, 345
–375
(2019
).12.
X.
Wang
,
N.
Deng
,
G. Y.
Cornejo Maceda
, and
B. R.
Noack
, “
Cluster-based control for net drag reduction of the fluidic pinball
,” Phys. Fluids
35
, 023601
(2023
).13.
M.
Bergmann
and
L.
Cordier
, “
Optimal control of the cylinder wake in the laminar regime by trust-region methods and pod reduced-order models
,” J. Comput. Phys.
227
, 7813
–7840
(2008
).14.
A.
Barbagallo
,
D.
Sipp
, and
P. J.
Schmid
, “
Closed-loop control of an open cavity flow using reduced-order models
,” J. Fluid Mech.
641
, 1
–50
(2009
).15.
M. J.
Lighthill
, “
Large-amplitude elongated-body theory of fish locomotion
,” Proc. R. Soc. London, Ser. B: Biol. Sci.
179
, 125
–138
(1971
).16.
M.
Porez
,
F.
Boyer
, and
A.
Ijspeert
, “
Improved lighthill fish swimming model for bio-inspired robots: Modeling, computational aspects and experimental comparisons
,” Int. J. Rob. Res.
33
, 1322
–1341
(2014
).17.
S.
Verma
and
J.-X.
Xu
, “
Data-assisted modeling and speed control of a robotic fish
,” IEEE Trans. Ind. Electron.
64
, 4150
–4157
(2016
).18.
S.
Verma
and
J.-X.
Xu
, “
Analytic modeling for precise speed tracking of multilink robotic fish
,” IEEE Trans. Ind. Electron.
65
, 5665
–5672
(2018
).19.
H.
Yu
,
B.
Liu
,
C.
Wang
,
X.
Liu
,
X.-Y.
Lu
, and
H.
Huang
, “
Deep-reinforcement-learning-based self-organization of freely undulatory swimmers
,” Phys. Rev. E
105
, 045105
(2022
).20.
I.
Mandralis
,
P.
Weber
,
G.
Novati
, and
P.
Koumoutsakos
, “
Learning swimming escape patterns for larval fish under energy constraints
,” Phys. Rev. Fluids
6
, 093101
(2021
).21.
L.
Hewing
,
J.
Kabzan
, and
M. N.
Zeilinger
, “
Cautious model predictive control using Gaussian process regression
,” IEEE Trans. Contr. Syst. Technol.
28
, 2736
–2743
(2020
).22.
Z.
Wu
,
A.
Tran
,
D.
Rincon
, and
P. D.
Christofides
, “
Machine learning-based predictive control of nonlinear processes. Part I: Theory
,” AIChE J.
65
, e16729
(2019
).23.
J.
Lore
,
S.
De Pascuale
,
P.
Laiu
,
B.
Russo
,
J.-S.
Park
,
J.
Park
,
S.
Brunton
,
J.
Kutz
, and
A.
Kaptanoglu
, “
Time-dependent SOLPS-ITER simulations of the tokamak plasma boundary for model predictive control using SINDy
,” Nucl. Fusion
63
, 046015
(2023
).24.
S. L.
Brunton
,
J. L.
Proctor
, and
J. N.
Kutz
, “
Discovering governing equations from data by sparse identification of nonlinear dynamical systems
,” Proc. Natl. Acad. Sci. U. S. A.
113
, 3932
–3937
(2016
).25.
U.
Fasel
,
J. N.
Kutz
,
B. W.
Brunton
, and
S. L.
Brunton
, “
Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control
,” Proc. R. Soc. A
478
, 0904
(2022
).26.
D. S.
Barrett
,
M. S.
Triantafyllou
,
D. K. P.
Yue
,
M. A.
Grosenbaugh
, and
M. J.
Wolfgang
, “
Drag reduction in fish-like locomotion
,” J. Fluid Mech.
392
, 183
–212
(1999
).27.
C.
Connaboy
,
S.
Coleman
, and
R. H.
Sanders
, “
Hydrodynamics of undulatory underwater swimming: A review
,” Sports Biomech.
8
, 360
–380
(2009
).28.
M.
Bergmann
and
A.
Iollo
, “
Modeling and simulation of fish-like swimming
,” J. Comput. Phys.
230
, 329
–348
(2011
).29.
P.
Angot
,
C.-H.
Bruneau
, and
P.
Fabrie
, “
A penalization method to take into account obstacles in incompressible viscous flows
,” Numer. Math.
81
, 497
–520
(1999
).30.
A. J.
Chorin
, “
On the convergence of discrete approximations to the Navier–Stokes equations
,” Math. Comp.
23
, 341
–353
(1969
).31.
R.
Témam
, “
Sur l'approximation de la solution des équations de Navier–Stokes par la méthode des pas fractionnaires (II)
,” Arch. Rational Mech. Anal.
33
, 377
–385
(1969
).32.
M.
Bergmann
,
J.
Hovnanian
, and
A.
Iollo
, “
An accurate Cartesian method for incompressible flows with moving boundaries
,” Commun. Comput. Phys.
15
, 1266
–1290
(2014
).33.
M.
Bergmann
,
A.
Iollo
, and
R.
Mittal
, “
Effect of caudal fin flexibility on the propulsive efficiency of a fish-like swimmer
,” Bioinspiration Biomimetics
9
, 046001
(2014
).34.
S.
Kern
and
P.
Koumoutsakos
, “
Simulations of optimized anguilliform swimming
,” J. Exp. Biol.
209
, 4841
–4857
(2006
).35.
A. P. S.
Bhalla
,
R.
Bale
,
B. E.
Griffith
, and
N. A.
Patankar
, “
Fully resolved immersed electrohydrodynamics for particle motion, electrolocation, and self-propulsion
,” J. Comput. Phys.
256
, 88
–108
(2014
).36.
E.
Kaiser
,
J. N.
Kutz
, and
S. L.
Brunton
, “
Sparse identification of nonlinear dynamics for model predictive control in the low-data limit
,” Proc. Math. Phys. Eng. Sci.
474
, 20180335
(2018
).37.
R.
Tibshirani
, “
Regression shrinkage and selection via the Lasso
,” J. R. Stat. Soc. Ser. B
58
, 267
–288
(1996
).38.
K.
Champion
,
P.
Zheng
,
A. Y.
Aravkin
,
S. L.
Brunton
, and
J. N.
Kutz
, “
A unified sparse optimization framework to learn parsimonious physics-informed models from data
,” IEEE Access
8
, 169259
–169271
(2020
).39.
A.
Kaptanoglu
,
B.
de Silva
,
U.
Fasel
,
K.
Kaheman
,
A.
Goldschmidt
,
J.
Callaham
,
C.
Delahunt
,
Z.
Nicolaou
,
K.
Champion
,
J.-C.
Loiseau
,
J.
Kutz
, and
S.
Brunton
, “
Pysindy: A comprehensive Python package for robust sparse system identification
,” J. Open Source Softw.
7
, 3994
(2022
).40.
L. T.
Biegler
, “
Solution of dynamic optimization problems by successive quadratic programming and orthogonal collocation
,” Comput. Chem. Eng.
8
, 243
–247
(1984
).41.
C.
Hargraves
and
S.
Paris
, “
Direct trajectory optimization using nonlinear programming and collocation
,” J. Guidance Control Dyn.
10
, 338
–342
(1987
).42.
H. G.
Bock
and
K. J.
Plitt
, “
A multiple shooting algorithm for direct solution of optimal control problems*
,” IFAC Proc. Volumes
17
, 1603
–1608
(1984
).43.
M.
Kelly
, “
An introduction to trajectory optimization: How to do your own direct collocation
,” SIAM Rev.
59
, 849
–904
(2017
).44.
C.
Kirches
, “
The direct multiple shooting method for optimal control
,” in Fast Numerical Methods for Mixed-Integer Nonlinear Model-Predictive Control
(
Vieweg+Teubner Verlag
, 2011
), pp. 13
–29
.45.
J. A. E.
Andersson
,
J.
Gillis
,
G.
Horn
,
J. B.
Rawlings
, and
M.
Diehl
, “
CasADi: A software framework for nonlinear optimization and optimal control
,” Math. Prog. Comp.
11
, 1
–36
(2018
).46.
A.
Wächter
and
L. T.
Biegler
, “
On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming
,” Math. Program.
106
, 25
–57
(2005
).47.
A.
Wächter
and
L. T.
Biegler
, “
Line search filter methods for nonlinear programming: Local convergence
,” SIAM J. Optim.
16
, 32
–48
(2005
).48.
F.
Fiedler
,
B.
Karg
,
L.
Lüken
,
D.
Brandner
,
M.
Heinlein
,
F.
Brabender
, and
S.
Lucia
, “
DO-MPC: Towards fair nonlinear and robust model predictive control
,” Control Eng. Practice
140
, 105676
(2023
).49.
M.
Bergmann
, “
Numerical modeling of a self-propelled dolphin jump out of water
,” Bioinspiration Biomimetics
17
, 065010
(2022
).50.
M.
Gazzola
,
M.
Argentina
, and
L.
Mahadevan
, “
Scaling macroscopic aquatic locomotion
,” Nat. Phys.
10
, 758
–761
(2014
).51.
G.
Li
,
I.
Ashraf
,
B.
François
,
D.
Kolomenskiy
,
F.
Lechenault
,
R.
Godoy-Diana
, and
B.
Thiria
, “
Burst-and-coast swimmers optimize gait by adapting unique intrinsic cycle
,” Commun. Biol.
4
, 40
(2021
).52.
G.
Li
,
D.
Kolomenskiy
,
H.
Liu
,
R.
Godoy-Diana
, and
B.
Thiria
, “
Intermittent versus continuous swimming: An optimization tale
,” Phys. Rev. Fluids
8
, 013101
(2023
).53.
A.
Marchetti
,
G.
François
,
T.
Faulwasser
, and
D.
Bonvin
, “
Modifier adaptation for real-time optimization-methods and applications
,” Processes
4
, 55
(2016
).54.
H.
Mania
,
M. I.
Jordan
, and
B.
Recht
, “
Active learning for nonlinear system identification with guarantees
,” J. Mach. Learn. Res.
23
, 1
–30
(2022
).© 2024 Author(s). Published under an exclusive license by AIP Publishing.
2024
Author(s)
You do not currently have access to this content.
