In this article, we present a proof-of-concept for microfluidic systems with high functional variability using reinforcement learning. By mathematically defining the objective of tasks, we demonstrate that the system can autonomously learn to behave according to its objectives. We applied Q-learning to a peristaltic micropump and showed that two different tasks can be performed on the same platform: adjusting the flow rate of the pump and manipulating the position of the particles. First, we performed typical micropumping with flow rate control. In this task, the system is rewarded according to the deviation between the average flow rate generated by the micropump and the target value. Therefore, the objective of the system is to maintain the target flow rate via an operation of the pump. Next, we demonstrate the micromanipulation of a small object (microbead) on the same platform. The objective was to manipulate the microbead position to the target area, and the system was rewarded for the success of the task. These results confirmed that the system learned to control the flow rate and manipulate the microbead to any randomly chosen target position. In particular, the manipulation technique is a new technology that does not require the use of structures such as wells or weirs. Therefore, this concept not only adds flexibility to the system but also contributes to the development of novel control methods to realize highly versatile microfluidic systems.

1.
N.
Convery
and
N.
Gadegaard
, “
30 years of microfluidics
,”
Micro Nano Eng.
2
,
76
91
(
2019
).
2.
R.
Dong
,
Y.
Liu
,
L.
Mou
,
J.
Deng
, and
X.
Jiang
, “
Microfluidics-based biomaterials and biodevices
,”
Adv. Mater.
31
,
1805033
(
2019
).
3.
E. K.
Sackmann
,
A. L.
Fulton
, and
D. J.
Beebe
, “
The present and future role of microfluidics in biomedical research
,”
Nature
507
,
181
189
(
2014
).
4.
G. M.
Whitesides
, “
The origins and the future of microfluidics
,”
Nature
442
,
368
373
(
2006
).
5.
S.
Dekker
,
W.
Buesink
,
M.
Blom
,
M.
Alessio
,
N.
Verplanck
,
M.
Hihoud
,
C.
Dehan
,
W.
Cesar
,
A. L.
Nel
,
A. V. D.
Berg
, and
M.
Odijk
, “
Standardized and modular microfluidic platform for fast Lab on Chip system development
,”
Sens. Actuators B: Chem.
272
,
468
478
(
2018
).
6.
K.
Leung
,
H.
Zahn
,
T.
Leaver
,
K. M.
Konwar
,
N. W.
Hanson
,
A. P.
Page
,
C.-C.
Lo
,
P. S.
Chain
,
S. J.
Hallam
, and
C. L.
Hansen
, “
A programmable droplet-based microfluidic device applied to multiparameter analysis of single microbes and microbial communities
,”
Proc. Natl. Acad. Sci. U.S.A.
109
(
20
),
7665
7670
(
2012
).
7.
K. A.
Shaikh
,
K. S.
Byu
,
E. D.
Goluch
,
J.-M.
Nam
,
J.
Liu
,
C. S.
Thaxton
,
T. N.
Chiesl
,
A. E.
Barron
,
Y.
Liu
,
C. A.
Mirkin
, and
C.
Lin
, “
A modular microfluidic architecture for integrated biochemical analysis
,”
Proc. Natl. Acad. Sci. U.S.A.
102
(
28
),
9745
9750
(
2005
).
8.
H.
Wang
,
Y.-L.
Zhang
,
D.-D.
Han
,
W.
Wang
, and
H.-B.
Sun
, “
Laser fabrication of modular superhydrophobic chips for reconfigurable assembly and self-propelled droplet manipulation
,”
PhotoniX
2
,
17
(
2021
).
9.
L. M.
Fidalgo
and
S. J.
Maerkl
, “
A software-programmable microfluidic device for automated biology
,”
Lab Chip
11
,
1612
(
2011
).
10.
R. S.
Sutton
and
A. G.
Barto
,
Reinforcement Learning an Introduction
(
MIT Press
,
1998
).
11.
V.
Mnih
,
K.
Kavukcuoglu
,
D.
Silver
,
A. A.
Rusu
,
J.
Veness
,
M. G.
Bellemare
,
A.
Graves
,
M.
Riedmiller
,
A. K.
Fidjeland
,
G.
Ostrovski
,
S.
Petersen
,
C.
Beattie
,
A.
Sadik
,
I.
Antonoglou
,
H.
King
,
D.
Kumaran
,
D.
Wierstra
,
S.
Legg
, and
D.
Hassabis
, “
Human-level control through deep reinforcement learning
,”
Nature
518
,
529
533
(
2015
).
12.
D.
Silver
,
A.
Huang
,
C. J.
Maddison
,
A.
Guez
,
L.
Sifre
,
G. V. D.
Driessche
,
J.
Schrittwieser
,
I.
Antonoglou
,
V.
Panneershelvam
,
M.
Lanctot
,
S.
Dieleman
,
D.
Grewe
,
J.
Nham
,
N.
Kalchbrenner
,
I.
Sutskever
,
T.
Lillicrap
,
M.
Leach
,
K.
Kavukcuoglu
,
T.
Graepel
, and
D.
Hassabis
, “
Mastering the game of Go with deep neural networks and tree search
,”
Nature
529
,
484
489
(
2016
).
13.
I.
Akkaya
,
M.
Andrychowicz
,
M.
Chociej
,
M.
Litwin
,
B.
McGrew
,
A.
Petron
,
A.
Paino
,
M.
Plappert
,
G.
Powell
, and
R.
Ribas
, “Solving Rubik’s cube with a robot hand,” arXiv:1910.07113 (
2019
).
14.
L.
Pinto
,
M.
Andrychowicz
,
P.
Welinder
,
W.
Zaremba
, and
P.
Abbeel
, “Asymmetric actor critic for image-based robot learning,” arXiv:1710.06542 (
2017
).
15.
E. A.
Galan
,
H.
Zhao
,
X.
Wang
,
Q.
Dai
,
W. T. S.
Huck
, and
S.
Ma
, “
Intelligent microfluidics: The convergence of machine learning and microfluidics in materials science and biomedicine
,”
Matter
3
(
6
),
1893
1922
(
2020
).
16.
J.
Zheng
,
T.
Cole
,
Y.
Zhang
,
J.
Kim
, and
S.-Y.
Tang
, “
Exploiting machine learning for bestowing intelligence to microfluidics
,”
Biosens. Bioelec.
194
(
15
),
113666
(
2021
).
17.
V.
Rizzuto
,
A.
Mencattini
,
B. A.
Gonzalez
,
D. D.
Giuseppe
,
E.
Martinelli
,
D. B.
Pastor
,
M. D. M. M.
Pereira
,
M. J. L.
Martinez
, and
J.
Samitier
, “
Combining microfluidics with machine learning algorithms for RBC classification in rare hereditary hemolytic anemia
,”
Sci. Rep.
11
,
13553
(
2021
).
18.
S.
Ota
,
R.
Horisaki
,
M.
Ugawa
,
I.
Sato
,
K.
Hashimoto
,
R.
Kamesawa
,
K.
Setoyama
,
S.
Yamaguchi
,
K.
Fujiu
,
K.
Waki
, and
H.
Noji
, “
Ghost cytometry
,”
Science
360
(
6394
),
1246
1251
(
2018
).
19.
N.
Nitta
,
T.
Sugimura
,
A.
Isozaki
,
H.
Mikami
,
K.
Hiraki
,
S.
Sakuma
,
T.
Lino
,
F.
Arai
,
T.
Endo
,
Y.
Fjiwaki
,
H.
Fukuzawa
,
M.
Hase
,
T.
Hayakawa
,
K.
Hiramatsu
,
Y.
Hoshino
,
M.
Inaba
,
T.
Ito
,
H.
Karakawa
,
Y.
Kasai
,
K.
Koizumi
,
S. W.
Lee
,
C.
Lei
,
M.
Li
,
T.
Maeno
,
S.
Matsusaka
,
D.
Murakami
,
A.
Nakagawa
,
Y.
Oguchi
,
M.
Olikawa
,
T.
Ota
,
K.
Shiba
,
H.
Shintaku
,
Y.
Shirasaki
,
K.
Suga
,
Y.
Suzuki
,
N.
Suzuki
,
Y.
Tanaka
,
H.
Tezuka
,
C.
Toyokawa
,
Y.
Yalikun
,
M.
Yamada
,
M.
Yamagishi
,
T.
Yamano
,
A.
Yasumoto
,
Y.
Yatomi
,
M.
Yazawa
,
D. D.
Carlo
,
Y.
Hosokawa
,
S.
Uemura
,
Y.
Ozeki
, and
K.
Goda
, “
Intelligent image-activated cell sorting
,”
Cell
175
(
1
),
266
276.e13
(
2018
).
20.
B.
Guo
,
C.
Lei
,
H.
Kobayashi
,
T.
Ito
,
Y.
Yalikun
,
Y.
Jiang
,
Y.
Tanaka
,
Y.
Ozeki
, and
K.
Goda
, “
High-throughput, label-free, single-cell, microalgal lipid screening by machine-learning-equipped optofluidic time-stretch quantitative phase microscopy
,”
Cytomet. Part A
91
,
494
502
(
2017
).
21.
X.
Huang
,
Y.
Jiang
,
X.
Liu
,
H.
Xu
,
Z.
Han
,
H.
Rong
,
H.
Yang
,
M.
Yan
, and
H.
Yu
, “
Machine learning based single-frame super-resolution processing for lensless blood cell counting
,”
Sensors
16
,
1836
(
2016
).
22.
S. A.
Damiati
,
D.
Rossi
,
H. N.
Joensson
, and
S.
Damiati
, “
Artificial intelligence application for rapid fabrication of size-tunable PLGA microparticles in microfluidics
,”
Sci. Rep.
10
,
19517
(
2020
).
23.
Y.
Mahdi
and
K.
Daoud
, “
Microdroplet size prediction in microfluidic systems via artificial neural network modeling for water-in-oil emulsion formulation
,”
J. Dispers. Sci. Technol.
38
,
1501
1508
(
2017
).
24.
D.
Stoecklein
,
K. G.
Lore
,
M.
Davies
,
S.
Sarkar
, and
B.
Ganapathysubramanian
, “
Deep learning for flow sculpting: Insights into efficient learning using scientific simulation data
,”
Sci. Rep.
7
,
46368
(
2017
).
25.
Z.
Zhou
,
X.
Li
, and
R. N.
Zare
, “
Optimizing chemical reactions with deep reinforcement learning
,”
ACS Cent. Sci.
3
,
1337
1344
(
2017
).
26.
B. A.
Rizkin
,
A. S.
Shkolnik
,
N. J.
Ferraro
, and
R. L.
Hartman
, “
Combining automated microfluidic experimentation with machine learning for efficient polymerization design
,”
Nat. Mach. Intell.
2
,
200
209
(
2020
).
27.
O. B.
Mercan
,
V.
Kılıç
, and
M.
Sen
, “
Machine learning-based colorimetric determination of glucose in artificial saliva with different reagents using a smartphone coupled PAD
,”
Sens. Actuators B: Chem.
329
(
15
),
129037
(
2021
).
28.
O. J.
Dressler
,
P. D.
Howes
,
J.
Choo
, and
A. J.
deMello
, “
Reinforcement learning for dynamic microfluidic control
,”
ACS Omega
3
,
10084
10091
(
2018
).
29.
T.
Abe
,
S.
Oh-hara
, and
Y.
Ukita
, “
Adoption of reinforcement learning for the intelligent control of a microfluidic peristaltic pump
,”
Biomicrofluidics
15
,
034101
(
2021
).
30.
D. C.
Duffy
,
J. C.
McDonald
,
O. J. A.
Schueller
, and
G. M.
Whitesides
, “
Rapid prototyping of microfluidic systems in poly(dimethylsiloxane)
,”
Anal. Chem.
70
,
4974
4984
(
1998
).

Supplementary Material

You do not currently have access to this content.