Cancer spatial and temporal heterogeneity fuels resistance to therapies. To realize the routine assessment of cancer prognosis and treatment, we demonstrate the development of an Intelligent Disease Detection Tool (IDDT), a microfluidic-based tumor model integrated with deep learning-assisted algorithmic analysis. IDDT was clinically validated with liquid blood biopsy samples (n = 71) from patients with various types of cancers (e.g., breast, gastric, and lung cancer) and healthy donors, requiring low sample volume (∼200 μl) and a high-throughput 3D tumor culturing system (∼300 tumor clusters). To support automated algorithmic analysis, intelligent decision-making, and precise segmentation, we designed and developed an integrative deep neural network, which includes Mask Region-Based Convolutional Neural Network (Mask R-CNN), vision transformer, and Segment Anything Model (SAM). Our approach significantly reduces the manual labeling time by up to 90% with a high mean Intersection Over Union (mIoU) of 0.902 and immediate results (<2 s per image) for clinical cohort classification. The IDDT can accurately stratify healthy donors (n = 12) and cancer patients (n = 55) within their respective treatment cycle and cancer stage, resulting in high precision (∼99.3%) and high sensitivity (∼98%). We envision that our patient-centric IDDT provides an intelligent, label-free, and cost-effective approach to help clinicians make precise medical decisions and tailor treatment strategies for each patient.

1.
I.
Dagogo-Jack
and
A. T.
Shaw
, “
Tumour heterogeneity and resistance to cancer therapies
,”
Nat. Rev. Clin. Oncol.
15
,
81
94
(
2018
).
2.
B. L.
Khoo
et al, “
Liquid biopsy and therapeutic response: Circulating tumor cell cultures for evaluation of anticancer treatment
,”
Sci. Adv.
2
,
e1600274
(
2016
).
3.
R. B.
Corcoran
, “
Liquid biopsy versus tumor biopsy for clinical-trial recruitment
,”
Nat. Med.
26
,
1815
1816
(
2020
).
4.
M.
Uddin
,
Y.
Wang
, and
M.
Woodbury-Smith
, “
Artificial intelligence for precision medicine in neurodevelopmental disorders
,”
npj Digit. Med.
2
,
112
(
2019
).
5.
M. Y.
Lu
et al, “
AI-based pathology predicts origins for cancers of unknown primary
,”
Nature
594
,
106
110
(
2021
).
6.
A.
Esteva
et al, “
Dermatologist-level classification of skin cancer with deep neural networks
,”
Nature
542
,
115
118
(
2017
).
7.
R.
Poplin
et al, “
Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning
,”
Nat. Biomed. Eng.
2
,
158
164
(
2018
).
8.
G. A.
Bello
et al, “
Deep-learning cardiac motion analysis for human survival prediction
,”
Nat. Mach. Intell.
1
,
95
104
(
2019
).
9.
M.
Subramanian
et al, “
Precision medicine in the era of artificial intelligence: Implications in chronic disease management
,”
J. Transl. Med.
18
,
1
12
(
2020
).
10.
K.-H.
Yu
,
A. L.
Beam
, and
I. S.
Kohane
, “
Artificial intelligence in healthcare
,”
Nat. Biomed. Eng.
2
,
719
731
(
2018
).
11.
E. J.
Topol
, “
High-performance medicine: The convergence of human and artificial intelligence
,”
Nat. Med.
25
,
44
56
(
2019
).
12.
B. L.
Khoo
et al, “
Short-term expansion of breast circulating cancer cells predicts response to anti-cancer therapy
,”
Oncotarget
6
,
15578
(
2015
).
13.
B. L.
Khoo
et al, “
Expansion of patient-derived circulating tumor cells from liquid biopsies using a CTC microfluidic culture device
,”
Nat. Protoc.
13
,
34
58
(
2018
).
14.
W.
Li
,
Y.
Zhou
,
Y.
Deng
, and
B. L.
Khoo
, “
Early predictor tool of disease using label-free liquid biopsy-based platforms for patient-centric healthcare
,”
Cancers
14
,
818
(
2022
).
15.
O.
Ronneberger
,
P.
Fischer
, and
T.
Brox
, “
U-net: Convolutional networks for biomedical image segmentation
,” in
International Conference on Medical Image Computing and Computer-Assisted Intervention
(
Springer
, 2015), pp.
234
241
.
16.
Z.
Zhou
,
M. M. R.
Siddiquee
,
N.
Tajbakhsh
, and
J.
Liang
,
Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support
(
Springer
,
2018
), pp.
3
11
.
17.
F. I.
Diakogiannis
,
F.
Waldner
,
P.
Caccetta
, and
C.
Wu
, “
ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data
,”
ISPRS J. Photogramm. Remote Sens.
162
,
94
114
(
2020
).
18.
D.
Jha
et al, in
IEEE International Symposium on Multimedia (ISM)
(
IEEE
,
2019
), pp.
225
2255
.
19.
R.
Deng
et al, “
Segment anything model (SAM) for digital pathology: Assess zero-shot segmentation on whole slide imaging
,” arXiv:2304.04155 (
2023
).
20.
K.
He
,
G.
Gkioxari
,
P.
Dollár
, and
R.
Girshick
, “
Mask R-CNN,
,” in
Proceedings of the IEEE International Conference on Computer Vision
(
IEEE
, 2017), pp.
2961
2969
.
21.
A.
Dosovitskiy
et al, “
An image is worth 16×16 words: Transformers for image recognition at scale
,” arXiv:2010.11929 (
2020
).
22.
N. J.
Schork
, “
Personalized medicine: Time for one-person trials
,”
Nature
520
,
609
611
(
2015
).
23.
Z.
Zhang
et al, “
Label-free estimation of therapeutic efficacy on 3D cancer spheres using convolutional neural network image analysis
,”
Anal. Chem.
91
,
14093
14100
(
2019
).
24.
Y.-C.
Chen
,
Z.
Zhang
, and
E.
Yoon
, “
Early prediction of single-cell derived sphere formation rate using convolutional neural network image analysis
,”
Anal. Chem.
92
,
7717
7724
(
2020
).
25.
D.
Ho
et al, “
Enabling technologies for personalized and precision medicine
,”
Trends Biotechnol.
38
,
497
518
(
2020
).
26.
E. K.
Sackmann
,
A. L.
Fulton
, and
D. J.
Beebe
, “
The present and future role of microfluidics in biomedical research
,”
Nature
507
,
181
189
(
2014
).
27.
Y.
Deng
,
S. Y.
Liu
,
S. L.
Chua
, and
B. L.
Khoo
, “
The effects of biofilms on tumor progression in a 3D cancer-biofilm microfluidic model
,”
Biosens. Bioelectron.
180
,
113113
(
2021
).
28.
C. D.
Chin
et al, “
Microfluidics-based diagnostics of infectious diseases in the developing world
,”
Nat. Med.
17
,
1015
1019
(
2011
).
29.
B.
Dai
et al, “
Fully-functional semi-automated microfluidic immunoassay platform for quantitation of multiple samples
,”
Sens. Actuators, B
300
,
127017
(
2019
).
30.
B.
Dai
et al, “
A flux-adaptable pump-free microfluidics-based self-contained platform for multiplex cancer biomarker detection
,”
Lab Chip
21
,
143
153
(
2021
).
31.
Y.
Fu
et al, “
Diagnosis of mixed infections with swine viruses using an integrated microfluidic platform
,”
Sens. Actuators, B
312
,
128005
(
2020
).
32.
S. N.
Bhatia
and
D. E.
Ingber
, “
Microfluidic organs-on-chips
,”
Nat. Biotechnol.
32
,
760
772
(
2014
).
33.
B.
Dai
,
L.
Zhang
,
C.
Zhao
et al, “
Biomimetic apposition compound eye fabricated using microfluidic-assisted 3D printing
,”
Nat. Commun.
12
,
6458
(
2021
).
34.
N. V.
Menon
,
S. B.
Lim
, and
C. T.
Lim
, “
Microfluidics for personalized drug screening of cancer
,”
Curr. Opin. Pharmacol.
48
,
155
161
(
2019
).
35.
D. D.
Truong
et al, “
A human organotypic microfluidic tumor model permits investigation of the interplay between patient-derived fibroblasts and breast cancer cells
,”
Cancer Res.
79
,
3139
3151
(
2019
).
36.
P.
Courtiol
et al, “
Deep learning-based classification of mesothelioma improves prediction of patient outcome
,”
Nat. Med.
25
,
1519
1525
(
2019
).
37.
X.
Li
,
L.
Liu
,
J.
Zhou
, and
C.
Wang
, “
Heterogeneity analysis and diagnosis of complex diseases based on deep learning method
,”
Sci. Rep.
8
,
6155
(
2018
).
38.
Z.
Chen
et al, “
Automated evaluation of tumor spheroid behavior in 3D culture using deep learning-based recognition
,”
Biomaterials
272
,
120770
(
2021
).
39.
D. K.
Singh
,
C. C.
Ahrens
,
W.
Li
, and
S. A.
Vanapalli
, “
Label-free, high-throughput holographic screening and enumeration of tumor cells in blood
,”
Lab Chip
17
,
2920
2932
(
2017
).
40.
H.
Hashemzadeh
et al, “
A combined microfluidic deep learning approach for lung cancer cell high throughput screening toward automatic cancer screening applications
,”
Sci. Rep.
11
,
9804
(
2021
).
41.
A. A.
Ponomaryova
et al, “
Potentialities of aberrantly methylated circulating DNA for diagnostics and post-treatment follow-up of lung cancer patients
,”
Lung Cancer
81
,
397
403
(
2013
).
42.
B.
Korbar
et al, “
Deep learning for classification of colorectal polyps on whole-slide images
,”
J. Pathol. Inf.
8
,
30
(
2017
).
43.
Z.
Song
et al, “
Automatic deep learning-based colorectal adenoma detection system and its similarities with pathologists
,”
BMJ Open
10
,
e036423
(
2020
).
44.
P.
Korfiatis
et al, “
Residual deep convolutional neural network predicts MGMT methylation status
,”
J. Digital Imaging
30
,
622
628
(
2017
).
45.
F.
Cheng
,
L.
Su
, and
C.
Qian
, “
Circulating tumor DNA: A promising biomarker in the liquid biopsy of cancer
,”
Oncotarget
7
,
48832
(
2016
).
46.
Y.-H.
Wang
,
Z.
Song
,
X.-Y.
Hu
, and
H.-S.
Wang
, “
Circulating tumor DNA analysis for tumor diagnosis
,”
Talanta
228
,
122220
(
2021
).
47.
A. L.
Allan
and
M.
Keeney
, “
Circulating tumor cell analysis: Technical and statistical considerations for application to the clinic
,”
J. Oncol.
2010
,
426218
.

Supplementary Material

You do not currently have access to this content.