In the current pandemic, lung ultrasound (LUS) played a useful role in evaluating patients affected by COVID-19. However, LUS remains limited to the visual inspection of ultrasound data, thus negatively affecting the reliability and reproducibility of the findings. Moreover, many different imaging protocols have been proposed, most of which lacked proper clinical validation. To address these problems, we were the first to propose a standardized imaging protocol and scoring system. Next, we developed the first deep learning (DL) algorithms capable of evaluating LUS videos providing, for each video-frame, the score as well as semantic segmentation. Moreover, we have analyzed the impact of different imaging protocols and demonstrated the prognostic value of our approach. In this work, we report on the level of agreement between the DL and LUS experts, when evaluating LUS data. The results show a percentage of agreement between DL and LUS experts of 85.96% in the stratification between patients at high risk of clinical worsening and patients at low risk. These encouraging results demonstrate the potential of DL models for the automatic scoring of LUS data, when applied to high quality data acquired accordingly to a standardized imaging protocol.

1.
World Health Organization
, “
WHO coronavirus disease (COVID-19) Dashboard
,” https://covid19.who.int/ (Last viewed February 15,
2021
).
2.
World Health Organization
, “
What are the symptoms of COVID-19?
,” https://www.who.int/emergencies/diseases/novel-coronavirus-2019/question-and-answers-hub/qa-detail/q-a-coronaviruses (Last viewed December 21, 2020).
3.
N.
Zhu
,
D.
Zhang
,
W.
Wang
,
X.
Li
,
B.
Yang
,
J.
Song
,
X.
Zhao
,
B.
Huang
,
W.
Shi
,
R.
Lu
,
P.
Niu
,
F.
Zhan
,
X.
Ma
,
D.
Wang
,
W.
Xu
,
G.
Wu
, and
G.
Gao
, “
A novel coronavirus from patients with pneumonia in China, 2019
,”
New England J. Med.
382
(
8
),
727
733
(
2020
).
4.
World Health Organization
, “
Laboratory testing for coronavirus disease (COVID-19) in suspected human cases
,” https://apps.who.int/iris/bitstream/handle/10665/331501/WHOCOVID-19-laboratory-2020.5eng.pdf?sequence=1&isAllowed=y (Last viewed December 21,
2020
).
5.
S.
Woloshin
,
N.
Patel
, and
A. S.
Kesselheim
, “
False negative tests for SARS-CoV-2 Infection—Challenges and implications
,”
New England J. Med.
383
(
6
),
e38
(
2020
).
6.
B.
Xu
,
Y.
Xing
,
J.
Peng
,
Z.
Zheng
,
W.
Tang
,
Y.
Sun
,
C.
Xu
, and
F.
Peng
, “
Chest CT for detecting COVID-19: A systematic review and meta-analysis of diagnostic accuracy
,”
Eur. Radiol.
30
(
10
),
5720
5727
(
2020
).
7.
G.
Soldati
,
A.
Smargiassi
,
R.
Inchingolo
,
D.
Buonsenso
,
T.
Perrone
,
D. F.
Briganti
,
S.
Perlini
,
E.
Torri
,
A.
Mariani
,
E. E.
Mossolani
,
F.
Tursi
,
F.
Mento
, and
L.
Demi
, “
Is there a role for lung ultrasound during the COVID-19 pandemic?
,”
J. Ultrasound Med.
39
(
7
),
1459
1462
(
2020
).
8.
G.
Soldati
,
M.
Demi
,
A.
Smargiassi
,
R.
Inchingolo
, and
L.
Demi
, “
The role of ultrasound lung artifacts in the diagnosis of respiratory diseases
,”
Expert Rev. Respiratory Med.
13
(
2
),
163
172
(
2019
).
9.
E.
Picano
and
P. A.
Pellikka
, “
Ultrasound of extravascular lung water: A new standard for pulmonary congestion
,”
Eur. Heart J.
37
(
27
),
2097
2104
(
2016
).
10.
G.
Soldati
,
M.
Demi
,
R.
Inchingolo
,
A.
Smargiassi
, and
L.
Demi
, “
On the physical basis of pulmonary sonographic interstitial syndrome
,”
J. Ultrasound Med.
35
(
10
),
2075
2086
(
2016
).
11.
R.
Copetti
,
G.
Soldati
, and
P.
Copetti
, “
Chest sonography: A useful tool to differentiate acute cardiogenic pulmonary edema from acute respiratory distress syndrome
,”
Cardiovasc. Ultrasound
6
(
1
),
16
(
2008
).
12.
G.
Soldati
,
A.
Smargiassi
,
L.
Demi
, and
R.
Inchingolo
, “
Artifactual lung ultrasonography: It is a matter of traps, order, and disorder
,”
Appl. Sci.
10
(
5
),
1570
(
2020
).
13.
L.
Demi
,
M.
Demi
,
R.
Prediletto
, and
G.
Soldati
, “
Real-time multi-frequency ultrasound imaging for quantitative lung ultrasound – first clinical results
,”
J. Acoust. Soc. Am.
148
(
2
),
998
1005
(
2020
).
14.
F.
Mento
,
G.
Soldati
,
R.
Prediletto
,
M.
Demi
, and
L.
Demi
, “
Quantitative lung ultrasound spectroscopy applied to the diagnosis of pulmonary fibrosis: The first clinical study
,”
IEEE Trans. Ultrason. Ferroelectr. Freq. Control
67
(
11
),
2265
2273
(
2020
).
15.
F.
Mento
and
L.
Demi
, “
On the influence of imaging parameters on lung ultrasound B-line artifacts, in vitro study
,”
J. Acoust. Soc. Am.
148
(
2
),
975
983
(
2020
).
16.
L.
Demi
,
W.
van Hoeve
,
R. J. G.
van Sloun
,
G.
Soldati
, and
M.
Demi
, “
Determination of a potential quantitative measure of the state of the lung using lung ultrasound spectroscopy
,”
Sci. Rep.
7
(
1
),
12746
(
2017
).
17.
K.
Mohanty
,
J.
Blackwell
,
T.
Egan
, and
M.
Muller
, “
Characterization of the lung parenchyma using ultrasound multiple scattering
,”
Ultrasound Med. Biol.
43
(
5
),
993
1003
(
2017
).
18.
X.
Zhang
,
T.
Osborn
,
B.
Zhou
,
D.
Meixner
,
R. R.
Kinnick
,
B.
Bartholmai
,
J. F.
Greenleaf
, and
S.
Kalra
, “
Lung ultrasound surface wave elastography: A pilot clinical study
,”
IEEE Trans. Ultrason. Ferroelectr. Freq. Control
64
(
9
),
1298
1304
(
2017
).
19.
G.
Soldati
,
A.
Smargiassi
,
R.
Inchingolo
,
D.
Buonsenso
,
T.
Perrone
,
D. F.
Briganti
,
S.
Perlini
,
E.
Torri
,
A.
Mariani
,
E. E.
Mossolani
,
F.
Tursi
,
F.
Mento
, and
L.
Demi
, “
Proposal for international standardization of the use of lung ultrasound for patients with COVID-19
,”
J. Ultrasound Med.
39
(
7
),
1413
1419
(
2020
).
20.
S.
Roy
,
W.
Menapace
,
S.
Oei
,
B.
Luijten
,
E.
Fini
,
C.
Saltori
,
I.
Huijben
,
N.
Chennakeshava
,
F.
Mento
,
A.
Sentelli
,
E.
Peschiera
,
R.
Trevisan
,
G.
Maschietto
,
E.
Torri
,
R.
Inchingolo
,
A.
Smargiassi
,
G.
Soldati
,
P.
Rota
,
A.
Passerini
,
R. J. G. V.
Sloun
,
E.
Ricci
, and
L.
Demi
, “
Deep learning for classification and localization of COVID-19 markers in point-of-care lung ultrasound
,”
IEEE Trans. Med. Imag.
39
(
8
),
2676
2687
(
2020
).
21.
L.
Carrer
,
E.
Donini
,
D.
Marinelli
,
M.
Zanetti
,
F.
Mento
,
E.
Torri
,
A.
Smargiassi
,
R.
Inchingolo
,
G.
Soldati
,
L.
Demi
,
F.
Bovolo
, and
L.
Bruzzone
, “
Automatic pleural line extraction and COVID-19 scoring from lung ultrasound data
,”
IEEE Trans. Ultrason. Ferroelectr. Freq. Control
67
(
11
),
2207
2217
(
2020
).
22.
F.
Mento
,
T.
Perrone
,
V. N.
Macioce
,
F.
Tursi
,
D.
Buonsenso
,
E.
Torri
,
A.
Smargiassi
,
R.
Inchingolo
,
G.
Soldati
, and
L.
Demi
, “
On the impact of different lung ultrasound imaging protocols in the evaluation of patients affected by coronavirus disease 2019
,”
J. Ultrasound Med.
(published online
2020
).
23.
A.
Smargiassi
,
G.
Soldati
,
E.
Torri
,
F.
Mento
,
D.
Milardi
,
P. D.
Giacomo
,
G.
De Matteis
,
M. L.
Burzo
,
A. R.
Larici
,
M.
Pompili
,
L.
Demi
, and
R.
Inchingolo
, “
Lung ultrasound for COVID-19 patchy pneumonia: Extended or limited evaluations?
,”
J. Ultrasound Med.
40
,
521
(
2021
).
24.
T.
Perrone
,
G.
Soldati
,
L.
Padovini
,
A.
Fiengo
,
G.
Lettieri
,
U.
Sabatini
,
G.
Gori
,
F.
Lepore
,
M.
Garolfi
,
I.
Palumbo
,
R.
Inchingolo
,
A.
Smargiassi
,
L.
Demi
,
E. E.
Mossolani
,
F.
Tursi
,
C.
Klersy
, and
A. D.
Sabatino
, “
A new lung ultrasound protocol able to predict worsening in patients affected by severe acute respiratory syndrome coronavirus 2 pneumonia
,”
J. Ultrasound Med.
(published online
2020
).
25.
ICLUS Team and BlueTensor
, “
ICLUS web application
,” https://iclus-web.bluetensor.ai/ (Last viewed February 15,
2021
).
You do not currently have access to this content.