Three-dimensional (3D/4D) ultrasound (US) imaging of the tongue has emerged as a useful instrument for articulatory studies. However, extracting quantitative measurements of the shape of the tongue surface remains challenging and time-consuming. In response to these challenges, this paper documents and evaluates the first automated method for extracting tongue surfaces from 3D/4D US data. The method draws on established methods in computer vision, and combines image phase symmetry measurements, eigen-analysis of the image Hessian matrix, and a fast marching method for surface evolution towards the automatic detection of the sheet-like surface of the tongue amidst noisy US data. The method was tested on US recordings from eight speakers and the resulting automatically extracted tongue surfaces were generally found to lie within 1 to 2 mm from their corresponding manually delineated surfaces in terms of mean-sum-of-distances error. Further experiments demonstrate that the accuracy of 2D midsagittal tongue contour extraction is also improved using 3D data and methods. This is likely because the additional information afforded by 3D US compared to 2D US images strongly constrains the possible location of the midsagittal contour. Thus, the proposed method seems appropriate for immediate practical use in the analysis of 3D/4D US recordings of the tongue.

1.
Berry
,
J.
,
Fasel
,
I.
,
Fadiga
,
L.
, and
Archangeli
,
D.
(
2012
). “
Training deep nets with imbalanced and unlabeled data
,” in
Interspeech
, pp.
1754
1757
.
2.
Charles
,
S.
, and
Lulich
,
S. M.
(
2018
). “
Case study of Brazilian Portuguese laterals using a novel articulatory-acoustic methodology with 3D/4D ultrasound
,”
Speech Commun.
103
,
37
48
.
3.
Csapó
,
T. G.
,
Deme
,
A.
,
Gráczi
,
T. E.
, and
Markó
,
A.
(
2017
). “
Comparison of distance measures in tongue contour traces of ultrasound images
,” in
Ultrafest VIII
, pp.
65
66
.
4.
Csapó
,
T. G.
, and
Lulich
,
S. M.
(
2015
). “
Error analysis of extracted tongue contours from 2D ultrasound images
,” in
Interspeech
, pp.
2157
2161
.
5.
Descoteaux
,
M.
,
Audette
,
M.
,
Chinzei
,
K.
, and
Siddiqi
,
K.
(
2006
). “
Bone enhancement filtering: Application to sinus bone segmentation and simulation of pituitary surgery
,”
Comput. Aided Surg.
11
(
5
),
247
255
.
6.
Ester
,
M.
,
Kriegel
,
H. P.
,
Sander
,
J.
, and
Xu
,
X.
(
1996
). “
A density-based algorithm for discovering clusters in large spatial databases with noise
,” in
KDD
, pp.
226
231
.
7.
Fabre
,
D.
,
Hueber
,
T.
,
Bocquelet
,
F.
, and
Badin
,
P.
(
2015
). “
Tongue tracking in ultrasound images using eigentongue decomposition and artificial neural networks
,” in
Interspeech
, pp.
2410
2414
.
8.
Fasel
,
I.
, and
Berry
,
J.
(
2010
). “
Deep belief networks for real-time extraction of tongue contours from ultrasound during speech
,” in
International Conference on Pattern Recognition
, pp.
1493
1496
.
9.
Gick
,
B.
,
Bird
,
S.
, and
Wilson
,
I.
(
2005
). “
Techniques for field application of lingual ultrasound imaging
,”
Clin. Linguist. Phonet.
19
(
6-7
),
503
514
.
10.
Goldman
,
R.
, and
Fristoe
,
M.
(
2015
).
Goldman-Fristoe Test of Articulation
, 3rd ed. (
Pearson
,
New York
).
11.
Hacihaliloglu
,
I.
,
Abugharbieh
,
R.
,
Hodgson
,
A. J.
,
Rohling
,
R. N.
, and
Guy
,
P.
(
2012
). “
Automatic bone localization and fracture detection from volumetric ultrasound images using 3-D local phase features
,”
Ultrasound Med. Biol.
38
(
1
),
128
144
.
12.
Hwang
,
Y.
,
Charles
,
S.
, and
Lulich
,
S. M.
(
2019
). “
Articulatory characteristics and variation of Korean laterals
,”
Phonet. Speech Sci.
11
,
19
27
.
13.
Jaumard-Hakoun
,
A.
,
Xu
,
K.
,
Roussel-Ragot
,
P.
,
Dreyfus
,
G.
, and
Denby
,
B.
(
2015
). “
Tongue contour extraction from ultrasound images based on deep neural network
,” in
ICPhS
.
14.
Karimi
,
E.
,
Ménard
,
L.
, and
Laporte
,
C.
(
2019
). “
Fully-automated tongue detection in ultrasound images
,”
Comput. Biol. Med.
111
,
103335
.
15.
Kovesi
,
P.
(
1997
). “
Symmetry and asymmetry from local phase
,” in
Tenth Australian Joint Conference on Artificial Intelligence
, Vol.
190
, pp.
2
4
.
16.
Laporte
,
C.
, and
Lulich
,
S. M.
(
2017
). “
Towards automated segmentation of the moving tongue surface in 3D ultrasound
,” in
Ultrafest VIII
, pp.
65
66
.
17.
Laporte
,
C.
, and
Ménard
,
L.
(
2018
). “
Multi-hypothesis tracking of the tongue surface in ultrasound video recordings of normal and impaired speech
,”
Med. Image Anal.
44
,
98
114
.
18.
Li
,
M.
,
Kambhamettu
,
C.
, and
Stone
,
M.
(
2005
). “
Automatic contour tracking in ultrasound images
,”
Clin. Linguist. Phonet.
19
(
6-7
),
545
554
.
19.
Lulich
,
S. M.
,
Berkson
,
K. H.
, and
de Jong
,
K.
(
2018
). “
Acquiring and visualizing 3D/4D ultrasound recordings of tongue motion
,”
J. Phonet.
71
,
410
424
.
20.
Lulich
,
S. M.
,
Charles
,
S.
, and
Lulich
,
B.
(
2017
). “
The relation between tongue shape and pitch in clarinet playing using ultrasound measurements
,”
J. Acoust. Soc. Am.
141
(
3
),
1759
1768
.
21.
Lulich
,
S. M.
, and
Pearson
,
W. G.
(
2019
). “
Three-/four-dimensional ultrasound technology in speech research
,”
Perspect. ASHA Special Interest Groups
4
,
733
747
.
22.
Mozaffari
,
M. H.
,
Ratul
,
M. A. R.
, and
Lee
,
W.-S.
(
2019
). “
Irisnet: Deep learning for automatic and real-time tongue contour tracking in ultrasound video data using peripheral vision
,” arXiv:1911.03972.
23.
Preston
,
J. L.
,
Byun
,
T. M.
,
Boyce
,
S. E.
,
Hamilton
,
S.
,
Tiede
,
M.
,
Phillips
,
E.
,
Rivera-Campos
,
A.
, and
Whalen
,
D. H.
(
2017
). “
Ultrasound images of the tongue: A tutorial for assessment and remediation of speech sound errors
,”
J. Vis. Exp.
119
,
e55123
.
24.
Quader
,
N.
,
Hodgson
,
A. J.
,
Mulpuri
,
K.
,
Schaeffer
,
E.
, and
Abugharbieh
,
R.
(
2017
). “
Automatic evaluation of scan adequacy and dysplasia metrics in 2-D ultrasound images of the neonatal hip
,”
Ultrasound Med. Biol.
43
(
6
),
1252
1262
.
25.
Roussos
,
A.
,
Katsamanis
,
A.
, and
Maragos
,
P.
(
2009
). “
Tongue tracking in ultrasound images with active appearance models
,” in
IEEE Image Proc
, pp.
1733
1736
.
26.
Sato
,
Y.
,
Nakajima
,
S.
,
Shiraga
,
N.
,
Atsumi
,
H.
,
Yoshida
,
S.
,
Koller
,
T.
,
Gerig
,
G.
, and
Kikinis
,
R.
(
1998
). “
Three-dimensional multi-scale line filter for segmentation and visualization of curvilinear structures in medical images
,”
Med. Image Anal.
2
(
2
),
143
168
.
27.
Scobbie
,
J. M.
,
Wrench
,
A. A.
, and
van der Linden
,
M. L.
(
2008
). “
Head-probe stabilisation in ultrasound tongue imaging using a headset to permit natural head movement
,” in
ISSP
, pp.
373
376
.
28.
Sethian
,
J. A.
(
1999
).
Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science
(
Cambridge University Press
,
Cambridge, UK
), Vol. 3.
29.
Stone
,
M.
(
2005
). “
A guide to analysing tongue motion from ultrasound images
,”
Clin. Linguist. Phonet.
19
(
6-7
),
455
501
.
30.
Tang
,
L.
,
Bressmann
,
T.
, and
Hamarneh
,
G.
(
2012
). “
Tongue contour tracking in dynamic ultrasound via higher-order MRFS and efficient fusion moves
,”
Med. Image Anal.
16
(
8
),
1503
1520
.
31.
Xu
,
K.
,
Yang
,
Y.
,
Stone
,
M.
,
Jaumard-Hakoun
,
A.
,
Leboullenger
,
C.
,
Dreyfus
,
G.
,
Roussel
,
P.
, and
Denby
,
B.
(
2016
). “
Robust contour tracking in ultrasound tongue image sequences
,”
Clin. Linguist. Phonet.
30
(
3-5
),
313
327
.
32.
Zharkova
,
N.
, and
Hewlett
,
N.
(
2009
). “
Measuring lingual coarticulation from midsagittal tongue contours: Description and example calculations using English /t/ and /a/
,”
J. Phonet.
37
(
2
),
248
256
.
33.
Zhu
,
J.
,
Styler
,
W.
, and
Calloway
,
I.
(
2019
). “
A CNN-based tool for automatic tongue contour tracking in ultrasound images
,” arXiv:1907.10210.

Supplementary Material

You do not currently have access to this content.