The goal of this project is to use acoustic signatures to detect, classify, and count the calls of four acoustic populations of blue whales so that, ultimately, the conservation status of each population can be better assessed. We used manual annotations from 350 h of audio recordings from the underwater hydrophones in the Indian Ocean to build a deep learning model to detect, classify, and count the calls from four acoustic song types. The method we used was Siamese neural networks (SNN), a class of neural network architectures that are used to find the similarity of the inputs by comparing their feature vectors, finding that they outperformed the more widely used convolutional neural networks (CNN). Specifically, the SNN outperform a CNN with 2% accuracy improvement in population classification and 1.7%–6.4% accuracy improvement in call count estimation for each blue whale population. In addition, even though we treat the call count estimation problem as a classification task and encode the number of calls in each spectrogram as a categorical variable, SNN surprisingly learned the ordinal relationship among them. SNN are robust and are shown here to be an effective way to automatically mine large acoustic datasets for blue whale calls.

1.
Bergler
,
C.
,
Schröter
,
H.
,
Cheng
,
R. X.
,
Barth
,
V.
,
Weber
,
M.
,
Nöth
,
E.
,
Hofer
,
H.
, and
Maier
,
A.
(
2019
). “
ORCA-SPOT: An automatic killer whale sound detection toolkit using deep learning
,”
Sci. Rep.
9
(
1
),
10997
.
2.
Bianco
,
M. J.
,
Gerstoft
,
P.
,
Traer
,
J.
,
Ozanich
,
E.
,
Roch
,
M. A.
,
Gannot
,
S.
, and
Deledalle
,
C. A.
(
2019
). “
Machine learning in acoustics: Theory and applications
,”
J. Acoust. Soc. Am.
146
,
3590
3628
.
3.
Branch
,
T. A.
,
Matsuoka
,
K.
, and
Miyashita
,
T.
(
2004
). “
Evidence for increases in Antarctic blue whales based on Bayesian modelling
,”
Mar. Mammal Sci.
20
,
726
754
.
4.
Branch
,
T. A.
,
Stafford
,
K. M.
,
Palacios
,
D. M.
,
Allison
,
C.
,
Bannister
,
J. L.
, et al (
2007
). “
Past and present distribution, densities and movements of blue whales Balaenoptera musculus in the southern hemisphere and northern Indian Ocean
,”
Mammal Review
37
,
116
175
.
5.
Cerchio
,
S.
,
Willson
,
A.
,
Leroy
,
E. C.
,
Muirhead
,
C.
,
Al Harthi
,
S.
,
Baldwin
,
R.
,
Cholewiak
,
D.
,
Collins
,
T.
,
Minton
,
G.
,
Rasoloarijao
,
T.
,
Rogers
,
T. L.
, and
Willson
,
M. S.
(
2020
). “
A new blue whale song-type described for the Arabian Sea and Western Indian Ocean
,”
Endanger. Species Res.
43
,
495
515
.
6.
Cooke
,
J.
(
2019
).
Balaenoptera musculus
(errata version published in 2019). Technical Report. The IUCN Red List of Threatened Species, 2018 (International Union for Conservation of Nature and Natural Resources, Cambridge, UK).
7.
Cummings
,
W. C.
, and
Thompson
,
P. O.
(
1971
). “
Underwater sounds from the blue whale, Balaenoptera musculus
,”
J. Acoust. Soc. Am.
50
,
1193
1198
.
8.
Fournet
,
M. E.
,
Szabo
,
A.
, and
Mellinger
,
D. K.
(
2015
). “
Repertoire and classification of non-song calls in Southeast Alaskan humpback whales (Megaptera novaeangliae)
,”
J. Acoust. Soc. Am.
137
,
1
10
.
9.
Garland
,
E. C.
,
Castellote
,
M.
, and
Berchok
,
C. L.
(
2015
). “
Beluga whale (Delphinapterus leucas) vocalizations and call classification from the eastern Beaufort Sea population
,”
J. Acoust. Soc. Am.
137
,
3054
3067
.
10.
Gavrilov
,
A. N.
, and
McCauley
,
R.
(
2013
). “
Acoustic detection and long-term monitoring of pygmy blue whales over the continental slope in southwest Australia
,”
J. Acoust. Soc. Am.
134
,
2505
2513
.
11.
Hoffer
,
E.
, and
Ailon
,
N.
(
2015
). “
Deep metric learning using triplet network
,” in
Similarity-based Pattern Recognition
, edited by
A.
Feragen
,
M.
Pelillo
, and
M.
Loog
(
Springer
,
New York
).
12.
Huang
,
G.
,
Liu
,
Z.
, and
Weinberger
,
K. Q.
(
2016
). “
Densely connected convolutional networks
,” arXiv:1608.06993.
13.
Ibrahim
,
A. K.
,
Zhuang
,
H.
,
Cherubin
,
L. M.
,
Schärer-Umpierre
,
M. T.
, and
Erdol
,
N.
(
2018
). “
Automatic classification of grouper species by their sounds using deep neural networks
,”
J. Acoust. Soc. Am.
144
,
196
202
.
14.
Ichihara
,
T.
(
1966
). “
The pygmy blue whale, Balaenoptera musculus brevicauda, a new subspecies from the Antarctic
,” in Whales, Dolphins, and Porpoises, edited by
K. S.
Norris
(
University of California Press
,
Berkeley/LA, CA
)
79
111
.
15.
International Whaling Commission
(
2020
). “
IWC (2020) report of the scientific committee, virtual meetings
,” 11-24 Section 8.2.1 (International Whaling Commission, Cambridge, UK).
16.
Kirsebom
,
O. S.
,
Frazao
,
F.
,
Simard
,
Y.
,
Roy
,
N.
,
Matwin
,
S.
, and
Giard
,
S.
(
2020
). “
Performance of a deep neural network at detecting North Atlantic right whale upcalls
,”
J. Acoust. Soc. Am.
147
,
2636
2646
.
17.
Koch
,
G.
,
Zemel
,
R.
, and
Salakhutdinov
,
R.
(
2015
). “
Siamese neural networks for one-shot image recognition
,” in
Proceedings of the 32nd International Conference on Machine Learning
, July 7–9, Lille, France.
18.
Kowarski
,
K. A.
, and
Moors-Murphy
,
H.
(
2021
). “
A review of big data analysis methods for baleen whale passive acoustic monitoring
,”
Mar. Mamm. Sci.
37
,
652
673
.
19.
Küsel
,
E. T.
,
Mellinger
,
D. K.
,
Thomas
,
L.
,
Marques
,
T. A.
,
Moretti
,
D.
, and
Ward
,
J.
(
2011
). “
Cetacean population density estimation from single fixed sensors using passive acoustics
,”
J. Acoust. Soc. Am.
129
(
6
),
3610
3622
.
20.
Leroy
,
E. C.
,
Samaran
,
F.
,
Stafford
,
K. M.
,
Bonnel
,
J.
, and
Royer
,
J.-Y.
(
2018
). “
Broad-scale study of the seasonal and geographic occurrence of blue and fin whales in the Southern Indian Ocean
,”
Endanger. Species Res.
37
,
289
300
.
21.
Marques
,
T. A.
,
Thomas
,
L.
,
Martin
,
S. W.
,
Mellinger
,
D. K.
,
Ward
,
J. A.
,
Moretti
,
D. J.
,
Harris
,
D.
, and
Tyack
,
P. L.
(
2013
). “
Estimating animal population density using passive acoustics
,”
Biol. Rev. Cambr. Philosoph. Soc.
88
(
2
),
287
309
.
22.
McClain
,
C. R.
,
Balk
,
M. A.
,
Benfield
,
M. C.
,
Branch
,
T. A.
,
Chen
,
C.
,
Cosgrove
,
J.
,
Dove
,
A. D. M.
,
Helm
,
R. R.
,
Hochberg
,
F. G.
,
Gaskins
,
L. C.
,
Lee
,
F. B.
,
Marshall
,
A.
,
McMurray
,
S. E.
,
Schanche
,
C.
,
Stone
,
S. N.
, and
Thaler
,
A. D.
(
2015
). “
Sizing ocean giants: Patterns of intraspecific size variation in marine megafauna
,”
PeerJ
3
(
2
),
e715
.
23.
McDonald
,
M. A.
,
Hildebrand
,
J. A.
, and
Mesnick
,
S. L.
(
2006
). “
Biogeographic characterization of blue whale song worldwide: Using song to identify populations
,”
J. Cetacean Res. Manage.
8
,
55
65
.
24.
McDonald
,
M. A.
,
Hildebrand
,
J. A.
, and
Mesnick
,
S.
(
2009
). “
Worldwide decline in tonal frequencies of blue whale songs
,”
Endanger. Species Res.
9
,
13
21
.
25.
Mouy
,
X.
,
Bahoura
,
M.
, and
Simard
,
Y.
(
2009
). “
Automatic recognition of fin and blue whale calls for real-time monitoring in the St. Lawrence
,”
J. Acoust. Soc. Am.
126
,
2918
2928
.
26.
Royer
,
J.-Y.
(
2009
).
OHASISBIO—Hydroacoustic Observatory for the Seismicity and Biodiversity in the Indian Ocean
, Technical Report (University of Brest, Brest, France).
27.
Samaran
,
F.
,
Adam
,
O.
, and
Guinet
,
C.
(
2010a
). “
Detection range modeling of blue whale calls in Southwestern Indian Ocean
,”
Applied Acoustics
71
,
1099
1106
.
28.
Samaran
,
F.
,
Stafford
,
K. M.
,
Branch
,
T. A.
,
Gedamke
,
J.
,
Royer
,
J.-Y. P.
,
Dziak
,
R. P.
, and
Guinet
,
C.
(
2013
). “
Seasonal and geographic variation of southern blue whale subspecies in the Indian Ocean
,”
PLoS ONE
8
,
e71561
.
29.
Schroff
,
F.
,
Kalenichenko
,
D.
, and
Philbin
,
J.
(
2015
). “
FaceNet: A unified embedding for face recognition and clustering
,” in
Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition
, June 7–12, Boston, MA, pp.
815
823
.
30.
Shiu
,
Y.
,
Palmer
,
K. J.
,
Roch
,
M. A.
,
Fleishman
,
E.
,
Liu
,
X.
,
Nosal
,
E.-M.
,
Helble
,
T.
,
Cholewiak
,
D.
,
Gillespie
,
D.
, and
Klinck
,
H.
(
2020
). “
Deep neural networks for automated detection of marine mammal species
,”
Sci. Rep.
10
,
607
.
31.
Širović
,
A.
,
Hildebrand
,
J. A.
,
Wiggins
,
S. M.
, and
Thiele
,
D.
(
2009
). “
Blue and fin whale acoustic presence around Antarctica during 2003 and 2004
,”
Mar. Mamm. Sci.
25
,
125
136
.
32.
Socheleau
,
F.-X.
,
Leroy
,
E.
,
Carvallo Pecci
,
A.
,
Samaran
,
F.
,
Bonnel
,
J.
, and
Royer
,
J.-Y.
(
2015
). “
Automated detection of Antarctic blue whale calls
,”
J. Acoust. Soc. Am.
138
,
3105
3117
.
33.
Stafford
,
K. M.
,
Bohnenstiehl
,
D. R.
,
Tolstoy
,
M.
,
Chapp
,
E.
,
Mellinger
,
D. K.
, and
Moore
,
S. E.
(
2004
). “
Antarctic-type blue whale calls recorded at low latitudes in the Indian and eastern Pacific Oceans
,”
Deep Sea Res. Part I Oceanogr. Res. Pap.
51
,
1337
1346
.
34.
Stafford
,
K. M.
,
Chapp
,
E.
,
Bohnenstiel
,
D. R.
, and
Tolstoy
,
M.
(
2011
). “
Seasonal detection of three types of ‘pygmy’ blue whale calls in the Indian Ocean
,”
Mar. Mamm. Sci.
27
,
828
840
.
35.
Stafford
,
K. M.
,
Fox
,
C. G.
, and
Clark
,
D. S.
(
1998
). “
Long-range acoustic detection and localization of blue whale calls in the northeast Pacific Ocean
,”
J. Acoust. Soc. Am.
104
,
3616
3625
.
36.
Torterotot
,
M.
,
Royer
,
J.-Y.
, and
Samaran
,
F.
(
2019
). “
Detection strategy for long-term acoustic monitoring of blue whale stereotyped and non-stereotyped calls in the Southern Indian Ocean
,” in
Proceedings of OCEANS 2019—Marseille
, June 17–20, Marseille, France.
37.
Torterotot
,
M.
,
Samaran
,
F.
,
Stafford
,
K. M.
, and
Royer
,
J.-Y.
(
2020
). “
Distribution of blue whale populations in the Southern Indian Ocean based on a decade of acoustic monitoring
,”
Deep-Sea Res. Part II Top. Stud. Oceanogr.
179
,
104874
.
38.
van der Maaten
,
L.
, and
Hinton
,
G.
(
2008
). “
Visualizing Data using t-SNE
,”
J. Mach. Learn. Res.
9
,
2579
2605
.
39.
Yang
,
W.
,
Luo
,
W.
, and
Zhang
,
Y.
(
2020
). “
Classification of odontocete echolocation clicks using convolutional neural network
,”
J. Acoust. Soc. Am.
147
(
1
),
49
55
.
40.
Zhong
,
M.
,
Castellote
,
M.
,
Dodhia
,
R.
,
Lavista Ferres
,
J.
,
Keogh
,
M.
, and
Brewer
,
A.
(
2020
). “
Beluga whale acoustic signal classification using deep learning neural network models
,”
J. Acoust. Soc. Am.
147
(
3
),
1834
1841
.
You do not currently have access to this content.