A large amount of social media data hosted on platforms like Twitter, Instagram, Facebook, etc. are event-based and hold a substantial amount of real-world data. Event-based information can appear on any social media site in the form of news items, images, videos, audio clips, status updates, etc. The task of event detection refers to identifying data relevant to an event and the classification of this relevant data to different event types. Traditional social media event detection techniques focused mainly on a single modality as the data shared were mostly homogenous. However, the current social media data is multimodal and includes text, images, audio, and video clips, and geolocations. Multimodal event detection techniques are essential for handling such heterogeneous data. Among all the social media sites Twitter is the most popular as users share event-related short messages and photos in real-time generating several thousands of tweets very frequently. In this paper, we focus on providing a comprehensive survey of event detection from social media, especially from the widely used platform, Twitter. The survey focuses mainly on research done on event detection using the two main modalities single and multimodality. At the end of the paper, we discuss the relevance of multimodal event detection from social media data which currently spans multiple dimensions.

1.
Sitaram
Asur
and
Bernardo A.
Huberman
, “
Predicting the future with social media
”,
Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology
, vol.
01
, WI-IAT’, pp.
492
499
, Washington, DC, USA,
2010
.
2.
Lei
Tang
and
Huan
Liu
, “
Toward predicting collective behavior via social dimension extraction
”,
IEEE Intelligent Systems
, vol.
25
no.
4
, pp.
19
25
, July
2010
.
3.
James
Allan
,
Jaime G
Carbonell
,
George
Doddington
,
Jonathan
Yamron
, and
Yiming
Yang
, “
Topic detection and tracking pilot study: Final report
”,
Proceedings DARPA Broadcast News Transcription and Understanding Workshop
, pp.
194
218
, USA,
1998
.
4.
Yiming
Yang
,
Tom
Pierce
, and
Jaime
Carbonell
, “
A study of retrospective and on-line event detection
”,
Proceedings 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
, pp.
28
36
, New York, NY, USA,
1998
.
5.
Raphaël
Troncy
,
Bartosz
Malocha
, and
André TS
Fialho
, “
Linking events with media
”,
Proceedings 6th ACM International Conference on Semantic Systems
, Article-42, pp.
1
4
, New York, NY, USA,
2010
.
6.
Hila
Becker
,
Dan
Iter
,
Mor
Naaman
, and
Luis
Gravano
, “
Identifying content for planned events across social media sites
”,
Proceedings Fifth ACM International Conference on Web Search and Data Mining, WSDM
, pp.
533
542
, New York, NY, USA,
2012
.
7.
Rui
Li
,
Kin Hou
Lei
,
Ravi
Khadiwala
, and
Kevin Chen-Chuan
Chang
, “
Tedas, A Twitter-based event detection, and analysis system
”,
Proceedings 28th IEEE International Conference on Data Engineering
, pp.
1273
1276
, USA,
2012
.
8.
James
Benhardus
and
Jugal
Kalita
, “
Streaming trend detection in Twitter
”,
International Journal of Web-Based Communities
, vol.
9
no.
1
, pp.
122
139
,
2013
.
9.
Sven
Rill
,
Dirk
Reinel
,
Jörg
Scheidt
, and
Roberto V Zicari.
Politwi
, “
Early detection of emerging political topics on Twitter and the impact on concept-level sentiment analysis
”,
Knowledge-Based Systems
, vol.
69
no.
1
, pp.
24
33
,
2014
.
10.
Saša
Petrović
,
Miles
Osborne
, and
Victor
Lavrenko
, “
Streaming first story detection with application to Twitter
”,
Proceedings 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics (ACL),
pp.
181
189
, Stroudsburg, PA, USA,
2010
.
11.
Muhammad Aslam
Jarwar
,
Rabeeh Ayaz
Abbasi
,
Mubashar
Mushtaq
,
Onaiza
Maqbool
,
Naif R
Aljohani
,
Ali
Daud
,
Jalal S
Alowibdi
,
JR
Cano
, S García, and
Ilyoung
Chong
, “
CommuniMents: A framework for detecting community-based sentiments for events
”,
International Journal on Semantic Web and Information Systems (IJSWIS),
vol.
13
, no.
2
, pp.
87
108
,
2017
.
12.
James
Benhardus
and
Jugal
Kalita
, “
Streaming trend detection in Twitter
”,
International Journal of Web-Based Communities
, vol.
9
, No.
1
pp.
122
139
,
2013
.
13.
D.
Boyd
,
S.
Golder
, and
G.
Lotan
, “
Tweet, tweet, retweet: Conversational aspects of retweeting on Twitter
”,
IEEE 43rd Hawaii International Conference on System Sciences
, pp.
1
10
, USA, Jan
2010
.
14.
Mahmud
Hasan
,
Mehmet A
Orgun
, and
Rolf
Schwitter
,” A survey on real-time event detection from the Twitter data stream“,
Journal of Information Science
,
2017
.
15.
Anuradha
Goswami
and
Ajey
Kumar
,” A survey of event detection techniques in online social networks”,
Social Network Analysis and Mining
, vol.
6
, no.
1
:
107
, Nov
2016
.
16.
Farzindar
Atefeh
and
Wael
Khreich
, “
A survey of techniques for event detection in Twitter
”,
Comput. Intell.
, ol.
31
.no.
1
. pp.
132
164
, February
2015
.
17.
Tobias
Schreck
,
Daniel
Kein
, “
Visual Analysis of Social Media Data
”,
IEEE Computer Society
, vol.
46
, no.
5
, May
2013
.
18.
Yingcai
Wu
,
Nan
Cao
,
David
Gotz
,
Yap-Peng
Tan
, and
Daniel A.
Kei
, “
A Survey on Visual Analytics of Social Media Data
”,
IEEE Transactions on Multimedia
, vol.
18
, no.
11
,
2016
.
19.
Huma
Parveen
,
Shikha
Pandey
, “
Sentiment analysis on Twitter Data-set using Naive Bayes algorithm
”,
Proceedings IEEE 2ⁿᵈ International Conference on Applied and Theoretical Computing and Communication Technology
”, pp.
416
419
, Bangalore, India,
2016
.
20.
Vinayakumar
R
,
Mamoun
Alazab
,
Alireza
Jolfaei
,
Soman
K.P
,
Prabaharan
Poornachandran
, “
Ransomware Triage Using Deep Learning: Twitter as a Case Study
”,
Proceedings IEEE Conference on Cybersecurity and Cyber-forensics,
pp.
67
73
, Melbourne, Australia,
2019
.
21.
J.
weng
and
B.S.
Lee
, “
Event detection in Twitter
”,
Proceedings Fifth International AAAI Conference on Weblogs and Social Media
, vol.
5
no.
1
,
2011
.
22.
A.H.
Hossny
and
L.
Mitchell
, “
Event Detection in Twitter: A Keyword Volume Approach
”,
Proceedings IEEE International Conference on Data Mining Workshop (ICDMW)
, pp.
1200
1208
, Singapore,
2018
.
23.
Sun
,
H.
,
Wang
,
Z.
,
Wang
,
J.
,
Huang
,
Z.
,
Carrington
,
N.
,
Liao
,
J
, “
Data-driven power outage detection by social sensors
”,
IEEE Transactions on Smart Grid
, vol
7
, no.
5
, pp.
2516
2524
,
2016
24.
Sajeev
Udayakumar
,
Damith Chamalke
Senadeera
,
Selvaraj
Yamunarani
,
NaJin
Cheon
, “Demographics Analysis of Twitter Users who Tweeted on Psychological Articles and Tweets Analysis”,
Procedia Computer Science
, vol.
144
, pp.
96
104
,
Elsevier
,
2018
.
25.
Anantharam
,
P.
,
Barnaghi
,
P.
,
Thirunarayan
,
K.
,
Sheth
,
A
, “
Extracting city traffic events from social streams
”,
ACM Transaction Intelligent System Technology (TIST)
Vol.
6
no.
4
, pp.
43
,
2015
.
26.
Zafar
Saee
,
Rabeeh Ayaz
Abbas
,
Imran
Razza
,
Onaiza
Maqboo
,
Abida
Sada
,
Guandong
Xu
, “Enhanced Heartbeat Graph for emerging event detection on Twitter using Time Series Networks”,
Expert Systems with Application
, vol.
136
, pp.
115
132
,
Elsevier
,
2019
.
27.
Aarzoo
Dhiman
and
Durga
Toshnewal
, “
An Approximate Model for Event Detection from Twitter Data
”,
IEEE Access
, vol.
8
, pp.
122168
122184
,
2020
.
28.
Pang
,
J.
,
Jia
,
F.
,
Zhang
,
C.
,
Zhang
,
W.
,
Huang
,
Q.
,
Yin
,
B.
, “
Unsupervised web topic detection using a ranked clustering-like pattern across similarity cascades
”,
IEEE Transactions on Multimedia
, vol.
17
no.
6
, pp.
843
853
,
2015
.
29.
Yasmeen
George
,
Shanika
Karunasekera
,
Aaron
Harwood
, and
Kwan Hui
Lim
, “
Spatio-temporal Event Detection using Poisson Model and Quad-tree on Geotagged Social Media
”,
Proceedings IEEE International Conference on Big Data
, pp.
2247
225
, Los Angeles, CA, USA
2019
.
30.
Chuang
Gan
, et.al., “
DevNet: A Deep Event Network for Multimedia Event Detection and Evidence Recounting
”,
Proceedings IEEE Conference on Computer vision and Pattern Recognition (CVPR)
, Boston, USA,
2015
, pp.
2568
2577
.
31.
Zhang
,
C.
,
Xiao
,
W.
,
Shyu
,
M.-L.
,
Peng
,
Q
, “
Integration of visual temporal information and textual distribution information for news web video event mining
”,
IEEE Transactions on Human-Machine System
, vol.
46
, No.
1
, pp.
124
135
,
2016
.
32.
Mahdyar
Ravanbakhsh
, et.al., “
Abnormal Event Detection in videos using generative adversarial nets
”,
Proceedings IEEE International Conference on Image Processing (ICIP),
China,
2017
, pp.
1577
1581
.
33.
Takamu
Kaneko
,
Keiji
Yanai
, “
Event photo mining from Twitter using keyword bursts and image clustering
”,
Neurocomputing
, vol.
172
, pp.
142
158
,
2015
.
34.
Zaheer
,
M.Z.
,
Mahmood
,
A.
,
Shin
,
H.
, and
Lee
,
S.I.
, “
A Self-Reasoning Framework for Anomaly Detection Using Video-Level Labels
”,
IEEE Signal Processing Letters
,
27
, pp.
1705
1709
,
2020
.
35.
Deepti
Deshwal
,
Pardeep
Sangwan
and
Divya
Kumar
, “
Feature extraction methods in language identification, A survey
”,
Wireless Personal Communication
,
Springer
, Vol.
107
, No.
4
, pp.
2071
2103
,
2019
36.
Annamaria
Mesaros
,
Toni
Heittola
,
Antti
Eronen
, and
Tuomas
Virtanen
, “
Acoustic Event Detection in Real-Life Recordings
”,
Proceedings 18ᵗʰ IEEE European Signal Processing Conference (EUSIPCO),
Denmark,
2010
, pp.
1267
1271
.
37.
Xianjun
Xia
, et.al., “
Multi-Task Learning for Acoustic Event Detection Using Event and Frame Position Information
”,
IEEE Transactions on Multimedia
, Vol.
22
, No.
3
, pp.
569
578
,
2020
.
38.
Kay L
O'Halloran
,
Sabine
Tan
,
Duc-Son
Pham
,
John
Bateman
,
Andrew Vande
Moere
, “
A digital mixed methods research design: Integrating multimodal analysis with data mining and information visualization for big data analytics
”,
Journal of Mixed Methods Research
, Vol.
12
, no.
1
,
2018
, pp.
11
30
.
39.
Imran
,
M.
,
Ofli
,
F.
,
Caragea
,
D.
and
Torralba
,
A.
, “
Using AI and Social Media Multimodal Content for Disaster Response and Management: Opportunities, Challenges, and Future Directions
”,
Information Processing & Management
, vol.
57
, no.
5
,
2020
.
40.
Fei
Wu
,
Hong
Zhang
,
Yueting
Zhuang
, “
Learning semantic correlations for cross-media retrieval
”,
Proceedings IEEE International Conference on Image Processing,
16-19 September 2007, San Antonio, Texas, pp.
1465
1468
.
41.
Nikhil
Rasiwasia
,
Jose Costa
Pereira
,
Emanuele
Coviello
,
Gabriel
Doyle
,
Gert RG
Lanckriet
,
Roger
Levy
,
Nuno
Vasconcelos
, “
A new approach to cross-modal multimedia retrieval
”,
Proceedings 18th ACM International Conference on Multimedia
, 25-29 October 2010, Firenze, Italy, pp.
251
260
.
42.
Jose Costa
Pereira
,
Emanuele
Coviello
,
Gabriel
Doyle
,
Nikhil
Rasiwasia
,
Gert RG
Lanckriet
,
Roger
Levy
,
Nuno
Vasconcelos
, “
On the role of correlation and abstraction in cross-modal multimedia retrieval
”,
IEEE Transactions on Pattern Analysis and Machine Intelligence,
vol.
36
, no.
3
, pp.
521
535
,
2014
.
43.
Shotaro
Akaho
, “
A kernel method for canonical correlation analysis
”, arXiv preprint cs/0609071,
2006
.
44.
Galen
Andrew
,
Raman
Arora
,
Jeff
Bilmes
,
Karen
Livescu
, “
Deep canonical correlation analysis
”,
Proceedings International Conference on Machine Learning,
June 16 – 21, Atlanta, USA,
2013
, pp.
1247
1255
.
45.
Kah
Seng
,
Li Minn
Ang
,
Chien
Ooi
, “
A combined rule-based and machine learning audio-visual emotion recognition approach
”,
IEEE Transactions on Affective Computing,
vol.
9
, no.
1
, pp.
99
,
2018
.
46.
Yun
Ma
,
Qing
Li
,
Zhenguo
Yang
,
Zheng
Lu
,
Haiwei
Pan
,
Antoni B.
Chan
, “
An SVD-based multimodal clustering method for social event detection
”,
Proceedings IEEE International Conference on Data Engineering Workshops
, April 13 -17,
2015
, Seoul, South Korea, pp.
202
209
.
47.
Sunil Kumar
Gupta
,
Dinh
Phung
,
Brett
Adams
,
Svetha
Venkatesh
, “
Regularized nonnegative shared subspace learning
”,
Data Mining and Knowledge Discovery
, Vol.
26
, No.
1
, pp.
57
97
,
2013
.
48.
Daniel
Ramage
,
Paul
Heymann
,
Christopher D
Manning
,
Hector
Garcia-Molina
, “
Clustering the tagged web
”,
Proceedings Second ACM International Conference on Web Search and Data Mining
, 9-12 February 2009, Barcelona Spain, pp.
54
63
.
49.
Jitao
Sang
,
Changsheng
Xu
, “
Right buddy makes the difference: An early exploration of social relation analysis in multimedia applications
”,
Proceedings 20th ACM International Conference on Multimedia
, October 29- November 2nd, 2012, Nara, Japan, pp.
19
28
.
50.
Jingwen
Bian
,
Yang
Yang
,
Hanwang
Zhang
,
Tat-Seng
Chua
, “
Multimedia summarization for social events in microblog stream
”,
IEEE Transactions on Multimedia
, vol.
17
, No.
2
, pp.
216
228
,
2015
.
51.
Hongyun
Cai
,
Yang
Yang
,
Xuefei
Li
,
Zi
Huang
, “
What are popular: exploring Twitter features for event detection, tracking, and visualization
”,
Proceedings 23rd ACM International Conference on Multimedia
, 26 - 30 October 2015, Brisbane Australia, pp.
89
98
.
52.
Nitish
Srivastava
,
Ruslan R.
Salakhutdinov
, “
Multimodal learning with deep Boltzmann machines
”,
Advances in Neural Information Processing Systems
,
2012
, pp.
2222
2230
.
53.
Jiquan
Ngiam
,
Aditya
Khosla
,
Mingyu
Kim
,
Juhan
Nam
,
Honglak
Lee
,
Andrew Y.
Ng
, “
Multimodal deep learning
”,
Proceedings 28ᵗʰ International Conference on Machine Learning (ICML-11)
, June 28-2, 2011, Bellevue, Washington, USA, pp.
689
696
.
54.
Weiran
Wang
,
Raman
Arora
,
Karen
Livescu
,
Jeff
Bilmes
, “
On deep Multiview representation learning
”,
Proceedings International Conference on Machine Learning,
July 6 – 11,
2015
, Lille, France, pp.
1083
1092
.
55.
Sungeun
Hong
,
Woobin
Im
,
Hyun S.
Yang
, “
Content-based video-music retrieval using soft intra-modal structure constraint
”,
Proceedings ACM International Conference on Multimedia
, June 11-14, 2018, Yokohama, Japan, pp.
353
361
.
56.
Y.
Wei
,
Y.
Zhao
,
C.
Lu
,
S.
Wei
,
L.
Liu
,
Z.
Zhu
,
S.
Yan
, “
Cross-modal retrieval with CNN visual features: A new baseline
”,
IEEE Transactions on Cybernetics
, Vol.
47
, No.
2
, pp.
449
460
,
2016
.
57.
Gao
,
W.
,
Li
,
L.
,
Zhu
,
X.
and
Wang
,
Y.
, “
Detecting Disaster-Related Tweets Via Multimodal Adversarial Neural Network
”,
IEEE MultiMedia
, vol.
27
, no.
4
, pp.
28
37
,
2020
.
58.
Singhal
,
S.
,
Kabra
,
A.
,
Sharma
,
M.
,
Shah
,
R.R.
,
Chakraborty
,
T.
and
Kumaraguru
,
P.
, “
Spotfake+: A multimodal framework for fake news detection via transfer learning
”,
Proceedings AAAI Conference on Artificial Intelligence
, February 7-12,
2020
, New York, USA, Vol.
34
, No.
10
, pp.
13915
13916
.
This content is only available via PDF.
You do not currently have access to this content.