Internet of Thing (IoT) environment deals with large amounts of data from sensors and other things. The main problem with Internet of Things based applications right now, is security threats with respect to the large amount of data in transit and at rest. Theft or snooping of information or fault in IoT device can cause the data to have some anomalies. We can scan the network to get details of type of packets being transferred, type of protocols being used, source and destination, etc. These data can be analysed to get meaningful inferences and detect intrusion in a network. Most of the work done in this field is single system, single core system based application. This paper focuses on proposing a network anomaly detection mechanism using Machine Learning method, Random Forest in a distributed setup (cluster of machines) using SPARK. Such a mechanism should detect anomalies in large IoT data in an accurate and efficient manner on an IoT dataset called UNSW- NB15.

Topics
Information technology, Machine learning
1.
Fahim
,
M.
 and
Sillitti
,
A.
,
2019
.
Anomaly detection, analysis and prediction techniques in iot environment: A systematic literature review
.
IEEE Access
,
7
, pp.
81664
–
81681
.
https://doi.org/10.1109/ACCESS.2019.2921912
Google Scholar
Crossref
Search ADS
 
2.
Butun
,
I.
,
Morgera
,
S.D.
 and
Sankar
,
R.
,
2013
.
A survey of intrusion detection systems in wireless sensor networks
.
IEEE communications surveys & tutorials
,
16
(
1
), pp.
266
–
282
.
https://doi.org/10.1109/SURV.2013.050113.00191
Google Scholar
Crossref
Search ADS
 
3.
Hasan
,
M.
,
Islam
,
M.M.
,
Zarif
,
M.I.I.
 and
Hashem
,
M.M.A.
,
2019
.
Attack and anomaly detection in IoT sensors in IoT sites using machine learning approaches
.
Internet of Things
,
7
, p.
100059
.
https://doi.org/10.1016/j.iot.2019.100059
Google Scholar
Crossref
Search ADS
 
4.
Abdallah
,
A.
,
Maarof
,
M.A.
 and
Zainal
,
A.
,
2016
.
Fraud detection system: A survey
.
Journal of Network and Computer Applications
,
68
, pp.
90
–
113
.
https://doi.org/10.1016/j.jnca.2016.04.007
Google Scholar
Crossref
Search ADS
 
5.
Ahmed
,
M.
,
Mahmood
,
A.N.
 and
Hu
,
J.
,
2016
.
A survey of network anomaly detection techniques
.
Journal of Network and Computer Applications
,
60
, pp.
19
–
31
.
https://doi.org/10.1016/j.jnca.2015.11.016
Google Scholar
Crossref
Search ADS
 
6.
Jakkula
,
V.R.
 and
Cook
,
D.J.
,
2011
.
Detecting Anomalous Sensor Events in Smart Home Data for Enhancing the Living Experience
.
Artificial intelligence and smarter living
,
11
(
201
), p.
1
.
Google Scholar
 
7.
Mahmoud
,
S.M.
,
Lotfi
,
A.
 and
Langensiepen
,
C.
,
2012
, June.
User activities outlier detection system using principal component analysis and fuzzy rule-based system
. In
Proceedings of the 5th International Conference on PErvasive Technologies Related to Assistive Environments
(pp.
1
–
8
).
Google Scholar
Crossref
Search ADS
 
8.
Yin
,
J.
,
Yang
,
Q.
 and
Pan
,
J.J.
,
2008
.
Sensor-based abnormal human-activity detection
.
IEEE Transactions on Knowledge and Data Engineering
,
20
(
8
), pp.
1082
–
1090
.
https://doi.org/10.1109/TKDE.2007.1042
Google Scholar
Crossref
Search ADS
 
9.
Hüllermeier
,
E.
,
2015
.
Does machine learning need fuzzy logic?
.
Fuzzy Sets and Systems
,
281
, pp.
292
–
299
.
https://doi.org/10.1016/j.fss.2015.09.001
Google Scholar
Crossref
Search ADS
 
10.
Chandola
,
V.
,
Banerjee
,
A.
 and
Kumar
,
V.
,
2010
.
Anomaly detection for discrete sequences: A survey
.
IEEE transactions on knowledge and data engineering
,
24
(
5
), pp.
823
–
839
.
https://doi.org/10.1109/TKDE.2010.235
Google Scholar
Crossref
Search ADS
 
11.
Garcia-Teodoro
,
P.
,
Diaz-Verdejo
,
J.
,
Maciá-Fernández
,
G.
 and
Vázquez
,
E.
,
2009
.
Anomaly-based network intrusion detection: Techniques, systems and challenges
.
computers & security
,
28
(
1-2
), pp.
18
–
28
.
https://doi.org/10.1016/j.cose.2008.08.003
Google Scholar
Crossref
Search ADS
 
12.
Mohamudally
,
N.
 and
Peermamode-Mohaboob
,
M.
,
2018
.
Building an anomaly detection engine (ADE) for Iot smart applications
.
Procedia computer science
,
134
, pp.
10
–
17
.
https://doi.org/10.1016/j.procs.2018.07.138
Google Scholar
Crossref
Search ADS
 
13.
Ding
,
X.
,
Li
,
Y.
,
Belatreche
,
A.
 and
Maguire
,
L.P.
,
2014
.
Novelty detection using level set methods
.
IEEE transactions on neural networks and learning systems
,
26
(
3
), pp.
576
–
588
.
https://doi.org/10.1109/TNNLS.2014.2320293
Google Scholar
Crossref
Search ADS
 
14.
Song
,
X.
,
Wu
,
M.
,
Jermaine
,
C.
 and
Ranka
,
S.
,
2007
.
Conditional anomaly detection
.
IEEE Transactions on knowledge and Data Engineering
,
19
(
5
), pp.
631
–
645
.
https://doi.org/10.1109/TKDE.2007.1009
Google Scholar
Crossref
Search ADS
 
15.
Mahmoud
,
S.M.
,
Lotfi
,
A.
 and
Langensiepen
,
C.
,
2011
, May.
Abnormal behaviours identification for an elder's life activities using dissimilarity measurements
. In
Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments
(pp.
1
–
5
).
Google Scholar
Crossref
Search ADS
 
16.
Cardinaux
,
F.
,
Brownsell
,
S.
,
Hawley
,
M.
 and
Bradley
,
D.
,
2008
, September. Modelling of behavioural patterns for abnormality detection in the context of lifestyle reassurance. In
Iberoamerican Congress on Pattern Recognition
(pp.
243
–
251
).
Springer
,
Berlin, Heidelberg
.
Google Scholar
 
17.
Moustafa
,
N.
,
Turnbull
,
B.
 and
Choo
,
K.K.R.
,
2018
.
An ensemble intrusion detection technique based on proposed statistical flow features for protecting network traffic of internet of things
.
IEEE Internet of Things Journal
,
6
(
3
), pp.
4815
–
4830
.
https://doi.org/10.1109/JIOT.2018.2871719
Google Scholar
Crossref
Search ADS
 
This content is only available via PDF.
© 2022 Author(s).
2022
Author(s)
You do not currently have access to this content.