Logo PTI Logo FedCSIS

Proceedings of the 18th Conference on Computer Science and Intelligence Systems

Annals of Computer Science and Information Systems, Volume 35

Cybersecurity Threat Detection in the Behavior of IoT Devices: Analysis of Data Mining Competition Results

, , , , , , , , , ,

DOI: http://dx.doi.org/10.15439/2023F3089

Citation: Proceedings of the 18th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 35, pages 1289–1293 ()

Full text

Abstract. The paper discusses a data science competition centered around the development of an anomaly detection system for IoT devices. The competition utilized an unique environment that allowed for the operation and monitoring of real IoT devices, including scheduling of attacks on these devices. The environment was used to collect the data, which included both normal and attack-induced behavior of IoT devices. The paper presents the background of the competition, top models submitted, and a discussion of the competition results. The paper includes discussion of restrictions related to using synthetic attack data as an input for creating anomaly-detecting systems.

References

  1. Creech G. and Hu J., “Generation of a new IDS test dataset: Time to retire the KDD collection,” in 2013 IEEE Wireless Communications and Networking Conference (WCNC), Apr. 2013, pp. 4487–4492, ISSN: 1558-2612.
  2. Stratosphere, “Stratosphere laboratory datasets,” 2020, Retrieved March 15, 2021, from https://www.stratosphereips.org/datasets-overview.
  3. CAIDA, “Center of applied internet data analysis,” 1998-2013, Retrieved March 16, 2021, from https://www.caida.org/catalog/datasets/completed-datasets/.
  4. Koroniotis N., Moustafa N., Sitnikova E., and Turnbull B., “Towards the development of realistic botnet dataset in the internet of things for network forensic analytics: Bot-iot dataset,” 2018.
  5. Sangster B., O’Connor T. J., Cook T., Fanelli R., Dean E., Adams W. J., Morrell C., and Conti G., “Toward instrumenting network warfare competitions to generate labeled datasets,” in Proceedings of the 2nd Conference on Cyber Security Experimentation and Test, USA, 2009, CSET’09, p. 9, USENIX Association.
  6. MIT Lincoln Laboratory MIT, “Mit lincoln laboratory - darpa datasets,” 1998-1999, Retrieved March 16, 2021, from https://www.ll.mit.edu/r-d/datasets.
  7. Stolfo S.J., Fan W., Lee W., Prodromidis A., and Chan P.K., “Cost-based modeling for fraud and intrusion detection: results from the jam project,” in Proceedings DARPA Information Survivability Conference and Exposition. DISCEX’00, 2000, vol. 2, pp. 130–144 vol.2.
  8. Ullah I. and Mahmoud Q. H., “A technique for generating a botnet dataset for anomalous activity detection in iot networks,” in 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2020, pp. 134–140.
  9. Shiravi A., Shiravi H., Tavallaee M., and Ghorbani A. A., “Toward developing a systematic approach to generate benchmark datasets for intrusion detection,” Computers & Security, vol. 31, no. 3, pp. 357–374, 2012.
  10. Kyoto University, “Traffic data from kyoto university’s honeypots,” 2015, Retrieved March 17, 2021, from https://www.takakura.com/Kyoto_data/.
  11. Tavallaee M., Bagheri E., Lu W., and Ghorbani A. A., “A detailed analysis of the kdd cup 99 data set,” in 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, Jul 2009, p. 1–6.
  12. Wawrowski Ł., Michalak M., Białas A., Kurianowicz R., Sikora M., Uchroński, and Kajzer A., “Detecting anomalies and attacks in network traffic monitoring with classification methods and xai-based explainability,” Procedia Comput. Sci., vol. 192, no. C, pp. 2259–2268, jan 2021.
  13. Wawrowski Ł, Białas A., Kajzer A., Kozłowski A., Kurianowicz R., Sikora M., Szymańska-Kwiecień A, Uchroński M., M. Białczak, Olejnik M., and Michalak M., “Anomaly detection module for network traffic monitoring in public institutions,” Sensors, vol. 23, no. 6, 2023.
  14. Moustafa N., Ahmed M., and Ahmed S., “Data analytics-enabled intrusion detection: Evaluations of ton_iot linux datasets,” in 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), 2020, pp. 727–735.
  15. Sperotto A., Sadre R., van Vliet F., and Pras A., “A labeled data set for flow-based intrusion detection,” in IP Operations and Management, Giorgio Nunzi et al., Eds., Netherlands, Oct. 2009, Lecture Notes in Computer Science, pp. 39–50, Springer, 9th IEEE International Workshop on IP Operations and Management, IPOM 2009, IPOM ; Conference date: 29-10-2009 Through 30-10-2009.
  16. Moustafa N. and Slay J., “Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set),” in 2015 Military Communications and Information Systems Conference (MilCIS), 2015, pp. 1–6.
  17. Yang S., Kurose J., and Levine B., “Disambiguation of residential wired and wireless access in a forensic setting,” in 2013 Proceedings IEEE INFOCOM, 04 2013, pp. 360–364.
  18. Maede Zolanvari, “Wustl-iiot-2021 dataset for iiot cybersecurity research,” https://www.cse.wustl.edu/~jain/iiot2/index.html, 2021.
  19. Ferrag M.A., Friha O., Hamouda D., Maglaras L., and Janicke H., “Edge-iiotset: A new comprehensive realistic cyber security dataset of iot and iiot applications: Centralized and federated learning,” https://dx.doi.org/10.21227/mbc1-1h68, 2022.
  20. Garcia S., Parmisano A., and Erquiaga M. J., “IoT-23: A labeled dataset with malicious and benign IoT network traffic,” Jan. 2020.
  21. Janusz A. and Ślęzak D., “KnowledgePit Meets BrightBox: A Step Toward Insightful Investigation of the Results of Data Science Competitions,” in Proceedings of the 17th Conference on Computer Science and Intelligence Systems, FedCSIS 2022, Sofia, Bulgaria, September 4-7, 2022, 2022, vol. 30 of Annals of Computer Science and Information Systems, pp. 393–398.
  22. Andrzej Janusz, Daniel Kałuża, Agnieszka Chadzyńska-Krasowska, Bartek Konarski, Joel Holland, and Dominik Ślęzak, “IEEE BigData 2019 Cup: Suspicious Network Event Recognition,” in 2019 IEEE International Conference on Big Data (IEEE BigData), Los Angeles, CA, USA, December 9-12, 2019. 2019, pp. 5881–5887, IEEE.
  23. Andrzej Janusz, Mateusz Przyborowski, Piotr Biczyk, and Dominik Ślęzak, “Network Device Workload Prediction: A Data Mining Challenge at Knowledge Pit,” in Proceedings of the 2020 Federated Conference on Computer Science and Information Systems, FedCSIS 2020, Sofia, Bulgaria, September 6-9, 2020, 2020, vol. 21 of Annals of Computer Science and Information Systems, pp. 77–80.
  24. Andrzej Janusz, Marek Sikora, Łukasz Wróbel, Sebastian Stawicki, Marek Grzegorowski, Piotr Wojtas, and Dominik Ślęzak, “Mining data from coal mines: Ijcrs’15 data challenge,” in Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing: 15th International Conference, RSFDGrC 2015, Tianjin, China, November 20-23, 2015, Proceedings. Springer, 2015, pp. 429–438.
  25. Adamczyk B., Brzȩczek M., Michalak M., Kostorz I., WawrowskiŁ., Hermansa M., Czerwiński M., and Jamiołkowski A., “Dataset generation framework for evaluation of iot linux host–based intrusion detection systems,” in 2022 IEEE International Conference on Big Data (Big Data), 2022, pp. 6179–6187.
  26. “MITRE ATT&CK® Adversarial Tactics, Techniques, and Common Knowledge,” https://attack.mitre.org/versions/v13/, 2023.