Logo PTI Logo FedCSIS

Communication Papers of the 21st Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 49

Data collection, aggregation, and analysis in the Internet of Everything environment – a case study

, , , ,

DOI: http://dx.doi.org/10.15439/2026F2726

Citation: Andrzej Paszkiewicz, , , ,

Full text

Abstract. This article presents a practical approach to topics related to the Internet of Everything (IoE) environment, specifically describing the main mechanisms and challenges associated with the collection, aggregation, and analysis of data from multiple sources. In distributed IoE environments, data is transmitted using various communication protocols and at different frequencies. Both network packets and logs transmitted by IoE devices are specific to a given device type and often vary depending on the manufacturer. In large-scale IoE environments, the very large volume of data transmitted in various formats requires the implementation of effective mechanisms to aggregate it in near real time, especially when considering its analysis in the context of cybersecurity. This article presents a general operational diagram of a system enabling the analysis of data from a distributed IoE environment, incorporating mechanisms for data acquisition from multiple sources, caching, scaling, and multi-stage data analysis that includes machine learning and artificial intelligence support.

References

  1. E. Siow, T. Tiropanis, W. Hall, “Analytics for the Internet of Things: A Survey”, ACM Computing Surveys, vol. 51, no. 4, pp. 1–36, 2018. DOI: 10.1145/3204947.
  2. M. Ge, H. Bangui, B. Buhnova, “Big Data for Internet of Things: A Survey”, Future Generation Computer Systems, vol. 87, pp. 601–614, 2018. DOI: 10.1016/j.future.2018.04.053.
  3. Y. Hajjaji, W. Boulila, I. R. Farah, I. Romdhani, A. Hussain, “Big Data and IoT-Based Applications in Smart Environments: A Systematic Review”, Computer Science Review, vol. 39, 100318, 2021. DOI: 10.1016/j.cosrev.2020.100318.
  4. D. Fawzy, S. M. Moussa, N. L. Badr, “The Internet of Things and Architectures of Big Data Analytics: Challenges of Intersection at Different Domains”, IEEE Access, vol. 10, pp. 4969–4992, 2022. DOI: 10.1109/ACCESS.2022.3140409.
  5. A. Naghib, N. Jafari Navimipour, M. Hosseinzadeh, A. Sharifi, “A Comprehensive and Systematic Literature Review on the Big Data Management Techniques in the Internet of Things”, Wireless Networks, vol. 29, pp. 1085–1144, 2023. DOI: 10.1007/s11276-022-03177-5.
  6. I. Ali, S. Sabir, E. Khan, “Privacy-Preserving Data Aggregation in Resource-Constrained Sensor Nodes in Internet of Things: A Review”, Future Computing and Informatics Journal, vol. 3, no. 1, pp. 41–50, 2018. DOI: 10.1016/j.fcij.2017.11.
  7. J. Hou, L. Qu, W. Shi, “A Survey on Internet of Things Security from Data Perspectives”, Computer Networks, vol. 148, pp. 295–306, 2019. DOI: 10.1016/j.comnet.2018.11.026.
  8. G. Calderon, G. del Campo, E. Saavedra, A. Santamaría, “Monitoring Framework for the Performance Evaluation of an IoT Platform with Elasticsearch and Apache Kafka,” Information Systems Frontiers, vol. 26, pp. 2373–2389, 2024. DOI: 10.1007/s10796-023-10409-2.
  9. T. P. Raptis, A. Passarella, “A Survey on Networked Data Streaming with Apache Kafka”, IEEE Access, vol. 11, pp. 85333–85350, 2023. DOI: 10.1109/ACCESS.2023.3303810.
  10. A. Emer, M. Unterhofer, E. Rauch, “A Cybersecurity Assessment Model for Small and Medium-Sized Enterprises”, IEEE Engineering Management Review, vol. 49, no. 2, pp. 98–109, 2021. DOI: 10.1109/EMR.2021.3078077.
  11. N. Rawindaran, A. Jayal, E. Prakash, “Machine Learning Cybersecurity Adoption in Small and Medium Enterprises in Developed Countries,” Computers, vol. 10, no. 11, 150, 2021. DOI:10.3390/computers10110150.
  12. M. Ulrich, F. Frank, A. S. Klug, “An AI Adoption Model for SMEs: A Conceptual Framework,” IFAC-PapersOnLine, vol. 54, no. 1, pp. 702–708, 2021. DOI: 10.1016/j.ifacol.2021.08.082.
  13. A. Chatterjee, B. S. Ahmed, “IoT Anomaly Detection Methods and Applications: A Survey”, Internet of Things, vol. 19, 100568, 2022. DOI: 10.1016/j.iot.2022.100568.
  14. K. DeMedeiros, A. Hendawi, M. Alvarez, “A Survey of AI-Based Anomaly Detection in IoT and Sensor Networks”, Sensors, vol. 23, no. 3, 1352, 2023. DOI: 10.3390/s23031352.
  15. J. Kinyua, L. Awuah, “AI/ML in Security Orchestration, Automation and Response: Future Research Directions”, Intelligent Automation & Soft Computing, vol. 28, no. 2, pp. 527–545, 2021. DOI: 10.32604/iasc.2021.016240.