Logo PTI Logo FedCSIS

Proceedings of the 18th Conference on Computer Science and Intelligence Systems

Annals of Computer Science and Information Systems, Volume 35

Optimum Large Sensor Data Filtering, Networking and Computing

, ,

DOI: http://dx.doi.org/10.15439/2023F5436

Citation: Proceedings of the 18th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 35, pages 431–440 ()

Full text

Abstract. In this paper we consider filtering and processing large data streams in intelligent data acquisition systems. It is assumed that raw data arrives in discrete events from a single expensive sensor. Not all raw data, however, comprises records of interesting events and hence some part of the input must be filtered out. The intensity of filtering is an important design choice because it determines the complexity of filtering hardware and software and the amount of data that must be transferred to the following processing stages for further analysis. This, in turn, dictates needs for the following stages communication and computational capacity. In this paper we analyze the optimum intensity of filtering and its relationship with the capacity of the following processing stages. A set of generic filtering intensity, data transfer, and processing archetypes are modeled and evaluated.

References

  1. M. Moges and T. Robertazzi, “Wireless sensor networks: Scheduling for measurement and data reporting,” IEEE Transactions on Aerospace and Electronic Systems, vol. 42, no. 1, pp. 327–340, 2006. http://dx.doi.org/10.1109/TAES.2006.1603426
  2. J. Berlińska, “A comparison of priority rules for minimizing the maximum lateness in tree data gathering networks,” Engineering Optimization, vol. 54, pp. 218–231, 2022. http://dx.doi.org/10.1080/0305215X.2020.1861263
  3. ——, “Scheduling in data gathering networks with background communication,” Journal of Scheduling, vol. 23, pp. 681–691, 2020. http://dx.doi.org/10.1007/s10951-020-00648-5
  4. ——, “Heuristics for scheduling data gathering with limited base station memory,” Annals of Operations Research, vol. 285, pp. 149–159, 2020. http://dx.doi.org/10.1007/s10479-019-03185-3
  5. ——, “Scheduling for data gathering networks with data compression,” European Journal of Operations Research, vol. 246, pp. 744–749, 2015. http://dx.doi.org/10.1016/j.ejor.2015.05.026
  6. T. Colombo, “Trigger & DAQ at the LHC, filtering data from 50 TB/s to 1 GB/s,” https://indico.cern.ch/event/825688/attachments/1872900/3082664/trigger_daq_at_lhc.pdf, CERN EP/LBC, July 2019, accessed 31/3/2022.
  7. Committee on U.S.-Based Electron-Ion Collider Science Assessment, An Assessment of U.S.-Based Electron-Ion Collider Science. Washington D.C.: The National Academy of Science, Engineering and Medicine, The National Academy Press, 2018.
  8. C. Toth and C. Jóźków, “Remote sensing platforms and sensors: A survey,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 115, pp. 22–36, 2016. http://dx.doi.org/10.1016/j.isprsjprs.2015.10.004
  9. Y.-C. Cheng and T. Robertazzi, “Distributed computation with communication delay,” IEEE Transactions on Aerospace and Electronic Systems, vol. 24, no. 6, pp. 700–712, 1988. http://dx.doi.org/10.1109/7.18637
  10. V. Bharadwaj, D. Ghose, V. Mani, and T. Robertazzi, Scheduling Divisible Loads in Parallel and Distributed Systems. Los Alamitos, CA: IEEE Computer Society Press, 1996.
  11. T. Robertazzi, “Ten reasons to use divisible load theory,” IEEE Computer, vol. 36, no. 5, pp. 63–68, 2003. http://dx.doi.org/10.1109/MC.2003.1198238
  12. M. Drozdowski, Scheduling for Parallel Processing. London: Springer, 2009.
  13. H. Casanova, A. Legrand, and Y. Robert, Parallel Algorithms. London, UK: CRC Press, Taylor and Francis, 2009.
  14. I. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, “A survey on sensor networks,” IEEE Communications Magazine, pp. 102–114, 2002. http://dx.doi.org/10.1109/MCOM.2002.1024422
  15. P. Pereira, A. Grilo, and F. Rocha, End-to-End Reliability in Sensor Networks: Survey and Research Challenges in P. Pereira (ed), EuroFGI Workshop in IP Qos and Traffic Control. Academia, 2007.
  16. T. Muhammed and A. Shaikh, “An analysis of fault detection strategies in wireless sensor networks,” Journal of Network and Computer Applications, vol. 78, pp. 267–287, 2017. http://dx.doi.org/10.1016/j.jnca.2016.10.019
  17. M. Dener, “Security analysis in wireless sensor networks,” International Journal of Distributed Sensor Networks, vol. 10, no. 10, p. 303501, 2014. http://dx.doi.org/10.1155/2014/303501
  18. W. Meng, L. Xie, and W. Xiao, “Optimality analysis of sensorsource geometries in heterogeneous sensor networks,” IEEE Transactions on Wireless Communication, vol. 12, pp. 1958–1967, 2013. http://dx.doi.org/10.1109/twc.2013.021213.121269
  19. L. Cao, Y. Cai, and Y. Yue, “Swarm intelligence-based performance optimization for mobile wireless sensor networks: Survey, challenges, and future directions,” IEEE Access, vol. 7, pp. 161 524–161 553, 2019. http://dx.doi.org/10.1109/access.2019.2951370
  20. W. Luo, B. Gu, and G. Lin, “Communication scheduling in data gathering networks of heterogeneous sensors with data compression: Algorithms and empirical experiments,” European Journal of Operational Research, vol. 271, pp. 462–473, 2018. http://dx.doi.org/10.1016/j.ejor.2018.05.047
  21. W. Luo, Y. Xu, B. Gu, W. Tong, R. Goebel, and G. Lin, “Algorithms for communication scheduling in data gathering network with data compression,” Algorithmica, vol. 80, pp. 3158–3176, 2018. http://dx.doi.org/10.1007/s00453-017-0373-6
  22. C. Li and W. Luo, “Exact and approximation algorithms for minimizing energy in wireless sensor data gathering network with data compression,” American Journal of Mathematical and Management Sciences, vol. 41, no. 4, pp. 305–315, 2022. http://dx.doi.org/10.1080/01966324.2021.1960226
  23. J. Berlińska and M. Drozdowski, “Scheduling divisible mapreduce computations,” Journal of Parallel and Distributed Computing, vol. 71, no. 3, pp. 450–459, 2011. http://dx.doi.org/10.1016/j.jpdc.2010.12.004
  24. ——, “Comparing load-balancing algorithms for mapreduce under zipfian data skews,” Parallel Computing, vol. 72, pp. 14–28, 2018. http://dx.doi.org/10.1016/j.parco.2017.12.003
  25. Wikipedia contributors, “Lambert W function,” https://en.wikipedia.org/wiki/Lambert_W_function, [Online; accessed 5-August-2022].