Logo PTI Logo FedCSIS

Proceedings of the 18th Conference on Computer Science and Intelligence Systems

Annals of Computer Science and Information Systems, Volume 35

Continual learning of a time series model using a mixture of HMMs with application to the IoT fuel sensor verification

, , ,

DOI: http://dx.doi.org/10.15439/2023F1856

Citation: Proceedings of the 18th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 35, pages 259–264 ()

Full text

Abstract. This paper presents an application of a mixture of Hidden Markov Models (HMMs) as a tool for verification of IoT fuel sensors. The IoT fuel sensors report the level of fuel in tanks of a petrol station, and are a key component for monitoring system reliability (billing), safety (fuel/oil leak detection) and security (theft prevention). We propose an algorithm for learning a mixture of HMMs based on a continual learning principle, i.e. it adapts the model while monitoring a sensor over time, signalling unexpected or anomalous sensor reports. We have tested the proposed approach on a real-life data of 15 fuel tanks being monitored with the FuelPrime system, where it has shown a very good performance (average area under ROC curve of 0.94) of detecting anomalies in the sensor data. Additionally we show that the proposed method can be used for trend monitoring and present qualitative analysis of the short and long term learning performance. The proposed method has promising performance score, the resulting model has a high degree of explainability, limited memory and computation requirements and can be easily generalized to other domains of sensor verification.

References

  1. R. Krishnamurthi, A. Kumar, D. Gopinathan, A. Nayyar, and B. Qureshi, “An overview of IoT sensor data processing, fusion, and analysis techniques,” Sensors, vol. 20, no. 21, 2020. http://dx.doi.org/10.3390/s20216076
  2. M. Henry and G. Wood, “Sensor validation: principles and standards,” atp International, vol. 3, no. 2, 2005.
  3. A. Blázquez-García, A. Conde, U. Mori, and J. A. Lozano, “A review on outlier/anomaly detection in time series data,” ACM Computing Surveys, vol. 54, no. 3, 2021. http://dx.doi.org/10.1145/3444690
  4. M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuytelaars, “A continual learning survey: Defying forgetting in classification tasks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3366–3385, 2022. http://dx.doi.org/10.1109/TPAMI.2021.3057446
  5. J. M. Ramírez, F. Díez, P. Rojo, V. Mancuso, and A. Fernández-Anta, “Explainable machine learning for performance anomaly detection and classification in mobile networks,” Computer Communications, vol. 200, pp. 113–131, 2023. http://dx.doi.org/10.1016/j.comcom.2023.01.003
  6. N. Aslam, I. U. Khan, A. Alansari, M. Alrammah, A. Alghwairy, R. Alqahtani, R. Alqahtani, M. Almushikes, and M. A. Hashim, “Anomaly detection using explainable random forest for the prediction of undesirable events in oil wells,” Applied Computational Intelligence and Soft Computing, vol. 2022, p. 1558381, 2022. http://dx.doi.org/10.1155/2022/1558381
  7. F. Dama and C. Sinoquet, Time Series Analysis and Modeling to Forecast: a Survey. https://arxiv.org/abs/2104.00164 [cs.LG], 2021.
  8. M. Staniszewski, A. Skorupa, Ł. Boguszewicz, M. Sokół, and A. Polański, “Quality control procedure based on partitioning of NMR time series,” Sensors, vol. 18, no. 3, p. 792, 2018. http://dx.doi.org/10.3390/s18030792
  9. M. Staniszewski, A. Skorupa, Ł. Boguszewicz, A. Michalczuk, K. Wereszczyński, M. Wicher, M. Konopka, M. Sokół, and A. Polański, “Application of reiteration of hankel singular value decomposition in quality control,” AIP Conference Proceedings, vol. 1863, no. 1, p. 400006, 2017. http://dx.doi.org/10.1063/1.4992575
  10. L. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proceedings of the IEEE, vol. 77, no. 2, pp. 257–286, 1989. http://dx.doi.org/10.1109/5.18626
  11. E. Lach, D. Grzechca, A. Polański, J. Rutkowski, and M. Staniszewski, “Analysis of semestral progress in higher technical education with HMM models,” in Computational Science – ICCS 2021. Springer, 2021. http://dx.doi.org/10.1007/978-3-030-77967-2_18 pp. 214–228.
  12. M. Cholewa and P. Głomb, “Natural human gestures classification using multisensor data,” in Proceedings 3rd IAPR Asian Conference on Pattern Recognition ACPR 2015. IEEE, 2015. http://dx.doi.org/10.1109/ACPR.2015.7486553 pp. 499–503.
  13. J. Ying, T. Kirubarajan, K. Pattipati, and A. Patterson-Hine, “A hidden Markov model-based algorithm for fault diagnosis with partial and imperfect tests,” IEEE Transactions on Systems, Man, and Cybernetics, Part C, vol. 30, no. 4, pp. 463–473, 2000. http://dx.doi.org/10.1109/5326.897073
  14. W. Zhao, T. Shi, and L. Wang, “Fault diagnosis and prognosis of bearing based on Hidden Markov Model with multi-features,” Applied Mathematics and Nonlinear Sciences, vol. 5, no. 1, pp. 71–84, 2020. http://dx.doi.org/10.2478/amns.2020.1.00008
  15. F. van de Pol and R. Langeheine, “Mixed Markov latent class models,” Sociological Methodology, vol. 20, pp. 213–247, 1990. http://dx.doi.org/10.2307/271087
  16. M. D. R. de Chaumaray, M. Marbac, and F. Navarro, “Mixture of hidden Markov models for accelerometer data,” Annals of Applied Statistics, vol. 14, no. 4, pp. 1834 – 1855, 2020. http://dx.doi.org/10.48550/arXiv.1906.01547
  17. D. Vidotto, J. K. Vermunt, and K. V. Deun, “Multiple imputation of longitudinal categorical data through bayesian mixture latent Markov models,” Journal of Applied Statistics, vol. 47, no. 10, pp. 1720–1738, 2020. http://dx.doi.org/10.1080/02664763.2019.1692794
  18. W. Khreich, E. Granger, A. Miri, and R. Sabourin, “Adaptive ROC-based ensembles of HMMs applied to anomaly detection,” Pattern Recognition, vol. 45, no. 1, pp. 208–230, 2012. http://dx.doi.org/10.1016/j.patcog.2011.06.014
  19. P. R. Cavalin, R. Sabourin, C. Y. Suen, and A. S. Britto Jr., “Evaluation of incremental learning algorithms for HMM in the recognition of alphanumeric characters,” Pattern Recognition, vol. 42, no. 12, pp. 3241–3253, 2009. http://dx.doi.org/10.1016/j.patcog.2008.10.012
  20. W. Khreich, E. Granger, A. Miri, and R. Sabourin, “A survey of techniques for incremental learning of HMM parameters,” Information Sciences, vol. 197, pp. 105–130, 2012. http://dx.doi.org/10.1016/j.ins.2012.02.017
  21. K. Faber, R. Corizzo, B. Sniezynski, and N. Japkowicz, “Vlad: Task-agnostic vae-based lifelong anomaly detection,” Neural Networks, vol. 165, pp. 248–273, 2023. http://dx.doi.org/10.1016/j.neunet.2023.05.032
  22. D. Samariya and A. Thakkar, “A comprehensive survey of anomaly detection algorithms,” Annals of Data Science, vol. 10, no. 3, pp. 829–850, 2023. http://dx.doi.org/10.1007/s40745-021-00362-9
  23. L. Wang, X. Zhang, H. Su, and J. Zhu, A Comprehensive Survey of Continual Learning: Theory, Method and Application. https://arxiv.org/abs/2104.00164 [cs.LG], 2023.
  24. P. Głomb, M. Romaszewski, A. Sochan, and S. Opozda, “Unsupervised parameter selection for gesture recognition with vector quantization and hidden markov models,” in Human-Computer Interaction – INTERACT 2011, P. Campos, N. Graham, J. Jorge, N. Nunes, P. Palanque, and M. Winckler, Eds. Springer Berlin Heidelberg, 2011. http://dx.doi.org/10.1007/978-3-642-23768-3_14 pp. 170–177.
  25. P. Foszner, A. Gruca, and J. Bularz, “Fuel pipeline thermal conductivity in automatic wet stock reconciliation systems,” in Advances in Data Mining. Applications and Theoretical Aspects: 16th Industrial Conference. Springer, 2016. http://dx.doi.org/10.1007/978-3-319-41561-1_22 pp. 297–310.
  26. G. Schwarz, “Estimating the dimension of a model,” Annals of Statistics, vol. 6, pp. 461–464, 1978. http://dx.doi.org/10.1214/aos/1176344136