Logo PTI Logo FedCSIS

Proceedings of the 19th Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 39

Learning from the COVID-19 Pandemic to Improve Critical Infrastructure Resilience using Temporal Fusion Transformers

, , , ,

DOI: http://dx.doi.org/10.15439/2024F2959

Citation: Proceedings of the 19th Conference on Computer Science and Intelligence Systems (FedCSIS), M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 39, pages 375–384 ()

Full text

Abstract. During the COVID-19 pandemic, traditional demand prediction models drastically failed mostly due to altered consumption patterns. Accurate forecasts are essential for ensuring grid stability. This paper analyzes the performance of the Temporal Fusion Transformer (TFT) model during the COVID-19 pandemic aiming to build resilient demand prediction models. Through detailed analysis, we identify which features may contribute to improved performance during large-scale events such as pandemics. During lockdowns, consumption patterns change significantly, leading to substantial errors in existing demand prediction models. We explore the impact of features such as mobility and special day considerations (e.g., lockdown days) on enhancing model performance. We demonstrate that periodic updates on a monthly basis make the model more resilient to changes in consumption patterns during future pandemics. Moreover, we show how improvements in prediction accuracy translate to real-world benefits, such as enhanced grid stability and economic advantages, including reduced energy waste. Additionally, we discuss the implications for energy-critical infrastructure, considering disruptive scenarios like future pandemics.

References

  1. M. Kwiatkowska and X. Zhang, “When to trust ai: Advances and challenges for certification of neural networks,” in Proceedings of the 18th Conference on Computer Science and Intelligence Systems, ser. Annals of Computer Science and Information Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Ślęzak, Eds., vol. 35. IEEE, 2023. http://dx.doi.org/10.15439/2023F2324 pp. 25—-37.
  2. S. D. Li, C. and T. Reindl, “Real-time scheduling of time-shiftable loads in smart grid with dynamic pricing and photovoltaic power generation,” In: 2015 IEEE Innovative Smart Grid Technologies - Asia (ISGT ASIA), pp. 1–6, 2015.
  3. Y. W. Chen, Y. and B. Zhang, “Using mobility for electrical load forecasting during the covid-19 pandemic,” 2020.
  4. B. Lim, S. Arık, N. Loeff, and T. Pfister, “Temporal fusion transformers for interpretable multi-horizon time series forecasting,” International Journal of Forecasting, vol. 37, no. 4, pp. 1748–1764, 2021.
  5. T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” CoRR, vol. abs/1603.02754, 2016. [Online]. Available: http://arxiv.org/abs/1603.02754
  6. A. V. Dorogush, V. Ershov, and A. Gulin, “Catboost: gradient boosting with categorical features support,” 2018.
  7. J. Jenko, “Development and analysis of new activation based load profiles,” Master’s thesis, University of Ljubljana, Faculty of Electical Engineering, 2023.
  8. J. Jakob and J. Pita Costa, “Using temporal fusion transformer predictions to maximise use of renewable energy sources,” In: Proceedings of the International Workshop on Artificial Intelligence and Machine Learning for Energy Transformation (AIE), IEEE, 2024.
  9. B. Lim, S. O. Arik, N. Loeff, and T. Pfister, “Temporal fusion transformers for interpretable multi-horizon time series forecasting,” 2020.
  10. Optuna. (2024) Optuna documentation. [Online]. Available: https://optuna.readthedocs.io/en/stable/
  11. Pytorch. (2024) Quantileloss documentation. [Online]. Available: https://pytorch-forecasting.readthedocs.io/en/stable/api/pytorch_forecasting.metrics.quantile.QuantileLoss.html
  12. Meteostat. (2024) Weather and climate database. [Online]. Available: https://meteostat.net/en/
  13. Open-meteo. (2024) Free weather api. [Online]. Available: https://open-meteo.com
  14. Openweathermap. (2024) Open weather history forecast bulk. [Online]. Available: https://openweathermap.org/api/history-forecast-bulk
  15. ENTSO-E. (2024) Entso-e transparency platform. [Online]. Available: https://transparency.entsoe.eu/
  16. C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del Ro, M. Wiebe, P. Peterson, P. Gerard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, H. Abbasi, C. Gohlke, and T. E. Oliphant, “Array programming with numpy,” Nature, vol. 585, no. 7825, p. 357 to 362, 2020. http://dx.doi.org/10.1038/s41586-020-2649-2. [Online]. Available: https://doi.org/10.1038/s41586-020-2649-2
  17. W. McKinney et al., “Data structures for statistical computing in python,” in Proceedings of the 9th Python in Science Conference, vol. 445. Austin, TX, 2010, p. 51 to 56.
  18. J. D. Hunter, “Matplotlib: A 2d graphics environment,” Computing in Science and Engineering, vol. 9, no. 3, p. 90 to 95, 2007. doi: 10.1109/MCSE.2007.55
  19. P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, İ. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors, “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python,” Nature Methods, vol. 17, p. 261 to 272, 2020. http://dx.doi.org/10.1038/s41592-019-0686-2
  20. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An open source machine learning framework,” arXiv preprint https://arxiv.org/abs/1802.02611, 2018.
  21. PyTorch Forecasting Developers, “Pytorch forecasting documentation,” https://pytorch-forecasting.readthedocs.io/en/stable/getting-started.html, 2024.
  22. P. Jowett. (2024) Slovenia’s new solar additions hit 400 mw in 2023. [Online]. Available: https://www.pv-magazine.com/2024/02/19/slovenias-2023-solar-additions-hit-400-mw/
  23. Swinand, G. Peter, Natraj, and Ashwini, “The value of lost load (voll) in european electricity markets: Uses, methodologies, future directions,” in 2019 16th International Conference on the European Energy Market (EEM), 2019. http://dx.doi.org/10.1109/EEM.2019.8916400 pp. 1–6.
  24. D. R. M. L. Quang Hieu Vu, Ling Cen, “Key factors to consider when predicting the costs of forwarding contracts,” in Proceedings of the 18th Conference on Computer Science and Intelligence Systems, ser. Annals of Computer Science and Information Systems, vol. 30. IEEE, 2022. http://dx.doi.org/10.15439/2022F293 pp. 447–450.
  25. K. Rasul, A. Ashok, A. R. Williams, H. Ghonia, R. Bhagwatkar, A. Khorasani, M. J. D. Bayazi, G. Adamopoulos, R. Riachi, N. Hassen, M. Biloš, S. Garg, A. Schneider, N. Chapados, A. Drouin, V. Zantedeschi, Y. Nevmyvaka, and I. Rish, “Lag-llama: Towards foundation models for probabilistic time series forecasting,” 2024.
  26. A. F. Ansari, L. Stella, C. Turkmen, X. Zhang, P. Mercado, H. Shen, O. Shchur, S. S. Rangapuram, S. P. Arango, S. Kapoor, J. Zschiegner, D. C. Maddix, H. Wang, M. W. Mahoney, K. Torkkola, A. G. Wilson, M. Bohlke-Schneider, and Y. Wang, “Chronos: Learning the language of time series,” 2024.
  27. A. Das, W. Kong, R. Sen, and Y. Zhou, “A decoder-only foundation model for time-series forecasting,” 2024.
  28. A. Garza, C. Challu, and M. Mergenthaler-Canseco, “Timegpt-1,” 2024