Logo PTI Logo FedCSIS

Proceedings of the 19th Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 39

Efficient Maritime Healthcare Resource Allocation Using Reinforcement Learning

, , , ,

DOI: http://dx.doi.org/10.15439/2024F8855

Citation: Proceedings of the 19th Conference on Computer Science and Intelligence Systems (FedCSIS), M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 39, pages 615–620 ()

Full text

Abstract. The allocation of healthcare resources on ships is crucial for safety and well-being due to limited access to external aid. Proficient medical staff on board provide a mobile healthcare facility, offering a range of services from first aid to complex procedures. This paper presents a system model utilizing Reinforcement Learning (RL) to optimize doctor-patient assignments and resource allocation in maritime settings. The RL approach focuses on dynamic, sequential decision-making, em- ploying Q-learning to adapt to changing conditions and maximize cumulative rewards. Our experimental setup involves a simulated healthcare environment with variable patient conditions and doctor availability, operating within a 24-hour cycle. The Q- learning algorithm iteratively learns optimal strategies to enhance resource utilization and patient outcomes, prioritizing emergency cases while balancing the availability of medical staff. The results highlight the potential of RL in improving healthcare delivery on ships, demonstrating the system's effectiveness in dynamic, time-constrained scenarios and contributing to overall maritime safety and operational resilience.

References

  1. M. Ciampi, A. Coronato, M. Naeem, and S. Silvestri, “An intelligent environment for preventing medication errors in home treatment,” Expert Systems with Applications, vol. 193, p. 116434, 2022.
  2. C. Hetherington, R. Flin, and K. Mearns, “Safety in shipping: The human element,” Journal of Safety Research, vol. 37, no. 4, pp. 401–411, 2006. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0022437506000818
  3. S. Nazim, V. K. Shukla, F. Beena, and S. Dubey, “Smart intelligent approaches for healthcare management,” in Computational Intelligence in Urban Infrastructure. CRC Press, 2024, pp. 189–211.
  4. D. Martínez-Méndez and M. Bravo-Acosta, “The challenges faced after a major trauma at an expedition ship at a remote area. report of one case,” Revista Medica de Chile, vol. 151, no. 2, pp. 255–258, 2023.
  5. K. Zong and C. Luo, “Reinforcement learning based framework for covid-19 resource allocation,” Computers Industrial Engineering, vol. 167, p. 107960, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0360835222000304
  6. M. M. S. L. T. R. Benjamin Rolf, Ilya Jackson and D. Ivanov, “A review on reinforcement learning algorithms and applications in supply chain management,” International Journal of Production Research, vol. 61, no. 20, pp. 7151–7179, 2023.
  7. P. Fiorucci, F. Gaetani, R. Minciardi, R. Sacile, and E. Trasforini, “Real time optimal resource allocation in natural hazard management,” 2004.
  8. C. Yu, J. Liu, S. Nemati, and G. Yin, “Reinforcement learning in healthcare: A survey,” ACM Computing Surveys (CSUR), vol. 55, no. 1, pp. 1–36, 2021.
  9. E. Aktaş, F. Ülengin, and Ş. Ö. Şahin, “A decision support system to improve the efficiency of resource allocation in healthcare management,” Socio-Economic Planning Sciences, vol. 41, no. 2, pp. 130–146, 2007.
  10. O. Elfahim, E. M. B. Laoula, M. Youssfi, O. Barakat, and M. Mestari, “Deep reinforcement learning approach for emergency response management,” in 2022 International Conference on Intelligent Systems and Computer Vision (ISCV), 2022, pp. 1–7.
  11. J. Zhang, M. Zhang, F. Ren, and J. Liu, “An innovation approach for optimal resource allocation in emergency management,” IEEE Transactions on Computers, 2016.
  12. T. Ø. Kongsvik, K. Størkersen, and S. Antonsen, “The relationship between regulation, safety management systems and safety culture in the maritime industry,” Safety, reliability and risk analysis: Beyond the horizon, pp. 467–473, 2014.
  13. M. Naeem, A. Coronato, and G. Paragliola, “Adaptive treatment assisting system for patients using machine learning,” in 2019 sixth international conference on social networks analysis, management and security (SNAMS). IEEE, 2019, pp. 460–465.
  14. A. Coronato and M. Naeem, “A reinforcement learning based intelligent system for the healthcare treatment assistance of patients with disabilities,” in International Symposium on Pervasive Systems, Algorithms and Networks. Springer, 2019, pp. 15–28.
  15. A. Coronato, M. Naeem, G. De Pietro, and G. Paragliola, “Reinforcement learning for intelligent healthcare applications: A survey,” Artificial Intelligence in Medicine, vol. 109, p. 101964, 2020.
  16. S. I. H. Shah, A. Coronato, M. Naeem, and G. De Pietro, “Learning and assessing optimal dynamic treatment regimes through cooperative imitation learning,” IEEE Access, vol. 10, pp. 78 148–78 158, 2022.
  17. G. Paragliola, A. Coronato, M. Naeem, and G. De Pietro, “A reinforcement learning-based approach for the risk management of e-health environments: A case study,” in 2018 14th international conference on signal-image technology & internet-based systems (SITIS). IEEE, 2018, pp. 711–716.
  18. S. I. H. Shah, M. Naeem, G. Paragliola, A. Coronato, and M. Pechenizkiy, “An ai-empowered infrastructure for risk prevention during medical examination,” Expert Systems with Applications, vol. 225, p. 120048, 2023.
  19. M. Cinque, A. Coronato, and A. Testa, “A failure modes and effects analysis of mobile health monitoring systems,” in Innovations and advances in computer, information, systems sciences, and engineering. Springer, 2012, pp. 569–582.
  20. M. Bakhouya, R. Campbell, A. Coronato, G. d. Pietro, and A. Ranganathan, “Introduction to special section on formal methods in pervasive computing,” pp. 1–9, 2012.
  21. M. Cinque, A. Coronato, and A. Testa, “Dependable services for mobile health monitoring systems,” International Journal of Ambient Computing and Intelligence (IJACI), vol. 4, no. 1, pp. 1–15, 2012.
  22. I. Sanz et al., “Resource allocation in home care services using reinforcement learning,” in Artificial Intelligence Research and Development: Proceedings of the 25th International Conference of the Catalan Association for Artificial Intelligence, vol. 375. IOS Press, 2023, p. 173.
  23. M. Fiorino, M. Naeem, M. Ciampi, and A. Coronato, “Defining a metric-driven approach for learning hazardous situations,” Technologies, vol. 12, no. 7, p. 103, 2024.
  24. M. Naeem, S. T. H. Rizvi, and A. Coronato, “A gentle introduction to reinforcement learning and its application in different fields,” IEEE access, vol. 8, pp. 209 320–209 344, 2020.
  25. F. Masroor, A. Gopalakrishnan, and N. Goveas, “Machine learning-driven patient scheduling in healthcare: A fairness-centric approach for optimized resource allocation,” in 2024 IEEE Wireless Communications and Networking Conference (WCNC). IEEE, 2024, pp. 01–06.
  26. S. Bharti, D. S. Kurian, and V. M. Pillai, “Reinforcement learning for inventory management,” in Innovative Product Design and Intelligent Manufacturing Systems: Select Proceedings of ICIPDIMS 2019. Springer, 2020, pp. 877–885.
  27. G. Paragliola and M. Naeem, “Risk management for nuclear medical department using reinforcement learning algorithms,” Journal of Reliable Intelligent Environments, vol. 5, pp. 105–113, 2019.
  28. T. Li, Z. Wang, W. Lu, Q. Zhang, and D. Li, “Electronic health records based reinforcement learning for treatment optimizing,” Information Systems, vol. 104, p. 101878, 2022.
  29. K. Gai and M. Qiu, “Optimal resource allocation using reinforcement learning for iot content-centric services,” Applied Soft Computing, vol. 70, pp. 12–21, 2018.
  30. A. Alelaiwi, “Resource allocation management in patient-to-physician communications based on deep reinforcement learning in smart healthcare services,” in 2020 IEEE International Conference on Multimedia Expo Workshops (ICMEW), 2020, pp. 1–5.
  31. C. Shyalika, T. Silva, and A. Karunananda, “Reinforcement learning in dynamic task scheduling: A review,” SN Computer Science, vol. 1, no. 6, p. 306, 2020.
  32. S. Liu, K. C. See, K. Y. Ngiam, L. A. Celi, X. Sun, and M. Feng, “Reinforcement learning for clinical decision support in critical care: comprehensive review,” Journal of medical Internet research, vol. 22, no. 7, p. e18477, 2020.
  33. E. Cabrera, M. Taboada, M. L. Iglesias, F. Epelde, and E. Luque, “Simulation optimization for healthcare emergency departments,” Procedia computer science, vol. 9, pp. 1464–1473, 2012.
  34. R. Fujimori, K. Liu, S. Soeno, H. Naraba, K. Ogura, K. Hara, T. Sonoo, T. Ogura, K. Nakamura, T. Goto et al., “Acceptance, barriers, and facilitators to implementing artificial intelligence–based decision support systems in emergency departments: quantitative and qualitative evaluation,” JMIR formative research, vol. 6, no. 6, p. e36501, 2022.
  35. N. Sahota, R. Lloyd, A. Ramakrishna, J. A. Mackay, J. C. Prorok, L. Weise-Kelly, T. Navarro, N. L. Wilczynski, R. Brian Haynes, and C. S. R. Team, “Computerized clinical decision support systems for acute care management: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes,” Implementation Science, vol. 6, pp. 1–14, 2011.
  36. M. Jamal, Z. Ullah, M. Naeem, M. Abbas, and A. Coronato, “A hybrid multi-agent reinforcement learning approach for spectrum sharing in vehicular networks,” Future Internet, vol. 16, no. 5, p. 152, 2024.
  37. E. B. Laber, K. A. Linn, and L. A. Stefanski, “Interactive model building for q-learning,” Biometrika, vol. 101, no. 4, pp. 831–847, 2014.
  38. E. Riachi, M. Mamdani, M. Fralick, and F. Rudzicz, “Challenges for reinforcement learning in healthcare,” arXiv preprint https://arxiv.org/abs/2103.05612, 2021.