Logo PTI Logo FedCSIS

Proceedings of the 19th Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 39

Disease Diagnosis On Ships Using Hierarchical Reinforcement Learning

, , , ,

DOI: http://dx.doi.org/10.15439/2024F3214

Citation: Proceedings of the 19th Conference on Computer Science and Intelligence Systems (FedCSIS), M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 39, pages 555–559 ()

Full text

Abstract. Every year about 30 million people travel by ships world wide often in extreme weather conditions and also in polluted environment due to ship's fuel combustion and many other factors that impacts the health of both passengers and crew staff so there is a need of medical staff but that's not always available so we introduce an a model based on Reinforcement learning(RL) that is used as the key approach in dialogue system.We incorporates Hierarchical reinforcement learning(HRL) model with the layers of Deep Q-Network for dialogue oriented diagnosis system.policy learning is integrated as policy gradients are already defined.We created two stage hierarchical strategy.We used the hierarchical structure with double layer policies for automatic disease diagnosis.Double layer means it splits the task into sub-tasks named as high-state strategy and low level strategy.It has a component called user simulator that communicates with patient for symptom collection low level agent inquire symptoms.Once its done collecting it sends results to high level agent which activates the D-classifier for last diagnosis.When its done its send back by user simulator to patients to verify diagnosis made.Every single diagnosis made has its own reward that trains the system.

References

  1. Qanita Bani Baker, Safa Swedat, and Kefah Aleesa. Automatic disease diagnosis system using deep q-network reinforcement learning. In 2023 14th International Conference on Information and Communication Systems (ICICS), pages 1–6, 2023.
  2. Mohamed Bakhouya, Roy Campbell, Antonio Coronato, Giuseppe de Pietro, and Anand Ranganathan. Introduction to special section on formal methods in pervasive computing, 2012.
  3. Marcello Cinque, Antonio Coronato, and Alessandro Testa. Dependable services for mobile health monitoring systems. International Journal of Ambient Computing and Intelligence (IJACI), 4(1):1–15, 2012.
  4. Marcello Cinque, Antonio Coronato, and Alessandro Testa. A failure modes and effects analysis of mobile health monitoring systems. In Innovations and advances in computer, information, systems sciences, and engineering, pages 569–582. Springer, 2012.
  5. Antonio Coronato and Muddasar Naeem. A reinforcement learning based intelligent system for the healthcare treatment assistance of patients with disabilities. In International Symposium on Pervasive Systems, Algorithms and Networks, pages 15–28. Springer, 2019.
  6. Antonio Coronato, Muddasar Naeem, Giuseppe De Pietro, and Giovanni Paragliola. Reinforcement learning for intelligent healthcare applications: A survey. Artificial Intelligence in Medicine, 109:101964, 2020.
  7. Antonio Coronato and Giovanni Paragliola. A structured approach for the designing of safe aal applications. Expert Systems with Applications, 85:1–13, 2017.
  8. Jonathan S Dillard, William Maynard, and Rahul Kashyap. The epidemiology of maritime patients requiring medical evacuation: a literature review. Cureus, 15(11), 2023.
  9. Mario Fiorino, Muddasar Naeem, Mario Ciampi, and Antonio Coronato. Defining a metric-driven approach for learning hazardous situations. Technologies, 12(7):103, 2024.
  10. Mansoor Jamal, Zaib Ullah, Muddasar Naeem, Musarat Abbas, and Antonio Coronato. A hybrid multi-agent reinforcement learning approach for spectrum sharing in vehicular networks. Future Internet, 16(5):152, 2024.
  11. Hao-Cheng Kao, Kai-Fu Tang, and Edward Chang. Context-aware symptom checking for disease diagnosis using hierarchical reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  12. Umamah bint Khalid, Muddasar Naeem, Fabrizio Stasolla, Madiha Haider Syed, Musarat Abbas, and Antonio Coronato. Impact of ai-powered solutions in rehabilitation process: Recent improvements and future trends. International Journal of General Medicine, pages 943–969, 2024.
  13. Kangenbei Liao, CHENG ZHONG, Wei Chen, Qianlong Liu, Baolin Peng, Xuanjing Huang, et al. Task-oriented dialogue system for automatic disease diagnosis via hierarchical reinforcement learning. 2021.
  14. Luigia Mocerino, Fabio Murena, Franco Quaranta, and Domenico Toscano. Validation of the estimated ships’ emissions through an experimental campaign in port. Ocean Engineering, 288:115957, 2023.
  15. Muddasar Naeem and Antonio Coronato. An ai-empowered home-infrastructure to minimize medication errors. Journal of Sensor and Actuator Networks, 11(1):13, 2022.
  16. Muddasar Naeem, Antonio Coronato, and Giovanni Paragliola. Adaptive treatment assisting system for patients using machine learning. In 2019 sixth international conference on social networks analysis, management and security (SNAMS), pages 460–465. IEEE, 2019.
  17. Muddasar Naeem, Antonio Coronato, Zaib Ullah, Sajid Bashir, and Giovanni Paragliola. Optimal user scheduling in multi antenna system using multi agent reinforcement learning. Sensors, 22(21):8278, 2022.
  18. Regina Padmanabhan, Nader Meskin, and Wassim M. Haddad. Learning-based control of cancer chemotherapy treatment**this publication was made possible by the gsra grant no. gsra1-1-1128-13016fromthe qatar national research fund (a member of qatar foundation). the findings achieved herein are solely the responsibility of the authors. IFAC-PapersOnLine, 50(1):15127–15132, 2017. 20th IFAC World Congress.
  19. Giovanni Paragliola, Antonio Coronato, Muddasar Naeem, and Giuseppe De Pietro. A reinforcement learning-based approach for the risk management of e-health environments: A case study. In 2018 14th international conference on signal-image technology & internet-based systems (SITIS), pages 711–716. IEEE, 2018.
  20. Yu-Shao Peng, Kai-Fu Tang, Hsuan-Tien Lin, and Edward Chang. Refuel: Exploring sparse features in deep reinforcement learning for fast disease diagnosis. Advances in neural information processing systems, 31, 2018.
  21. A.John Rush, Maurizio Fava, Stephen R Wisniewski, Philip W Lavori, Madhukar H Trivedi, Harold A Sackeim, Michael E Thase, Andrew A Nierenberg, Frederic M Quitkin, T.Michael Kashner, David J Kupfer, Jerrold F Rosenbaum, Jonathan Alpert, Jonathan W Stewart, Patrick J McGrath, Melanie M Biggs, Kathy Shores-Wilson, Barry D Lebowitz, Louise Ritz, George Niederehe, and for the STAR*D Investigators Group. Sequenced treatment alternatives to relieve depression (star*d): rationale and design. Controlled Clinical Trials, 25(1):119–142, 2004.
  22. Syed Ihtesham Hussain Shah, Antonio Coronato, Muddasar Naeem, and Giuseppe De Pietro. Learning and assessing optimal dynamic treatment regimes through cooperative imitation learning. IEEE Access, 10:78148–78158, 2022.
  23. Syed Ihtesham Hussain Shah, Muddasar Naeem, Giovanni Paragliola, Antonio Coronato, and Mykola Pechenizkiy. An ai-empowered infrastructure for risk prevention during medical examination. Expert Systems with Applications, 225:120048, 2023.
  24. Beata Sokołowska, Wiktor Świderski, Edyta Smolis-Bąk, Ewa Sokołowska, and Teresa Sadura-Sieklucka. A machine learning approach to evaluate the impact of virtual balance/cognitive training on fall risk in older women. Frontiers in Computational Neuroscience, 18:1390208, 2024.
  25. Milene Santos Teixeira, Vinícius Maran, and Mauro Dragoni. The interplay of a conversational ontology and ai planning for health dialogue management. In Proceedings of the 36th annual ACM symposium on applied computing, pages 611–619, 2021.
  26. Edvard Tijan, Marija Jović, Saša Aksentijević, and Andreja Pucihar. Digital transformation in the maritime transport sector. Technological Forecasting and Social Change, 170:120879, 2021.
  27. Abhisek Tiwari, Tulika Saha, Sriparna Saha, Shubhashis Sengupta, Anutosh Maitra, Roshni Ramnani, and Pushpak Bhattacharyya. Multi-modal dialogue policy learning for dynamic and co-operative goal setting. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2021.
  28. Jianhong Wang, Yuan Zhang, Tae-Kyun Kim, and Yunjie Gu. Modelling hierarchical structure between dialogue policy and natural language generator with option framework for task-oriented dialogue system. arXiv preprint https://arxiv.org/abs/2006.06814, 2020.
  29. Xin Wang, Wenhu Chen, Jiawei Wu, Yuan-Fang Wang, and William Yang Wang. Video captioning via hierarchical reinforcement learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4213–4222, 2018.
  30. Zhongyu Wei, Qianlong Liu, Baolin Peng, Huaixiao Tou, Ting Chen, Xuanjing Huang, Kam-fai Wong, and Xiangying Dai. Task-oriented dialogue system for automatic diagnosis. In Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 201–207, Melbourne, Australia, July 2018. Association for Computational Linguistics.
  31. Yuan Xia, Jingbo Zhou, Zhenhui Shi, Chao Lu, and Haifeng Huang. Generative adversarial regularized mutual information policy gradient framework for automatic diagnosis. Proceedings of the AAAI Conference on Artificial Intelligence, 34(01):1062–1069, Apr. 2020.
  32. Lin Xu, Lin Xu, Qixian Zhou, Qixian Zhou, , Ke Gong, Xiaodan Liang, Xiaodan Liang, Jianheng Tang, Jianheng Tang, Jianheng Tang, Lin Li, and Liang Lin. End-to-end knowledge-routed relational dialogue system for automatic diagnosis. null, 2019.
  33. Lin Xu, Qixian Zhou, Ke Gong, Xiaodan Liang, Jianheng Tang, and Liang Lin. End-to-end knowledge-routed relational dialogue system for automatic diagnosis. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 7346–7353, 2019.
  34. Ran Yan, Dong Yang, Tianyu Wang, Haoyu Mo, and Shuaian Wang. Improving ship energy efficiency: Models, methods, and applications. Applied Energy, 368:123132, 2024.
  35. Qian Zhang, Tianhao Li, Dengfeng Li, and Wei Lu. A goal-oriented reinforcement learning for optimal drug dosage control. Annals of Operations Research, pages 1–21, 2024.
  36. Cheng Zhong, Kangenbei Liao, Wei Chen, Qianlong Liu, Baolin Peng, Xuanjing Huang, Jiajie Peng, and Zhongyu Wei. Hierarchical reinforcement learning for automatic disease diagnosis. Bioinformatics, 38(16):3995–4001, 07 2022.
  37. Ying Zhu, Yameng Li, Yuan Cui, Tianbao Zhang, Daling Wang, Yifei Zhang, and Shi Feng. A knowledge-enhanced hierarchical reinforcement learning-based dialogue system for automatic disease diagnosis. Electronics, 12(24), 2023.
  38. Qijie Zou, Xiling Zhao, Bing Gao, Shuang Chen, Zhiguo Liu, and Zhejie Zhang. Relabeling and policy distillation of hierarchical reinforcement learning. International Journal of Machine Learning and Cybernetics, pages 1–17, 2024.