Logo PTI Logo FedCSIS

Proceedings of the 19th Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 39

The Interplay of Learning Analytics and Artificial Intelligence

DOI: http://dx.doi.org/10.15439/2024F5859

Citation: Proceedings of the 19th Conference on Computer Science and Intelligence Systems (FedCSIS), M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 39, pages 35–44 ()

Full text

Abstract. The widespread use of digital systems and tools in education has opened up opportunities for collecting, measuring, and analysing data about user (learner, teacher) interactions with a variety of learning resources and activities, with the ultimate objective of better understanding learning and advancing both learning outcomes and the overall learning experience. This promise motivated the development of Learning Analytics (LA) as a research and practical field and the use of insights derived from learning trace data for evidence-based decision making in a variety of educational settings. While LA has made a significant contribution to better understanding of learning and the environments in which it takes place, many open questions and challenges remain. Furthermore, new opportunities and challenges continue to emerge with the ever-changing modalities of teaching and learning, the latest of which are associated with the rapid development and accessi-bility of Artificial Intelligence (AI). Taking the cyclical model of LA as its exploration framework, this paper examines how key components of the LA model -- namely data, methods, and actions -- relate to and may benefit from the latest develop-ments in AI, and especially Generative AI. Aiming for evi-dence-based analysis and discussion of the interplay between LA and AI, the paper relies on the latest empirical research in LA and the related research fields of AI in Education and Educational Data Mining.

References

  1. C. Lang, A. F. Wise, A. Merceron, D. Gašević, and G. Siemens, “What is Learning Analytics?,” in The Handbook of Learning Analytics, 2nd ed., Vancouver, Canada: SoLAR, 2022, pp. 8–18. [Online]. Available: https://www.solaresearch.org/publications/hla-22/hla22-chapter1/
  2. H. Ouhaichi, D. Spikol, and B. Vogel, “Research trends in multimodal learning analytics: A systematic mapping study,” Computers and Education: Artificial Intelligence, vol. 4, p. 100136, Jan. 2023, http://dx.doi.org/10.1016/j.caeai.2023.100136
  3. P. H. Winne and A. E. Hadwin, “Studying as Self-Regulated Learning,” in Metacognition in Educational Theory and Practice, Routledge, 1998.
  4. P. H. Winne, “Cognition and Metacognition within Self-Regulated Learning,” in Handbook of Self-Regulation of Learning and Performance, 2nd ed., Routledge, 2017.
  5. D. Clow, “The learning analytics cycle: closing the loop effectively,” in Proceedings of the 2nd International Conference on Learning Analytics and Knowledge. New York, NY, USA: ACM, Apr. 2012, pp. 134–138. http://dx.doi.org/10.1145/2330601.2330636
  6. N. Hotz, “What is CRISP DM?,” Data Science Process Alliance. Accessed: Jul. 18, 2024. [Online]. Available: https://www.datascience-pm.com/crisp-dm-2/
  7. D. Gašević, S. Dawson, T. Rogers, and D. Gasevic, “Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success,” The Internet and Higher Education, vol. 28, pp. 68–84, Jan. 2016, http://dx.doi.org/10.1016/j.iheduc.2015.10.002.
  8. R. Conijn, C. Snijders, A. Kleingeld, and U. Matzat, “Predicting Student Performance from LMS Data: A Comparison of 17 Blended Courses Using Moodle LMS,” IEEE Transactions on Learning Technologies, vol. 10, no. 1, pp. 17–29, Jan. 2017, http://dx.doi.org/ 10.1109/TLT.2016.2616312
  9. J. Jovanović, M. Saqr, S. Joksimović, and D. Gašević, “Students matter the most in learning analytics: the effects of internal and instructional conditions in predicting academic success,” Computers & Education, vol. 172, p. 104251, Oct. 2021, http://dx.doi.org/10.1016/j.compedu.2021.104251
  10. D. Gašević, S. Dawson, and G. Siemens, “Let’s not forget: Learning analytics are about learning,” TECHTRENDS TECH TRENDS, vol. 59, no. 1, pp. 64–71, Dec. 2014, http://dx.doi.org/10.1007/s11528-014-0822-x
  11. S. Jarvela et al., “Hybrid Intelligence – Human-AI Co-Evolution and Learning in Multirealities (HI),” in HHAI 2023: Augmenting Human Intellect, IOS Press, 2023, pp. 392–394. http://dx.doi.org/10.3233/FAIA230107.
  12. I. Molenaar, “The concept of hybrid human-AI regulation: Exemplifying how to support young learners’ self-regulated learning,” Computers and Education: Artificial Intelligence, vol. 3, p. 100070, Jan. 2022, http://dx.doi.org/10.1016/j.caeai.2022.100070
  13. L. Yan, R. Martinez-Maldonado, and D. Gašević, “Generative Artificial Intelligence in Learning Analytics: Contextualising Opportunities and Challenges through the Learning Analytics Cycle,” in Proceedings of the 14th Learning Analytics and Knowledge Conference, Mar. 2024, pp. 101–111. http://dx.doi.org/10.1145/3636555.3636856
  14. A. Palanci, R. M. Yılmaz, and Z. Turan, “Learning analytics in distance education: A systematic review study,” Educ Inf Technol, May 2024, http://dx.doi.org/10.1007/s10639-024-12737-5
  15. B. Chen and O. Poquet, “Networks in Learning Analytics: Where Theory, Methodology, and Practice Intersect,” Journal of Learning Analytics, vol. 9, no. 1, Art. no. 1, Mar. 2022, http://dx.doi.org/10.18608/jla.2022.7697
  16. J. Jovanovic, D. Gašević, L. Yan, G. Baker, A. Murray, and D. Gasevic, “Explaining trace-based learner profiles with self-reports: The added value of psychological networks,” Journal of Computer Assisted Learning, 2024, http://dx.doi.org/10.1111/jcal.12968.
  17. J. Du, K. F. Hew, and L. Liu, “What can online traces tell us about students’ self-regulated learning? A systematic review of online trace data analysis,” Computers & Education, vol. 201, p. 104828, Aug. 2023, http://dx.doi.org/ 10.1016/j.compedu.2023.104828.
  18. R. Kaliisa, K. Misiejuk, S. López-Pernas, M. Khalil, and M. Saqr, “Have Learning Analytics Dashboards Lived Up to the Hype? A Systematic Review of Impact on Students’ Achievement, Motivation, Participation and Attitude,” in Proceedings of the 14th Learning Analytics and Knowledge Conference. New York, NY, USA: ACM, Mar. 2024, pp. 295–304. http://dx.doi.org/10.1145/3636555.3636884.
  19. M. Zhou and P. H. Winne, “Modeling academic achievement by self-reported versus traced goal orientation,” Learning and Instruction, vol. 22, no. 6, pp. 413–419, Dec. 2012, http://dx.doi.org/10.1016/j.learninstruc.2012.03.004
  20. D. Gasevic, J. Jovanovic, A. Pardo, and S. Dawson, “Detecting learning strategies with analytics: links with self-reported measures and academic performance,” Journal of Learning Analytics, vol. 4, no. 2, pp. 113–128, Jul. 2017.
  21. S. Dawson, S. Joksimovic, O. Poquet, and G. Siemens, “Increasing the Impact of Learning Analytics,” in Proceedings of the 9th International Conference on Learning Analytics & Knowledge. New York, NY, USA: ACM, Mar. 2019, pp. 446–455. http://dx.doi.org/10.1145/3303772.3303784.
  22. S. Zirkel, J. A. Garcia, and M. C. Murphy, “Experience-Sampling Research Methods and Their Potential for Education Research,” Educational Researcher, vol. 44, no. 1, pp. 7–16, Jan. 2015, http://dx.doi.org/10.3102/0013189X14566879
  23. E. I. Fried, F. Papanikolaou, and S. Epskamp, “Mental Health and Social Contact During the COVID-19 Pandemic: An Ecological Momentary Assessment Study,” Clinical Psychological Science, vol. 10, no. 2, pp. 340–354, Mar. 2022, http://dx.doi.org/10.1177/21677026211017839
  24. M. Saqr, R. Cheng, S. López-Pernas, and E. D. Beck, “Idiographic artificial intelligence to explain students’ self-regulation: Toward precision education,” Learning and Individual Differences, vol. 114, p. 102499, Aug. 2024, http://dx.doi.org/10.1016/j.lindif.2024.102499.
  25. O. Poquet, K. Kitto, J. Jovanovic, S. Dawson, G. Siemens, and L. Markauskaite, “Transitions through lifelong learning: Implications for learning analytics,” Computers and Education: Artificial Intelligence, vol. 2, p. 100039, Nov. 2021, http://dx.doi.org/10.1016/j.caeai.2021.100039
  26. M. Saqr, “Group-level analysis of engagement poorly reflects individual students’ processes: Why we need idiographic learning analytics,” Computers in Human Behavior, p. 107991, Oct. 2023, http://dx.doi.org/10.1016/j.chb.2023.107991.
  27. S. Hutt, J. F. Grafsgaard, and S. K. D’Mello, “Time to Scale: Generalizable Affect Detection for Tens of Thousands of Students across An Entire School Year,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM, May 2019, pp. 1–14. http://dx.doi.org/10.1145/3290605.3300726
  28. S. Liu, S. Liu, Z. Liu, X. Peng, and Z. Yang, “Automated detection of emotional and cognitive engagement in MOOC discussions to predict learning achievement,” Computers and Education, vol. 181, no. C, May 2022, http://dx.doi.org/10.1016/j.compedu.2022.104461
  29. L. Liao, G. H. Yang, and C. Shah, “Proactive Conversational Agents in the Post-ChatGPT World,” in Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, NY, USA: ACM, Jul. 2023, pp. 3452–3455. http://dx.doi.org/10.1145/3539618.3594250
  30. S. Joksimović, R. Marshall, T. Rakotoarivelo, D. Ladjal, C. Zhan, and A. Pardo, “Privacy-Driven Learning Analytics,” in Manage Your Own Learning Analytics: Implement a Rasch Modelling Approach, E. McKay, Ed., Springer International Publishing, 2022, pp. 1–22. http://dx.doi.org/10.1007/978-3-030-86316-6_1
  31. E. Yacobson, O. Fuhrman, S. Hershkovitz, and G. Alexandron, “De-identification is Insufficient to Protect Student Privacy, or – What Can a Field Trip Reveal?,” Journal of Learning Analytics, vol. 8, no. 2, Art. no. 2, Sep. 2021, http://dx.doi.org/10.18608/jla.2021.7353.
  32. Q. Liu, M. Khalil, J. Jovanovic, and R. Shakya, “Scaling While Privacy Preserving: A Comprehensive Synthetic Tabular Data Generation and Evaluation in Learning Analytics,” in Proceedings of the 14th Learning Analytics and Knowledge Conference. New York, NY, USA: ACM, Mar. 2024, pp. 620–631. http://dx.doi.org/10.1145/3636555.3636921.
  33. J. Jordon et al., “Synthetic Data -- what, why and how?,” May 06, 2022, https://arxiv.org/abs/ arXiv:2205.03257. http://dx.doi.org/10.48550/arXiv.2205.03257.
  34. C. Zhan, S. Joksimović, D. Ladjal, T. Rakotoarivelo, R. Marshall, and A. Pardo, “Preserving Both Privacy and Utility in Learning Analytics,” IEEE Transactions on Learning Technologies, vol. 17, pp. 1655–1667, 2024, http://dx.doi.org/10.1109/TLT.2024.3393766.
  35. D. Shan, D. Wang, C. Zhang, B. Kao, and C. K. K. Chan, “Annotating Educational Dialog Act with Data Augmentation in Online One-on-One Tutoring,” in Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky, Cham: Springer Nature Switzerland, 2023, pp. 472–477. http://dx.doi.org/10.1007/978-3-031-36336-8_73.
  36. J. Bao et al., “A Synthetic Data Generation Framework for Grounded Dialogues,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 10866–10882. http://dx.doi.org/10.18653/v1/2023.acl-long.608.
  37. S. Singhal, A. F. Zambrano, M. Pankiewicz, X. Liu, C. Porter, and R. S. Baker, “De-Identifying Student Personally Identifying Information with GPT-4,” in the Proceedings of the 17th International Conference on Educational Data Mining, 2024, pp. 559–565. http://dx.doi.org/10.5281/zenodo.12729884.
  38. C. Hou et al., “Prompt-based and Fine-tuned GPT Models for Context-Dependent and -Independent Deductive Coding in Social Annotation,” in Proceedings of the 14th Learning Analytics and Knowledge Conference. New York, NY, USA: ACM, Mar. 2024, pp. 518–528. http://dx.doi.org/10.1145/3636555.3636910.
  39. A. Barany et al., “ChatGPT for Education Research: Exploring the Potential of Large Language Models for Qualitative Codebook Development,” in Artificial Intelligence in Education, Cham: Springer Nature Switzerland, 2024, pp. 134–149. http://dx.doi.org/10.1007/978-3-031-64299-9_10.
  40. Y. Fan, “Learning and Regulating with ChatGPT: What an Experimental Study Tells Us,” Jun. 25, 2024. Accessed: Jul. 18, 2024. [Online]. Available: https://www.youtube.com/watch?v=YdWePjSYrzw
  41. A. Darvishi, H. Khosravi, S. Sadiq, D. Gašević, and G. Siemens, “Impact of AI assistance on student agency,” Computers & Education, p. 104967, Nov. 2023, http://dx.doi.org/10.1016/j.compedu.2023.104967.
  42. J. Brender, L. El-Hamamsy, F. Mondada, and E. Bumbacher, “Who’s Helping Who? When Students Use ChatGPT to Engage in Practice Lab Sessions,” in Artificial Intelligence in Education, Cham: Springer Nature Switzerland, 2024, pp. 235–249. http://dx.doi.org/10.1007/978-3-031-64302-6_17.
  43. D. Gasevic, “Reimagining Assessment in the Age of Artificial Intelligence,” May 13, 2024. Accessed: Jul. 18, 2024. [Online]. Available: https://www.youtube.com/watch?v=2Ea3oJysD6s
  44. Y. Cheng, K. Lyons, G. Chen, D. Gašević, and Z. Swiecki, “Evidence-centered Assessment for Writing with Generative AI,” in Proceedings of the 14th Learning Analytics and Knowledge Conference. New York, NY, USA: ACM, Mar. 2024, pp. 178–188. http://dx.doi.org/10.1145/3636555.3636866.
  45. L. Yan et al., “VizChat: Enhancing Learning Analytics Dashboards with Contextualised Explanations Using Multimodal Generative AI Chatbots,” in Artificial Intelligence in Education, Cham: Springer Nature Switzerland, 2024, pp. 180–193. http://dx.doi.org/10.1007/978-3-031-64299-9_13.
  46. G. M. Fernandez Nieto, K. Kitto, S. Buckingham Shum, and R. Martinez-Maldonado, “Beyond the Learning Analytics Dashboard: Alternative Ways to Communicate Student Data Insights Combining Visualisation, Narrative and Storytelling,” in LAK22: 12th International Learning Analytics and Knowledge Conference. New York, NY, USA: ACM, Mar. 2022, pp. 219–229. http://dx.doi.org/10.1145/3506860.3506895.
  47. R. Martinez-Maldonado, V. Echeverria, G. Fernandez Nieto, and S. Buckingham Shum, “From Data to Insights: A Layered Storytelling Approach for Multimodal Learning Analytics,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM, Apr. 2020, pp. 1–15. http://dx.doi.org/10.1145/3313831.3376148.
  48. M. E. Milesi et al., “‘It’s Really Enjoyable to See Me Solve the Problem like a Hero’: GenAI-enhanced Data Comics as a Learning Analytics Tool,” in Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM, May 2024, pp. 1–7. http://dx.doi.org/10.1145/3613905.3651111.
  49. B. Maheshi, W. Dai, R. Martinez-Maldonado, and Y.-S. Tsai, “Dialogic feedback at scale: Recommendations for learning analytics design,” Journal of Computer Assisted Learning, 2024, http://dx.doi.org/10.1111/jcal.13034.
  50. P. Sikström, C. Valentini, A. Sivunen, and T. Kärkkäinen, “How pedagogical agents communicate with students: A two-phase systematic review,” Computers & Education, vol. 188, p. 104564, Oct. 2022, http://dx.doi.org/10.1016/j.compedu.2022.104564.
  51. S. Hutt et al., “Feedback on Feedback: Comparing Classic Natural Language Processing and Generative AI to Evaluate Peer Feedback,” in Proceedings of the 14th Learning Analytics and Knowledge Conference. New York, NY, USA: ACM, Mar. 2024, pp. 55–65. http://dx.doi.org/10.1145/3636555.3636850.