Technology-Enhanced Multimodal Learning Analytics in Higher Education: a Systematic Literature Review

Abstract

Multimodal learning analytics (MMLA) is an emerging field of learning analytics and promises a more comprehensive analysis of the learning process thanks to advances in technological devices and data science. The purpose of this study was to explore technology-enhanced multimodal learning analytics in higher education systematically. A systematic literature review was performed using the PRISMA guidelines, and 45 studies published between January 2012 and June 2024 were determined. The findings demonstrated that China, the USA, Australia, and Chile were the leading contributors to MMLA research, with a notable surge in publications in 2021. Audio recorders, cameras, webcams, eye trackers, and wristbands were the most used devices. Most studies were conducted in experiment rooms or laboratories, though studies in authentic classroom settings have been growing. Data were primarily collected during activities such as programming, simulation exercises, presentations, discussions, writing, watching videos, reading, or exams, as well as throughout the entire instructional process, predominantly in computer science, health, and engineering courses. The studies were mainly predictive or descriptive whereas quite a few studies were prescriptive. Frequently tracked data types included audio, gaze, log, facial expression, physiological, and behavioral data. Traditional machine learning and basic statistics were the commonly used analytical methods whilst advanced statistics and deep learning were relatively less utilized. Test performance, engagement, emotional state, debugging performance, and learning experience were the popular target variables. The studies also pointed out several implications and future directions, with a significant portion highlighting the development of interventions, frameworks, or adaptive systems using MMLA. © 2013 IEEE.