---

Common LMS Log Indicators for Predicting Student Success: A Systematic Literature Review

---

---

1.Introduction

Research on Learning Management System (LMS) log indicators used to predict student success has become an increasingly important area of inquiry, driven by the growing adoption of digital learning environments and the large volume of data generated by platforms such as Moodle, Blackboard, and Brightspace [1], [2]. Over the past decade, the field has evolved from basic descriptive analytics to sophisticated predictive modeling using machine learning and educational data mining techniques [3], with LMS activity metrics reported to predict academic performance with accuracy often exceeding 80%, underscoring their potential for improving educational quality and reducing dropout rates [4], [5], [6].

Despite this progress, a persistent challenge is identifying which LMS log variables most reliably predict student success across educational contexts [1], [7]. Predictive power varies with course design, discipline, and student population characteristics [8], [9], and no consensus exists on the relative importance of generic versus course-specific log features, or on how to balance early predictive accuracy with interpretability and ethical considerations [2], [10]. Findings also diverge on whether demographic data and prior academic performance meaningfully improve predictive models beyond behavioral data alone [11], [12]. This inconsistency limits the generalizability and practical application of predictive analytics in LMS contexts and may delay timely intervention, with consequences for student retention [13], [14].

This review is grounded in Learning Analytics (LA) and Educational Data Mining (EDM), complementary approaches that leverage LMS log data to model student engagement and predict academic outcomes [15], [16]. LA emphasizes measuring and optimizing the learning process through data analysis, while EDM focuses on developing predictive models using machine learning algorithms [2], [17]. Core constructs include behavioral indicators such as time-on-task, study regularity, and interaction diversity, which prior work links to academic success through both statistical and computational methods [1], [11].

For the purposes of this review, student success refers to the academic outcome measures used in the reviewed studies, such as final course grade, grade point average, pass or fail status, and course completion. An at-risk student is a student whose LMS-based indicators, or whose model-predicted outcome, signal a high likelihood of failing or withdrawing from a course or dropping out, following the operational definition adopted in each study (for example, a predicted final grade below the passing threshold). Because these operational definitions may vary across studies, they are treated as study-level operationalizations and not as a single fixed standard.

Taken together, these gaps point to the need for a systematic synthesis of which LMS log indicators most consistently predict student success, and under what conditions. This review addresses the question: What are the most commonly and effectively used LMS log indicators for predicting student academic success, and how do different predictive modeling techniques compare in effectiveness? To answer this question, the review pursues three objectives:

(1) to evaluate current evidence on LMS log indicators significantly associated with student academic success;

(2) to compare predictive modeling techniques applied to LMS data for early identification of at-risk students; and

(3) to identify and synthesize key student engagement metrics derived from LMS logs in relation to learning outcomes.

The contribution of this review is to consolidate fragmented empirical findings and diverse methodological approaches into practical guidance for educators, researchers, and institutional stakeholders, while highlighting gaps to guide the development of more robust, equitable, and interpretable learning analytics systems.

2.Method

2.1.Research Design

This study used a systematic literature review approach following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to identify, select, and synthesize empirical evidence on LMS log indicators used to predict student success. This systematic approach was chosen to ensure transparency, reproducibility, and minimize bias in the literature selection and analysis process.

2.2.Protocol

This systematic review protocol was developed prior to conducting the literature search, defining the research question, eligibility criteria, search strategy, and data extraction methods. The primary research question guiding this review was: "What are the most commonly and effectively used LMS log indicators to predict student academic success, and how do different predictive modeling techniques compare in terms of effectiveness?" The question was elaborated into the three objectives stated in the Introduction: (1) evaluating LMS log indicators associated with academic success, (2) comparing predictive modeling techniques for the early identification of at-risk students, and (3) identifying and synthesizing key engagement metrics derived from LMS logs. The protocol was not registered in a public registry. To support reproducibility, the search statements, eligibility criteria, and selection stages are reported in this article.

2.3.Eligibility Criteria

Inclusion criteria for studies in this review included: (1) empirical research using log data from Learning Management Systems (LMS) such as Moodle, Blackboard, Canvas, or similar platforms; (2) studies using LMS log data to predict, classify, or explain student academic success, performance, or at-risk status; (3) studies reporting quantitative metrics on the performance of predictive models or the strength of the relationship between LMS indicators and academic outcomes; (4) studies published in English; and (5) research in the context of formal tertiary education (university, college, or higher education). Exclusion criteria included: (1) studies not using LMS log data as the primary data source; (2) studies that were purely descriptive in nature without predictive or inferential analysis; (3) review articles, opinion pieces, or conceptual papers without empirical data; (4) studies with very small sample sizes (n<30) inadequate for reliable statistical analysis; and (5) non-peer-reviewed publications such as theses or internal institutional reports without external validation. There is no restriction based on publication year to capture the methodological evolution in this field, although focus is given to publications in the last two decades (2005-2025) when LMS adoption became widespread.

2.4.Information Sources and Search Strategies

The literature search was conducted using Google Scholar as the primary data source, given its broad indexing coverage of publishers, institutional repositories, and preprint servers. However, the authors acknowledge that reliance on Google Scholar as the sole primary database carries well-documented limitations: its relevance-ranking algorithm is not fully transparent and may introduce ranking bias, its search results are not consistently reproducible across sessions or over time, and its coverage of peer-reviewed literature is less systematic than that of curated databases such as Scopus, Web of Science, ERIC, IEEE Xplore, or ScienceDirect. Ideally, this review would have incorporated parallel searches across these established databases to reduce bias and strengthen coverage. As a partial mitigation, all 125 finally included studies were cross-verified for indexing status in Scopus, and this limitation is revisited as a methodological limitation in the Conclusion, together with a recommendation that future replications of this review incorporate multi-database searching from the outset.

The search strategy was developed through systematic query development, which identified key keywords and their combinations. Five key search strings were constructed using combinations of Boolean operators:

Query 1: ("LMS log" OR "learning management system log" OR "Moodle log" OR "Blackboard log") AND ("student success" OR "academic performance" OR "student achievement") AND ("predict*" OR "indicator*")

Query 2: ("LMS data" OR "VLE data" OR "learning analytics") AND ("predictive model*" OR "machine learning" OR "classification") AND ("at-risk student*" OR "early warning" OR "dropout prediction")

Query 3: ("learning management system" OR "LMS" OR "Moodle" OR "Canvas" OR "Blackboard") AND ("engagement metric*" OR "activity log*" OR "clickstream") AND ("academic success" OR "performance prediction" OR "grade prediction")

Query 4: ("behavioral indicator*" OR "behavioral factor*" OR "learning behavior*") AND ("LMS" OR "online learning platform") AND ("predictive model*" OR "success prediction" OR "performance forecasting")

Query 5: ("student engagement" OR "learning engagement" OR "online participation") AND ("LMS log*" OR "activity data" OR "interaction data") AND ("academic outcome*" OR "learning outcome*" OR "course completion")

Each query used Boolean operators (AND, OR) to strategically combine term clusters, with a wildcard (*) to capture morphological variations of keywords. The search was conducted without publication year restrictions to capture methodological evolution in the field of learning analytics, with a particular focus on peer-reviewed publications in English.

2.5.Study Selection Process

The study selection process was conducted in stages following the PRISMA protocol with systematic screening and relevance assessment, as presented in Fig. 1. The first stage (identification) involved executing five query strings in Google Scholar, resulting in 591 candidate articles after applying initial filters for peer-reviewed and English-language publications. Automated deduplication was performed to eliminate articles appearing in multiple search results.

The second stage (screening through citation chaining) used two complementary strategies: backward citation chaining, which examined the reference lists of 50 core papers identified as highly relevant to find foundational studies that might have been missed in database searches, and forward citation chaining, which identified newer papers citing the core papers using Google Scholar's "Cited by" feature to capture emerging research and methodological advances. The citation chaining process yielded 26 additional articles, bringing the total candidate papers to 617.

The third stage (eligibility assessment) involved systematic relevance ranking, where 617 candidate papers were assessed based on their alignment with the research questions and eligibility criteria. Screening was conducted hierarchically: first based on the title and abstract for initial relevance determination, then full-text review for papers that passed the initial screening. From this process, four papers were excluded for not meeting the eligibility criteria (e.g., non-empirical, not using LMS log data, or having an inadequate sample size), resulting in 613 papers deemed relevant.

The fourth stage (final selection) involved in-depth quality assessment and relevance scoring on a scale of 1-5 based on: (1) directness of relevance to research questions, (2) methodological rigor, (3) sample size adequacy, (4) clarity of reporting, and (5) contribution to theoretical or practical understanding. Papers with relevance scores ≥4 (highly relevant) were selected for detailed analysis, resulting in a final sample of 125 studies that underwent comprehensive data extraction and synthesis.

2.6.Data Extraction and Synthesis Process

Data extraction was systematically performed from 125 highly relevant studies using a standard extraction form that included: (1) study characteristics (author, year, publication venue, country, educational context); (2) methodological details (sample size, LMS platform, course type, delivery mode, duration); (3) LMS log indicators studied (type, operationalization, measurement); (4) predictive modeling techniques used (algorithms, validation methods, performance metrics); (5) reported outcomes (accuracy, precision, recall, AUC, effect size, correlation coefficient); (6) temporal granularity (daily, weekly, monthly, semester-level aggregation); (7) feature importance and interpretability approach; and (8) ethical considerations and fairness assessments if reported. Extraction was performed with attention to detail to ensure accuracy and completeness, with cross-checking to minimize extraction errors.

Data synthesis was conducted through several approaches. First, a descriptive synthesis organized studies by categories such as LMS indicator type, predictive method, and context, with frequency counts and prevalence rates calculated. Second, a comparative synthesis identified agreement and differences across studies, examining factors that might explain variability in findings such as differences in course design, student population, or methodological approach. Third, a thematic synthesis identified recurring themes and patterns in the literature, organizing findings around key dimensions such as behavioral engagement, cognitive engagement, temporal patterns, and model performance. Fourth, a chronological synthesis tracked the evolution of research approaches from 2005 to 2025, documenting methodological advances and shifting priorities in the field. Meta-analytic techniques were considered but ultimately not fully implemented due to substantial heterogeneity in indicator operationalization, variation in outcome measures, and diversity of analytical approaches across studies, which made quantitative pooling problematic. Instead, a narrative synthesis with systematic tabulation of findings was used to provide a comprehensive picture while acknowledging contextual nuances. Correlation coefficients (r) are reported separately from classification performance metrics. Among the latter, the ranges and means in Table 3 and Table 4 summarize the primary metric reported by each study (accuracy, AUC, or F1-score) and are descriptive spreads, not pooled estimates, because the studies used different datasets, outcome definitions, and validation procedures.

3.Results and Discussion

3.1.LMS Log Indicators Correlated with Students’ Academic Success

According to an analysis of 125 empirical studies, there is strong agreement that LMS-based behavioral indicators are significant predictors of student academic success. Of the 125 studies, 105, or 84 percent, indicated that behavioral engagement metrics were the primary predictor. Table 1 shows a complete categorization of the identified indicators, their publication frequency, and the strength of their correlation with academic findings.

The findings in Table 1 demonstrate a clear hierarchical pattern in the predictive power of indicators. Engagement quality indicators, particularly those related to task completion and punctuality, exhibited the highest prevalence and correlation strength compared to simple activity frequency indicators. This aligns with the theoretical construct of self-regulated learning, where students who actively manage their time, access materials regularly, and complete assignments on time demonstrate characteristics of effective learners [19], [33]. Wang and Mousavi [1] found, through a meta-analysis, that regularity metrics exhibited moderate to high effect sizes, with stronger correlations than raw frequency measures, suggesting that not only the quantity but also the temporal pattern of engagement plays a critical role in academic success.

Timeliness of assignment submission emerged as one of the strongest predictors, reported in 72 studies with effect sizes ranging from moderate to very high. Guo et al. [4] reported that submission completion rates achieved the highest feature importance in their predictive model, achieving over 90% accuracy, with SHAP values demonstrating a dominant contribution to prediction. The regularity of study intervals showed that students with regular and consistent study patterns had a 2-3 times higher probability of success than those with sporadic patterns. Early engagement in weeks 1-3 was predictive in several studies [9], [28], enabling early identification of at-risk students for preventive intervention.

Despite general agreement, the analysis revealed significant variability in the predictive power of indicators based on educational context. Table 2 presents a systematic comparison of how the relevance of indicators varies across different educational contexts.

Predictive relevance of certain indicators. Hernández-García et al. [7] showed that the course delivery mode significantly influenced the predictive power of indicators, with clickstreams categorized by learning cycle providing better explanatory power. Conijn et al. [35], in an analysis of 17 blended courses, found that the portability of predictive models was very low across courses, with variance explained ranging from 11% to 68% depending on specific course characteristics. In STEM courses emphasizing problem-solving and practical application, indicators related to hands-on activities and repetitive exercises were more predictive, while in humanities courses emphasizing discussion and critical reflection, indicators of discussion participation and depth of interaction with texts were more informative [10].

Several studies have introduced more sophisticated metrics that go beyond simple frequency measures. Johnston et al. [43] developed an engagement diversity score that measures variation in the types of LMS activities students engage in, and proximity metrics that assess the temporal consistency of engagement in a more nuanced way, indicating more holistic and multidimensional learning. Hernández-García et al. [7] introduced a learning cycle-based classification that categorizes clickstreams by learning phase (exploration, elaboration, and assessment), improving the interpretability and explanatory power of regression models by 15–20% compared to simple frequency counts. You [19] and Paajanen [44] identified self-regulated learning indicators that measure planning, monitoring, and regulation behaviors such as early submission rates, revision patterns, and proactive help-seeking as stronger predictors, with correlations reaching r=0.65 with final grades.

Significant debate exists regarding the inclusion of demographic data in predictive analytics. Some studies show that combining behavioral and demographic data improves predictive accuracy by 10–15%, with variables such as socioeconomic status and prior academic performance explaining additional variance [45], [46], [47]. However, another group of researchers advocates a behavior-only approach to reduce bias and focus on factors that can be modified through intervention [2], [1]. Arizmendi et al. [2] demonstrated that elastic net regularization with behavioral features alone can achieve comparable accuracy with a sensitivity of ~75% and a specificity of ~80%, while improving fairness across demographic groups, reducing the risk of perpetuating systemic bias. This tension reflects a fundamental trade-off in predictive analytics between maximizing accuracy and promoting equity, with significant ethical implications for the implementation of early warning systems in educational institutions.

3.2.Comparison of Predictive Modeling Techniques for Identifying at-Risk Students

An evaluation of predictive modeling techniques revealed widespread adoption of machine learning algorithms in LMS data analysis. A total of 101 of 125 studies (80.8%) used a variety of supervised and unsupervised machine learning techniques. Table 3 presents a systematic comparison of the performance of different algorithm categories used in predicting student success.

The data in Table 3 reveals a nuanced performance hierarchy with a significant trade-off between accuracy and interpretability. Tree-based models, particularly Random Forest and Gradient Boosting, frequently emerged as a favorable choice for many LMS prediction applications, achieving high reported performance (75–96%) while maintaining a reasonable degree of interpretability through feature importance analysis. This is a general pattern across the reviewed studies and not a universal conclusion: reported performance depended on course design, prediction timing, class balance, and validation method. Santos and Henriques [26] reported that Random Forest and Gradient Boosting achieved AUCs of 0.75–0.92, with significant improvements when assessment scores were integrated into the model. Duch et al. [54] found that Random Forest provided the best performance, with a significant correlation between predictions and actual outcomes, with consistent validation across an entire academic year.

Deep learning architectures demonstrate superior capabilities in capturing complex temporal and sequential patterns in LMS data, with mean reported performance of 89-94% across different architectures (Table 3). Liu et al. [15] demonstrated that LSTM achieved approximately 90% accuracy in clickstream analysis with weekly and monthly feature sets, outperforming traditional models in capturing sequential dependencies. Moore and Shin [50] reported particularly impressive results with LSTM achieving 96.6% accuracy in cross-course engagement analysis, using SHAP values to identify feature importance despite architectural complexity. GRU studies reported the highest range across all categories (88-99%), with Al-Sulami et al. [52] reporting 93.65-98.9% accuracy in different contexts, albeit with significant trade-offs in interpretability and computational requirements.

Ensemble methods that combine predictions from multiple models were frequently reported to be robust, with a mean reported performance of 88.9%. Noorani et al. [17] achieved the highest reported accuracy of 97.4% through a combination of ensemble machine learning with appropriate feature selection, demonstrating that strategic feature selection can substantially improve performance. Ardianti et al. [55] found that bagging methods achieved 81.25% accuracy, while Kumari et al. [56] showed that ensemble approaches (bagging, boosting, stacking) consistently improved accuracy compared to single models, particularly in addressing the class imbalance common in at-risk student prediction datasets.

Temporal comparisons of predictions reveal important patterns regarding the trade-off between timeliness and accuracy. Table 4 shows how prediction accuracy evolved over the course duration.

The pattern in Table 4 shows consistent increases in reported performance as temporal data increases, but with diminishing returns after the mid-course point. Liz-Dominguez et al. [57] found that the classifier struggled with imbalanced baseline data, with early prediction accuracy ranging from 65-75%, but improved significantly when data from the first exam became available. Bernacki et al. [28] demonstrated that pre-exam digital behavior can predict poor grades with 75% accuracy, enabling interventions that improve outcomes.

The factors influencing model performance exhibit substantial complexity. The inclusion of assessment scores or prior performance data significantly improves predictive accuracy, with Santos and Henriques [26] reporting an increase in AUC from 0.75 to 0.92 when scores were integrated. Alhassan et al. [58] identified assessment scores as the most important feature in their Random Forest model, suggesting that prior performance is a strong predictor of future performance. However, the reliance on assessment scores creates tension with the goal of early intervention, as the first assessment scores are typically not available until several weeks into the course.

Dataset characteristics also play a critical role in model performance. Studies with small sample sizes (<500) generally report lower accuracy and higher variance, with Wakelam et al. [59] noting particular challenges in predicting student performance in small cohorts with minimal available attributes. Class imbalance, where the number of failing or at-risk students is significantly smaller than the number of successful students, poses significant challenges, with Liz-Dominguez et al. [57] and Buraimoh et al. [5] reporting that models exhibit low sensitivity in identifying minority at-risk groups despite high overall accuracy. Pongpaichet et al. [60] found that Random Forest with oversampling techniques performed best in addressing imbalanced data, while Buraimoh et al. [5] showed that CART and LDA achieved 86% accuracy with appropriate imbalanced data handling.

Prediction and correlation should also be distinguished. The correlations in Table 1 describe associations between an indicator and an outcome within a given sample, whereas the figures in Table 3 and Table 4 describe how well a model classifies students on the data used to validate it. High reported performance therefore does not by itself show that behavioral indicators predict success, and three factors limit the comparability of the reported figures. The first is class imbalance, discussed above, which can inflate overall accuracy while sensitivity for the at-risk minority remains low. The second is potential data leakage: information that would not be available at the moment of prediction, such as assessment scores, late-semester activity, or components of the final grade, may enter the feature set, or the same students and courses may appear in both training and test data. The highest values in Table 4 come from late-course and post-assessment data, and Santos and Henriques [26] reported that AUC increased from 0.75 to 0.92 once assessment scores were integrated, so these values should not be read as evidence that log indicators alone support early identification. Whether the reviewed studies guarded against leakage, for example through time-based or course-level hold-out validation, was not quantified in this review. The third is prediction timing: models built on early-course data (weeks 1-3) are the most useful for intervention but report the lowest performance (Table 4), so figures from different time points should not be ranked against each other.

Model validation and generalization show concerning variability. Conijn et al. [35], in a comprehensive analysis of 17 blended courses, found that predictive model portability was very low across courses, with LMS data limited for early intervention due to course-specific factors. Hernández-García et al. [7] showed that course delivery mode significantly impacted predictive performance, with models trained on online courses not transferring well to blended or face-to-face contexts. Dileep et al. [61] reported accuracies of 73.5% for online and 87.8% for face-to-face course sections in a calculus course, which suggests that model performance, and possibly the relevance of LMS indicators, differs across delivery modes. Table 2 illustrates that the most predictive indicators also differ between fully online, blended, and face-to-face courses.

Evidence on external validity, fairness, privacy, and intervention outcomes is thinner than evidence on internal predictive performance. For external validity and cross-course transfer, external validation on a separate institution or cohort was not quantified in this review, and the low portability reported by Conijn et al. [35] and Hernández-García et al. [7] suggests that transfer should not be assumed. The most predictive indicators also varied by discipline and delivery mode (Table 2; [10]). On fairness, Arizmendi et al. [2] showed that a behavior-only elastic net model achieved comparable accuracy while improving fairness across demographic groups, whereas adding demographic variables may raise accuracy at the risk of perpetuating systemic bias; because fairness assessments were extracted only where reported, they are not compared systematically across studies. Privacy was not systematically extracted from the reviewed studies and is therefore discussed as an implication and not as a finding: LMS logs are behavioral traces of identifiable students, so institutions deploying such models need informed consent or another lawful basis, data minimization, and access controls. Practical intervention outcomes are also rarely reported. Among the studies discussed here, Bernacki et al. [28] link prediction to support provided before students begin to fail, and Macfadyen and Dawson [27] present an early warning system for educators as a proof of concept, whereas most of the other studies report predictive performance only. Evidence on whether predictions improve student outcomes therefore remains limited.

Advanced interpretability techniques are beginning to be adopted to bridge the gap between model complexity and practical usability. Guo et al. [4] used SHAP values to identify assignment submission and content completion as the most influential features in their tree-based model, achieving >90% accuracy. Moore and Shin [50] applied SHAP analysis to their LSTM model, achieving 96.6% accuracy, revealing that cross-course engagement patterns provide substantial predictive power. Pérez-Suay et al. [62] used Gaussian Process Regression with ARD kernels (R=0.89) to automatically identify relevant variables, combining predictive accuracy with interpretable feature selection. Nimy et al. [14] proposed probabilistic logistic regression that not only predicts outcomes but also measures uncertainty, achieving 92.8% accuracy with quantification of prediction confidence, facilitating more nuanced decision-making.

4.Conclusion

A systematic review of 125 records on LMS log indicators for predicting student success found that behavioral indicators were the predictors most often reported as significant for academic success, with 84% of the records (105 of 125) identifying behavioral engagement metrics as significant. Timeliness of assignment submissions and number of assignment submissions were among the strongest predictors, with high to very high correlations in individual studies (r=0.65-0.88 and r=0.60-0.85, respectively), followed by regularity of study intervals, initial engagement in weeks 1-3, and content completion. These findings reinforce self-regulated learning theory and suggest that the quality of engagement is more predictive than the simple quantity of activity. In terms of modeling techniques, tree-based models such as Random Forest and Gradient Boosting frequently offered a favorable balance between high reported performance (75-96%) and adequate interpretability. This is a general pattern across the reviewed studies and not a universal conclusion, because performance depended on course design, prediction timing, class balance, and validation method. Deep learning architectures such as LSTM, CNN-LSTM, and GRU reported the highest mean performance (89-94%), especially in capturing complex temporal patterns, but with significant trade-offs in interpretability and computational requirements. Temporal analysis revealed a paradox between timeliness and accuracy: early predictions (weeks 1-3) allowed for more timely interventions but with lower reported performance (65-85%), while mid-semester predictions (weeks 4-8) achieved a better balance with 75-92% reported performance.

Overall, this review indicates that LMS log-based predictive analytics has significant potential to improve early identification of at-risk students and facilitate more timely and targeted interventions. However, fully realizing this potential requires careful attention to context-specific factors, a judicious balance between timeliness and accuracy in prediction timing, strategic selection of actionable engagement metrics, and the selection of modeling approaches that align with institutional capabilities and limitations. The predictive power of indicators exhibits significant contextual variability based on course delivery mode, discipline, and pedagogical characteristics, challenging the assumption of universality and underscoring the need for context-aware approaches. The methodological heterogeneity observed across studies underscores the urgent need for standardized protocols for operationalizing indicators, reporting results, and assessing model fairness and generalizability. Thus, the implementation of predictive analytics systems requires approaches that are not only technically sophisticated but also pedagogically meaningful and contextually relevant.

Supplementary Materials: -

Author Contribution: RT contributed to the conceptualization of the study, methodology development, data collection, data analysis, interpretation of the findings, and preparation of the original manuscript. EA and EI provided academic supervision and guidance throughout the research process, conducted quality control of the methodology and manuscript, and critically reviewed and revised the manuscript. All authors read and approved the final version of the manuscript.

Funding: -

Acknowledgment: -

Conflicts of Interest: “The authors declare no conflict of interest.”

References