Knowledge Tracing within Single Programming Practice Using Problem-Solving Process Data

Accurately tracing the state of learner knowledge contributes to providing high-quality intelligent support for computer-supported programming learning. However, knowledge tracing is difficult when learners have only had a few practice opportunities, which is often common in block-based programming. This article proposed two knowledge tracing models that can exploit the problem-solving process data generated by learners from a single programming task. A novel metric, the approaching index, was developed using the tree edit distance in abstract syntax trees to measure the similarities between the learners' intermediate solutions and the optimal solution. The proposed method allows for each learner's programming path to be represented as a raw approaching index sequence (AISeq) or as a single variable (AIScore) by averaging the AISeq. A logistic regression model was first designed to predict the learners' performances using their AIScore, the number of attempts, and their current performance. A second model, a recurrent neural network model, was also developed to directly use the AISeq and to make predictions. To verify the effectiveness of these models, a series of statistical analyses and experiments were conducted on two existing large-scale block-based programming datasets, the results from which revealed that the proposed models were competitive with four state-of-the-art models on multiple metrics, such as the precision-recall curve, accuracy, specificity, and Cohen's Kappa. Especially, the proposed models were found to be more robust than the compared models in predicting who would fail to complete the tasks.