PIPE: Predicting Logical Programming Errors in Programming Exercises

In colleges, programming is increasingly becoming a general education course of almost all STEM majors as well as some art majors, resulting in an emerging demand for scalable programming education. To support scalable education, teaching activities such as grading and feedback have to be automated. Recently, online judge systems have been extensively used for programming training, because they are able to automatically evaluate the correctness of programs in real time and thereby make grading work scalable. However, existing online judge systems lack of the ability to give effective feedback on logical programming errors. As such, instructors and teaching assistants are still overwhelmed by the work of helping students fix programs, especially for those novice students. To tackle the challenge, we develop PIPE, a deep learning model that is able to Predict logIcal Programming Errors in student programs. The model seamlessly integrates a representation learning model for obtaining the latent feature of a program and a multi-label classification model for predicting the error types in the program, thereby allowing end-to-end learning and prediction. We use the C programs submitted in our online judge system to train PIPE, and demonstrate its superior performance over the baseline models. We use PIPE to implement the error-feedback feature in our online judge system and enable automated feedback on logical programming errors to the students.