This paper presents an educational environment to learn how to program computers which integrates an automatic grading tool. The educational environment offers summative and formative feedback, by evaluating the functionality of computer programs. Given a programming problem, students must write a solution and validate it through the automatic grading tool of the educational environment. Summative feedback is offered through verdicts of the programs, indicating whether the syntax, semantics and efficiency of the program are correct or present some type of error. Based on these verdicts, a grade is assigned to the solutions proposed by the students. Moreover, the environment also offers formative feedback through different mechanisms that support the student in the improvement of the proposed programming solutions, before these solutions are submitted for assessment. These mechanisms include: a code editor with syntax highlighting and automatic autocompletion; automatic verification of good programming practices; interactive visualization of the program execution from its source code; a partial scoring system, which assigns partial credit to incomplete or not entirely correct solutions; a test module with personalized inputs; support for submitting programs with multiple source code files and projects; compatibility for several programming languages ​​including Java, Python, JavaScript and C/C++; a manual assessment module; and interactive statistical reports for students and teachers.

The state of the art of tools to support the learning of computer programming is comprised of learning environments and automatic grading systems. The first group includes: basic programming environments, interactive environments based on examples, visualizations/animations, and intelligent tutoring systems. However, generally these systems lack of grading tools, which could be useful to carry out formative evaluation processes while the students practice. On the other hand, in most cases the automatic grading systems focus only on the assessment of the programs (online judges), which are generally adaptations of the evaluation systems used in programming competitions. Although these systems have been shown to be effective in improving the practical skills of students and the competitive aspect has proven to be an excellent motivation for students, these systems fall short as formative assessment tools. This is due to the limited feedback given to the student and their lack of integration with content and tools typical from learning environments.

In this context, the educational environment presented in this paper extends an automatic grading tool for computer programs with new functionalities that make it a truly support system for learning computer programming.

This article presents a continuous assessment methodology for a computer programming course supported by an automatic assessment tool, applied to the practical programming exercises performed by the students. The interaction between the students and the assessment tool was studied through quantitative analyses. In particular, the solutions proposed by the students (computer programs) were analyzed using the verdicts given by the automatic assessment tool: correct solutions or incorrect solutions. In the case of incorrect solutions, the types of programming errors were studied. Additionally, interaction was also studied by analyzing the students’ success rate. This rate is the percentage of correct solutions among the total number of attempts (correct and incorrect). Moreover, the relationship between success rate and academic performance was analyzed. Furthermore, this research examines the students’ perceptions toward the assessment tool through interviews. The results of this study help understanding the benefits and perceptions of the students with respect to the use of an automatic assessment tool in a computer programming course.

This article describes the use of an automatic tool that supports both formative and summative evaluation, in an introductory computer programming course. The aim was to test the tool in a real scenario and to assess its potential benefits. The participants were 56 students in an engineering school. Different programming tasks were solved by students with the support of the tool. It provides summative evaluation using automatic grading, and formative evaluation through real-time checking of good coding practices, code execution visualisation, testing code with custom inputs, informative feedback on failed test cases, and student performance statistics for self-monitoring the learning process. Results of the experience show the students considerably used the tool, denoting high engagement to achieve the learning goals of the tasks. The students perceived the tool was highly useful to debug programs and carry out frequent practice of programming, thanks to the automatic feedback and test of solutions for programming assignments. Results offer evidence of the importance of providing comprehensive feedback to facilitate students finding mistakes and building knowledge about how to proceed in case of errors. Moreover, the tool shows some potential to help students developing learning strategies, e.g. metacognitive awareness and improving problem-solving skills.

Jupyter notebooks provide an interactive programming environment that allows writing code, text, equations, and multimedia resources. They are widely used as a teaching support tool in computer science and engineering courses. However, manual grading programming assignments in Jupyter notebooks is a challenging task, thus using an automatic grader becomes a must. This paper presents UNCode notebook auto-grader, that offers summative and formative feedback instantaneously. It provides instructors with an easy-to-use grader generator within the platform, without having to deploy a new server. Additionally, we report the experience of employing this tool in two artificial intelligence courses: Introduction to Intelligent Systems and Machine Learning. Several programming activities were carried out using the proposed tool. Analysis of students’ interactions with the tool and the students’ perceptions are presented. Results showed that the tool was widely used to evaluate their tasks, as a large number of submissions were performed. Students expressed positive opinions mostly, giving feedback about the auto-grader, highlighting the usefulness of the immediate feedback and the grading code, among other aspects that helped them to solve the activities. Results remarked on the importance of providing clear grading code and formative feedback to help the students to identify errors and correct them.

In recent years, learning analytics has emerged as one of the most important fields on the future of education. In the context of programming courses, the applications of learning analytics show high efficacy in giving directions for interventions, which help to promote better learning methods. However, few investigations have considered complex datasets where there are heterogeneous student's groups, letting out differential factors that can influence in the learning process. The main objective of this work is determining the relations between the measurements and metrics of the learning process with the academic performance of the computer programming students of the National University of Colombia. We apply a quantitative non-experimental methodological design, using as source of information the records of 2 years of student's interactions with an educational platform of automatic grading and feedback use in the course. In total 38 variables are considered in this work, that include the number of submissions, the results of each submission, software metrics, and the use rates of the tools available in the platform. The results show that the number of submissions, three types of results/verdicts, two verdict's rates, and one software metric have a positive correlation with the academic performance. Moreover, the runtime error rate, and the use of a good practices verification tool (i.e., Linter) have a negative correlation with the final performance of the students.

In this article, the authors present a case study of an intervention in an introductory programming course aimed at improving the learning experience of students through the use of three different tools: an interactive document format (Jupyter notebooks), an automatic assessment tool (UNCode), and an education-oriented forum platform (Piazza). The study included 100 students from three different sections of the Introduction to Computer Programming course at an engineering school. The study analysed both the use of the tools and the students’ perception of them. Usage was evaluated quantitatively using usage statistics reported by the forum and automatic evaluation tools. Student perception was assessed using a survey with closed-ended and open-ended questions. The results of the study show that all three tools had a positive impact on the students’ learning experience. In general, the participants found Jupyter notebooks, UNCode and Piazza forums easy to use, and useful to better understand the programming problems to be solved.

This article proposes a framework based on a sequential explanatory mixed-methods design in the learning analytics domain to enhance the models used to support the success of the learning process and the learner. The framework consists of three main phases: (1) quantitative data analysis; (2) qualitative data analysis; and (3) integration and discussion of results. Furthermore, we illustrated the application of this framework by examining the relationships between learning process metrics and academic performance in the subject of Computer Programming coupled with content analysis of the responses to a students’ perception questionnaire of their learning experiences in this subject.

Hardware Description Languages (HDL) have gained popularity in the field of digital electronics design, driven by the increasing complexity of modern electronic circuits. Consequently, supporting students in their learning of these languages is crucial. This work aims to address this need by developing an automated assessment software tool with feedback process to support the learning of HDL and making an educational intervention to support the learning process of students. The tool’s features were selected based on similar developments, and a prototype was designed and implemented. Additionally, an educational intervention was conducted over a five-week period in a Digital Electronics course at the National University of Colombia. Through analyzing students’ interactions with the tool and their perceptions of its usage, the study examined their learning experiences. Among the features highlighted by students as most beneficial for their HDL learning process were the online availability of the tool, the feedback system that helped them identify and correct errors in their code, the provision of immediate feedback, the online editor with syntax highlighting, and the graphical user interface. This work makes two significant contributions to the field of HDL teaching in engineering. Firstly, a publicly accessible HDL grading tool has been developed, offering students immediate formative and summative feedback through an automated grader. Secondly, empirical evidence has been provided regarding the benefits of using such a tool in enhancing students’ learning process.