by Claire Kenny and Claus Pahl
Active learning and training is an effective form of education. Here we discuss an automated learning and skills training system for database programming that promotes procedural knowledge acquisition. Automated tutoring can provide meaningful feedback, such as correction of student solutions and personalised guidance through recommendations. Automated synchronous feedback and accumulative recommendations based on personalised performance assessment are central features here. At the core of this tutoring system is an error correction component that analyses student input and diagnoses student weaknesses.
The Interactive Database Learning Environment (IDLE) has been developed for use by undergraduate students in the School of Computing, Dublin City University. The SQL tutor within IDLE provides the student with a range of SQL programming problems. A student answer to a given problem is received and semantically analysed by a correction component, and a guidance and recommendation component then provides suggestions for further study based on an assessment of the student's weaknesses.
SQL is a computer language, and its representation in terms of grammars and syntax trees provides the data and knowledge structures of our system. Correction and recommendation techniques provide feedback for the student based on problem answers and solutions centred on these data structures. The system's learning content model captures grammar-defined language expressions for answers and solutions in terms of syntax trees to enable analysis. The representation of content knowledge as structured data is the crucial aspect that enables intelligent feedback and interaction between student and system.
An empirical determination and definition of suitable SQL programming problems for the student was completed based on an educator's judgement and experience. For each problem the educator defined an ideal solution, and answers submitted by students are matched against this. The system then focuses on the correction of semantic errors in SQL statements.
Our approach is thus a syntax-driven correction and analysis technique to identify and correct errors based on ideal solutions:
- a notion of semantic equivalence on answer representations in terms of abstract syntax trees (AST) identifies correct answers that are not syntactically equal to the ideal solution
- a pattern-matching technique on syntax trees compares the student answer and the ideal solution by matching structural patterns in the two representations, and thus identifies correct and erroneous elements.
A weighted and faceted error categorization scheme allows the classification and explanation of errors. Although semantics is the issue, only an ideal solution in syntactical representation is required as it acts as a semantic solution when employed together with predefined equivalence rules. Abstracting student answers and ideal solutions from concrete syntax and from irrelevant information through patterns provides the basis of the comparison. Pattern matching on the AST structure provides a similarity measure between solution and answer. A two-step procedure is applied: firstly, error determination through semantic equality identification and, secondly, error localization and categorization through inequality-based error determination.
We distinguish two forms of feedback: immediate feedback based on correction that comprises error location, error explanation, hints and correct solutions; and accumulative feedback that comprises a diagnosis of past performance and recommendations for future studies. The recommendation determination has the following features. It is:
- diagnosis-based/personalized, ie derived from individual student performance weaknesses
- accumulated, ie based on weighted and ranked learner assessments,
- filtered, ie presented according to feedback-level preferences and scaffolding principles.
The recommendation algorithm groups errors into categories and calculates the number of errors within each category. A ranked list is made of the number of errors per category and the overall performance is graded based on the weight of each problem and the number and severity of errors made. For each error type, suitable problems are retrieved from a problem repository, thus providing suitable study material.
Feedback is a classical example of scaffolding (a support framework for learning). Correcting, providing feedback and recommendations are part of the scaffolding needed for knowledge-level interactions between student and tutor. Scaffolding can be implemented for computer language learning and training based on both language grammars as the structuring principle that defines learning content, and syntax trees as data structures that capture problem-based knowledge communicated between student and system.
Some difficulties need to be addressed in the implementation of automated tutoring systems. A systems accuracy and the students trust level, which is itself affected by accuracy, are important for its success. Designing and implementing a flawless correction method is, however, a challenge. Our experience shows that sufficient accuracy can be achieved for a language of the complexity of SQL.
Please contact:
Claus Pahl
Dublin City University, School of Computing / IUA
Tel:+353 1 700 5620
E-mail: Claus.Pahl
dcu.ie
