by Thibaud Latour and Sandrine Sarre
The ICASE project addresses issues related to computer-based assessment practices, analysed from the perspectives of IT, psychometrics and assessment, and is intended to open up new research strands.
ICASE is a research, development and innovation project undertaken by the Public Research Centre (CRP) Henri Tudor in Luxembourg. ICASE (Intelligent Computer-Assisted Skill Evaluation) is part of a project portfolio called the ESIS (Educational Systems and International Surveys) programme, which supports education policy makers in lifelong learning, teaching and assessment activities by offering products and services. The project started in 2008 and will end in 2011. The initial presentation made at the International Conference of Psychology (ICP 2008) has demonstrated interest in a cross-disciplinary approach. As a strategic project, it aims at setting up a long-term roadmap for the development of computer-based assessment (CBA) followed by the development of assessment tool prototypes and empirical investigations in the field.
CRP Henri Tudor is gaining international expertise and a strong reputation with TAO (Testing Assisté par Ordinateur), a testing platform created in 2002 in cooperation with the University of Luxembourg. TAO is used in the Programme for International Student Assessment (PISA), and in the Programme for the International Assessment for Adult Competencies (PIACC) international surveys for the OECD, which is led by ACER (Australian Council for Educational Research) and ETS (Educational Testing Services). Through this, hundreds of testing items translated into more than 20 languages are managed and supported.
Figure 1: Topics addressed by ICASE.
From a general point of view, assessment addresses a wide range of dimensions (political, economic, social, educational, psychological, linguistic and cultural). Each of these endorses characteristics that may vary considerably depending on national, organizational or individual contexts and requirements. The complexity of this area has increased significantly with the integration of new forms of assessment, tools and methodologies that have dramatically changed the landscape of skill assessment.
The participation of the TAO team in many meetings, workshops and seminars in the domain of CBA has led to the identification of five challenging research areas addressed by different work packages (WP). The exploitation of advanced results (WP1) is mainly based on data mining and is used to analyse candidates’ logs and extract hidden patterns from chronometric and behavioural data. Behavioural patterns dedicated to cheating detection are also considered. Security issues (WP2) concerning particularly high-stake and large-scale tests are still complex (eg brain dump, cheating, item protection etc), and so is identity management; only a few solutions have been provided so far. New forms of testing (WP3) encompass a wide range of mature ubiquitous technologies used for checking how to perform formative assessments, collaborative assessments (eg with serious games) as well as situational skill assessments (eg with mobile devices). Assessment of business-related skills (WP4) is challenging because, in a workplace context, assessments mostly lack scientific validity and consume too much time and money for both test takers and producers. Last but not least, management of e-assessment resources (WP5) covers metadata management and the storage, annotation, search, retrieval and exchange of e-testing resources within a Web-based distributed community.
The main expected deliverables include new software, methods, tools and standards that will significantly improve the quality of assessment practices and spread new efficient ways of assessing the new skills of the 21st century (eg collaborative assessment, portfolio assessment etc).
Amongst the successive phases planned for achieving these goals, the prototyping phase is undoubtedly of the utmost importance. It consists of designing and developing services with pragmatic features. These new components will be directly integrated into the TAO platform and evaluated in real-world applications. Scientific publications, roadmaps and technological developments will enable the CRP to define innovative projects in the future.
The centre plans to enlarge its network of international RDI collaborations by creating and reinforcing synergies at the scientific level with leading players worldwide. ICASE is therefore providing doctoral and post-doctoral opportunities and expects to include a wide variety of stakeholders affected by the future challenges of CBA, whatever their role (psychometricians, educators etc) or organizational setting (testing centres, research institutes etc).
Finally, our future activities will not only permit us to study how assessment practices in teaching and learning evolve towards formative assessments (eg supporting test takers in knowledge construction), but they will also allow us to better understand and integrate CBA into education in general. The ICASE project represents a starting point for the development of new business and research opportunities at the service of the European and international (e)-assessment community.
Link:
http://www.tao.lu
Please contact:
Thibaud Latour
Tel: + 352 42 59 91 327
E-mail:
Sandrine Sarre
Tel: + 352 42 59 91 825
E-mail:
