Designing an online rater training programme for EFL writing teachers.

Modalidad virtual

Congreso; XVL Congreso Anual de FAAPI: Primera Conferencia Latinoamericana sobre la Evaluación de la Lengua Inglesa; 2021

FAAPI

There seems to be general agreement that rater training and standardization are essential to ensure the validity and reliability of writing assessment (Kuiken & Vedder, 2020). In fact, several studies report on the generally positive effects of training on raters? behaviour and consistency while assessing or marking students? writing (Dawson, 2015; Weigle, 2002). However, the variety and specific characteristics of training methods have not been as frequently addressed in the field (Knock, Read & von Randow, 2007). One such training method is online rater training, which emerged to meet specific needs in cases in which evaluators are geographically dispersed, such as international examinations, or when the group of raters is too big, and arranging face-to-face meetings may be problematic (Hamilton, Reddel & Spratt, 2001).In this presentation, we will describe an online training programme for a group of EFL teachers in higher education designed within a larger research project whose aim is the validation of an assessment instrument for EFL academic writing, an Analytic Scoring Scale (ASS). In our present context of a global pandemic, we are immersed, which has forced researchers and educators alike to carry out their activities in a virtual environment, an online rater training programme seems to be the most feasible option. The programme is characterized by a rating cycle that includes both individual self-training and group sessions. The self-training component comprises a number of benchmark texts to be graded using the ASS. Raters have the possibility of comparing their marks with benchmark marks, and to include comments related to their own grading in relation to each of the categories. The group sessions through virtual meetings are aimed at clarifying confusing aspects and discrepancy points. The overall aim of this training programme is to increase inter-rater agreement, which may help reduce the problem of inter-rater variability as well as intra-rater consistency - two frequent issues in foreign language assessment. In addition, the rationale underlying training and the description of the concrete steps to be followed during the virtual training sessions may inform current pedagogical decisions related to assessment in other similar contexts and educational levels.