Article

The Effect of Rating Order on Raters’ Assessing of Speaking

---

Abstract

Rater subjectivity is a well-established issue in performance assessment. Among the factors leading to this subjectivity, rating order, the order in which different samples are rated, can play a significant role in raters’ judgments of performance samples. The current study was set to investigate the effect of the rating order on assessing speaking. To this end, 30 EFL learners with different proficiency levels sat an oral presentation test and their spoken productions were rated by 14 raters in seven different orders. Facets was used to analyze the data through the Rasch model. The results showed that the rating order affected raters’ rating of spoken samples, especially when they rated a lower-level test taker before a higher-level one. Also, interactions existed between the rating order and different facets with small effect sizes. The findings have implications for rater training programs to include rigorous training on rating speaking samples in different orders.

Keywords: Assessing speaking, rating order, test bias, technical knowledge

How to Cite:

Taghvafard, M. & Ahmadi, A., (2026) “The Effect of Rating Order on Raters’ Assessing of Speaking”, Practical Assessment, Research, and Evaluation 31(1): 16. doi: https://doi.org/10.7275/pare.2308