Article

Assessing the Assessment: Rubrics Training for Pre-service and New In-service Teachers

---

Abstract

Recent studies report that the use of rubrics may not improve the reliability of assessment if raters are not well trained on how to design and employ them effectively. The intent of this two-phase study was to test if training pre-service and new in-service teachers in the construction, use, and evaluation of rubrics would improve the reliability of their evaluations of writing assignments. Results of Phase 1 showed that raters were significantly influenced by mechanical characteristics of students’ writing, and that using rubrics may not improve the reliability of assessment attributes if raters are not well trained on how to design and employ them effectively. Researchers asserted that without high quality, intense training, participants (N=355) who use rubrics to assess students’ writing are usually just as subjective in their evaluative marks as those who do not use rubrics. In Phase 2, participants (N=55) received intense rubrics training prior to being presented with the two writing samples. Results of Phase 2 showed that rubrics training led to more reliable assessment of both writing samples. Accessed 10,198 times on https://pareonline.net from October 22, 2011 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.

Keywords: Evaluation Methods

How to Cite:

Lovorn, M. G. & Rezaei, A. R., (2011) “Assessing the Assessment: Rubrics Training for Pre-service and New In-service Teachers”, Practical Assessment, Research, and Evaluation 16(1): 16. doi: https://doi.org/10.7275/sjt6-5k13