Article

Combining Dual Scaling with Semi-Structured Interviews to Interpret Rating Differences

---

Abstract

Dual scaling, a variation of multidimensional scaling, can reveal the dimensions underlying scores, such as rater's judgments. This study illustrates the use of a dual scaling analysis with semi-structured interviews of raters to investigate the differences among the raters as captured by the dimensions. Thirty applications to a one-year post-Bachelor's degree teacher education program were rated by nine teacher educators. Eight of the raters were subsequently interviewed about how they rated the responses. A three-dimensional model was found to explain most of the variance in the ratings for two of the questions and a two-dimensional model was most interpretable for the third question. The interviews suggested that the dimensions reflected, in addition to differences in rater's stringency, differences in their beliefs about their roles as raters and about the types of insights that were required of applicants. Accessed 9,371 times on https://pareonline.net from May 26, 2009 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.

Keywords: Research Methodology, Statistical Analysis

How to Cite:

Childs, R. A., Ram, A. & Xu, Y., (2009) “Combining Dual Scaling with Semi-Structured Interviews to Interpret Rating Differences”, Practical Assessment, Research, and Evaluation 14(1): 11. doi: https://doi.org/10.7275/wtdw-1c03