Article

A Comparison of Three Approaches to Correct for Direct and Indirect Range Restrictions: A Simulation Study

---

Abstract

A common methodological problem in the evaluation of the predictive validity of selection methods, e.g. in educational and employment selection, is that the correlation between predictor and criterion is biased. Thorndike’s (1949) formulas are commonly used to correct for this biased correlation. An alternative approach is to view the selection mechanism as a missing data mechanism. The aim of this study was to compare Thorndike’s formulas for direct and indirect range restriction scenarios with two state-of-the-art approaches for handling missing data: full information maximum likelihood (FIML) and multiple imputation by chained equations (MICE). We conducted Monte-Carlo simulations to investigate the accuracy of the population correlation estimates in dependence of the selection ratio and the true population correlation in an experimental design. For a direct range restriction scenario, the three approaches are equally accurate. For an indirect range restriction scenario, the corrections using FIML and MICE are more precise than when using Thorndike’s formula. The higher the selection ratio and the true population correlation, the higher the precision of the population correlation estimates. Our findings indicate that both missing data approaches are alternative corrections to Thorndike’s formulas, especially in the case of indirect range restriction. Accessed 3,954 times on https://pareonline.net from March 24, 2016 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.

Keywords: Educational Research, Research Methodology, Statistical Analysis

How to Cite:

Pfaffel, A., Schober, B. & Spiel, C., (2016) “A Comparison of Three Approaches to Correct for Direct and Indirect Range Restrictions: A Simulation Study”, Practical Assessment, Research, and Evaluation 21(1): 6. doi: https://doi.org/10.7275/x4ep-fv42