Article

Do Loss Aversion and the Ownership Effect Bias Content Validation Procedures?

---

Abstract

In making validity arguments, a central consideration is whether the instrument fairly and adequately covers intended content, and this is often evaluated by experts. While common procedures exist for quantitatively assessing this, the effect of loss aversion—a cognitive bias that would predict a tendency to retain items—on these procedures has not been investigated. For more novel constructs, experts are typically drawn from adjacent domains. In such cases, a related cognitive bias, the ownership effect, would predict that experts would be more loss averse when considering items closer to their domains. This study investigated whether loss aversion and the ownership effect are a concern in standard content validity evaluation procedures. In addition to including promising items to measure a relatively novel construct, framing agency, we included distractor items linked to other areas of our evaluators’ expertise. Experts evaluated all items following procedures outlined by Lawshe (1975). We found on average, experts were able to distinguish between the intended items and distractor items. Likewise, on average, experts were somewhat more likely to reject distractor items closer to their expertise. This suggests that loss aversion and the ownership effect are not likely to bias content validation procedures.

Keywords: CVI, survey design, instrument design, content validity, expert panel, cognitive biases

How to Cite:

Svihla, V. & Gallup, A., (2021) “Do Loss Aversion and the Ownership Effect Bias Content Validation Procedures?”, Practical Assessment, Research, and Evaluation 26(1): 7. doi: https://doi.org/10.7275/34d8-qe13