Article

Fairness Concerns of Discrete Option Multiple Choice Items

---

Abstract

The Discrete Option Multiple Choice (DOMC) item format was introduced by Foster and Miller (2009) with the intent of improving the security of test content. However, by changing the amount and order of the content presented, the test taking experience varies by test taker, thereby introducing potential fairness issues. In this paper we investigated fairness concerns by evaluating the impact on test takers of the differing testing experiences when items are administered in the DOMC format. Specifically, we described the impact of the presentation order of the key on item difficulty and discrimination as well as the cumulative impact at the test level. We recommend not including DOMC items in exams until the methodology of scoring test takers on these items is revised to address specific fairness concerns identified in this paper. Accessed 1,375 times on https://pareonline.net from November 02, 2018 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.

Keywords: Test Construction

How to Cite:

Eckerly, C., Smith, R. & Sowles, J., (2018) “Fairness Concerns of Discrete Option Multiple Choice Items”, Practical Assessment, Research, and Evaluation 23(1): 16. doi: https://doi.org/10.7275/chaw-y360