Article

Deconstruction of Holistic Rubrics into Analytic Rubrics for Large-Scale Assessments of Students’ Reasoning of Complex Science Concepts

---

Abstract

Constructed responses can be used to assess the complexity of student thinking and can be evaluated using rubrics. The two most typical rubric types used are holistic and analytic. Holistic rubrics may be difficult to use with expert-level reasoning that has additive or overlapping language. In an attempt to unpack complexity in holistic rubrics at a large scale, we have developed a systematic approach called deconstruction. We define deconstruction as the process of converting a holistic rubric into defining individual conceptual components that can be used for analytic rubric development and application. These individual components can then be recombined into the holistic score which keeps true to the holistic rubric purpose, while maximizing the benefits and minimizing the shortcomings of each rubric type. This paper outlines the deconstruction process and presents a case study that shows defined concept definitions for a hierarchical holistic rubric developed for an undergraduate physiology-content reasoning context. These methods can be used as one way for assessment developers to unpack complex student reasoning, which may ultimately improve reliability and validation of assessments that are targeted at uncovering large-scale complex scientific reasoning. Accessed 398 times on https://pareonline.net from September 05, 2019 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.

Keywords: Test Construction, Standardized Tests, Test Format, Evaluation Methods, Student Evaluation

How to Cite:

Jescovitch, L. N., Scott, E. E., Cerchiara, J. A., Doherty, J. H., Wenderoth, M., Merrill, J. E., Urban-Lurain, M. & Haudek, K. C., (2019) “Deconstruction of Holistic Rubrics into Analytic Rubrics for Large-Scale Assessments of Students’ Reasoning of Complex Science Concepts”, Practical Assessment, Research, and Evaluation 24(1): 7. doi: https://doi.org/10.7275/9h7f-mp76