Article

Ensuring Breadth and Depth of Knowledge on Multiple-Choice Examinations for Board Certification

---

Abstract

Certification organizations aim to assess candidates on their breadth and depth of knowledge to determine eligibility for certification in their field of specialty. Assessments used for certification, when appropriately constructed, should use questions (or items) that assess the entirety of the field. However, comparing the plethora of the content of items to assess content coverage is a lengthy and time-consuming process. In an effort to become more aligned with the purpose of increasing content representativeness, organizations can implement a variety of Natural Language Processing (NLP) techniques with their items to ensure no one concept, medical condition, or scenario presents itself redundantly throughout each of its multiple-choice examinations. We provide an illustrative example from the American Board of Obstetrics and Gynecology (ABOG) of the NLP processes used to increase efficiencies and ensure content representativeness.

Keywords: natural language processing, board certification, validity

How to Cite:

Kincaid, H., Sparks, A., Shivraj, P., Holmes, J., Young, A. & Wendel Jr., G. D., (2025) “Ensuring Breadth and Depth of Knowledge on Multiple-Choice Examinations for Board Certification”, Practical Assessment, Research, and Evaluation 30(1): 12. doi: https://doi.org/10.7275/pare.2117