Article
Authors: Heath Kincaid (American Board of Obstetrics and Gynecology) , Anthony Sparks (American Board of Obstetrics and Gynecology) , Pooja Shivraj (American Board of Obstetrics and Gynecology) , Jill Holmes (American Board of Obstetrics and Gynecology) , Amy Young (American Board of Obstetrics and Gynecology) , George D. Wendel Jr. (American Board of Obstetrics and Gynecology)
Certification organizations aim to assess candidates on their breadth and depth of knowledge to determine eligibility for certification in their field of specialty. Assessments used for certification, when appropriately constructed, should use questions (or items) that assess the entirety of the field. However, comparing the plethora of the content of items to assess content coverage is a lengthy and time-consuming process. In an effort to become more aligned with the purpose of increasing content representativeness, organizations can implement a variety of Natural Language Processing (NLP) techniques with their items to ensure no one concept, medical condition, or scenario presents itself redundantly throughout each of its multiple-choice examinations. We provide an illustrative example from the American Board of Obstetrics and Gynecology (ABOG) of the NLP processes used to increase efficiencies and ensure content representativeness.
Keywords: natural language processing, board certification, validity
How to Cite: Kincaid, H. , Sparks, A. , Shivraj, P. , Holmes, J. , Young, A. & Wendel Jr., G. D. (2025) “Ensuring Breadth and Depth of Knowledge on Multiple-Choice Examinations for Board Certification”, Practical Assessment, Research, and Evaluation. 30(1). doi: https://doi.org/10.7275/pare.2117