Certificate Program: Applying Data Analytics to Assess Quality of Questions on Multiple-Choice Exam

Certificate Program: Applying Data Analytics to Assess Quality of Questions on Multiple-Choice Exam

  • Date: Wednesday, Apr 24, 2019
  • Start Time: 4:00 PM
  • End Time: 5:30 PM
  • Location: Pinn Hall Conference Center (formerly Jordan Hall), Charlottesville

NOTE: This workshop will be held in Room G1/G2, Pinn Hall Conference Center. (This is a change from the previously scheduled room 2005 in MR-5). 

Presented by: James Martindale, PhD, Director of Test Development and Associate Professor of Medical Education; Christine M. Peterson, MD, Assistant Dean for Medical Education and Associate Professor of Obstetrics and Gynecology; and Mary Kate Worden, PhD, Director of Curriculum Integration and Development and Associate Professor of Medical Education.Description/Learning Objectives: Faculty members teaching in the UVa SOM pre-clerkship curriculum are required to submit several exam questions for every educational activity for which they have responsibility.  But how does one measure the quality of these questions?  In this session, participants will review basic statistical and psychometric techniques used for determining the quality of exam items and exams as a whole.  Participants will work in groups to review example test items and the data output from them.  Emphasis will be placed on data interpretation and constructing a logical argument for the evidence of quality.  Given student performance data on assessment items from written MCQ exams, following this workshop, participants will be able to:  review and interpret the psychometric data associated with the exam (difficulty, discrimination, KR-20 value); justify whether individual test items should be dropped or kept from the scoring of the exam; explain how the data can inform decisions about modifying exam items to attempt to improve their difficulty and discrimination. (1.5 CME credits)