Development and Validation of Assessment Measures for a Newly Developed Physical Examination Simulator
- Correspondence and reprints: Carla M. Pugh, MD, PhD, 251 W. Campus Drive, MSOB Suite #232, Stanford Unversity School of Medicine, Stanford, CA 94305-5466; e-mail: <cpugh{at}stanford.edu>
- Received 17 February 2002
- Accepted 23 July 2002
Abstract
Objective Define, extract and evaluate potential performance indicators from computer-generated data collected during simulated clinical female pelvic examinations.
Design Qualitative and quantitative study analyzing computer generated simulator data and written clinical assessments collected from medical students who performed physical examinations on three clinically different pelvic simulators.
Setting Introduction to patient care course at a major United States medical school.
Participants Seventy-three pre-clinical medical students performed 219 simulated pelvic examinations and generated 219 written clinical assessments.
Measurements Cronbach’s alpha for the newly defined performance indicators, Pearson’s correlation of performance indicators with scored written clinical assessments of simulator findings.
Results Four novel performance indicators were defined: time to perform a complete examination, number of critical areas touched during the exam, the maximum pressure used, and the frequency at which these areas were touched. The reliability coefficients (alpha) were time = 0.7240, critical areas = 0.6329, maximum pressure = 0.7701, and frequency = 0.5011. Of the four indicators, three correlated positively and significantly with the written clinical assessment scores: critical areas, p < 0.01; frequency, p < 0.05; and maximum pressure, p < 0.05.
Conclusion This study demonstrates a novel method of analyzing raw numerical data generated from a newly developed patient simulator; deriving performance indicators from computer generated simulator data; and assessing validity of those indicators by comparing them with written assessment scores. Results show the new assessment measures provide an objective, reliable, and valid method of assessing students’ physical examination techniques on the pelvic exam simulator.
