Variation of SNOMED CT Coding of Clinical Research Concepts among Coding Experts
- Affiliations of the authors: School of Library and Information Science (JEA), University of South Florida; Pediatrics Epidemiology Center (RLR, JK), University of South Florida, Tampa, FL
- Correspondence and reprint requests to: James E. Andrews, PhD, School of Library and Information Science, University of South Florida, 4202 E. Fowler Ave., CIS 1040 , Tampa FL 33620; e-mail: <jandrews{at}cas.usf.edu>
- Received 10 January 2007
- Accepted 9 April 2007
Abstract
Objective To compare consistency of coding among professional SNOMED CT coders representing three commercial providers of coding services when coding clinical research concepts with SNOMED CT.
Design A sample of clinical research questions from case report forms (CRFs) generated by the NIH-funded Rare Disease Clinical Research Network (RDCRN) were sent to three coding companies with instructions to code the core concepts using SNOMED CT. The sample consisted of 319 question/answer pairs from 15 separate studies. The companies were asked to select SNOMED CT concepts (in any form, including post-coordinated) that capture the core concept(s) reflected in the question. Also, they were asked to state their level of certainty, as well as how precise they felt their coding was.
Measurements Basic frequencies were calculated to determine raw level agreement among the companies and other descriptive information. Krippendorff’s alpha was used to determine a statistical measure of agreement among the coding companies for several measures (semantic, certainty, and precision).
Results No significant level of agreement among the experts was found.
Conclusion There is little semantic agreement in coding of clinical research data items across coders from 3 professional coding services, even using a very liberal definition of agreement.
Footnotes
-
The project described was supported by Grant Number RR019259 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of NCRR or NIH. The authors wish to thank Heather Guillette, MS of the Pediatrics Epidemiology Center (USF) for providing the test sample, and all of the investigators and researchers of the RDCRN, who collectively provided data collection forms that motivated this study. Also, we wish to thank the NIH Office of Rare Diseases, and Drs. Kent Spackman and Asif Syed (SNOMED, Intl., College of American Pathologists) for their helpful suggestions on the coding instructions.
-
↵* The network is supported by several NIH components, including the Office of Rare Diseases (ORD), National Center for Research Resources (NCRR), National Institute of Neurological Disorders and Stroke (NINDS), National Institute of Child Health and Human Development (NICHD), National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS), and the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK).
