Identifying Wrist Fracture Patients with High Accuracy by Automatic Categorization of X-ray Reports
- Affiliations of the authors: National Research Council Canada, Institute for Information Technology (BdeB, JDM), Ottawa Hospital Research Institute (AC, AJF, SO’D), Department of Medicine, University of Ottawa (AC, AJF), Ottawa, Ontario, Canada
- Correspondence and reprints: Berry de Bruijn, Ph.D., NRC-IIT, 1200 Montreal Road, Building M-50, Ottawa ON, Canada K1A 0R6. email: <berry.debruijn{at}nrc.gc.ca>
- Received 21 October 2005
- Accepted 10 July 2006
Abstract
The authors performed this study to determine the accuracy of several text classification methods to categorize wrist x-ray reports. We randomly sampled 751 textual wrist x-ray reports. Two expert reviewers rated the presence (n = 301) or absence (n = 450) of an acute fracture of wrist. We developed two information retrieval (IR) text classification methods and a machine learning method using a support vector machine (TC-1). In cross-validation on the derivation set (n = 493), TC-1 outperformed the two IR based methods and six benchmark classifiers, including Naive Bayes and a Neural Network. In the validation set (n = 258), TC-1 demonstrated consistent performance with 93.8% accuracy; 95.5% sensitivity; 92.9% specificity; and 87.5% positive predictive value. TC-1 was easy to implement and superior in performance to the other classification methods.
Footnotes
-
Supported by a grant from Canadian Institutes of Health Research (CIHR), Institute of Musculoskeletal Health and Arthritis (Health Services and Policy Research Themes). Dr. Cranney is supported by a salary award from CIHR. Dr. Forster is the PSI Foundation Fellow for Innovative Health Services Research and is supported by a Clinician Scientist award from the Ministry of Health. The Research Ethics Board approved this study.
