A Pilot Study of Contextual UMLS Indexing to Improve the Precision of Concept-based Representation in XML-structured Clinical Radiology Reports
- Yang Huang, MS,
- Henry J Lowe, MD,
- William R Hersh, MD
- Affiliations of the authors: Stanford Medical Informatics and The Office of Information Resources and Technology, Stanford University School of Medicine, Stanford, California (YH, HJL); Division of Medical Informatics & Outcomes Research, Oregon Health & Science University, Portland, Oregon (WRH)
- Correspondence and reprints: Yang Huang, MS, Stanford Medical Informatics, MSOB X200, 251 Campus Drive, Stanford University, Stanford, CA 94305; e-mail: <yhuang{at}smi.stanford.edu>
- Received 21 March 2003
- Accepted 30 June 2003
Abstract
Objective Despite the advantages of structured data entry, much of the patient record is still stored as unstructured or semistructured narrative text. The issue of representing clinical document content remains problematic. The authors' prior work using an automated UMLS document indexing system has been encouraging but has been affected by the generally low indexing precision of such systems. In an effort to improve precision, the authors have developed a context-sensitive document indexing model to calculate the optimal subset of UMLS source vocabularies used to index each document section. This pilot study was performed to evaluate the utility of this indexing approach on a set of clinical radiology reports.
Design A set of clinical radiology reports that had been indexed manually using UMLS concept descriptors was indexed automatically by the SAPHIRE indexing engine. Using the data generated by this process the authors developed a system that simulated indexing, at the document section level, of the same document set using many permutations of a subset of the UMLS constituent vocabularies.
Measurements The precision and recall scores generated by simulated indexing for each permutation of two or three UMLS constituent vocabularies were determined.
Results While there was considerable variation in precision and recall values across the different subtypes of radiology reports, the overall effect of this indexing strategy using the best combination of two or three UMLS constituent vocabularies was an improvement in precision without significant impact of recall.
Conclusion In this pilot study a contextual indexing strategy improved overall precision in a set of clinical radiology reports.
Footnotes
-
Supported by contract N01-LM-9-3522 from the U.S. National Library of Medicine (NLM) and by the Office of Information Resources and Technology at Stanford University School of Medicine.
