Improved Identification of Noun Phrases in Clinical Radiology Reports Using a High-Performance Statistical Natural Language Parser Augmented with the UMLS Specialist Lexicon
- Affiliations of the authors: Stanford Medical Informatics and Stanford Center for Clinical Informatics, Stanford University School of Medicine, Stanford, CA (YH, HJL); Computer Science Division, University of California at Berkeley, Berkeley, CA (DK); Department of Medicine, University of California, San Francisco, San Francisco, CA (RJC)
- Correspondence and reprints: Yang Huang, MS, Stanford Medical Informatics, MSOB X215, 251 Campus Drive, Stanford, CA 94305-5479; e-mail: <huangy{at}stanford.edu>
- Received 9 September 2004
- Accepted 28 December 2004
Abstract
Objective The aim of this study was to develop and evaluate a method of extracting noun phrases with full phrase structures from a set of clinical radiology reports using natural language processing (NLP) and to investigate the effects of using the UMLS® Specialist Lexicon to improve noun phrase identification within clinical radiology documents.
Design The noun phrase identification (NPI) module is composed of a sentence boundary detector, a statistical natural language parser trained on a nonmedical domain, and a noun phrase (NP) tagger. The NPI module processed a set of 100 XML-represented clinical radiology reports in Health Level 7 (HL7)® Clinical Document Architecture (CDA)–compatible format. Computed output was compared with manual markups made by four physicians and one author for maximal (longest) NP and those made by one author for base (simple) NP, respectively. An extended lexicon of biomedical terms was created from the UMLS Specialist Lexicon and used to improve NPI performance.
Results The test set was 50 randomly selected reports. The sentence boundary detector achieved 99.0% precision and 98.6% recall. The overall maximal NPI precision and recall were 78.9% and 81.5% before using the UMLS Specialist Lexicon and 82.1% and 84.6% after. The overall base NPI precision and recall were 88.2% and 86.8% before using the UMLS Specialist Lexicon and 93.1% and 92.6% after, reducing false-positives by 31.1% and false-negatives by 34.3%.
Conclusion The sentence boundary detector performs excellently. After the adaptation using the UMLS Specialist Lexicon, the statistical parser's NPI performance on radiology reports increased to levels comparable to the parser's native performance in its newswire training domain and to that reported by other researchers in the general nonmedical domain.
Footnotes
-
The authors thank Albert Chan, MD, and Todd Ferris, MD, for their assistance in the evaluation component of this study. The authors also thank Dr. Robert Newcombe for providing a method to calculate confidence intervals and Haoyi Wang for helpful discussions on sentence boundary detection.
