Do Online Information Retrieval Systems Help Experienced Clinicians Answer Clinical Questions?
- Affiliations of the authors: Centre for Health Informatics, University of New South Wales, Kensington, Australia (JIW, EWC); and Department of Psychology, Royal Holloway College, University of London, Surrey, United Kingdom (ASG)
- Correspondence and reprints: Johanna I. Westbrook, PhD, Clinical Centre for Health Informatics, Cliffbrook Campus, University of New South Wales, Kensington, 2052, Australia; e-mail: <j.westbrook{at}unsw.edu.au>
- Received 11 October 2004
- Accepted 4 January 2005
Abstract
Objective To assess the impact of clinicians' use of an online information retrieval system on their performance in answering clinical questions.
Design Pre-/post-intervention experimental design.
Measurements In a computer laboratory, 75 clinicians (26 hospital-based doctors, 18 family practitioners, and 31 clinical nurse consultants) provided 600 answers to eight clinical scenarios before and after the use of an online information retrieval system. We examined the proportion of correct answers pre- and post-intervention, direction of change in answers, and differences between professional groups.
Results System use resulted in a 21% improvement in clinicians' answers, from 29% (95% confidence interval [CI] 25.4–32.6) correct pre- to 50% (95% CI 46.0–54.0) post-system use. In 33% (95% CI 29.1–36.9) answers were changed from incorrect to correct. In 21% (95% CI 17.1–23.9) correct pre-test answers were supported by evidence found using the system, and in 7% (95% CI 4.9–9.1) correct pre-test answers were changed incorrectly. For 40% (35.4–43.6) of scenarios, incorrect pre-test answers were not rectified following system use. Despite significant differences in professional groups' pre-test scores [family practitioners: 41% (95% CI 33.0–49.0), hospital doctors: 35% (95% CI 28.5–41.2), and clinical nurse consultants: 17% (95% CI 12.3–21.7; χ2 = 29.0, df = 2, p < 0.01)], there was no difference in post-test scores. (χ2 = 2.6, df = 2, p = 0.73).
Conclusions The use of an online information retrieval system was associated with a significant improvement in the quality of answers provided by clinicians to typical clinical problems. In a small proportion of cases, use of the system produced errors. While there was variation in the performance of clinical groups when answering questions unaided, performance did not differ significantly following system use. Online information retrieval systems can be an effective tool in improving the accuracy of clinicians' answers to clinical questions.
Footnotes
-
Supported by a grant from the National Institute of Clinical Studies.
-
Dr. Gosling participated in this research while a Senior Research Fellow at the Centre for Health Informatics, University of New South Wales.
-
The authors thank K. Vaughan, K. Lintern, M. Gazarian, S. Stapleton, V. Sinchenko, and B. Booth for sharing their expertise. They are grateful to N. Creswick who assisted with scenario testing, recruitment, and data collection and to M. Wensley, who assisted with recruitment. Thanks to the team responsible for the information retrieval system, M. Walther, K. Nguyen, V. Vickland, and F. Magrabi, the development of which was partly funded by an Australian Research Council Grant.
