Influence of Case and Physician Characteristics on Perceptions of Decision Support Systems

Abstract

Objective This study examines how characteristics of clinical cases and physician users relate to the users' perceptions of the usefulness of the Quick Medical Reference (QMR) and their confidence in their diagnoses when supported by the decision support system.

Methods A national sample (N = 108) of 67 internists, 35 family physicians, and 6 other U.S. physicians used QMR to assist in the diagnosis of written clinical cases. Three sets of eight cases stratified by diagnostic difficulty and the potential of QMR to produce high-quality information were used. A 2 × 2 repeated-measures analysis of variance was used to test whether these factors were associated with perceived usefulness of QMR and physicians' diagnostic confidence after using QMR. Correlations were computed among physician characteristics, ratings of QMR usefulness, and physicians' confidence in their own diagnoses, and between usefulness or confidence and actual diagnostic performance.

Results The analyses showed that QMR was perceived to be significantly more useful (P < 0.05) on difficult cases, on cases where QMR could provide high-quality information, by non-board-certified physicians, and when diagnostic confidence was lower. Diagnostic confidence was higher when comfort with using certain QMR functions was higher. The ratings of usefulness or diagnostic confidence were not consistently correlated with diagnostic performance.

Conclusions The results suggest that users' diagnostic confidence and perceptions of QMR usefulness may be associated more with their need for decision support than with their actual diagnostic performance when using the system. Evaluators may fail to find a diagnostic decision support system useful if only easy cases are tested, if correct diagnoses are not in the system's knowledge base, or when only highly trained physicians use the system.