Using Commercial Knowledge Bases for Clinical Decision Support
Opportunities, Hurdles, and Recommendations
- Affiliations of the authors: New York Presbyterian Hospital (GJK), Weill-Cornell Medical College, Columbia University College of Physicians and Surgeons, New York; NY Center for Healthcare Quality and Effectiveness, BJC Healthcare and Department of Medicine (RMR,TCB), Washington University School of Medicine, St. Louis, MO
- Correspondence and reprints: Gilad J. Kuperman, MD, PhD, Director, Quality Informatics, NewYork-Presbyterian Hospital, 525 E. 68th St., Box 298, New York, NY 10021; e-mail: gkuperman{at}nyp.org
- Received 11 January 2006
- Accepted 26 March 2006
The quality and safety of health care leaves much to be desired.1 2 Automated clinical decision support (CDS) tools embedded in clinical information systems (CISs) such as computer provider order entry (CPOE) and electronic health records (EHR) applications have the potential to improve care and should be part of any comprehensive approach to improve quality.3 4 5 6 Medication prescribing is a component of health care with well documented quality and safety problems that can be improved by CDS.7 8 9
Medication-related CDS requires that pharmaceutical knowledge be represented in a computable, explicit and unambiguous form. Creating an automated representation of medical knowledge often is the most time consuming step in the development of a CDS system and is known as the “knowledge acquisition bottleneck.”10 For a time, it was hoped that the move toward explicit guidelines in medicine would decrease the knowledge acquisition effort,11 but that has not happened.12 Experiments on data sharing from over a decade ago have not progressed.13 As a result, just a few organizations, primarily academic medical centers, are creating rules and benefiting from CDS,14 but most health care organizations do not have the expertise or resources to create such knowledge bases themselves.
One potential solution to the problem of access to automated medication-related knowledge is the set of commercial vendors that supply medication-related knowledge bases for pharmacy and prescribing applications. These vendors' products contain such knowledge as drug-drug and drug-disease interactions, minimum and maximum dosing suggestions, drug-allergy cross-sensitivity groupings, and groupings of medications by therapeutic class. Developers of CISs (either vendor-based or “homegrown”), with appropriate licensing, can incorporate commercial knowledge bases into their products. The knowledge base vendor receives a licensing fee for each CIS implementation and can amortize the costs of knowledge creation over several clients. Any given health care organization pays just a fraction of the total knowledge development costs and is spared the effort of developing the knowledge itself.
The CIS developers can create CDS features based on the commercial knowledge base, for example, a drug-drug interaction checking feature within a CPOE application. To the end-user of the CIS, it may not be evident that the CIS application and the knowledge contained therein are two distinct components. The CIS developer may provide end-user configuration tools that allow certain decision support features to be turned on and off.
However, several end-user organizations have reported that, when used as the basis for CDS within clinical systems, commercial knowledge bases generate excessive number of alerts, many of which are clinically unhelpful.15 Specific examples include overly sensitive drug-allergy interaction checking,16 drug-drug interaction checking,17 and dose limit checks.18 Such frequent alerts are bothersome to practicing clinicians who deem them to be “nuisance” or “nonsense” alerts. Such nuisance alerts are problematic in various ways. First, they are not helpful in the particular circumstance. Second, they degrade clinician confidence in the entire category of alerts (e.g., a clinically irrelevant maximum dose alert can decrease clinicians' confidence in all maximum dose alerts). Third, excessive nuisance alerts can decrease confidence in the alerting system and CDS as a whole and can cause clinicians to ignore clinically relevant alerts. Fourth, inappropriate CDS may cause clinicians to be dissatisfied with the CIS within which the CDS resides.19 20 21 Such dissatisfaction may impede the nation's progress towards the policy goal22 of rapid and broad dissemination of health information technology.
Another problem with the use of commercial knowledge bases is that the tools within the CIS that allow the CDS features to be turned on and off may be inadequate to achieve the alerting functionality desired by the end-user. For example, the tools may permit all class-based duplicate medication alerts to be turned off, but the same tools may not have the capability to turn off the alerting feature for some medications (e.g., antibiotics) while leaving it on for others (e.g., potassium preparations).
The result of all these problems is that, in at least some instances, decision support features in CISs that are based on commercial medication knowledge bases have not been well received by the clinician community.15 19
Perhaps it should not be surprising that the use of commercial knowledge bases within CISs results in a large number of alerts. Knowledge base vendors and CIS developers may have incentives (legal as well as financial) to point out when any possible breach of reasonable behavior has occurred. A naïve perspective may be that “any information is good” and some parties may not appreciate the costs of false positive alerts and the human factors involved when an excessive number of alerts are generated.
Some health care organizations have found that despite the fact that commercial knowledge bases generate a large number of alerts if used in an unfiltered way, the number of alerts can be reduced and the overall value of the alerting functionality can be increased if the behavior of the knowledge base is customized to be in alignment with local preferences (Figure 1). Such organizations have created filters to reduce the total number of alerts generated by the commercial knowledge base. In addition to filters, the organizations also have created automated knowledge structures that allow local preferences to be represented and folded into the alerting logic. Examples of such filters and other local customizations include excluding certain drug-allergy cross-sensitivity rules,16 increasing maximum drug dose levels,18 and filtering out certain drug-drug interactions.23 In addition, these organizations have found that with just a moderate amount of additional knowledge engineering and technical effort, a vendor-supplied knowledge base can be made much more useful.18 Such organizations are leveraging the knowledge base vendor's substantial effort in knowledge engineering, and are customizing it to meet local requirements.
Model for Customizing the Behavior of a Vendor Knowledge Base. (KB = knowledge base, CDS = clinical decision support, CPOE = computer provider order entry)
Health care organizations that wish to enhance a commercial knowledge base with local customizations face several challenges. First, the organization must have in place the management processes to decide on the specific CDS functionality that it wishes to implement. Second, the organization needs to build the components that can alter the usual behavior of the vendor's knowledge within the CIS. Such components may include filters that prevent certain alerts from firing18 and/or auxiliary knowledge tables that permit additional or different rules to fire. An organization that builds such components must have detailed knowledge of the structure of the vendor knowledge base as well as the architecture of the CIS. The architecture of the CIS must be open. Developing such tools may be impossible if the architecture of the CIS is closed. Third, the health care organization needs to assure that its knowledge base customizations will not be erased or altered when the vendor sends updated versions of the knowledge base. Fourth, because of the absence of knowledge representation and medical concept standards, one health care organization usually cannot share its knowledge base customizations with other organizations.9 Fifth, there may be liability concerns. For example, if a potentially life-saving alert is rendered non-functional by a local customization and a patient suffers harm that might otherwise have been prevented, the CIS developer, the knowledge base vendor and/or the health care provider organization theoretically could be liable.24 Sixth, customizations to knowledge base behavior could complicate legal agreements among knowledge base vendors, the CIS developers, and end-user organizations.
Given the foregoing concerns and complexities, it is unlikely that end-user customization of commercial knowledge bases will meaningfully increase the use of CDS. There are, however, some actions that CIS and knowledge base vendors could take to allow end-user organizations to more easily customize knowledge base behavior and realize attendant benefits. We recommend that CIS and knowledge base vendors:
-
Provide tools that allow end-user organizations to more easily customize the native knowledge bases in a way that decreases the frequency of clinically unhelpful alerts.16 17 18 Giving users the ability to customize severity levels of interactions and/or filter out specific classes of interactions would be one place to start. Another desirable customization feature would be to allow end-users to add to the knowledge base, e.g., to add a drug-drug interaction not in the vendor's knowledge base. The end-user customizations should not be affected by successive updates to the commercial products.
-
Provide better tools for browsing the knowledge.
-
Articulate the “knowledge architecture,” i.e., the set of knowledge-related data structures and software services, within which their products operate. It should be clear how the knowledge base and the CIS interact.
-
Design their products so that customizations made by one health care organization can be exported to other health care organizations that use the same products.
-
Adopt standards for knowledge base representation and concept identifiers so that in the long term such customizations could span knowledge base products and CISs.
-
Facilitate communications among organizations so that when one organization has made a set of customizations to a commercial knowledge base that it believes is clinically useful, such customizations can be evaluated and refined by other organizations.
We further recommend that:
-
To minimize risk, healthcare organizations create explicit policies and procedures regarding knowledge editing. Such policies should require that rule specifications be documented explicitly and that an identified individual or clinical committee be responsible for the content of the rule.25 26 The organization must assure that the edited knowledge is behaving as intended.27
-
Research should be targeted towards understanding how best to use commercial knowledge bases for CDS that is well accepted by practicing clinicians
-
Work on sharable standards for knowledge structures and concept identifiers should continue to be supported.
Conclusion
If the benefits of CDS are to be more broadly realized, we will need to accelerate the availability of clinically useful knowledge bases. Knowledge base vendors and CIS developers have an important role to play but health care organizations will realize value only if they have the ability to customize the knowledge to be more suited to their local clinical environment. Purchasers of knowledge bases and CISs should encourage the vendors to supply sophisticated knowledge customization tools and to be explicit about the knowledge architecture in which they operate.
Footnotes
-
The authors gratefully acknowledge helpful comments made by Jane Metzger on an early draft of this manuscript.
