Provider and pharmacist responses to warfarin drug–drug interaction alerts: a study of healthcare downstream of CPOE alerts
- 1Department of Pharmaceutical Services, University of California San Francisco, San Francisco, California, USA
- 2Pharmacy Service, San Francisco Veterans Affairs Medical Center, San Francisco, California, USA
- 3Department of Medicine, San Francisco Veterans Affairs Medical Center, San Francisco, California, USA
- Correspondence to Dr Allison Miller, Department of Pharmaceutical Services, University of California San Francisco, 505 Parnassus Ave, Rm M39C, Box 0622, San Francisco, CA 94143, USA; allison.miller{at}ucsfmedctr.org
- Received 20 March 2011
- Accepted 23 August 2011
- Published Online First 28 October 2011
Abstract
Objective To categorize the appropriateness of provider and pharmacist responses to warfarin critical drug–drug interaction (cDDI) alerts, assess responses and actions to the cDDI, and determine the occurrence of warfarin adverse drug events (ADE) after alerts.
Design An 18-month, retrospective study of acute care admissions at a single Veterans Affairs medical center using computerized provider order entry (CPOE).
Measurements Patients included had at least one warfarin cDDI alert. Chart reviews included baseline laboratory values and demographics, provider actions, patient outcomes, and associated factors, including other interacting medications and number of simultaneously processed alerts.
Results 137 admissions were included (133 unique patients). Amiodarone, vitamin E in a multivitamin, sulfamethoxazole, and levothyroxine accounted for 75% of warfarin cDDI. Provider responses were clinically appropriate in 19.7% of admissions and pharmacist responses were appropriate in 9.5% of admissions. There were 50 ADE (36.6% of admissions) with warfarin; 80% were rated as having no or mild clinical effect. An increased number of non-critical alerts at the time of the reference cDDI alert was the only variable associated with an inappropriate provider response (p=0.01).
Limitations This study was limited by being a retrospective review and the possibility of confounding variables, such as other interacting medications.
Conclusion The large number of CPOE alerts may lead to inappropriate responses by providers and pharmacists. The high rate of ADE suggests a need for improved medication management systems for patients on warfarin. This study highlights the possibility of alert fatigue contributing to the high prevalence of inappropriate alert over-ride text responses.
Introduction
Implementing computerized provider order entry (CPOE) with clinical decision support was a primary recommendation of the Institute of Medicine's seminal ‘To Err Is Human’ report to help minimize the number of patients harmed or killed as a result of medical errors. Studies have shown that these systems can reduce prescriber error and decrease adverse drug events (ADE) compared with written orders.1–3 However, CPOE remains sporadically implemented, partly because information is needed to determine how to optimize these systems, particularly with respect to how medication alerts are processed.4
Background
Department of Veterans Affairs (VA) medical centers have had a CPOE system implemented for more than 10 years in 150 hospital environments. At the San Francisco VA Medical Center (SFVAMC), CPOE with basic decision support is utilized throughout the institution, bar code recognition technology is in place for verification and documentation of inpatient medication administration, and the patient medical record is fully integrated and available online. Providers are prompted when an ordered medication interacts with medications already included on the patient's active inpatient and outpatient medication profile. At the time of this study, these interaction alerts provide only information about interaction severity. When the interaction is categorized as a potentially critical drug–drug interaction (cDDI), providers are required to either cancel the order or enter an explanation as free text before over-riding the alert. Providers have a single text box to enter the reason for over-ride for all order checks being over-ridden at that time. When a pharmacist subsequently processes the order with a cDDI alert, they view the provider's over-ride response and are prompted to document their own intervention before processing the order; however, pharmacists are able to bypass this last step.
Results from a 2007 study of prescriber responses to CPOE critical alerts across six VA medical centers showed that few prescribers enter meaningful information when over-riding alerts.5 During a 1-year study period, 206 309 cDDI alerts occurred and no reasons were provided for 53% of the over-rides. Overall, 80% of the reasons provided by prescribers when over-riding cDDI alerts were characterized as non-useful.
VA Puget Sound published a study comparing rates of prescriber over-rides in 2001 and in a follow-up in 2006.6–8 Approximately 2.5% of inpatient orders generated critical alerts. In 2001, 88% of drug–drug interactions (DDI) and 69% of drug–allergy interactions (DAI) alerts were over-ridden. In 2006, a similar percentage of alerts, 87% of DDI and 81% of DAI, were over-ridden. Studies looking at clinical outcomes related to alert processing are less common. A study of over-rides of DAI and ADE alerts at Brigham and Women's Hospital found that 80% of DAI alerts were over-ridden. Six percent of the over-rides resulted in ADE.9 Of these, 47% were categorized as serious, but all over-rides that resulted in ADE were nevertheless considered clinically justified. Similarly, a study from Beth Israel Deaconess Medical Center in Boston found that 89% of high-severity DDI and 91% of DAI alerts were over-ridden in the primary care setting.10 They found no ADE when physicians did not prescribe the alerted medication and ADE in 2.5% of patients (a total of three patients) when the alerts were over-ridden. Two of the three ADE involved warfarin, a high-risk medication because its frequent interactions can lead to serious adverse events.11 12
In the SFVAMC CPOE system, 40 medications have cDDI with warfarin. One of The Joint Commission's National Patient Safety Goals is to reduce harm associated with anticoagulation in the inpatient setting.13 Thus, using warfarin as an ‘index’ medication for the study of CPOE systems and DDI alerts is warranted.
Studies focused on CPOE alerts and warfarin management to date have shown mixed results on provider actions. A study of alerts in the long term care setting found that 12% of the DDI alerts seen by providers were for warfarin interactions and that providers were only slightly more likely to take appropriate action if they were alerted versus a non-alerted control.14 Similarly, a customized CPOE alert requiring provider responses when warfarin and non-steroidal anti-inflammatory agents (NSAIDs) were co-prescribed did not reduce concomitant prescription rates compared to a usual standard of care.15 Conversely, a CPOE alert system providing alternatives to medications being co-prescribed with warfarin decreased co-prescription rates quickly but modestly and only for a single drug, acetaminophen.16
The objective of this study was to investigate the utility of cDDI alert processes by categorizing provider and pharmacist responses to warfarin cDDI alerts as clinically appropriate, tracking providers' actions after cDDI alerts, and determining the occurrence of warfarin ADE following the alerts.
Methods
Study location
The VA Veterans Health Information Systems and Technology Architecture (VISTA) database was queried for patients who had at least one cDDI alert to warfarin over-ridden during an acute care hospital stay. Admissions were included if the patient received warfarin and the interacting medication during their admission, were admitted from January 1, 2007 to June 30, 2008, and had an admission lasting at least 72 h. Admissions were excluded if the patient was on both warfarin and the interacting medication before admission or no INR was calculated during admission before warfarin was started. The alert of interest (AOI) was the first cDDI alert with warfarin that met all inclusion criteria.
This study was approved by the institutional review board at University of California, San Francisco and the Research Office at SFVAMC.
Data collection
The electronic medical records of included patients were reviewed to gather demographic information on age, sex, admitting service, length of stay, goal INR, admission INR, INR at the time of the AOI, indication for anticoagulation, other interacting medications and disease states, and warfarin dose at admission and at the time of the AOI. Also collected were provider and pharmacist responses to alerts, provider actions after the alert, and the occurrence of an ADE related to warfarin during the admission. Provider actions, including warfarin dose adjustments, interacting medication adjustment, documentation of the interaction in the medical record, and INR monitoring, were counted as related to the cDDI alert if they occurred within 72 h of the AOI. Pharmacists' responses were collected from an electronic database into which pharmacists are prompted to enter their interventions following a cDDI alert in the CPOE system. Non-critical alerts were classified as alerts within the VA CPOE system if identified as less severe than critical, such as duplicate order alerts, which are also shown to the provider at the time of order completion.
Data categorization
Provider and pharmacist responses when over-riding the cDDI alerts were categorized as clinically appropriate (hereafter referred to as appropriate), inappropriate, or absent based on criteria published by Grizzle et al.5 An appropriate response was used as a marker for the utility of the information provided in the alert. A provider was defined as a healthcare provider able to prescribe, such as a physician or nurse practitioner. For a provider response to be categorized as appropriate, the provider must have addressed adjusting the warfarin dose, adjusting the interacting medication, or INR monitoring. Pharmacist responses were categorized as appropriate when the pharmacist indicated they had contacted the provider to adjust the warfarin dose, adjust the interacting medication, or monitor INR. Other responses were categorized as inappropriate. Lack of a documented response was categorized as absent. All categorizations were conducted by one rater and verified by a second rater. Discrepancies were discussed by both raters. In situations where no consensus could be reached, a third rater was consulted. Due to the small number of subjects predicted to be enrolled in this study, inter-rater variability was not assessed.
An ADE was defined as an INR outside the goal range for the patient, administration of fresh frozen plasma or vitamin K that was not related to a surgical procedure, a bleeding event either noted in the medical record or as defined by the Thrombolysis In Myocardial Infarction (TIMI) trials, or a clotting event noted in the medical record.17 18 The TIMI trials defined a minor bleed as a ≥3 g/dl drop in hemoglobin and a major bleed as a ≥5 g/dl drop in hemoglobin. ADE severity was categorized as mild, moderate, or severe based on VA criteria.19 A mild ADE required a dose adjustment or medication discontinuation. A moderate ADE required an additional medication to reverse the ADE, such as vitamin K. A severe ADE required an escalation of the level of care or an increased length of stay. ADE were assessed until patient discharge from the acute care setting.
Statistical analysis
Descriptive statistics were used to summarize data found, including means, SD, and ranges. T tests and χ2 tests were used for univariate analysis to compare demographics and provider actions for admissions with versus without ADE and alert responses that were appropriate versus inappropriate. Binary logistic regressions were used for multivariate analysis. Goodness-of-fit for the logistic regression was assessed using Pearson and Hosmer–Lemeshow tests. The significance level was set at 0.05. Sigma Stat 2.03 and Microsoft Excel 2003 were used for the statistical analysis. The time period of 18 months was selected to include approximately 12 subjects per month with an anticipated sample size of 200. As this study was designed to be descriptive, no power studies were conducted prior to data collection.
Results
During the study period, there were 555 acute care admissions with a cDDI alert to warfarin over-ridden during the hospital stay. After chart review, 137 admissions were included in this study, accounted for by 133 unique patients. A total of 418 admissions were excluded, some due to multiple reasons. Grounds for exclusion included being on both medications prior to admission (294), failure to receive both medications together during the admission (171), admission lasting less than 72 h (109), or other miscellaneous reasons.7
The included population was mostly elderly, male veterans. Table 1 describes the demographic characteristics.
Demographic characteristics (n=137)
Table 2 describes the total number of cDDI alerts for admissions included and the medications implicated in the cDDI. Providers were presented with an average of six simultaneous alerts to warfarin at the time of the AOI.
Critical drug–drug interaction (cDDI) prevalence and interacting medications (n=137)
Amiodarone, vitamin E in a multivitamin, sulfamethoxazole in combination with trimethoprim, and levothyroxine were the most common medications interacting with warfarin in this study. Eighty-one percent of patients were newly starting warfarin or being restarted after a period off warfarin at the time of the AOI. Forty-seven percent of the population was on clopidogrel and/or aspirin at the time of the AOI.
The categorization of provider and pharmacist responses to cDDI alerts is given in table 3. Response categories were mutually exclusive.
Categorization of provider and pharmacist responses to and provider actions following critical drug–drug interaction (cDDI) alerts to warfarin (n=137)
The most common categories for responses are listed. For providers, the most common responses were ‘OK’ or ‘MD aware’ which were categorized as ‘Provider aware but no additional reason.’ Due to the set up of the system, providers were not allowed to have an absent response. For pharmacists, no response was most common, categorized as absent. The only variable found to be associated with inappropriate provider response was an increased number of non-critical alerts at the time of the AOI (p=0.01). Alerts with an inappropriate response had an average of four simultaneous non-critical alerts versus 2.4 simultaneous non-critical alerts for over-rides with appropriate responses. On multivariate analysis as well, the only factor associated with an inappropriate response was the number of simultaneous non-critical alerts. Providers were no more likely to have an appropriate response to amiodarone than to vitamin E, the two most common medications involved in the cDDI. Analysis was not completed on factors associated with a inappropriate or absent pharmacist response due to the small number of text responses.
Table 3 also shows the actions taken by the provider within 72 h of the alert. In 42% of admissions, providers decreased, held, or stopped warfarin within 72 h of the AOI. Providers adjusted or stopped the interacting medication in 11% of admissions.
There were 50 ADE related to warfarin documented in this study, representing 36.6% of admissions. Descriptions of the ADE are provided in table 4.
Adverse events following critical drug–drug interactions (cDDIs) to warfarin (n=50)
No INRs fell below the therapeutic window. Bleeding events occurred in nine (6.6%) of the admissions. No clotting events were noted. There was no association between ADE and an inappropriate provider response. Factors significantly associated with an ADE were increased total number of cDDI alerts to warfarin during admission (p=0.035), increased INR at admission (p=0.043), and increased INR at the time of cDDI alert (p<0.001).
Discussion
This study demonstrates that while cDDI alerts are common when warfarin is prescribed in the inpatient setting, mandatory responses to over-ride alerts are rarely helpful. This brings the necessity of obtaining a required over-ride reason into question, especially since in this study there was no association of an inappropriate response with ADE (see Box 1 for a summary of important findings).
Summary of important findings
-
Increased numbers of concurrent provider alerts are associated with fewer appropriate responses to critical drug–drug interaction alerts.
-
Inclusion of less clinically important medications in alert system may make providers discount important drug–drug interaction alerts.
-
Pharmacists are currently not using the Veterans Affair Intervention Package to document their involvement with warfarin critical drug–drug interaction alerts.
Quantity and quality of alerts
This study highlights the possibility of alert fatigue contributing to the high prevalence of inappropriate alert over-ride text entries in providers entering orders in the VA CPOE system. The only factor associated with an inappropriate response to cDDI alerts was an increased number of non-critical alerts, implying that the providers pay less attention if extra information of lesser significance is provided.
Providers had to over-ride a large number of CPOE alerts during the course of a patient's hospitalization. When signing orders with the VA CPOE system, providers are simultaneously shown all order checks for newly entered orders during that session, including both cDDI and non-critical alerts. Providers over-rode an average of 21 cDDI alerts to warfarin during a patient hospitalization. This is an underestimation of the total number of cDDI alerts over-ridden during the course of the hospitalization, as this study only assessed cDDI alerts to warfarin. This does not take into account the non-critical alerts shown to the provider, not requiring a response, which were not totaled as part of this study.
Some of the alerts could be considered clinically insignificant. The second most commonly involved medication in this study was vitamin E. All vitamin E over-rides from this study were from a multivitamin which contains 60 units of vitamin E at SFVAMC. To interact with warfarin, 400 units of vitamin E is required.20 21 This common but not clinically relevant cDDI could be exacerbating alert fatigue. While providers likely do not know the concentration of vitamin E in the multivitamin, the frequency of this interaction and perception by medical staff that an interaction with a multivitamin is not relevant may cause providers to discount other truly significant cDDI that are categorized in the same manner. This study showed providers had a similar rate of appropriate responses for vitamin E as for amiodarone, medications of different clinical importance. It was determined that the SFVAMC had locally upgraded the vitamin E–warfarin DDI from nationally set significant DDI to cDDI. Based on the low vitamin E content in multivitamins, the DDI was downgraded and no longer requires providers to enter a comment. However, in a study from Kaiser Permanente where cDDI warfarin alerts were specifically tailored to just five co-prescribed medications to help limit alert fatigue, the only significant decrease in prescription rates was for acetaminophen, the most controversial co-prescribed agent in terms of clinical relevance.16
Providers had a low rate of entering appropriate responses when over-riding cDDI alerts. One text box is supplied in which to write responses to all alerts detected by the system at the time of signing the orders. Results from this study show that providers were over-riding on average six interactions to warfarin at the time of completing each medication ordering session. This does not account for simultaneous order checks to medications other than warfarin ordered at the same time. Limited space and the large number of order checks could account for providers' failure to consistently document why they are over-riding the alerts.
Type of alert system
Because of its large number of known interactions, warfarin itself may be related to a lower likelihood of provider comment entry. A small study at New York-Presbyterian hospital noted that alerts including warfarin elicited the highest proportion of ‘content-free’ provider comments.22 However, other studies, including that of Singh et al, have suggested that anticoagulants are associated with the highest risk of potential ADE in the VA environment and are probably deserving of cDDI alerts with improved usability, to improve patient safety.23
Some data have recently been published regarding the optimization of CPOE alert systems. A recent study demonstrated that CPOE with basic decision support, such as the system at the VA, decreases medication errors but not adverse events.24 This could be related to the results of another recent study showing that new providers, in this study medical and surgical residents, often responded to basic alerts appropriately but their underlying reasons were incorrect.25 With regard to the actual structure of the alert system, some recent literature has demonstrated that non-interruptive alerts are not helpful in terms of medication effect monitoring.26 Others suggest that a ‘tiered alert’ strategy is associated with higher compliance rates with DDI alerts.27 28 In a warfarin specific study, a system with enhanced alerts for antibiotics that interact with warfarin showed that fewer providers prescribed interacting medications with enhanced alerts than with standard alerts, but were not more likely to re-check an INR within 7 days, if prescribed.29 Other studies of highly customized alerts specific to warfarin within CPOE systems appear to have limited impact on prescribing patterns and even sophisticated warfarin dose-modification CPOE tools improve dosing only a minority of the time.15 16 30 More intelligent systems to improve warfarin dosing safety are clearly warranted.
Both this study and that completed by Grizzle et al showed that providers have appropriate responses when over-riding cDDI in only 20% of VA CPOE alerts.5 It is interesting to note the consistency of these results. The Grizzle study looked at many more types of medications. Even when looking at orders involving a high alert medication such as warfarin, providers have a difficult time documenting their rationale when over-riding the cDDI alert. This suggests that the current system within the VA does not accurately capture why providers are over-riding these alerts. As pointed out by both Grizzle et al and Singh et al, the system is not being utilized by providers as a communication tool with pharmacists, as was intended. Communication between the two services appears to not be documented, if it occurs at all. The design of this system, with limited space available to communicate about multiple medications, may be inadequate for its intended purpose.
Providers may lack knowledge about the DDI, the consequences to the patient, the likelihood of an interaction, or the immediacy of the interaction. The current system does not provide any decision support on the direction of expected INR change, time course, or relative severity of the DDI. Such support could improve providers' knowledge on how to manage patients facing such interactions. Indeed, increasing the specificity of alerts has been suggested as the most important step in improving alert utility and decreasing high over-ride percentages.4 23
Patient outcomes
Overall, there was good patient follow-up after the cDDI alerts. Over 97% of admissions had INR checks within 72 h of the cDDI alert. However, for the four patients who did not have their INR checked within the 72 h window, their INR remained unchecked for the rest of their hospitalization. Since it is known that hospitalization is associated with fewer days in the therapeutic range, ideally all inpatients on warfarin, especially after a cDDI, should have their INR checked.31 Even with appropriate patient monitoring, a large percentage, over one third, had their INR rise above their goal range or had other adverse events. The quality of anticoagulation therapy and degree of INR variability could be tightened.
Pharmacist responses
This study found a very low rate of pharmacists providing reasons when over-riding cDDI alerts, with just under 10% of admissions having an appropriate alert response by the pharmacist, and most having no response. To our knowledge, no previous study has been conducted assessing pharmacist responses to DDI alerts. This study may have underestimated the number of pharmacist actions in response to the cDDI alerts. It is possible for pharmacists to bypass the electronic intervention documentation system. Pharmacists could have contacted providers when they processed the order to make a recommendation about monitoring or dose adjustment but not entered the intervention into the system. Recommendations by clinical pharmacy staff interacting with providers on patient wards were not captured in this study. Despite all of these possible other ways pharmacists may have been involved in patient care, our institution requests pharmacists processing inpatient orders to document their dialog with the provider for the safety of the patient. It currently is not clear that pharmacists are providing necessary input to providers. This study has prompted an increased emphasis within the pharmacy service to require pharmacists to complete this documentation step.
Limitations
There were a number of additional limitations to this study. ADE may not have been directly related to the AOI. For some admissions, more than one medication may have fit the inclusion criteria of a new medication to interact with warfarin. The first medication to appear on the data output was selected as the reference medication. For example, if a patient already on a multivitamin with vitamin E and fluconazole was newly started on warfarin, both medications would have a cDDI to warfarin. If vitamin E appeared first on the data output, it was selected as the reference medication, although its clinical implication would most likely be less than that of fluconazole. The alerts would have appeared to the providers in this same order.
While hospitalized, many other clinical factors could have made patients more susceptible to elevated INRs and other ADE. Alterations in diet and changes to other medications may have affected patients' INR and bleeding risk.32 Many of the patients were new starts or restarts on warfarin at the time of the AOI. ADE may have been related to aggressive warfarin dosing or a naive patient's sensitivity to warfarin, not the cDDI. This study looked at ADE that occurred during the patient's hospitalization. Some interactions may have been delayed and not become apparent until after the patient's discharge.
As a retrospective chart review, this study depended on accurate documentation in the medical record of patient care and adverse events. There may have been an underestimation of the number of bleeding events or clotting events. Also, providers may have been planning to modify the dose of warfarin or of the interacting medication but did not document these plans in their notes. Actions taken by the provider before the cDDI alert were not collected. If a potentially interacting medication was discontinued at the time of or immediately before the initiation of warfarin, the patient would not have been captured for inclusion in this study.
Conclusion
The large number of CPOE alerts that providers and pharmacists must respond to in the VA CPOE system is associated with inappropriate responses to cDDI alerts, even with high risk medications such as warfarin. This decision support tool is not being fully utilized by providers at the point of care and review by pharmacists does not improve documentation of over-ride rationale. While most of the patients in this study had appropriate follow-up, the high rate of ADE suggests a need to improve the quality of appropriate medication management following cDDI alerts. Although the rationale for DDI alerts is reasonable, improvements in design are needed to overcome the prevalence of inappropriate responses to alert over-rides. This study provides additional support for the necessity of improving alert utility in existing and future CPOE systems.
Acknowledgments
Thanks to Kathy Schulz, Nancy Fong, PharmD, Lisa Tong, PharmD, and Michael Winter, PharmD for their assistance with study design.
Footnotes
-
The work was completed at the San Francisco Veterans Affairs Medical Center, in San Francisco, California, within the departments of pharmacy and information technology. Dr Miller currently works at the Department of Pharmaceutical Services, University of California San Francisco Medical Center.
-
This research was presented in part at the 2009 American Society of Health Systems Pharmacists Midyear Clinical Meeting and the 2010 VA eHealth University conference.
-
Competing interests None.
-
Ethics approval The institutional review board at University of California, San Francisco and the Research Office at San Francisco VA Medical Center both provided ethics approval for this study.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement The authors would be happy to share data on request.
