Errors associated with outpatient computerized prescribing systems
- Karen C Nanji1,
- Jeffrey M Rothschild2,3,
- Claudia Salzberg3,
- Carol A Keohane3,
- Katherine Zigmont3,
- Jim Devita4,
- Tejal K Gandhi2,3,5,
- Anuj K Dalal2,3,
- David W Bates2,3,5,
- Eric G Poon2,3,5
- 1Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA
- 2Harvard Medical School, Boston, Massachusetts, USA
- 3Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA
- 4CVS Pharmacy, Woonsocket, Rhode Island, USA
- 5Partners Healthcare, Boston, Massachusetts, USA
- Correspondence to Dr Karen C Nanji, Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, 55 Fruit Street, Boston, MA 02114, USA;
Contributors KCN had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
- Received 22 February 2011
- Accepted 2 May 2011
- Published Online First 29 June 2011
Objective To report the frequency, types, and causes of errors associated with outpatient computer-generated prescriptions, and to develop a framework to classify these errors to determine which strategies have greatest potential for preventing them.
Materials and methods This is a retrospective cohort study of 3850 computer-generated prescriptions received by a commercial outpatient pharmacy chain across three states over 4 weeks in 2008. A clinician panel reviewed the prescriptions using a previously described method to identify and classify medication errors. Primary outcomes were the incidence of medication errors; potential adverse drug events, defined as errors with potential for harm; and rate of prescribing errors by error type and by prescribing system.
Results Of 3850 prescriptions, 452 (11.7%) contained 466 total errors, of which 163 (35.0%) were considered potential adverse drug events. Error rates varied by computerized prescribing system, from 5.1% to 37.5%. The most common error was omitted information (60.7% of all errors).
Discussion About one in 10 computer-generated prescriptions included at least one error, of which a third had potential for harm. This is consistent with the literature on manual handwritten prescription error rates. The number, type, and severity of errors varied by computerized prescribing system, suggesting that some systems may be better at preventing errors than others.
Conclusions Implementing a computerized prescribing system without comprehensive functionality and processes in place to ensure meaningful system use does not decrease medication errors. The authors offer targeted recommendations on improving computerized prescribing systems to prevent errors.
- Patient safety
- quality of care
- improving healthcare workflow and process efficiency
- developing/using clinical decision support (other than diagnostic) and guideline systems
- measuring/improving patient safety and reducing medical errors
- decision support
- data exchange
- medical informatics
- decision support
- healthcare information technology
Medication errors and adverse drug events (ADEs) are common and often preventable in the ambulatory setting,1 2 and electronic prescribing (e-prescribing) has been highlighted by researchers,2 3 policy makers,4 and payers5 as an important strategy to prevent them. While the bulk of the evidence suggests that enthusiasm for this patient safety technology is well founded,6–8 research has also demonstrated the potential for new technologies to result in unintended consequences.1 9–13 In the case of electronic prescribing in the ambulatory setting, chief among these unintended consequences is the potential for computer-generated prescriptions to introduce new errors. Furthermore, computerized generation of prescriptions is unlikely to prevent all errors made with manual paper prescriptions. For example, wrong patient or wrong diagnosis prescription errors may not be prevented by electronic prescribing systems. Errors associated with inpatient electronic prescribing have been addressed in the literature,14 but outpatient prescribing has been less well studied.15 Outpatient pharmacies have increasingly expressed concerns about errors, since they may cause serious patient injuries and often result in unnecessary delays in the dispensing process.4 16 If left unaddressed, such errors may harm patients and could delay the universal adoption of electronic prescribing.
The federal government has put in place major incentives for providers to adopt electronic health records through the meaningful use incentives, in part because it is believed they will improve safety, quality, and efficiency. Many of these benefits are expected to come as the result of electronic prescribing. However, many of these benefits will not be realized if the electronic prescribing applications are not mature and either do not catch or even cause new medication errors.
The aims of this study were to characterize the errors associated with computer-generated prescriptions in the community setting, understand their contributing factors, and develop a framework to classify the errors to determine which strategies have the greatest potential to both prevent errors and improve patient safety.
Materials and methods
Overall study design
Following Institutional Review Board approval, we reviewed 3898 prescriptions received by a commercial outpatient pharmacy chain across three states. The pharmacy anonymized all computer-generated prescriptions received over a 4-week period in 2008 and faxed them to our research group. A total of 48 prescriptions were excluded due to illegible facsimile transmission, resulting in 3850 eligible prescriptions. All of the prescriptions originated from outpatient computerized prescribing systems that are used in physician offices by providers caring for patients outside the hospital.
Each computer-generated prescription was reviewed by a panel of independent clinical reviewers (see Medication error review and classification) using a previously described method.1 17 Clinician reviewers identified medication errors, classified each error by type, and determined both its potential for patient harm and the severity of the potential harm. An error with the potential for harm was considered a potential ADE. The severity of potential ADEs was classified as significant, serious, or life-threatening.
Our primary outcomes were the incidence of medication errors and potential ADEs found on electronic prescriptions. Our secondary outcome was the rate of prescribing errors by error type. Error types included dose errors, duration errors, frequency errors, route errors, formulation errors, drug product errors, and patient instruction errors. We also compared error rates and types of errors across various computerized prescribing systems to determine whether these varied among the different systems in our sample.
Medication error review and classification
We used a two-tier approach for identifying medication errors and potential ADEs. All prescriptions underwent an initial review by two independent clinician reviewers who identified and categorized suspected medication errors. Subsequently, prescriptions with suspected medication errors underwent a second evaluation by two independent physician reviewers to confirm the presence of an error and to identify the potential for each error to result in patient harm. Medication errors with potential for harm were considered potential ADEs. A potential ADE was termed significant if the event could cause symptoms that, while harmful to the patient, pose little or no threat to the patient's life function. Examples include a rash, diarrhea, headache, and nausea and/or vomiting. A potential ADE was considered serious if the event could cause symptoms that are associated with a serious but non-life-threatening level of risk or a persistent alteration of life function. Examples include altered mental status, symptomatic hypoglycemia, bradycardia, dizziness, and syncope. Potential ADEs were considered life-threatening if the event could cause symptoms or changes that, if not treated, would put the patient at risk of death. Examples include cardiac arrest, anaphylaxis, and respiratory failure.
We disregarded any hand-written annotations on prescriptions made either by the prescriber or by the pharmacist. We did not exclude any prescriptions based on the mode of transmission to the pharmacy (eg, hand delivery, facsimile, or other electronic transmission). Disagreements between the two physician reviewers on error classification and/or severity were resolved by consensus. To ensure consistency, classification rules were established for disagreements that surfaced frequently during the review process (available upon request). We further classified errors and potential ADEs by type of computerized prescribing system used to generate the prescription for the 12 most frequently encountered computerized prescribing systems in our sample.
Finally, we recommended strategies to reduce the frequency of errors that we identified. To do this, we iteratively regrouped the error types to create a mutually exclusive and collectively exhaustive framework that describes the major cognitive missteps that contributed to the prescription errors. Based on this framework, we recommended strategies to reduce the errors associated with each type of cognitive misstep during the electronic prescribing process.
We compared medication error and potential ADE rates among the different computerized prescribing systems in our sample. Comparisons are presented as counts with percentages, and p values were calculated using the χ2 test.
Role of the funding source
This project was supported by grant number U18HS016970 from the Agency for Healthcare Research and Quality (Rockville, Maryland) and in part by a grant from the Harvard Risk Management Foundation, Cambridge, Massachusetts. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.
We reviewed a total of 3898 prescriptions, of which 48 were excluded due to illegible facsimile transmission. In our final sample of 3850 prescriptions, 1195 (30.7%) originated from Florida, 845 (21.7%) from Arizona, and 1858 (47.7%) from Massachusetts. Of these eligible prescriptions, 452 (11.7%) contained a total of 466 errors (examples shown in appendix, online). We classified 163 (35.0%) of these errors as potential ADEs, so 4.2% of prescriptions contained potential ADEs. Of the potential ADEs, 95 (58.3%) were significant, 68 (41.7%) were serious, and none were life-threatening. A breakdown of the errors by drug category (table 1) showed that the four most common classes of drug containing medication errors were anti-infectives (40.3%), nervous-system drugs (13.9%), and respiratory-system drugs (8.6%). The most common drug classes associated with potential ADEs were nervous-system drugs (27.0%), cardiovascular drugs (13.5%), and anti-infectives (12.3%). The six most commonly encountered anti-infectives in the sample accounted for 17.3% of medication errors.
Figure 1 shows the total number of errors and potential ADEs by prescribing system. We found a significant variation by prescribing system in prescribing errors (p<0.001) and potential ADEs (p<0.002). Prescribing error rates ranged widely from 5.1% (95% CI 0.3% to 9.9% errors) to 37.5% (95% CI 23.5% to 51.5% errors). Table 2 describes the total errors and potential ADEs for each system by error type. Systems A, B, and C were used in Massachusetts; H, I, J, and K in both Florida and Arizona; and L only in Florida. Systems D, E, F, and G were used across all three states. Different systems were associated with different error types. For example, the three most common types of errors associated with system A were omitted duration of treatment (27.3%), miscellaneous errors (27.3%), and omitted dose (12.1%); for system F, the most common errors were inappropriate abbreviations (25.0%), quantity to dispense not matching the directions (25.0%), and miscellaneous errors (20.8%). The frequency of potential ADEs also varied among systems (p<0.002). For example, although system B's total medication error rate (8.5%) was less than that of system G (13.0%), system B had substantially more potential ADEs than system G (3.9% of errors vs 0).
We iteratively grouped the errors found in the computer-generated prescriptions to create a framework to describe the underlying cause of the errors (figure 2). The most common cause for error was omitted information (60.7% of total errors and 50.9% of potential ADEs). The most likely information to be omitted was duration, dose, or frequency, and omitted dose was the most likely to result in a potential ADE, accounting for 35% of all potential ADEs in our sample. If an error's cause did not involve omitted information, then the information was unclear (16.1% of total errors, 19.6% of potential ADEs), conflicting (15.7% of total errors, 16.0% of potential ADEs), or clinically incorrect (7.5% of total errors, 13.5% of potential ADEs). Examples of errors involving unclear information included instructions to take a drug ‘as directed’ without further specification. Examples of prescriptions with conflicting information included those where the information in the sig or instructions section (eg, take 200 mg PO twice daily) conflicted with the expressed directions (eg, take 400 mg PO twice daily for 1 day, then 200 mg PO twice daily). Examples of prescriptions that were clinically incorrect included potential overdoses such as instructions to take ‘Vicodin 5/500 mg 1–2 tabs PO q4–6 h prn.’ Table 3 shows the total number of errors and potential ADEs made in each category.
Providers appear to be rapidly adopting electronic health records and computerized prescribing, and one of the major anticipated benefits is expected to be through medication-error reduction. However, many of these benefits may not be realized if the computerized prescribing applications are not mature. In this study of computerized prescriptions received by an outpatient pharmacy chain in three states, we found that just over one in 10 computer-generated prescriptions contained errors, and about a third of the errors in prescriptions had the potential for harm. Roughly six in 10 errors related to omitted information, which should be relatively easy to eliminate. The rates of errors varied by vendor, and some had higher potential ADE rates than others.
Our results in terms of error frequency with electronic prescription are consistent with outpatient handwritten and electronic prescription error rates that are reported in the literature, although we evaluated more systems than in prior reports.2 In one study, Gandhi et al reported that 7.6% of computer-generated and handwritten outpatient prescriptions contained errors, of which 43% were potential ADEs. In that study, the use of basic computerized prescribing systems was not associated with reduced error rates. Similarly, Devine et al found an 8.2% prescription error rate after implementation of a CPOE system that included basic decision support and dosage calculators in a community-based, multispecialty health system.18
Although we found an average error rate that is consistent with these studies, we found in addition that the number, type, and severity of prescribing errors varied significantly according to which computerized prescribing system was used, suggesting that either the system designs differed, for example, with a superior user interface or more advanced functionality; or implementation varied, since, for example, better clinician training may result in safer prescribing.19
Based on our framework (figure 2), we identified several strategies to minimize the errors associated with computer-generated prescriptions, including errors that may not be seen in hand-written prescriptions. These strategies include both computer-based interventions and provider-based interventions.
Examples of computer-based interventions are outlined below, and include forcing functions, specific drug decision-support systems such as maximum dose checkers, and calculators.
Forcing functions can be designed to prevent omitted information, incomplete drug names, medications, with instructions to be taken as needed without a specific indication, and inappropriate abbreviations. In our sample, forcing functions could have eliminated 71.7% of total errors and 63.2% of potential ADEs.
Specific drug decision support, including features such as maximum dose checking, have the capacity to eliminate clinical errors such as wrong dose or frequency, which comprised 7.5% of errors and 13.5% of potential ADEs associated with the computer-generated prescriptions in our sample.20 21 These represent some of the errors that have been most likely to result in harm in the inpatient setting, suggesting that this may be an important priority.22
Calculators can resolve inconsistent quantity errors by eliminating redundant data entry. For example, instead of entering the final quantity to dispense, the system calculates this quantity from the duration of treatment and the frequency of administration entered by the physician. In our sample, a simple calculator could have eliminated 5.6% of total errors and 1.2% of potential ADEs, namely those errors where the quantity to dispense does not match the quantity in the patient directions. Although this seems like a small number of errors prevented, 3.5 billion prescriptions are written per year, and with a 10% error rate, 6% of errors prevented is equivalent to 21 million errors.
When implementing any computer-based intervention, the benefits of reduced errors and improved patient safety must be weighed against the cost of physician resistance to using an inflexible system while under significant time pressure. If the intended benefits are well conveyed, the system is well designed and flexible enough to allow for non-standard specifications, and training is sufficient, acceptance of the system can be very good.11 23–25 However, specific issues around human-factors design in the decision support should receive careful consideration. For example, alerts should steer providers in the right direction early on instead of providing retrospective warnings after users have invested time entering the information in question.21 In order to incorporate computer-based interventions into current systems without generating significant physician resistance, the content for decision support should be developed and tested with clinicians in the field. User groups may be a good forum for iterative testing and clinician feedback on system design.
Unlike computer-based interventions, which focus on eliminating errors on a per-prescription basis, provider-based interventions focus on ensuring that the computerized prescribing system's design and implementation support the elimination of errors. Provider-based interventions may include rigorous vendor selection, aligned financial incentives, and strong training.
To ensure a minimum standard of system functionality, rigorous vendor selection should begin with vendors who have certified electronic prescribing systems. However, certified electronic prescribing systems alone do not guarantee success, as the current certification criteria are pass/fail for all criteria, and remain subject to change. For example, while the criteria require drug–drug interaction checking, there is no check of whether or not the key interactions are in place or whether the warnings when delivered adhere to human factors design principles. As the criteria for certification continue to evolve, it is important not only to choose certified systems but also to monitor total errors (see figure 2) and error distribution (see table 3) on an ongoing basis. The vendor selection process should also eliminate vendors that are unwilling to commit to resolving problems as they arise through system revisions such as the addition of forcing functions.26 27 Finally, implementation and adoption may be facilitated if vendors commit to providing long-term, on-site training that covers all shifts in order to minimize workflow disruptions.27 Internists prescribe an especially large proportion of prescriptions, and thus should be especially sensitive to how good the prescribing application is in the electronic health record they select.
In addition to vendor selection, adoption and implementation of electronic prescribing systems could include financial incentives to encourage physician adherence to meaningful system use. Simply purchasing a certified, fully functional electronic prescribing system is not enough to achieve improvements in quality and patient safety. Key system functionality must be used in order to improve safety and quality. Beginning in 2011, providers will be rewarded by the government for achieving ‘meaningful use’ of certified systems.28 These financial incentives will likely improve adoption of advanced electronic prescribing systems with forcing functions, which can be expected to result in better error-reduction rates. For example, the prevention of incomplete prescription information (eg, drug name, dose, frequency, duration of treatment, and quantity to dispense) could have resulted in the elimination of more than 70% of prescribing errors in our sample. Pharmacy feedback to providers on error rates may serve as a mechanism for monitoring meaningful use as a basis for financial incentives. Future meaningful-use regulations in this area should also include some testing of whether key safety checks are included in provider system implementations, since data from the inpatient setting have demonstrated that there is substantial variability in whether such checks are implemented.29
Training also represents a key determinant of successful adoption. While the vendor's training capabilities should be part of the vendor selection process, organizations need to ensure themselves that enough training of adequate types occurs.27 Training when a new electronic health record is implemented typically should account for 30–40% of costs, yet is often given short shrift (JP Glaser, personal communication, 2010). Tracking of how users are using electronic prescribing after implementation is also likely to be helpful, as some users may adopt less slowly or have specific issues with the system.
Our study has several limitations. First, we were not able to distinguish between true electronic prescriptions (ie, those that were electronically transmitted directly to pharmacies) and computer-generated prescriptions that were printed and either faxed or hand-delivered by the patient to the pharmacy. However, we believe that our results apply regardless of mode of prescription transmission. Second, although we identified a difference in error rate between various electronic prescribing systems in our sample, we were not able to assess whether this difference was due to variations in system design or the implementation process. This limitation occurred because these data represent many different systems being used by providers across three states, and the details on specific system functionality and implementation process were not available to us. Further research should investigate specific system characteristics that are associated with increased error rates. Third, our sample, while large, did not include all vendors and also included systems that appeared to be home-grown and not commercially available. Fourth, this study focuses on prescribing errors and was not designed to investigate whether the errors were intercepted at the pharmacy level, or other pharmacy workflow issues. Future studies should investigate subsequent error interception and resulting patient outcomes. Finally, this study was not designed to identify errors that are not evident on a prescription, such as the wrong patient receiving a medication, an incorrect diagnosis causing the wrong medication to be prescribed, or a drug–drug interaction. When considering these other sources of error, actual prescription error rate may be higher than the 11.7% we have identified.
In summary, about one in 10 computer-generated prescriptions in our sample included one or more errors, about two-thirds of which involved omissions. The number, type, and severity of prescribing errors varied significantly according to which computerized prescribing system was used, suggesting that systems with more advanced functionality, or those used by physicians with improved computer training, were better able to prevent errors. Implementing a computerized prescribing system without comprehensive functionality and processes in place to ensure meaningful use of the system does not decrease medication errors. To enable stakeholders to realize more of the potential benefits of computerized prescribing systems, vendors and healthcare providers may consider implementing several of the outlined computer-based and provider-based interventions, which combined have the potential to eliminate more than 80% of the errors. These data are especially important now because providers are currently rapidly adopting electronic health records, yet may not realize the full range of benefits if the prescribing applications have some of the issues we identified.
Funding This project was supported by grant number U18HS016970 from the Agency for Healthcare Research and Quality (Rockville MD) and in part by a grant from the Harvard Risk Management Foundation, Cambridge, Massachusetts.
Competing interests None.
Ethics approval Ethics approval was provided by Partners Human Research Committee.
Provenance and peer review Not commissioned; externally peer reviewed.