rss
J Am Med Inform Assoc 13:547-556 doi:10.1197/jamia.M2042
  • Original Investigation
  • Research Paper

Types of Unintended Consequences Related to Computerized Provider Order Entry

  1. Emily M Campbell,
  2. Dean F Sittig,
  3. Joan S Ash,
  4. Kenneth P Guappone,
  5. Richard H Dykstra
  1. Affiliations of the authors: Department of Medical Informatics and Clinical Epidemiology (EMC, DFS, JSA, KPG, RHD), Oregon Health & Science University, Portland, OR; Department of Medical Informatics (DFS), Northwest Permanente, P.C., Portland, OR; Kaiser Sunnyside Medical Center (RHD), Portland, OR
  1. Correspondence and reprints: Joan Ash, PhD, Department of Medical Informatics and Clinical Epidemiology, Mail Code: BICC, 3181 SW Sam Jackson Park Road, Portland, OR 97239-3098; e-mail:<ash{at}ohsu.edu>
  • Received 21 December 2005
  • Accepted 6 June 2006

Abstract

Objective To identify types of clinical unintended adverse consequences resulting from computerized provider order entry (CPOE) implementation.

Design An expert panel provided initial examples of adverse unintended consequences of CPOE. The authors, using qualitative methods, gathered and analyzed additional examples from five successful CPOE sites.

Methods Using a card sort method, the authors developed a categorization scheme for the 79 unintended consequences initially identified and then iteratively modified the scheme to categorize 245 additional adverse consequences resulting from fieldwork. Because the focus centered on consequences requiring prevention or remedial action, the authors did not further analyze reported unintended beneficial (positive) consequences.

Results Unintended adverse consequences (UACs) fell into nine major categories (in order of decreasing frequency): 1) more/new work for clinicians; 2) unfavorable workflow issues; 3) never ending system demands; 4) problems related to paper persistence; 5) untoward changes in communication patterns and practices; 6) negative emotions; 7) generation of new kinds of errors; 8) unexpected changes in the power structure; and 9) overdependence on the technology. Clinical decision support features introduced many of these unintended consequences.

Conclusion Identifying and understanding the types and in some instances the causes of unintended adverse consequences associated with CPOE will enable system developers and implementers to better manage implementation and maintenance of future CPOE projects.

Introduction

Computerized provider order entry (CPOE), narrowly defined, is the process by which physicians or their surrogates (but not intermediaries) directly enter medical orders into a computer application. CPOE systems commonly exist as one of many integrated clinical applications in larger institutions’ information systems; the other applications offer complementary functionality such as real-time clinical decision support, on-line clinical documentation, and electronic message transmission. This study refers to CPOE systems as containing, at a minimum, electronic order entry capabilities, whether or not this functionality is part of a larger, more complex information system.

Health care organizations often implement CPOE as part of their approach to improve medication safety and reduce health care costs.1 2 3 Yet, several studies indicate that unpredictable, emergent problems, or unintended adverse consequences (UACs) can surround CPOE implementation and maintenance.4 5 Careful identification, description, and categorization of UACs can provide insight into the unexpected outcomes of placing CPOE systems into complex health care work environments.

The purpose of this study was to identify and describe the major types of UACs related to CPOE implementation. Because CPOE implementations affect many different types of personnel in the health care environment, their evaluation must encompass multiple, discrete perspectives. The current study focuses not on the impact of CPOE on clinical outcomes for patients, but instead on impacts affecting health care personnel who use, maintain, or manage CPOE systems. Specifically, we gathered perspectives regarding CPOE from three groups: clinical end-users, IT staff, and administrators. We broadly define clinical end-users as those health care providers and other clinical staff (e.g., physicians, pharmacists, nurses, ward secretaries, etc.) who work with CPOE systems. IT staff includes those who implement, configure, maintain, and support CPOE systems, whether or not their primary professional background is technical or clinical in nature. Finally, administrative staff refers to those who manage organizational implementation of CPOE, through establishing policies and procedures, assuring compliance with local and federal guidelines, and making high-level CPOE-related resource allocation decisions.

Background

Theoretical Framework

Diffusion of Innovations (DOI) theory served as the framework for this study. Diffusion has been defined by Everett Rogers as “the process by which an innovation is communicated through certain channels over time among the members of a social system,” and an innovation is “an idea, practice, or objective perceived as new by an individual, a group, or an organization.”6 While adoption of any innovation inevitably generates consequences, Rogers notes that such consequences constitute the least studied aspect of diffusion of innovation. DOI theory suggests that consequences can be desirable or undesirable, and anticipated or unanticipated. In this study, unintended consequences refer to events that are neither anticipated or the specific goals of the associated CPOE project. Although unintended most often connotes consequences that are both unanticipated and undesirable, we have found, in our prior work, numerous unintended desirable positive, beneficial consequences of CPOE as well.7 While the current study focuses on unintended adverse consequences of CPOE adoption, it is important to remember that unintended consequences are not uniformly errors or mistakes: they are simply surprises that can span a spectrum from lucky to unfortunate. Errors and adverse events comprise a subset of all consequences.

Methods

Selection of Sites

The initial expert panel helped the authors to identify five hospitals in three different organizations where successful CPOE implementation occurred (See Table 1) To study a range of successful CPOE implementations, the project selected both academic and non-academic hospitals of differing sizes and geographic locations. Each site had used distinct CPOE products (either developed “in house,” or purchased commercially) for varied implementation durations. All had high overall percentages of orders directly entered by clinicians.

Table 1

Description of Sites Studied

Data Collection Methods

Institutional Review Board approval for the study was granted by the Oregon Health & Science University and all study sites. Initial expert group discussions focusing on the unintended consequences surrounding the implementation of CPOE led to subsequent fieldwork using participant observation and semi-structured oral history interviews. In April 2004, at the rural Menucha Conference Center in Corbett, OR, the project team and informatics leaders (technical and clinical experts) participated in a two-day conference focused on identifying and understanding unintended adverse consequences of CPOE implementation. The conference discussions were recorded and transcribed.8

The project team’s multidisciplinary informatics researchers (two physicians, a nurse, a pharmacist, a librarian, a public health researcher, and a technically-oriented informaticist) carried out participant observation at the five field sites (CPOE hospitals) by accompanying hospital staff (resident physicians, pharmacists, nurses and allied health care providers) during the course of their daily activities. Researchers unobtrusively observed the hospital staff as they interacted with the CPOE systems. Project members documented the staff’s activities and comments; only rarely did researchers interrupt the staff to ask for clarifications. Project members recorded observations in field journals that were later transcribed into formal field notes. Occasionally, circumstances allowed informal interviews of the observed clinicians. If taped, these interviews were subsequently transcribed. The project selected certain clinicians, and hospital and IT administrators, based on their long history and involvement with both the institution and the implementation of the CPOE systems, to undergo debriefing through formal oral history interviews with open-ended questioning. Rather than observing these clinicians interacting with CPOE systems, project members instead asked them to describe the development and adoption of these systems in their respective organizations. Interviews were audiotaped and transcribed by experienced oral-history transcriptionists.

During nine months of field data collection (August, 2004 through April, 2005) project team members spent 390 total hours observing roughly 95 clinical providers interacting with CPOE systems in various settings. The 32 semi-structured interviews totaled approximately 43 hours. Transcripts from these interviews and the Menucha conference, and field notes comprised 1,894 single spaced, typed pages. The project team collected and compiled the field notes and interview transcriptions using qualitative research software (N6, QSR International Pty. Ltd., Melbourne, Australia, 2002).

Data Analysis

The project research team of six individuals met 36 times to analyze data. Using a card sort method,9 researchers developed a categorization scheme for the 79 unintended adverse consequences identified by the expert panel. Individual team members identified UACs in specifically assigned transcripts. During team meetings, consensus developed regarding which quotes represented UACs and how these UACs could be categorized. Using a grounded theory approach, the categories emerged from the data, rather than from preconceived expectations.10 Researchers then iteratively modified the initial categorization scheme as they reviewed the 245 additional unintended consequences identified during fieldwork. After several months of analysis, common themes emerged. The team formalized its list of UAC categories. The final list was both simple (consisting of only nine categories) and comprehensive (the categories directly covered all observed UACs).

Results

Introduction

Nine major types of unintended consequences emerged from the data. Table 2 lists the UACs and their frequencies of occurrence. A detailed discussion of each UAC type follows below, including direct quotations from speakers who articulated an issue particularly well. Table 3 (available as a JAMIA on-line supplement at www.jamia.org) includes additional quotes relevant to each type of UAC. Because study subjects were promised confidentiality, researchers edited statements to protect confidentiality whenever original statements potentially identified either speakers or the observation site.

Table 2

Unintended Consequences and Their Frequencies of Occurrence

Types of Unintended Adverse Consequences

Type 1: More/New Work for Clinicians

Clinical systems can potentially create new work for all staff members (e.g., both clinical and non-clinical staff). The present UAC focuses on the ever-increasing workload of clinicians. Despite the common CPOE implementation goal of providing a better “patient overview” to the clinician, many CPOE systems make clinicians do more work to get this overview than before CPOE implementation. The CPOE systems may engender new work by requiring that clinicians: (a) enter new information (e.g., justification for a treatment selection) not previously required; (b) respond to excessive alerts that may contain non-helpful information (e.g., non-specific medication interactions with no application to the current patient); or (c) expend extra time in completing non-routine, complex orders (e.g., selecting among differing doses and types of insulin to be administered at different times for a diabetic patient).

Many CPOE systems slow the speed at which clinicians can carry out the clinical documentation and ordering processes.11 This loss of efficiency often recovers over time.12 Simply learning to use CPOE takes time and attention away from demanding schedules. If their patient loads are not decreased temporarily during training periods, clinicians work longer hours to complete their combined electronic and clinical work.13 The indiscriminant, excessive generation of clinical alerts by CPOE systems can also slow clinicians as they pause to decipher alerts, deliberate on whether and how to respond, and potentially document reasons for not complying with alerts.

Administrators and researchers commonly leverage CPOE to collect information not directly related to patient care. The time burden for doing so usually falls on clinicians. One noted: “It seems like every new organizational mandate filters down to the… fingertips…of our primary care physicians in the form of something else that needs to be entered through the computer and the feeling is ‘Well, they have a computer, so it’s easy for them to do that’ but the cumulative effect [on the physicians] of all those tasks is not fully appreciated.”

When CPOE systems are poorly integrated with other clinical information systems, clinicians find it time-consuming to log in to different systems using different account names and passwords. In some cases, data from one system must be entered manually into another, doubling the work. In addition, built-in functionality such as “cut and paste” may proliferate redundant text in electronic records that clinicians must navigate in order to have a complete picture of the patient. One physician said: “There is no way for me to really know what’s new, but I keep seeing chunks of the same text over and over so I have to read every word. Most of it isn’t useful.”

Type 2: Unfavorable Workflow Issues

Clinical information systems (CIS) in general, and some CPOE systems in particular, by rigidly modeling work processes according to the “letter of the law” (as set forth in organizational policy and procedure) can dramatically highlight mismatches between intended and actual work processes in real-world clinical settings. These systems can add to ineffective or dysfunctional workflows when CPOE developers’ computational models do not reflect actual clinical practices. Such failures can shed light on issues related to clinical role boundaries (due to confusion, misunderstanding, or duplication), and uncover misconceptions among clinical team members regarding what specific work processes actually entail. Furthermore, if CPOE designers have not considered the appropriate range of workflow perspectives (e.g., those of the nursing or clerical staff, as well as of physicians), the resulting technological system cannot accommodate comprehensive, fully integrated clinical workflows. When the system fails to support all the individual role-players who must interact with it, work-shifting to others may occur, leading to resentment and ineffective work activity synchronization. The following quote from a physician illustrates these combined workflow issues: “We found that [the labor and delivery area] was one of the most complex places in the hospital because the patients are going from the screening room to the pod room to a labor room to the delivery room to postpartum, and each of those [is] a different level of care and so orders need to be rewritten…and although nurses are good about blending the orders as necessary…the computer is trying to execute JCAHO rules about changing orders for every level of care [and they aren’t] nearly so flexible.”

The clinical ordering process might appear to follow predictable steps: e.g., a clinician places (enters on the computer) an order, the system routes it to the desired destination, the order is processed, and the requested action occurs. However, in actual clinical practice the process is much more adaptable, and includes a variety of checks, balances, interventions, and exceptions.14 This non-computerized process of placing an order is multi-threaded, and consists of several concurrent and asynchronous steps, each of which may modify, terminate, or intervene in the processing of a given medical order. Many CPOE implementations change or eliminate these multiple interdependent steps, resulting in fewer process reviews and greater potential difficulties.

The project team noted many instances where fewer process reviews led to problems. In one case, x-ray orders were unnecessarily duplicated: “We probably underestimated the gatekeeper function that the clerical staff [provided]…One of the first symptoms is that patients had daily chest x-ray orders in many units, and the clerk had sort of provided a function of questioning after a certain amount of time, based on what he or she knew about the patient, [if] this was still appropriate. Once we automated those daily chest x-ray orders, [they] went on ad infinitum until we came up with an intervention to address that.” In another instance, a double check was eliminated: “The process before wasn’t just the clerk writing down the allergies. The process was the clerk writing down the allergy and then the physician reviewing it before anyone took any action on it…that physician review was not kept in the [new CPOE] system.”

One of the most often cited benefits of CPOE is the ability for clinicians to enter orders from anywhere in the hospital, or even from home. However, such new workflows can cause unexpected duplications or contradictions among orders, to the point of endangering patient care. For example, we heard…“We had a lot more instances of within thirty seconds of each other, two, sometimes three providers would enter the same order at approximately the same time [from different locations] and so it really forced us to go back and really do more education on being careful to look and see what’s active before you enter a new order.”

Type 3: Never-Ending Demands for System Changes

Never-ending system demands arise regarding hardware and software purchases, implementation tasks, and maintenance issues. They represent UACs from the perspective of both administration and IT staff. Implementation of CPOE requires advanced hardware platforms that can support clinical software. Purchasing or upgrading hardware is not a one-time event, as future technology advances make this an ongoing need. As clinical software systems increase in scope and capabilities, more users require more computer access time, via more computers. Clinicians complain when contention for access to computers interferes with accomplishing clinical tasks, particularly during the busiest hours (e.g., after morning rounds).

Software application demands are also never-ending. One clinical development group moved upgrade releases from a weekly to monthly schedule because “the testing requirements… [were] becoming unbearable.” In another instance, JCAHO recommendations to eliminate the use of common abbreviations meant that “there [were] over 4000 occurrences of the abbreviation ‘QD’ in various order entry templates that would have to be manually changed,” and this was only one of a list of about 20 changes. “I can’t imagine how much work it is going to take to review all of the screens to find them, and what the incidence of new errors might be during the fix, such as eliminating an element in a pick list by accident, or making a typo in some drug name.” Overhead in maintaining systems and data increases regularly. When CPOE systems allow clinicians to create their own order sets, disparate single-user sets proliferate. It becomes progressively difficult to standardize, update, or maintain these over time. It is also difficult to reconcile old order sets with new institutional initiatives to streamline care processes and to follow the most recent evidence-based clinical practice guidelines.

CPOE adoption transforms some acute problems into subtle and insidious ones. One author calls these the “revenge effects” of technology.15 During system implementation, the organization necessarily focuses on system go-live activities. But after go-live, ongoing work begins. The system must be tuned, upgraded, tested, interfaced with other systems, and backed up regularly. As the clinical staff increasingly depends on the technology for their daily work, pressure to keep the system operational 24×7×365 increases. Round-the-clock help-desk support becomes necessary. All employees must be trained in system use, and retrained after substantive system changes. Backup systems must operate if the primary system fails. The burden on the technical support staff rarely levels off. Although these consequences can be anticipated, their extent is typically underestimated. One study participant referred to CPOE maintenance as “repairing a jet engine in flight” because the consequences of making mistakes with these systems are “orders of magnitude” greater than for less-integrated, less closely-coupled clinical systems.

As a CPOE system evolves, users rely more completely on the software and demand ever more sophisticated functionality for clinical support. As medical practices evolve, corresponding new features must be added to the original implementation. Over time, complex interactions among the numerous software features can make the installation both unmanageable and outdated, such that the system needs to be replaced with a newer (and “cleaner”) version: “The fact that you develop a critical mass of code, doing anything radically different becomes extremely difficult when you have an installed user base that you are supporting. So a lot of the early rapid flexibility and leeway you had in the early years of implementation you get stuck with … and it isn’t easy to sort of wipe things clean and start over.”

Type 4: Problems Related to Paper Persistence

Many CPOE vendors advertise products as helping an organization to “go paperless.” While eliminating the paper-based medical record has clear advantages, including “improved legibility; simultaneous, remote access; and integration with other information sources”16 one should not confuse this concept with eliminating the use of paper by clinicians in their efforts to take care of patients. Instead, the key issue is to decrease or eliminate the dependency on ineffective, paper-based processes that form barriers to optimal health care delivery, and in this regard, CPOE systems can be particularly effective.

Use of paper was endemic in the five institutions we visited. Use was especially pronounced when paper interfaces substituted for lack of electronic CPOE system integration with other clinical systems (e.g., medication administration recording, pharmacy dispensing, or laboratory ordering). In hospitals where CPOE and ancillary system integration was incomplete, we observed computerized orders being printed out in the processing departments, then re-entered into the local department’s clinical information system. We typically saw nurses manually transcribing allergy, blood type, and medication information from the CPOE system to paper-based medication administration records.

We observed providers using paper for temporary, handwritten data storage for later entry into the computer and conversely as a portable, disposable, computer output display medium for quick reference use during their workdays. In hospitals where CPOE systems generated and printed patient summary sheets, we noticed some providers documenting patient progress notes on these printouts. Despite an explicit directive on CPOE printouts not to do so, and counter to current recommendations by the American Hospital Information Management Association,17 clinicians placed annotated printouts in the patient’s chart as formal documentation. These are but a few examples of paper persistence and proliferation in all areas of patient care.

By contrast, paper often serves as a necessary, sometimes superior, cognitive memory aid. As one clinician noted: “I like to have the information on paper where I can hold onto it.“ Furthermore, paper remains the most malleable, flexible, and easily transportable data medium available. Organizations are understandably hard pressed to limit its use. Personnel need only point and click to print ”hard copies“ of stored information. Paper-based clinical record storage will become obsolete, but use of paper in the clinical setting will not. One leader indicated that his institution uses roughly ”…1.6 million pieces of paper per month—printed or copied—and we think half is related to clinical care…we print and destroy 40% of that paper.”

Type 5: Untoward Changes in Communication Patterns and Practices

CPOE systems often dramatically alter traditional communication patterns among care providers, ancillary services, and clinical departments. Installation of CPOE replaces the nexus of previously interpersonal conversations regarding provision of care with a computer system. Some describe CPOE as providing an “illusion of communication”18 because it promotes the belief that entry of an order into the system ensures that the proper people will see it and act upon it. This unfounded belief is especially problematic for “stat” (emergency) orders, because their execution in a timely fashion should depend on interpersonal communication (even post CPOE implementation), but many CPOE users assume that electronic transmission will be efficient, and do not understand that fast computer or network transmission does not guarantee fast, or accurate, notification of the person who must take care of the order. We observed many instances where emergency orders were not only placed in the CPOE system but were also (redundantly) phoned in to assure they took place immediately.

Doctors, nurses and other providers consistently report that clinical systems like CPOE can cause unsatisfactory reductions in face-to-face communication regarding patient care. The providers further suggest that reduction in communication increases the likelihood of errors due to miscommunication, delayed initiation and execution of orders, and fewer team-wide discussions regarding planning and coordination of care. For example, order entry sessions may precede or remotely follow ICU rounding sessions, when the attending and consulting physicians, the respiratory therapist, and the nurse are together to discuss patient care. Rapid and significant changes in the patient’s condition in the time interval between order discussion and order entry may lead to omission or delayed entry of some relevant orders. CPOE systems can exacerbate problems related to the use of verbal orders in conjunction with system entry; some institutions have gone to the extreme of banning verbal orders except in the case of emergencies: “It is not uncommon for a physician to enter an order which has also been verbally stated to the nurse. The nurse, acting on that verbal order, then goes back to, say, mix an IV. The doctor in the meantime changes his mind [about the IV admixture, then places the order] and does not tell the nurse until the bag has been hung, resulting in a waste of a $100 bag of IV fluid.”

Type 6: Negative Emotions

Organizational change is never easy, and shifting clinical practices and workflows can engender enormous emotional resentment in end-users. Sittig and colleagues have suggested that “a specific event or series of events that either cause the person to succeed or fail in reaching his or her goal(s)” triggers many emotions, and that such emotions can affect one’s ability to carry out complex physical and cognitive tasks.19 Shifting from paper-based order generation to CPOE is bound to evoke strong emotional responses as users struggle to adapt to the new technology.

We noted a wide variety of emotional responses to CPOE, including both strongly negative and highly positive emotions. Negative comments predominated. The amount of time a CPOE system had been in use strongly correlated with the level of positive emotions the system elicited. For example, one nurse described her first impression of a CPOE system in this manner: “At first we hated every second of it. I mean we were all like ’I have sick patients here. I’m busy. I don’t have time to sit here [at the computer] for twenty minutes.’ It was a pain.” Most agree that the high level of negative emotions decreases over time: “It gets better.”

Type 7: Generation of New Kinds of Errors

Studies have indicated that CPOE adoption can generate new kinds of health care–practice related errors, while others have described roles for CPOE in both preventing and causing medication errors.(4 20 21 22 23 24 25) Here, we focus on new types of errors that emerge when CPOE replaces paper-based ordering.

New CPOE-related errors result from: problematic electronic data presentations; confusing order option presentations and selection methods; inappropriate text entries; misunderstandings related to test, training, and production versions of the system; and workflow process mismatches. System designs (including poor data organization, data omissions, etc.), and end-user confusion about system functionality contribute to new forms of errors. When users make data entry selections from pick lists (drop down lists), a new class of “juxtaposition errors” results from making a wrong selection without realizing it. For example, long, dense pick lists predispose a provider to selecting a patient name adjacent to the intended name. The system should provide adequate feedback on who was selected (e.g., displaying the selected name in large letters on the next screen). If this does not occur, the user may proceed to enter an entire set of orders on the wrong patient. “Backing out” such erroneous orders before they are executed can be problematic. Similar errors occur whenever pick lists facilitate selection of other order parameters.

CPOE systems manage massive amounts of clinical information. However, CPOE workstation screens cannot display large amounts of data simultaneously. Thus, clinicians must learn to navigate serially through CPOE interface screens to perform their work. When busy clinicians cannot readily find the “correct” data entry location, they tend to enter data where it might fit, such as in a “miscellaneous” section. Although such information resides in the system, it may be stored in a manner that makes categorizing, cross-checking, processing, and acting upon it more difficult. Furthermore, improper data placement may impede other clinicians from finding important information: “The biggest problem with orders [is that] people get frustrated finding the right spot to put something, or don’t see what they need immediately, then end up entering orders in the miscellaneous section. This makes it easy to miss things and hard to capture data on the orders being entered.”

Poor coordination in deploying test, training, and production versions of CPOE systems can create new kinds of errors. For example, unless safeguards are in place that only allow obviously artificial patient names in “test” and “train” modes, it may be possible for a clinician to use a test system for a patient name that by chance matched an actual patient. The user would not know that the orders entered will not be acted upon, because they were outside the production system. Similarly, without appropriate safeguards, a provider might, during a training session, enter orders for a “test” patient who is actually a “live” patient in the production system. Resulting “test” orders will be processed and have consequences. Finally, problems may emerge when test patients are only identified by a cute, simple name (e.g., “Tom E. Test,” rather than a safer “ZZZTest, ZZZTom”), especially when an actual patient with a last name of “Test” is admitted.

Type 8: Unexpected and Unintended Changes in Institutional Power Structure

As CPOE systems enforce specific clinical practice patterns, while at the same time monitoring clinicians’ behaviors, they may induce changes in the power structure and culture of an organization. Power, whether formally or informally delegated, plays an extremely important role in any work environment.26 This is especially true in health care, where lines of authority emanate from a tradition based on educational hierarchies, various providers’ roles, differences between general practitioners and specialists, and more.27 For these reasons, CPOE systems may subtly bring important power issues to the surface.

CPOE system configurations control who may do what (and when) through the use of clinical, role-based authorizations. While narrowly defined authorizations may lead to much needed role standardizations that reduce unnecessary clinical practice overlaps, the constraints may also redistribute work in unexpected ways, causing frustration. Physicians may resent the need to enter orders directly into a computer, especially when they view this work as a clerical task. Nurses may refuse to take verbal orders except in cases of emergency, or insist that the physician enter orders into the CPOE system before any order will be carried out. We heard many nurses express comments such as: “It really isn’t a nursing job to convince the doctor to use the system—and if we must use it, they should too.”

Physicians report loss of professional autonomy when CPOE systems prevent them from ordering the types of tests or medications they prefer, or force them to comply with clinical guidelines they may not embrace, or limit their narrative flexibility through structured rather than free-text clinical documentation. It is irritating when the system directly rewords clinical orders to standardize them: “I didn’t realize how important nomenclature was in ordering…Our system has a very robust synonym process where you can use synonyms; we have synonyms for…common misspellings. The doctor puts [an order] in and [it] goes into the patient’s chart with the kind of ‘accepted’ name. They’ll come back later and look at that and say, ‘I didn’t order that. It doesn’t look like anything I ordered.’ In the preceding example, the clinician might object to the forced use of awkward terminology. Such terminology not only disrupts clinicians’ workflows, but might also compromise patient care. While order standardization may benefit the organization, it may confuse clinicians. This represents a non-trivial domain in which clinicians’ professional autonomy may be circumvented.

Whether the power base is centralized or decentralized plays an important role in occurrence of UACs. Centralized power structures use top-down, hierarchical formats to mandate compliance with organizational rules and to enforce procedure standardization. Decentralized arrangements lead to greater variations in CPOE system configuration and utilization, and increased competition and conflict among departments. Conflicts create significant problems for IT departments, and lead to problems with the application consistency, clinical coordination, and evaluation of impact on patient care.

When many departments participate in CPOE implementation, significant unanticipated power shifts occur. Viewed as the new enforcer of standards, the IT department gains power, even when other departments mandate the standards. This can be frustrating to others: “CPOE should be a clinical project, not an IT project, but it’s still amazing how much I think it comes out of the IT department.” Administration and quality assurance departments gain power by requiring users to comply with CPOE-based directives: “The doctor can’t place that order unless he fills in that field. They’re [administration] really happy and I think that’s really a problem for the doctors.”

Type 9: Overdependence on Technology

As CPOE technology diffuses and becomes entrenched within organizations, clinical care delivery becomes inextricably dependent upon it.28 29 System failures increasingly wreak havoc when paper backup systems are not in place: “It’s funny now. When the system goes down, we don’t remember how to work with paper.” Prolonged system failures (lasting hours) can so dramatically halt the flow of clinical information that outpatient activities may be curtailed or canceled and emergency rooms at trauma centers may divert admissions until vital systems are restored. The more widely and deeply diffused the technology, the more difficult it becomes to work without it.

Although limited experience exists, embedding clinical decision support (CDS) within CPOE systems may increase clinician-users’ access to educational material30 31 and may affect learning and retention.32 Clinicians who have only worked with CPOE systems using CDS technologies face new and interesting problems when they transfer to work settings without this technology. Important knowledge gaps might emerge for the clinician who relied on CDS to provide real-time information and/or error prevention. The clinician might have trouble remembering standard dosages, hospital formulary recommendations, and medication contraindications. On the other hand, prior use of CDS may actually promote learning, through repeated and consistent presentation of the sorts of information mentioned in real-time alerts and reminders. Thus, depending on an individual’s learning style and the type and amount of available decision support, CDS may actually enrich clinical training.

Finally: “Our society is geared so that if it’s in the computer, it must be accurate and complete, and as we know, it just isn’t so.” For example, certain free-text fields cannot be processed by CDS components of CPOE. Allergies to medications, entered as atypical abbreviations (e.g., “PCN”, “SMTX”), are a common example. Clinicians assume the computer “knows” the information. This can be especially worrisome for clinically vital information such as allergies.

Decision Support Systems

During analysis of unintended consequences, the researchers learned that clinical decision support generates a disproportionately large number (over 25%) of UACs, spanning all 9 types above. Clinical decision support functionality, viewed as necessary to make CPOE beneficial, often lacks relevance to many specific clinical situations. Accordingly, CPOE-based CDS is not consistently useful for clinicians, is impractical to maintain, and is less than fully reliable in complex situations.19 24 33 34 35 36 37 38

Discussion

This study identified types of unintended, negative (adverse) consequences resulting from CPOE implementation. Project team members observed end-users interacting with CPOE, interviewed key players involved in implementations, and analyzed transcripts of meetings designed to elicit UAC-related information. The project identified hundreds of UACs and grouped them into nine categories. The project team came to the realization that the degree of undesirability of each UAC depends to a great extent on one’s perspective. We found that the nine types of unintended consequences emerged with surprising regularity at the sites visited. Many UACs originated with attempts to implement clinical decision support.

The project team learned significant lessons. First, UACs occur during all CPOE implementations, though not all institutions experience all of the types of UACs. Second, the nine types of UACs occur in a widespread manner. The UACs pose significant consequences for clinicians, technical staff, and organizations as a whole. Finally, the project’s typology establishes a framework for systematic approaches to address these issues.

  • More work/new work: CPOE systems can significantly increase clinician workload, and improved system design may not reduce the amount of new work such systems require. Great care must be taken to balance the risks of over-alerting with not alerting. Developers should rework clinical system interfaces to: (a) reduce collection of redundant information; (b) display relevant information in logical locations; and (c) reduce the amount of required typing. The lesson is that more work for the clinician is inevitable, and must be addressed in the planning process. Successful implementations balance required new work with system-based reductions in old work to make use of the systems by clinicians tolerable.

  • Workflow: Clinical workflows are complex, and clinical computer technology integration significantly impacts health care workflows.12 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 Modeling clinical workflows is difficult because clinical practice is so inherently complex, interruption-driven, and constantly changing. No CPOE system fits all workflows of a given hospital perfectly. Even if a system initially did so, it would not eliminate the need for constant system adaptation to changing workflows in the future. Whenever there are adjustments, there will be unintended consequences.

  • Never-ending system demands: CPOE systems evolve (i.e., are reconfigured, enhanced, or replaced) over time, making hardware and software upgrades a necessity. With each change, implementers should expect unintended consequences. As changes occur, users must be retrained and quality assurance measures must be reassessed. The lesson is that planning must allocate adequate resources for ongoing improvements.

  • Paper persistence: While electronic medical record systems trend toward “going paperless,” health care organizations, as a whole, do not. Vendors and administrators alike must understand differences between having a paperless record system and a paperless office. Organizations must delineate what constitutes legal documentation in the presence of an electronic medical record.53 A likely reduction in paper use will follow full integration of disparate clinical information systems and widespread deployment of clinical computing workstations or wireless, handheld devices. Paper is here to stay for utilitarian, as opposed to permanent record-keeping purposes, and attempts to limit its pragmatic use in health care are often misguided.

  • Changes in communication patterns and practices: Computerized systems are unlikely to replicate the richness of face-to-face communication, but computer-based communication systems must improve. Improvements in system interface design must pay special attention to the communication needs of health care providers. The lesson is that CPOE implementation changes clinical communication patterns. In addition, a comprehensive communication plan that reaches all levels of the organization must be part of any CPOE project management plan.

  • Emotions: Emotional responses to change are inevitable. These responses can point out significant problems with the system design, and can lead to solutions. Training and open communication can help to promote better understanding, which may reduce the negative emotional responses to CPOE.

  • New kinds of errors: CPOE systems prevent some types of errors while creating or propagating new ones.4 5 21 22 24 25 33 54 55 56 57 58 59 60 61 Many new errors result from straightforward system interface design problems, such as dense pick lists that cause juxtaposition problems. Recognizing current unintended consequences should encourage system designers to optimize human computer interface design, and to exert caution when implementing new alerts.

  • Changes in the power structure: Because CPOE-related power changes affect organizational and personal autonomy, they often cause significant UACs for end-users. Most often it is the physician who loses power: this must be recognized and dealt with explicitly during the CPOE planning process.

  • Overdependence on technology: Health care is increasingly dependent on technology and this is unlikely to change. Dependence on technology must never become so great that basic medical care cannot be provided in its absence. Planning for management of unexpected downtime is critical.

Decision Support Systems and CPOE

The project identified decision-support-based clinical alerts as a common source of UACs. There are optimal and less optimal approaches to designing and implementing decision support. The suboptimal approaches to CDS can have widespread negative impacts on clinical practices. Poorly designed alerts constantly interrupt providers, often with trivial, redundant, or already known information. Alerts present a major workflow process issue, adding to the steps required to enter an order. Clinicians report inappropriate alerts as highly frustrating nuisances. Alerts can beneficially and adversely change the power structure, and present challenges to professional expertise or autonomy. Because alerts appear to be from “the system,” they may be viewed as correct when they are erroneous. Conversely, when appropriate alerts are ignored, error prevention may not occur, and redundant or unnecessary test avoidance diminishes. Building and maintaining an appropriate set of CPOE alerts based on an up-to-date evidence base is an onerous, never-ending task.

Looking to the Future

Improvements are both warranted and attainable with respect to unintended consequences. Currently, CPOE technology is immature and rapidly evolving. The field must prioritize ongoing efforts to better understand the nature of clinical work. Designers must build systems that respect health care providers’ burdens, and constant feedback from providers should guide system implementation and evolution. Workflow improvements require experience and time. As the adoption of CPOE systems increases, so will the wealth of knowledge about how to use the systems to improve care. Improved interface designs may eliminate or reduce the possibility of juxtapositions and other related errors. As technology evolves, dramatic improvements in providing clinicians with pertinent data will occur. Such technological advances first and foremost require clinical involvement to better support clinical work. Only through a careful combination of CPOE-related research, design, feedback, and understanding will many of the unintended consequences be reduced or eliminated.

Conclusion

It is important to view UACs as the result of a constellation of factors. Their causative agents almost never occur in isolation. UACs emerge in all aspects of health care, whether or not the task from which they emerge is technological. Not all types of UACs may occur with every CPOE implementation. The goal is to discover and understand causative factors leading to UACs to allow future CPOE developers and implementers to mediate or eliminate UACs that are preventable or remediable. Doing so will reduce the negative impacts CPOE systems have on providers, patients, and administrators. It is only through careful evaluation of UACs that we can gain better insight into how to best approach these problems.

Footnotes

  • This research was funded by research grant LM06942 and training grant ASMM10031 from the U.S. National Library of Medicine, National Institutes of Health.

  • The authors thank all the individuals who allowed us to observe or interview them and the experts who participated in the Menucha Conference. Special thanks go to the site principal investigators J. Marc Overhage, MD, PhD, Eric G. Poon, MD, MPH, and Rainu Kaushal, MD. We also thank Cody Curtis, MBA, the reviewers, and the JAMIA Editorial Board for assistance with this manuscript.

References

Free Sample

This recent issue is free to all users to allow everyone the opportunity to see the full scope and typical content of JAMIA.
View free sample issue >>

Access policy for JAMIA

All content published in JAMIA is deposited with PubMed Central by the publisher with a 12 month embargo. Authors/funders may pay an Open Access fee of $2,000 to make the article free on the JAMIA website and PMC immediately on publication.

All content older than 12 months is freely available on this website.

AMIA members can log in with their JAMIA user name (email address) and password or via the AMIA website.