Reevaluating Recovery: Perceived Violations and Preemptive Interventions on Emergency Psychiatry Rounds

Introduction

Six years ago, the Institute of Medicine’s report “To Err Is Human”1 suggested that between 44,000 and 98,000 people die each year because of preventable medical error. This report has attracted considerable public and research attention to the issue of medical error. Despite this attention, in assessment of the progress towards safety since the release of this report, leading error researchers Leape and Berwick note that while the report raised awareness of medical error “little evidence exists that systematic improvements in safety are widely available.”2 According to these authors, amongst the key barriers to progress in medical error reduction, “the first challenge is complexity” in reference to the complex nature of healthcare work. This highlights the need for approaches to error that can address the complexity of medical practice. The framework of individual accountability is poorly suited to the problem, as it isolates erroneous action from its larger context. In addition, and in keeping with contemporary human error research in other domains,3 we propose that approaches seeking to eradicate error fail to recognize that error, error detection, and error recovery are integral to cognitive work in a complex context.

According to Danish error expert Jens Rasmussen, complexity is a distinguishing feature of the modern workplace.3 The traditional workplace is characterized by repetitive tasks, an example being part assembly on a construction line. The constraints of such repetitive tasks allow for the definition of human errors as deviations from a normative sequence of correct actions. In contrast, Rasmussen highlights the multitude of options available to workers in the modern workspace. In such complex environments, workers learn the boundaries of acceptable work behavior in an adaptive, exploratory process. Health care work has been characterized as the product of a complex socio-technical system in which roles and responsibilities often defy formal definitions such as job descriptions.4 In the hospital setting, much clinical work is performed by residents and trainee nurses. As errors are to be expected as part of the exploratory learning process, the elimination of error in this context is not a realistic goal. Furthermore, error research in other domains suggests that error is not eliminated once expertise is attained.5 Rather, experts’ errors are more easily detected and corrected. The importance of error recovery has been illustrated in a naturalistic study of the critical care environment.6

Reason defines error as the failure of a planned sequence of mental or physical actions to achieve the intended outcome when this failure cannot be attributed to chance.7 Recovered errors often do not fit this definition, as they are captured before resulting in an unintended outcome. In the research presented here, we focus on audio-recorded evidence of perceived violations of the boundaries of accepted practice. Figure 1 illustrates the sequential relation between perceived violations and medical error. First, the bounds of safe practice are violated. An error curtailed at this stage results in a “near miss,” part of normal system function, particularly at high productivity. If undetected, a near miss crosses another boundary, culminating in an unintended outcome. The study of perceived violations captures potential error early in its evolution, and has implications for error prevention regardless of the actual consequences of each event. This research aims to identify and describe perceived violations of the boundaries of accepted practice in a PED, and determine how these potential errors are resolved in practice.

View larger version:

• In a new window

• Download as PowerPoint Slide

Figure 1

The evolution of medical error. Adapted from Patel (2006).8

Background

Theoretical Framework and Related Research

Distributed cognition is a framework developed by Hutchins.9 According to distributed cognition theory, cognitive process is not confined to the thoughts of individuals, but rather is distributed across groups of individuals and artifacts such as clinical notes, which form part of a larger computational system with greater capacity than any of its individual components. Cohen and his colleagues applied the framework of distributed cognition to the study of work and information flow in the PED,10 a high-volume clinical unit dealing exclusively with the acute phases of psychiatric crises. The primary objective of this work was to characterize the cognitive system underlying decision-making, and consequently error, in the PED. Ethnographic and interview data were analyzed in order to characterize the distribution of cognitive work and information flow, revealing latent systemic flaws that are vulnerable to error. These latent flaws were related to the distribution of cognition across teams, time, space, and artifacts reflecting the complex work-flow dynamics of the PED. In our past studies, the following vulnerabilities were found in the PED: (1) Cognition is distributed across teams. The staff of the PED aggregate into temporary “mini-teams” attached to each patient. Mini-teams’ plans may compete with one another when resources such as referral beds are limited. (2) Cognition is distributed across time. Various team members work in asynchronous shifts; consequently information loss can occur at shift change. (3) Cognition is distributed across space. The PED is also responsible for patients residing in the medical emergency area. However, this responsibility is shared with medical residents and medical attending physicians, who have less experience with psychiatric patients. (4) Cognition is distributed across artifacts: external representations may promote error. For example, handwriting may be misinterpreted, or unrecognized abbreviations may be used. These findings enhance our understanding of error in the PED, and inform the collection and analysis of the prospective audio-recorded data presented in this manuscript. Zhang and his colleagues define a cognitive taxonomy of medical error.11 The intention of this taxonomy is to classify errors according to their underlying mechanism. The taxonomy approaches medical error from a top-down, theory-driven cognitive perspective. This analysis is informed by the work of Zhang and his colleagues on the cognitive mechanisms of error which suggests that perception (or mis-perception) is directly linked to action (erroneous action).11 In turn, the data gathered in several naturalistic studies of different critical care contexts, including the research presented here, will inform the further development of this taxonomy, to ensure it can accommodate cognitive mechanisms of medical error that are observed in the field.

Naturalistic Study of Error Recovery in Critical Care

Naturalistic studies of medical decision-making attempt to characterize clinical decisions in their natural context.12 The importance of contextual factors to the understanding of medical error has motivated naturalistic studies of several critical care environments, including the cardio-thoracic intensive care unit13 and PED.10 Kubose, Patel, and Jordan conducted a field study of error detection and recovery in the intensive care unit (ICU),6 using an innovative approach combining observation, shadowing of ICU team members, audio-recording, and analysis of infusion pump keystroke logs. Four handovers (in which information is exchanged between clinicians at shift-change) and rounds (in which the team gathers to reassess the management plan for a particular patient) for six patients were captured. Recorded protocols were analyzed and coded for error detection and recovery. Both handovers and rounds exhibited error detection, with a mean of 10.5 errors per handover and 5.6 per round. Most errors detected during rounds and handover were recovered (mean of 7.25 per handover and 5.25 per round respectively). Further studies were conducted to determine the relationship between expertise and error correction, by selectively shadowing clinicians of different levels of experience. The results suggest that clinicians of all levels of expertise make mistakes; however experts are better able to detect and recover from error.

Cognitive Ethnography

Cognitive research occupies a broad spectrum ranging from controlled laboratory experiments to naturalistic studies of cognitive activities in context. Ethnographic methods such as observation and audio-recording are essential to the study of cognitive activities in context. However, the use of these methods in cognitive research differs from their traditional use, for example in anthropology. Ball and Ormerod highlight three features of cognitive ethnography that deviate from the fundamental tenets of traditional ethnography: observational specificity, purposiveness, and verifiability.14 In contrast to the exhaustive data collection required to meet the holistic descriptive goals of traditional ethnography, cognitive ethnography utilizes selective sampling of representative “time-slices” of activity. This selection process is based on the goals of the cognitive study, which often include some form of intervention. In addition, cognitive ethnography seeks to verify observations across observers, methods, and data-sets. For example, many of the findings of the research presented here are verified by their consistency with the findings of our previous research which applied observational and interview methods in this context.10 Traditional ethnography is personalized: the ethnographer’s feelings are included as data, which precludes validation. These differences are appropriate to the research goals of each discipline. While the goal of traditional ethnography is purely descriptive, cognitive ethnography seeks to evaluate specific research hypotheses or theoretical concepts in a naturalistic context.

Domain Description

PED patients present in crisis, with limited information available at the outset to guide management. The primary tasks are acute-phase diagnosis and management followed by disposition planning. Patients may be observed for a period of up to 72 hours, against their will if necessary, in an extended observation bed. This observation period allows for acute phase management which may obviate the need for inpatient admission, and gives clinicians time to gather information to support clinical decisions. Eventually, a decision is taken to admit to an inpatient unit or discharge with a suitable disposition plan. Several disparate sources of information must be integrated to support this decision. While in the PED, short-term management is initiated to stabilize the patient. This may include medication, supportive therapy, and family sessions, which may obviate the need for admission.

Methods

Approval for data collection was provided by the institutional review board once a certificate of confidentiality protecting research data from forced disclosure was obtained. Nine morning rounds (of approximately forty-five minutes duration) and five handover rounds (approximately a half-hour in length) were audio recorded. In total, seven hours of recorded transcripts were analyzed. For morning rounds, the entire team gathers for an in-depth discussion of every patient on the ward. In addition to facilitating information transfer, morning rounds facilitate teaching and coordination of the clinical team. Handover rounds are relatively informal. Essential information is imparted by a clinician about to leave the PED, and discussion is limited. While some of the rounds occurred on consecutive days, there were only two instances in which an incident that was captured overlapped between rounds. Linguistic cues suggesting a clinician was critical of the action of a colleague were used as markers for segments of dialogue containing perceived violations of the bounds of accepted practice. These segments were then transcribed, and analyzed using QSR N-VIVO software for qualitative data analysis and a coding system developed by the researchers (Table 1).

To develop the coding system, key concepts were extracted from excerpts of recordings describing error-related incidents, as well as our previous research on the cognitive mechanisms of error in the PED. In this way, our analysis emerged from the data itself, in keeping with the principles of grounded theory as defined by Corbin and Strauss.15 These principles guide exploratory qualitative research so as to ensure that theory development is grounded in the data. The codes used reflect the inherent subjectivity of the data. The data often reflect the perspective of a single clinician, and it would be misleading to label individual perception as medical error. Rather, we have used the code “perceived violation” to denote an apparent violation of the bounds of correct practice, in accordance with the perspective of the clinician captured on audio-tape. It is not the intention of this research to prove that particular clinicians committed erroneous actions, and we do not believe that our evidence is sufficient to support such assertions. The resolution of such controversies requires an in-depth investigation of each incident which is beyond the scope of this research. However, the evidence that one clinician found fault with the action of another is seldom ambiguous. For example, the following statements are excerpts from our recorded transcripts:

• “Went back to (another department) where they proceeded to neglect him.”

• “(another service) has been lame recently.”

• “Hadn’t taken any vitals for like 12 hours.”

Coding was conducted by a team of four researchers. Each transcript was coded independently by two coders. Initial agreement between raters was adequate, with a Kappa score of 0.72 across all coded segments prior to conflict resolution. Any discrepancies were resolved through discussion until an agreement was reached. Collaborative coding in this manner is a recognized technique in this kind of research and it has shown to be reasonably successful in our previous work (for example, 16). Perceived violations were further classified according to whether they were self-detected, validated by consensus, and justifiable according to accepted clinical practice. In addition, the seriousness of the potential consequences (stated in the data, or implied) of each violation was categorized by a researcher with psychiatric clinical experience. Incidents and codes were reviewed by a domain expert in the field of clinical psychiatry. In addition, wherever possible clinical literature and recorded evidence of peer consensus were used to support the perspective of the clinician captured on audiotape. Miscommunication incidents were further classified by mechanism, according to the characterization of the distribution of cognitive work across teams, time, space, and artifacts from our previous research in the PED. This prior research was based on observation and interview data and is described in detail in Cohen and colleagues.10

Results

A total of nineteen perceived violations and eleven incidents of miscommunication were extracted from nine morning rounds and six handover rounds. Of these incidents, only one, a drug overdose had been formally reported and resulted in an internal inquiry.

Perceived Violations: Validated and Potentially Harmful

Ten of the nineteen perceived violations had serious potential or actual consequences (Table 2). Of these ten violations, three were validated by spoken consensus on the ward round. In addition, seven were supported by comparison to published clinical recommendations. Seven of these ten violations involved perceived violations attributed to clinicians in the medical (n = 6) and pediatric (n = 1) emergency departments. Of these, three concerned “medical clearance” in which a patient is deemed to be medically stable with a primary psychiatric disorder. The potential problems with this process are explored in some detail in our prior research,10 which characterized the distribution of cognition, in this case expertise, across space. In essence, each unit is specialized to accommodate the needs of a particular group of patients, and the inappropriate transfer of a medical patient to the psychiatric ward may result in suboptimal care for his or her medical condition. The converse, sub-optimal care of psychiatric problems in the medical department, is suggested by the three perceived omissions attributed to the medical department. In two instances, appropriate medication was not prescribed to patients in alcohol withdrawal, an omission which can lead to seizures and the life-threatening condition of delirium tremens.17 In one instance a patient in severe withdrawal was not appropriately monitored. The remaining three potentially harmful perceived violations occurred within the PED. In one incident, an overdose of the anti-epileptic drug Lamictal was prescribed and administered. These two incidents are analyzed in greater detail in our prior research.10 The recorded data from patient rounds provided additional information suggesting the error was initiated by the misinterpretation of a handwritten note. Finally, the anti-Parkinsonian agent Sinemet was prescribed to a patient with a primary psychotic disorder. This patient had a prior diagnosis of Parkinson’s disease, but this was considered to be inaccurate. Sinemet conflicts pharmacologically with the anti-psychotic medication that was also prescribed to this patient, and should be used with caution in patients with pre-existing psychosis.18

Perceived Violations: Delays and Diagnostic Disputes

Table 3 shows nine perceived violations that were either considered less potentially dangerous, or were difficult to validate given the limitations of our data set. In five of these instances, these perceived violations resulted in delayed management. One incident is an example of an expert detecting and correcting his or her own error, omitting to order a laboratory test for thyroid stimulating hormone. Another suggests the process of exploratory learning: a resident e-mails a nearby shelter to notify them about a patient’s admission. It is suggested on the round that phoning, rather than e-mailing is the correct procedure. The last incident on Table 3 shows the detection of a perceived error in which the antidepressant bupropion (Wellbutrin) was prescribed without reference to a note in the chart that suggested this patient should not receive bupropion on account of a prior concussion. Although this was not discussed on the round, this suggestion is itself erroneous, as bupropion is not contra-indicated in concussion (unless there is a predisposition to seizures).

Miscommunication

Eleven incidents of miscommunication were detected in our data (Table 4). These were classified according to their relationship to the distribution of cognitive work in the PED that we characterized in our previous research.10 Some of these incidents are causally related to the perceived violations described earlier in this manuscript.

Six of these incidents (highlighted) are related to the limitations of the artifacts (such as notes, folders, and the communal whiteboard) that are used in the PED. The first three incidents relate to the misinterpretation of handwriting. The central shared representation in the PED is the whiteboard, which provides an overview of all the patients the PED is responsible for. Each patient is represented by a short vignette written with removable marker. In two of these incidents, the handwriting on the board was not readily interpretable. There were no serious consequences, as another clinician on the round was able to act as a translator. However, our data suggest that the first incident of handwriting misinterpretation resulted in a potentially dangerous overdose of the anti-epileptic drug Lamictal being prescribed. In two other incidents notes with important information were not available at the time of the ward round. In a third, information recorded in a patient’s folder was not verbally communicated. These incidents illustrate the limitations of the external representations that are used in this context. The whiteboard is only useful as a shared representation if it is readily interpretable. Information written on notes may be lost to the decision-making process if the notes are missing, or if this information is not verbally communicated in addition to being recorded.

Discussion

Recommendations Based on Results

A total of thirty incidents of perceived violation or miscommunication were identified through the analysis of clinical discourse captured during patient rounds. Because of the specific nature of the work in critical care psychiatry, as well as our focus on perceived violations rather than an outcomes-based definition of error, the number of incidents detected cannot be meaningfully compared to prospective studies of error quantity in the literature. Incident counts are less valuable than in-depth qualitative data describing the nature of perceived error in this context. Incidents were interpreted within the context of prior research, in which we identified the potential vulnerabilities inherent in the distribution of cognitive work in the PED. In addition to confirming the predictions of this prior work, these data provide support for the following recommendations to improve patient safety in the PED:

• The number of perceived violations related to the management of alcohol withdrawal in the medical area suggests that the implementation of a management guideline may be an appropriate intervention. These data suggest that the PED has the knowledge required to provide optimal management of patients in withdrawal, but that the location of these patients in the medical area hampers delivery of care for this condition. Representing this knowledge as a guideline would ensure that it is readily accessible at the bedside, where it is needed most. According to the PED clinical staff, a previous attempt to implement a paper-based guideline was unsuccessful on account of the inability of a static guideline to encode the flexibility required for the optimal management of alcohol withdrawal. This suggests there may be scope for the development of a dynamic computer-based guideline for the management of alcohol withdrawal. The implementation of computer-based guidelines may improve guideline adherence.22 An alternative intervention would be the implementation of training sessions to educate clinical staff on the management of alcohol withdrawal.

• The medication errors suggest that either manual or automated checking of prescriptions would enhance patient safety. Computerized Physician Order Entry (CPOE) systems have been shown to reduce medication errors.23 24 Such systems are able to facilitate automated checking for inaccurate dosage and undetected drug interactions. In hospitals where CPOE is not available, checking of prescriptions by a pharmacist is an effective alternative.25

• The recurrent difficulty in interpreting written information on the whiteboard suggests that clinicians should be made aware of the inherent limitations of this representational device. Information obtained from the whiteboard should be confirmed by accessing more reliable sources, such as written clinical notes. This should be emphasized during on-the-job training. In addition, the use of unfamiliar abbreviations should be discouraged. The implementation of an electronic whiteboard has been shown to improve the efficiency of emergency department communication,26 and may provide a solution to the problem of handwriting misinterpretation.

Innovation

The research described in this manuscript focuses on the analysis of clinical discourse captured during patient rounds in the PED. Audiotapes of rounds are interpreted in light of the characterization of cognitive work in the PED developed during the course of our previous research.10 Incidents of error detection and recovery are analyzed, in order to reveal perceived violations of the bounds of accepted practice. These data capture information that strongly suggests violations of the bounds of accepted practice within the PED. In addition, the data captured are prospective, and consequently less vulnerable to bias than retrospective data collected after an error has occurred. The analysis of real-time, prospective error-related data is a significant departure from the traditional approach to error analysis in medicine, which focuses on reported adverse events in an attempt to determine blame retrospectively. An obvious advantage of capturing prospective data is the extension of the data available for analysis beyond the occasional reported adverse event. This is particularly important in the domain of psychiatry, as there is little published research on the nature of error in this domain.27 Without the analysis of clinical discourse, recovered errors will remain an untapped source of data for the study of medical error.

Limitations

Although the methods of analysis applied in this study are objective, the data employed are inherently subjective. It represents the perspective of a single clinician expressed during a spoken ward round. These data do not allow for an in depth inquiry as is customary in investigations to determine the individual responsible for an error. In addition, patient rounds may not accurately reflect exactly what has happened in the PED. Patients’ conditions may change unexpectedly, necessitating the reversal of a decision taken on an earlier round. However, data gathering and analysis are prospective, avoiding the problem of bias which may affect data gathered retrospectively, after an error occurred. As our methods do not capture the eventual consequence of each perceived violation, we also cannot determine how many of these incidents would fit Reason’s definition of error as “a failure to achieve an intended outcome.” Chart review and patient follow up are beyond the scope of our study. One of the captured incidents resulted in a formal inquiry, but had no enduring harmful consequences for the patient concerned. Given that corrective actions were documented for more than half of the incidents with potential harmful consequences, it is unlikely that the majority of these had serious consequences for the patient concerned. However, the study of perceived violations captures potential error early, and these data have implications for error prevention regardless of the actual consequences of each event. These data strongly suggest incidents of error detection and correction, as in many cases the clinical opinion of the clinician on the round has been validated through consensus as well as through reference to published clinical practice recommendations. In addition errors that do not show evidence of serious consequences one context can have severe consequences in another.13 The execution of corrective actions further supports our assertion that several of these incidents represent perceived violations of the accepted bounds of practice. Two additional limitations are related to our methods of data collection and analysis. Initial inter-rater agreement for the coding of segments was adequate but not high, indicating some subjectivity in categorization prior to conflict resolution. In addition, audio-recorded rounds were selectively rather than exhaustively transcribed. Selection occurred on the basis of verbal cues that suggested a clinician believed a colleague had acted in error. Although the linguistic cues that suggest a perceived error are readily recognizable, some possibility remains that an event was missed during the transcription process.

Conclusions

Rasmussen describes a problem space of acceptable work behavior, the boundaries of which are explored in practice. From this perspective, each perceived violation helps to define the boundaries of acceptable practice and identifies opportunities for preemptive intervention. As we have illustrated, the capture and analysis of perceived violations can be used to support interventions to improve the quality of patient care. This analysis has implications for the study of medical error in other contexts. In particular, the methodology employed would translate well to other critical care environments, particularly in domains in which the margins of error are more precisely defined than those of psychiatry. Our methods have expanded the data available for error analysis in this context. One of the medication errors (encompassing two perceived violations and a miscommunication) was the only element of our data set that was formally reported. This is to be expected. As is evident from our data, error detection and correction are an integral part of any cognitive work. Only the occasional adverse event that slips through the recovery process is reported. The analysis of error detection and correction during patient rounds expands the data available for error analysis beyond these occasional adverse events, allowing for the prospective study of the evolution of error in its natural context.