Agreement between common goals discussed and documented in the ICU
- 1Department of Biomedical Informatics, Columbia University, New York, New York, USA
- 2School of Nursing, Columbia University, New York, New York, USA
- 3Centre for Health Informatics, University of New South Wales, Sydney, NSW, Australia
- 4School of Nursing, University of British Columbia, Vancouver, BC, Canada
- Correspondence to Dr Sarah A Collins, Department of Biomedical Informatics, 622 West 168th Street, VC-5, New York, NY 10032, USA;
- Received 27 May 2010
- Accepted 3 November 2010
- Published Online First 27 November 2010
Objective Meaningful use of electronic health records (EHRs) is dependent on accurate clinical documentation. Documenting common goals in the intensive care unit (ICU), such as sedation and ventilator management plans, may increase collaboration and decrease patient length of stay. This study analyzed the degree to which goals stated were present in the EHR.
Design Descriptive correlational study of common goals verbally stated during daily ICU interdisciplinary rounds compared with the presence of those goals, and actions related to those goals, documented in the EHR over the subsequent 24 h for 28 patients over 15 days. The study setting was a neurovascular ICU with a fully implemented electronic nursing and physician documentation system.
Measurements Descriptive statistics and χ2 analyses were used to assess differences in EHR documentation of stated goals and goal-related actions. Inter-coder reliability was performed on 16 (13%) of the 127 stated goals.
Results One-quarter of the stated goals were not documented in the EHR. If a goal was not documented, actions related to that goal were 60% less likely to be documented. The attending physician note contained 81% of the stated ventilator weaning goals, but only 49% of the sedation weaning goals; additionally, sedation goals were not part of the structured nursing documentation. Inter-coder reliability (κ) was greater than 0.82.
Limitations Observations in a single ICU setting at a large academic medical center using a commercial EHR.
Conclusion The current documentation tools available in EHRs may not be sufficient to capture common goals of ICU patient care.
Meaningful use of electronic health records (EHRs) is dependent on the quality of data contained therein.1 One ‘meaningful use’ goal is to implement processes to improve data quality and integrity within the EHR.1 A means of achieving such a goal is to identify and assess the patient data and clinical care processes that are not currently represented in the EHR.
Intensive care unit (ICU) interdisciplinary rounds and electronic documentation are the formal structures used in many ICUs for clinical communication and decision-making.2–5 Additionally, EHR information quality affects clinical communication and decision-making as well as secondary purposes, such as research, quality assurance, and policy planning.6 7 A 2008 scientific review called for studies to analyze the documentation artifacts created by healthcare professionals to inform future EHR development.7
ICU patient care is dependent on clinicians from many disciplines who simultaneously must work autonomously and collaboratively.8 During ICU interdisciplinary rounds, various clinicians contribute to the verbal discussion, and goals are defined that guide clinicians' actions until the next interdisciplinary rounds are held the following day.3 Surprisingly, these goals may not be explicitly represented in clinical documentation.9 Moreover, evidence suggests that the nurses' and physicians' shared understanding of common goals in the ICU may be limited.9
To improve collaboration, several groups have implemented a paper interdisciplinary ICU daily goals sheet.4 10 11 The goals sheet may help establish mutual understanding about the plan of care and increase collaboration among clinicians, thereby improving the quality of care.4 10 11 However, the use of goals sheets is not ubiquitous, and no evidence exists that such tools have been effectively integrated into EHRs. Identifying the extent to which interdisciplinary daily goals are documented in EHRs in the absence of a daily goals sheet is an important step to developing an electronic process. Therefore, the purpose of this study was to examine how accurately electronic documentation reflected clinical discussions and decisions of common goals of patient care.
Common ground, or mutual understanding, as it relates to the communication of common goals is a core concept of the distributed cognition and the clinical communication space theoretical frameworks.2 12 The combination of these theoretical frameworks informed the analysis of the communication and documentation of common goals among ICU nurses, physicians, and respiratory therapists. The clinical communication space2 describes the types of communication and information tasks used to meet clinical information needs related to common goals. This framework posits that clinicians choose unstructured tools (eg, face-to-face conversation) when there is a high degree of interactivity and much variability in what needs to be said, and that clinicians choose structured tools (eg, EHRs) when interactivity is low and the information to be exchanged can be formally modeled and coded.2
The concept of ‘common ground’ is described as a means to facilitate effective informal communication, where interactivity and time pressure is high.2 However, achieving common ground can be a challenge.2 12 Distributed cognition informs the description of individual and team-based human performance in healthcare (eg, decision-making strategies) and the design of technologies (eg, EHRs) to enhance this performance.8 13 Distributed cognition supports the notion of division of labor, and suggests that within an activity system (eg, ICU) it would be an inefficient use of cognitive processes for every individual to possess complete knowledge of all activities or information. During a face-to-face discussion, individuals can develop a mutual understanding; however, speech is transient and must be documented for the information to persist over time.12 14
The distributed cognition and the clinical communication space theoretical frameworks highlight the dependency between discussions (eg, rounds) and documentation (eg, EHR notes) during the process of establishing and maintaining common ground among clinicians. Therefore, the analysis of the accuracy with which electronic patient documentation reflects clinical discussions and decisions, an area of research that has not been studied, may have significant contributions to clinical care.
Setting and study population
This study took place on an 18-bed neurovascular ICU (NICU) at a large academic medical center in New York City. All nurses, resident physicians, attending physicians, and respiratory therapists in the NICU used the EHR for discipline-specific patient care documentation or order entry. Some clinicians carried paper-based personal ‘to-do’ notes or EHR printouts; a paper-based nursing care plan was available but was not used. The nurses and respiratory therapists could enter narrative EHR notes, although their documentation primarily consisted of structured EHR flow sheets for assessments, interventions, and one goal specific to this study (‘wean ventilator’). The physician EHR documentation consisted of order entry and narrative typed notes. Documentation by exception was not practiced on this unit.
The sample population consisted of all patients receiving mechanical ventilation, and the unit of analysis was distinct episodes of a stated common goal during ICU interdisciplinary rounds. A priori power analysis indicated that a sample size of 121 distinctly stated common goals for patient care was needed to detect a statistically significant difference between all goals discussed and all goals documented (medium effect size 0.3, α=0.05, β=0.20, df=3, χ2). Institutional review board approval was obtained for this study.
This descriptive correlational study analyzed the degree to which common goals stated during rounds were represented within the EHR. Based on our knowledge of ICU ventilator management, we defined five categories of ventilator management goals and iteratively refined and validated these during preliminary observations: (1) ventilator-related; (2) sedation; (3) extubation; (4) tracheostomy; (5) withdrawal of care. From these five categories we defined 11 explicit goals: (1) wean the ventilator; (2) do not wean ventilator; (3) wean sedation; (4) do not wean sedation; (5) extubate; (6) do not extubate; (7) perform a tracheostomy today; (8) perform a tracheostomy in the future; (9) place the patient on a tracheostomy collar; (10) discuss with the family about withdrawal of care; (11) withdraw care. Note that some goals within the same category were mutually exclusive (eg, ‘wean the ventilator’ and ‘do not wean the ventilator’), and others have inherent dependencies (eg, ‘weaning sedation’ increases success of ‘weaning the ventilator’). One of the authors (SC) observed ICU rounds and recorded each time one of the 11 ventilator management goals was stated by the ICU team for each patient receiving mechanical ventilation therapy. Exactly 24 h after rounds, we categorized these discussed goals as present or absent in the EHR. Inter-coder reliability during ICU rounds was performed on a targeted 10% of the stated goals with an informaticist who was an experienced neurovascular nurse. Cohen's κ statistic was calculated separately for the five categories because of differences in the number of options within each category.
The observations of ICU rounds took place on consecutive weekdays. A tablet computer was used for structured data entry to record all ventilator management goals stated for each patient, the types of clinicians present at rounds, and if the discussion of one patient was interrupted by the discussion of another patient. EHR documentation was analyzed for each patient from 07:00 on the day of each observation until 07:00 the following day, and a Microsoft Excel document was used to collect the data. EHR data collection included: whether goals stated during rounds were documented and how often; any goal-related actions documented (eg, decreased ventilator settings for the stated goal ‘wean the ventilator’); and the type of clinician (nurse, resident, attending, respiratory therapist) who documented each goal. Goals had to be stated explicitly and were not inferred from any documentation or computer provider order entry orders. Goals analyzed within the EHR were defined as: (1) ‘matching documentation’ if a structured field or narrative text accurately reflected the stated goal; (2) ‘conflicting documentation’ if a structured field or narrative text did not accurately reflect the stated goal (eg, stated goal was ‘do not wean ventilator’ and the documented goal was ‘wean ventilator’); or (3) ‘not documented’ if a stated goal was not present in any structured field or narrative text. Eight different data sources within the EHR were examined: (1) interdisciplinary plan of care flow sheet; (2) computer provider order entry; (3) nurses' notes; (4) physicians' notes; (5) respiratory therapists' notes; (6) treatment flow sheet; (7) respiratory flow sheet; and (8) intake and output flow sheet. The interdisciplinary flow sheet was the only interdisciplinary documentation form and included ventilator weaning goal checkboxes for nurses and respiratory therapists. Goals that may have been stated outside of the context of interdisciplinary morning rounds were excluded from the study.
Descriptive statistics were used to describe the stated goals, the documentation rates, the type of clinician who documented each goal, and the clinicians present when each goal was stated. Goals were analyzed by clinician type to capture the instances when one goal was mentioned multiple times in the documentation. χ2 Analyses and the Fisher exact test, when sample size was small, were used to examine differences in the electronic documentation of stated goals and goal-related actions.
Fifteen days of ICU rounds were observed for three to eight patients per day over the course of 4 weeks during April and May 2009. Data were collected from 31 nurses, 9 respiratory therapists, 10 residents, and 3 attending physicians. Patients whose length of stay exceeded 1 day and who remained on mechanical ventilation for multiple days were discussed during rounds on multiple days; therefore, most patients' charts were reviewed more than once. Twenty-eight patients' charts were reviewed for documentation of stated goals. A total of 127 goals were stated during 77 distinct discussions about a specific patient (mean=1.6 goals stated during each patient discussion). Cohen's Kappa statistic was computed for the inter-coder reliability of 16 (13%) stated goals (greater than the targeted 10% of goals were stated during inter-coder observations). The κ value was between 0.82 and 1.00 depending on the goal category: ventilator, κ=1.00; sedation, κ=0.82; extubation, κ=1.00; tracheostomy, κ=0.83; withdrawal of care, κ=1.00.
During the observations, the attending structured and summarized the goal decision-making discussion, yet all clinician types initiated discussions about patient problems and contributed to the decision-making process. The frequency and percentage of stated goals, documented goals, and documented goal-related actions are shown in table 1. The three most commonly stated goals were: (1) ‘wean ventilator’; (2) ‘wean sedation’; and (3) ‘do not wean sedation’. Overall, 75.6% of the stated goals were documented (95% CI 67.2 to 82.8) (‘wean ventilator’=100%; ‘do not wean ventilator’=90%; ‘wean sedation’=72.4%; ‘do not wean sedation’=35.7%).
Table 2 shows the number of stated goals by clinician type that had ‘matching documentation’, ‘conflicting documentation’, and were ‘not documented’. To perform the χ2 analysis, the frequency counts of the ‘conflicting documentation’ and the ‘not documented’ stated goals were combined, and these were then compared with the ‘matching documentation’ goals to assess differences between clinician types. There was a statistically significant difference in documentation by clinician type of stated ventilator goals (χ2=21.9, df=3, p<0.0001), stated sedation goals (χ2=38.8, df=3, p<0.0001), and stated extubation goals (χ2=12.1, df=3, p=0.007). The attending physician was the most likely clinician to document a ventilator goal (81% documented), and the nurse was the second most likely clinician to document a ventilator goal (71.4% documented). The attending was the most likely clinician to document sedation (48.9% documented) and extubation goals (66.7% documented), and the resident was the second most likely clinician to document sedation (21.0% documented) and extubation goals (46.7% documented). Conflicting information in resident and attending notes was evenly distributed among goals. Except for one instance, nurses' and respiratory therapists' conflicting documentation was for ventilator goals. Only three of the 127 total goals had ‘conflicting documentation’ without ‘matching documentation’ elsewhere; all goals that had conflicting documentation still had a correct action documented.
Overall, the documentation of goals and goal-related actions was significantly different (χ2=10.121, df=3, p=0.001) (table 3). If any goal was documented, it was more likely that a goal-related action was documented (82.9% vs 17.0%). If a ventilator goal was documented, a related ventilator action was neither more nor less likely to be documented (p=0.857, Fisher exact test). However, if a sedation goal was documented, it was more likely that a sedation goal-related action was documented (p=0.011, Fisher exact test) (table 3). The type of clinician that documented an action was not collected; it was assumed that the responsibility of actions is an established practice based on the clinician's role.
There was a difference between the rates of matching documentation for the goals ‘wean ventilator’ and ‘wean sedation’ (p=0.001, Fisher exact test) (table 4). The goal ‘wean ventilator’ was more likely to be documented than the goal ‘wean sedation’ (100% vs 72.4%). However, there was no difference between the documentation of actions for the goals ‘wean ventilator’ and ‘wean sedation’ (p=1.000, Fisher exact test).
Analysis of the EHR documentation indicated that the NICU nurses performed hourly documentation of assessments and interventions. However, nurses' documentation seldom explicitly mentioned goals established during rounds. For example, in one instance during rounds, the stated goal was ‘titrate the ventilator's oxygen setting based on the patient's brain oxygen level,’ which is calculated via a monitoring instrument placed inside the patient's brain. Analysis of the nurse's documentation indicated that she adjusted the ventilator oxygen setting at 5 to 30 min intervals. Knowing the titration goal, it was evident that adjustments were made in accordance with the stated goal, based on the brain oxygen level. Nevertheless, the nurse's documentation did not indicate that the minute-to-minute monitoring and ventilator adjustments were performed to meet a specific clinical goal.
There was no significant difference between the attending leading rounds and the number of goals stated per patient discussion (linear regression, R2=0.012, df=1, p=0.334), or ‘matching documentation’ rates (χ2=0.946, df=2, p=0.623). However, there was a difference between the documentation of goal-related actions and the attending leading rounds (χ2=7.142, df=2, p=0.028). There was no significant difference between ‘matching documentation’ rates when another patient was discussed in an interruptive manner during rounds (71.4%) versus when another patient was not discussed in an interruptive manner during rounds (76.8%) (p=0.362, Fisher exact test).
This study examined the degree to which electronic documentation reflected goals that were stated during interdisciplinary rounds in an NICU. A mean of 24.4% of stated goals were not present in the EHR. The range of goals that were present within the documentation was 14.3–100% depending on the type of goal. These findings are comparable to those of a related study by Chisholm et al that identified discrepancies between observations of emergency physicians' assessments and treatments of pain and their documentation.15 Chisholm et al found that the physicians documented 91.7% of their pain assessments, but documented only 31.7% of their pain treatments.15 If a stated goal was not documented, it was over 60% less likely that an action related to that goal was documented (17% vs 45%). However, all conflicting documentation still had a correct action documented. Although we cannot assume that documentation provides hard evidence of an action, it is likely that most documented activities were actually completed.7 These findings support the need for the explicit documentation of common goals that are discussed during rounds.
The attending's note was the most likely place for the stated goals to be documented, and the note reflected the discussions of all ICU team members. This finding suggests that, despite the fact that the EHR system observed in our study does not have a centralized interdisciplinary note, the NICU clinicians treated the attending note as a centralized patient-focused note. However, documentation rates depended on the type of goal and clinician. For instance, ventilator goals were documented about 1.5 times more often than sedation goals (97.6% vs 60.5%). Variable documentation rates may lead to inconsistent information within the EHR. Shaver et al's study, which analyzed the documentation rates of sedation-related events, also found similar documentation inconsistencies, with only 9% agreement between nursing and physician EHR documentation for sedation-related events (95% CI 2 to 16; κ=0.13).16 Therefore, to maintain EHR data integrity, data should be entered once, verified if needed, and reused. Once documented, information should be accessible to clinicians for various purposes (eg, ordering, listing, and planning), and patient updates should be automated throughout the EHR to avoid inefficiencies and errors in clinical care.17 18 Therefore, the practice of documenting goals in one location may be beneficial to information exchange.
The three most commonly stated goals were ‘wean ventilator’ (N=32), ‘wean sedation’ (N=29), and ‘do not wean sedation’ (N=14). Overall, documentation of ventilator goals was high for all clinician types. However, only 35.7% of the ‘do not wean sedation’ goals were documented. Given that weaning sedation is a prerequisite activity to allow a patient to breathe without a ventilator, it may be clinically important that 64.3% of the documentation to indicate sedation should not be weaned was missing. This result may be an artifact of the EHR, which supported nurses' structured documentation of ventilator goals, but not sedation goals.
The lack of documentation related to sedation goals was also observed in attending documentation. The attending note included a stated sedation goal (‘wean sedation’/‘do not wean sedation’) only 49% of the time, compared with a stated ventilator goal 81% of the time. Similarly, Shaver et al found low rates of EHR documentation of sedation-related events compared with an event-reporting system for sedation-related events.16
Nursing EHR documentation contained 40% of the sedation events (95% CI 28 to 53), and the physician documentation contained only 20% of the sedation events (95% CI 11 to 32). The efficacy of nurse-led sedation weaning has been demonstrated in a number of studies.19–21 It was observed that the nurses would suspend sedation before rounds to allow the team to assess the patient's neurological status without a sedative effect, and during rounds the nurses initiated many sedation-related discussions. Therefore, if sedation weaning is considered a nursing activity, then the attending, as well as the resident and respiratory therapist, may be less likely to document sedation goals versus ventilator goals in his or her note. Moreover, the attending's note is the supporting documentation used for professional billing22; if sedation weaning is not a billable goal or action, then attending physicians may be less likely to include this information in their notes. An alternative explanation for the different documentation rates of ‘wean ventilator’ versus ‘wean sedation’ is that the goal of weaning sedation may be implicitly understood to be a necessary part of the process of weaning the ventilator. This may be an example of charting by exception23 in that documenting the goal ‘wean sedation’ may be unnecessary, and it may only be necessary to explicitly state the goal ‘do not wean sedation’ when it is inappropriate to wean sedation. However, the rates at which the goals ‘wean sedation’ (72.4%) and ‘do not wean sedation’ (35.7%) were documented imply that charting by exception, with ‘do not wean sedation’ as the exception, may not be an explanation of the disparate documentation rates between sedation goals and ventilator goals. In other words, if charting by exception was occurring on the NICU, then the exception, ‘do not wean sedation’, should be documented at a higher rate than the normal plan of care. Such an example points to the difference between ‘continuation of care’ goals versus ‘change in care’ goals. However, the interpretation of a goal as a continuation or change is dependent on the patient's previous state; ‘wean ventilator’ may be considered either type of goal, depending on the patient. Further research should investigate ‘continuation’ versus ‘change’ goals.
The nurses' omission of information in documentation, such as in the example of the nurse weaning the ventilator based on the patient's brain oxygen level, may not only be a result of the lack of EHR structure for the explicit documentation of goals as they relate to actions. Keenan and Yakel also demonstrated that nurses omit information that was used in practice and communicated to others, such as judgments and decisions.24 Nurses may assume that these pieces of information are understood by others to be part of standard nursing practice and therefore do not warrant explicit documentation.24 Yet, billing codes are used to reflect the patient care delivered by a healthcare provider, and the completeness, accuracy, and preciseness of these codes are used to determine evidence to support clinical decision-making and healthcare policy.25 It is possible that nurses' omission of information in documentation may be a downstream effect of the fact that nurses do not bill independently for their services. Therefore, a nurse's documentation is not used for a secondary purpose that requires that it demonstrate a link between the nurse's professional practice judgments and activities and the patient care that was delivered. The act of not documenting information that is verbally communicated by nurses has implications for the nursing profession by concealing an important dimension of nurses' work.26 Additionally, the concealment of nurses' work, by omitting documentation of clinical judgments, may have clinical significance to patient care, nursing knowledge development, and efforts to improve patient care practices.
The use of the attending ICU note for multiple purposes may have some patient safety implications if these multiple purposes are not explicitly understood by all members of the ICU team. The use of the note for billing purposes is currently specific to the attending. A lack of common ground regarding the purpose and intended use of the attending ICU note exists if some members of the ICU team view the attending ICU note as the record of the interdisciplinary common goals discussed during ICU interdisciplinary morning rounds and the attending views the note as a billing record.22 The attending may not include some goals that he or she considers to be part of the nursing domain because he or she cannot bill for those goals; therefore, a night shift nurse may not be aware that the attending ICU note is an incomplete record of the interdisciplinary goals that were discussed during rounds. Furthermore, we concluded that the attending leading rounds and interruptions did not affect the documentation of goals; however, future research should investigate how other environmental and clinical factors affect documentation rates, such as overnight events, patient status, length of stay, or time lag to documentation.
The limitations to this study include the single setting of one specialty neurovascular ICU at a large academic medical center using a commercial EHR. The data collection was performed by one researcher, yet our inter-coder reliability (κ) was between 0.82 and 1.00 for 13% of the sample. Data regarding changes to the plan of care after rounds were not collected; therefore, our data did not reflect instances when a goal was changed or an action related to a goal was not performed for a clinically appropriate reason.
The rates of undocumented stated goals ranged from 0 to 85.7% (mean=24.4%) depending on goal type. Goals that were not documented were 60% less likely to have a goal-related action documented. Overall, stated goals were most commonly documented in the attending physician's ICU progress note, which was used as an unofficial, interdisciplinary centralized patient-focused plan of care. However, the attending's ICU note was an imperfect source of the goals stated during rounds. Sedation goals were not routinely present in nursing or physician EHR documentation. Inconsistent goal documentation indicates that the current EHR documentation structure may not be sufficient to capture common goals discussed during ICU interdisciplinary rounds. The purpose of EHR documentation should be aligned with how it is used by other clinicians who provide patient care. Future work should support and centralize the electronic documentation of interdisciplinary common goals of patient care in the ICU, analyze the impact of environmental and clinical factors on goal documentation, analyze the effect of goal documentation on patient outcomes, and implement decision support tools specific to common goals of patient care.
We thank the NICU clinicians who participated in the study, and Sunmoo Yoon, RN, PhD, for performing the inter-coder reliability.
Funding This project was supported by National Institute for Nursing Research (T32NR007969). SAC is supported by the National Library of Medicine (T15 LM 007079).
Competing interests None.
Ethics approval This study was conducted with the approval of Columbia University.
Provenance and peer review Not commissioned; externally peer reviewed.