Evaluation of a Physician Informatics Tool to Improve Patient Handoffs
- aVA HSR&D Center for Implementing Evidence-based Practice, Richard L Roudebush VA Medical Center, Indianapolis, IN
- bIU Center for Health Services and Outcomes Research, Regenstrief Institute, Inc, Indianapolis, IN
- cVA Getting at Patient Safety (GAPS) Center, Cincinnati, OH
- dOhio State University, Columbus, OH
- eRegenstrief Center for Healthcare Engineering, Purdue University, West Lafayette, IN
- fDivision of General Medicine and Geriatrics, Department of Medicine, Indiana University School of Medicine, Indianapolis, IN
- Correspondence: Mindy Flanagan, PhD, IU Center for Health Services and Outcomes Research, Regenstrief Institute, 410 W. 10th Street, Indianapolis, IN 46202; e-mail: < >.
- Received 12 June 2008
- Accepted 1 March 2009
Objective To facilitate patient handoffs between physicians, the computerized patient handoff tool (PHT) extracts information from the electronic health record to populate a form that is printed and given to the cross-cover physician. Objectives were to: (1) evaluate the rate at which data elements of interest were extracted from the electronic health record into the PHT, (2) assess the frequency for needing information beyond that contained in the PHT and where obtained, (3) assess physician's perceptions of the PHT, (4) identify opportunities for improvement.
Design Observational study.
Measurements This multi-method study included content coding of PHT forms, end of shift surveys of cross-cover resident physicians, and semi-structured interviews to identify opportunities for improvement. Thirty-five of 42 internal medicine resident physicians participated. Measures included: 1264 PHT forms coded for type of information, 63 end-of-shift surveys of cross-cover residents (residents could participate 2 times), and 18 semi-structured interviews.
Results For objective 1, patient identifiers and medications were reliably extracted (>98%). Other types of information—allergies and code status—were more variable (<50%). For objective 2, nearly a quarter of respondents required information from physician notes not available in the PHT. For objective 3, respondents found that the PHT supported handoffs but indicated that it often excluded the assessment and plan. For objective 4, residents suggested including treatment plans.
Conclusions The PHT reliably extracts information from the electronic health record. Respondents found the PHT to be suitable, although opportunities for improvement were identified.
According to the Institute of Medicine, up to 98,000 patients die and another 15 million are harmed in United States hospitals annually due to medical errors.1 Root cause analysis of reported sentinel events from 1994 to 2004 reveals that two-thirds of the errors were due to communication failures.2 One area of risk for communication failures is during “handoffs”, when a patient's care is transferred from one provider to another. For example, while caring for inpatients, resident physicians participate in handoffs as they transfer patient care to a cross-cover resident. Typically, the primary resident cares for patients during the day and a cross-cover resident assumes care at night. Even though patient handoffs are critical to ensuring knowledge transfer, they are often unstructured and haphazard; therefore, the information transmitted to the on-call provider varies considerably.3
The hazards of patient care transfers have been empirically documented in inpatient settings. In a matched case-control study of an inpatient service, the likelihood of preventable adverse events was significantly higher under the care of a cross-covering physician than under the admitting care team.4 Similarly, patients admitted to the hospital by a cross-covering physician (rather than primary physician) had longer inpatient stays and more laboratory tests.5
Improving and standardizing information transfer between physicians may reduce susceptibility to communication failures. Findings from interviews with twenty-six interns suggest that the preferred approach to handoffs is a combination of thorough face-to-face verbal handoffs that include discussion of anticipated issues with written handoffs that contain accurate and updated patient content, such as code status and anticipated medical issues.6 The Joint Commission included implementing standardized patient handoff procedures as one of its National Patient Safety Goals in 2006 and 2007. It specified that information transmitted during handoffs should include up-to-date data about care, treatment, services, condition, and recent or anticipated changes. It also specified that handoffs be interactive, with minimal interruptions, so the receiver can review and verify patient information, and ask questions.7
One electronic tool developed at the University of Washington—“UWCores”—has improved continuity of care and decreased length of time to prepare handoffs. UWCores uses the World Wide Web and consequently is accessible in the hospital as well as on personal Internet-connected computers. The tool allows residents to organize patient lists based on cross-cover assignments and automatically downloads patient data as part of the handoff form that is transferred to on-call residents.8 Another study testing a computerized handoff program reduced patient adverse events.9
Clinicians at the Indianapolis Veterans Affairs (VA) Medical Center use a computerized patient handoff tool (PHT).10 11 Similar to UWCores, this PHT generates a paper form after automatically extracting patient data from the VA's computerized patient record system (CPRS). The VHA developed this electronic medical record system (EMR) to enable clinicians to review and analyze patient clinical data, order laboratory tests and medications, document care, review radiology and other data and support clinical decision-making. The PHT is an application that displays a defined list of fields in EMR which extracts the most recent information available in the patient record. The information in the EMR comes from a variety of sources (active medications and code status are extracted from physician orders; allergy data are entered by the pharmacy). The resident physician typically completes a free text physician note (i.e., daily progress note) in the EMR after completing rounds, although attending physicians, specialist physicians, nurses, or other care providers can also write and sign a physician note. The physician note may contain an assessment of the patient's presentation, an active problem list, past medical history, venous/instrumentation access, laboratory data, tests, short-term concerns, long-term plan, and psychosocial concerns.
Typically, at the end of the day after completing physician notes, the outgoing physician prints a PHT paper form for each patient and gives these documents to the incoming physician covering the night shift. Physicians generally have a brief face-to-face conversation about patients as well.
At the time of this study, the structured PHT paper form included patient name, age, sex, Social Security number, hospital location, medicine team assignment, allergies, and active medications (see Fig 1 for PHT form). Some PHT forms included the assessment and plan from the physician note and/or information that the physician entered into an available text box for additional notes. The assessment and plan from the transferring physician's most recent note was included at the bottom, when certain terms (e.g., “assessment”, “plan”) were included in the text. The PHT included a text box for the outgoing physician to incorporate other key patient information, such as anticipated problems and follow-up tasks (these notes are not saved in the EMR), that printed out on the PHT paper form. Content in physician notes and PHT text box did not follow a standard format, which introduced some variability to the information that was transferred.
While our research team was not involved in designing the PHT, we sought to formally evaluate, modify, and improve it. We had four study objectives: (1) evaluate the rate at which data elements of interest were extracted from the electronic health record into the PHT, (2) assess the frequency of needing information beyond what was contained in the PHT and where it was obtained, (3) assess physicians' perceptions of the PHT, and (4) identify opportunities to improve the design of the PHT.
We designed a multimethod study, incorporating both quantitative and qualitative approaches. We collected three types of data. First, for Objective 1, information contained in the PHT forms was coded into categories (including coding the most recent physician note). Second, for Objective 2, cross-cover residents were surveyed at the end of their shift about information omitted from the handoff. Third, for objectives 3 and 4, semi-structured interviews were conducted to assess perceptions of the PHT and identify other improvement opportunities.
The research was conducted with internal medicine resident physicians at the Indianapolis VA Medical Center. Eligible residents were first year house staff officers completing month-long rotations either on an internal medicine inpatient ward or in the medical intensive care unit. Residents completed surveys either once or twice during their rotation (after a 4 pm−7am cross-cover shift) and had the option to participate in a semi-structured interview. For each month of the study (Apr 2006 to Jun 2006), 14 residents were eligible to participate (4 of whom were working in the medical intensive care unit), resulting in a potential sample of 42 subjects.
During a monthly orientation session for residents beginning service at the Indianapolis VA Medical Center, the researcher explained the purpose of the study and invited residents to participate. If willing, residents completed the consent form at orientation. The researcher then contacted participants at the end of cross-cover shifts to collect PHT forms and administer the postcall survey. Residents were asked to participate after their second cross-cover shift to ensure that they had some experience with the PHT. The researcher also invited participants to be interviewed about the PHT. The researcher opportunistically selected residents based on their availability to participate in the interviews (i.e., residents were in the team room and had time to answer questions). These interviews took place in the last week of their month on-service. The study was approved by the Indiana University Institutional Review Board and Indianapolis VA Medical Center Research Committee.
To address Objective 1, the information contained in PHT forms was coded using a structured coding algorithm. A physician trained the researcher to use the coding algorithm.
We recorded whether the most recent physician note was extracted from the EMR. For the physician note to be extracted, it had to be written in the patient's record and include specific note titles (i.e., “assessment”, “plan”). Information in the PHT forms was coded into 22 categories (see Table 1 for the complete list of categories). These categories came directly from the Solet et al recommendations for handoff content.11 Nine of the categories pertained to the assessment and plan from the physician note. For each category, the information was coded as either present or absent.
To compute interrater reliability, 15 forms were randomly selected from the first 80 PHT forms and were independently coded by both the physician and the researcher after clarification of the codes by the researcher. The coding between the two reviewers was consistent (interrater reliability k = 0.92). The researcher completed all remaining coding.
To address Objective 2, we developed a postcall survey, including both open- and closed-ended questions, to assess how busy residents were during the cross-cover shift, whether unanticipated patient events occurred, whether there was other information, if communicated during the handoff, that would have been useful for patient care, and where they sought patient information when it was not included in the handoff. The survey was reviewed for clarity by several researchers and physicians and then revised. Residents responded to “Was there information that would have been useful that you did not receive during sign-out to prepare you for caring for your patients last night?” if yes, respondents indicated which information source was most useful in obtaining the information required to handle the patient's care (e.g., physician note, phone call to fellow, laboratory data). Residents could select more than one source of information that they found helpful. All postcall survey items are listed in Table 2.
To address Objectives 3 and 4, a researcher conducted semi-structured interviews with residents to assess perceptions (e.g., satisfaction, ease of use) of the PHT and identify other potential ways to improve the PHT.12 These interviews were conducted in team rooms, which are semiprivate and allow limited interruptions.
The semi-structured nature of the interviews allowed two-way communication and follow-up questions to further clarify answers. The interviewer asked residents to think of a recent patient under their care and describe how the PHT might have been used more effectively for that patient. As part of this scenario-based discussion, the interviewer also asked “What information, if any, turned out to be critical to know at the beginning of the shift for this patient?” other questions included the following: “in thinking about a typical patient, how has the Patient Handoff tool not worked for certain patients?” and “Are there changes that you would like to see made to the changeover list?” The interview questions are shown in Table 3. The interviews were brief (lasting 5–10 min). They were tape-recoded and transcribed.
To address objectives 1 and 2, descriptive statistics for PHT coding and survey data were calculated. As another component to address Objective 2, a generalized estimating equations (GEE) approach was used to determine which PHT categories related to the dependent variable—resident reports of searching for patient information (not contained in the handoff) during the cross-cover shift. The dependent variable was dichotomous (0 = did not report missing patient information, 1 = did report missing patient information). Independent variables included nine categories coded from the PHT forms pertaining to the physician note along with the presence or absence of allergies and code status. The other categories were not included either because they were nearly always present or were not expected to influence information gathering (i.e., patient race). Additionally, the number of patients under the resident's care during the shift was included as a covariate. The GEE methodology, an extension of generalized linear models, is an important approach when analyzing correlated data. The GEE methodology calculates parameters and standard errors with consideration to within cluster variability rather than to only between group variation.13 In the present study, multiple PHT forms were gathered per resident, resulting in a clustered (or correlated) data structure. Alpha was set at 0.05. Statistical analyses were conducted using SAS version 9.1 (Cary, NC).
For objectives 3 and 4, we conducted qualitative analysis of the interview data using a process tracing analysis.14 Two coders independently reviewed the transcripts and coded the data for themes. For objective 3, we coded statements bearing on satisfaction and problems encountered with the PHT. We also coded information about residents' perceptions of the PHT (e.g., easy to use). For Objective 4, we coded information that residents mentioned as important in a handoff or ways to improve the current PHT. Any areas of disagreement were discussed and consensus reached.
Sixty-three surveys were completed by 35 (83% response rate) internal medicine residents. Of the seven residents who were not included in the study, one refused and the others provided incomplete data (because they did not have time to participate). Participants cared for an average of 18 patients per cross-cover shift (range = 7 to 36). During the study period, 1264 handoff forms were collected and coded. Of those, 1216 were generated by the PHT and were included in the analysis. Finally, we invited 18 residents to participate in semi-structured interviews; all 18 agreed to complete semi-structured interviews about their experiences with the PHT.
Objective (1) Information Contained in Patient Handoff Tool Forms
Over 90% of PHT forms included the following information: patient identifiers, patient demographics, ward location, current medication list, problem list, and short-term concerns. Fewer than half of the PHT forms included patient race, allergies, code status, IV access, laboratory test results, long-term plan, and psychosocial concerns (Table 1).
Of the PHT forms, 70% contained the assessment and plan from the most recent physician note. The PHT forms that contained the assessment and plan more frequently included the following: assessment, problem list, past medication history, IV access, laboratory data, tests, and short-term concerns.
Objective (2) Omitted Information in Handoff
Survey results indicated that about a fourth of the time (14 of 63 surveys) residents reported needing additional information that they did not receive during the handoff. Residents reported that they typically sought information contained in the patient's chart or on the computer. For those indicating that they sought information in the patient's chart, 88% reported referring to the resident progress note. For those seeking information on the computer, 97% reported using CPRS. For those cases in which either the resident progress note or CPRS was used to obtain information (36 of 63 surveys), 24 accessed both.
We tested which content in the PHT forms was related to needing additional information that was not received during the handoff (Table 4). When the patient assessment, code status, relevant laboratory data, and short-term concerns were included in the PHT form, residents were less likely to report missing information in the handoff. In contrast, inclusion of the problem list was positively related to missing information. This latter finding seems counter-intuitive but may be due to a relationship between the problem list and acuity. That is, a patient with a problem list included in the physician note is likely highly complex or in an unstable condition.
Objective (3) Resident Perceptions of the Patient Handoff Tool
During the interviews, residents provided preferences for content to be contained in the PHT. Twelve residents indicated that they valued having the patient medication list as part of the PHT form; and, 10 residents indicated they valued the assessment and plan from the most recent physician note included in the PHT form. When the assessment and plan were not included, residents reported having a harder time understanding the plan for a patient. Additionally, residents indicated a preference for receiving a list of anticipated problems and recommendations for treatment (“if-then” list).
Another issue that surfaced was how residents communicated the information on the PHTs to one another. Residents emphasized the importance of having a face-to-face conversation about patients as part of the handoff. However, they noted that face-to-face conversations did not occur as part of all handoffs due to competing demands (e.g., missing outgoing resident while in clinic).
Objective (4) Suggested Improvements for the Patient Handoff Tool
Residents identified several possibilities for improving the PHT. The primary suggestion was that PHT forms be organized by patient location (i.e., ward patients grouped together), which was mentioned by 8 of 18 residents. Additionally, residents tended to find the entire set of PHT forms (usually one patient per page) rather bulky and cumbersome to carry. Based on these concerns the PHT was revised so that patient lists could be organized by unit, and an export function was added that allows the patient information to be displayed in a Microsoft Excel spreadsheet.
Our Facility's PHT captures consistent domains of information to help resident physicians during cross-cover shifts. The PHT has gained wide acceptance among internal medicine residents at this facility as a reliable method for transferring patient information. We found that the PHT form's structure and content were consistent with others' recommendations. Specifically, including a problem list and if-then list that is up to date is important in a handoff.15 Also, a standard form reduces some of the variability in information transfer during handoffs.8
We found that even using a standardized tool, the content was often different from one PHT form to another. This difference in the content contained in the PHT forms was particularly pronounced when the physician note was not automatically included. We attribute this difference in content across PHT forms to be the result of limitations to the data sources (physician note missing key terms—“assessment” and “plan”) rather than a limitation to the PHT. Additionally, handoffs varied depending upon whether verbal communication was part of the handoff process. Due to competing demands at the end of shifts, residents were not always able to have face-to-face conversations. Residents indicated during the interviews that face-to-face conversations are important for communicating issues that might arise during the cross-cover shift. This information is not as clearly communicated in the PHT forms. The Joint Commission guidelines and a prior study of effective handoff strategies both highlight the importance of face-to-face conversations for effective information transfer.7 16 Although the present study focused primarily on the written portion of handoffs, face-to-face communication is equally important for conveying key information. Future research should examine how content is communicated verbally and non-verbally during handoffs.
Residents suggested improvements for creating more reliable and effective handoffs. They emphasized that the best handoffs included a current medication list, assessment and plan, patient identification, and code status. Additionally, they suggested that including anticipated problems for patients along with an if-then recommendation would help them during cross-cover shifts. For example, in an analysis of handoffs in high consequence settings (i.e., aviation, nuclear power sites), including a contingency plan was a key aspect of handoffs.16 Furthermore, similar to recommendations in the literature, residents suggested the addition of “tasks to be done” to handoffs.15 Residents in our study emphasized that having shift tasks would help them to provide better patient care. We have since revised the PHT based on these findings (see Figs 2 and 3).
Strengths and Limitations of the Study
To our knowledge, this study is one of the first evaluations of an electronic tool used in handoffs. Use of a multimethod design is a key strength of the present study, providing complementary information about the PHT. We learned how the tool is typically used through semi-structured interviews, identified what information was contained in the PHT forms, and collected survey data about characteristics of the cross-cover shift.
One important limitation was that we did not ask residents to identify which patients had missing information. Future studies would benefit from linking processes of care with patients for which additional information was needed. The handoff for these patients could then be compared to handoffs for patients not requiring additional information. Another potential limitation is that the participants came from one internal medicine residency program; therefore, the experiences with the PHT may not generalize to other settings and may be idiosyncratic to the setting.
A final limitation to this study is that we did not evaluate physician's satisfaction with data entry into the EMR which could influence the quality of information that was available in the patient record to be extracted by the PHT. A future study could broaden the scope of evaluating the PHT to also consider the influence of the EMR data input on the PHT.
Ensuring optimal handoffs may improve patient care by preventing communication errors. We found that a standardized handoff tool contained valuable information and was highly accepted by residents. Residency programs and health care systems should incorporate training for effective handoffs using a standardized tool. The PHT, marked for dissemination to other VA medical centers, has considerable potential for improving safety.
This research was supported by the Department of Veterans Affairs, Veterans Health Administration, HSR&D RRP 06–156 (Doebbeling PI).
The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.