Translating Knowledge into Practice: Passing the Hot Potato!

As important informatics interventions, clinical reminders (CRs) translate emerging knowledge into accessible advice at the point-of-care. In a timely fashion, CRs prompt clinicians to take specific actions that can improve or optimize patient care. To ensure that CRs achieve their desired impact, systems must deliver CRs to the clinicians most likely to act on them. Sadly, CRs many times pass from one clinician to another like a hot potato, in the hope that someone, somewhere will have the time, inclination, and ability to take action. Or the hot potato is dropped entirely.

Both CRs and similar informatics interventions succeed when they are well-integrated into the clinical workflow processes that they must support. Use of models from sociotechnical systems theory and Human-Computer Interaction (HCI) theory can facilitate CR-related research.1 2 3 4 These theoretical foundations identify common actors and processes as elements, including: a) individual users, b) interactions between users, c) technology, d) interactions between individuals and technology, e) tasks, f) specific contexts, and g) a developmental process that unfolds over time—e.g., users and technology mature as they interact over time. It is not sufficient to claim that a whole body of knowledge, say cognitive engineering, underpins a given study. Rather, researchers apply a theoretical work specifically to the problem at hand, to guide robust study design.

Saleem and colleagues5 identified effective deployment of clinical reminders (a task) as something that could benefit from effective design of computer interfaces (a technology) for delivering clinical reminders to clinicians (the individuals), as targeted for use in the Veterans’ Health Administration (VHA) ambulatory settings (the context). The authors examined prior evidence about problems that can interfere with CR implementation. The investigators believed that problems arising from nurse–physician communications and related to usability could be addressed through a carefully controlled laboratory study.

Saleem and colleagues conducted a laboratory study to evaluate the efficiency and ease of learning related to a redesign of a currently deployed user interface for CRs. The authors found that, in 2 of 5 scenarios, when CRs were presented in a particular way, nurses learned more quickly, and in doing so reduced their perceived mental workload and their frustration while improving their satisfaction. While we concur that much can be learned from laboratory investigations that provide insight into efficient design of applications for clinical practice, one must be sure that the investigations provide a sufficient representation of the tasks, contexts, individuals and interactions over time to be sufficiently generalizable.

Taking a broad view, the study by Saleem et al. focuses on: a) naïve users—new to both the old and new CR design and new to the VHA, b) relatively uncomplicated tasks (the CRs), c) a controlled laboratory setting as the context, and d) only the initial learning component of an interaction’s developmental trajectory. To address the main question at hand, all model elements should be considered together. Thus, the Saleem study answers questions about the newest users of new CRs, the newest employees in the VHA during the first minutes they learn the CRs, and on tasks in a controlled laboratory setting. The study outcomes therefore do not inform readers about: the impact once CRs are used by more experienced individuals through repeated practice; how current VHA users would employ or favor the new versus the old CR designs; or, how the CRs might be embedded in work processes (or not). Readers cannot know from the reported study whether the new user interface design affects outcomes after the users and technology move past the initial learning phase. Perhaps the efficiency, workload, frustration, and satisfaction effects disappear entirely after even moderate practice. Or perhaps, the negative effects increase when applied to a clinical setting. These are the crux of effective design considerations.

Surprisingly, the authors note that “(T)he baseline design A is the current VHA CPRS, not the improved ‘reengineered’ (JAVA) version currently being developed by the VHA, which will likely incorporate the recommendations stemming from this study.”5 They note correctly that it is the integration into practice that remains the major challenge. Yet, the focus of the reported study does not address this major challenge because it only addresses naïve users who are new to the VHA and new to CRs, and examines only CRs new to the context at hand.

The study’s design, which engaged 17 nurse participants in a laboratory experiment, was flawed due to the mixed sample of clinical nurses and a nurse practitioner. Within the VHA system, nurse practitioners often serve as primary care practitioners; thus, they function as physicians although they are identified as nurses. This lack of distinction among study participants’ work practices fundamentally confounds the study’s later designation of some guidelines as oriented for “N” (nurse) and “P” (physician). Use of a HCI or socio-technical model may have averted such confounding by directing investigators to characterize workers by the credentials they hold (individual), the role they serve in the organization (context), and the process (task) at hand.

In particular, usability addresses the fit of these elements (individual, context, and task) together. The first step in designing applications is to understand the individual. Therefore, understanding the scopes of practice for nurses, physicians, and their support personnel is essential to effective deployment of CRs into clinical practice. Next, it is crucial to understand the tasks that each group of individuals performs within their scopes of practice and within their specific contexts. It is especially important to identify which tasks are separate and which are integrated. After that, one can design solid CRs, and conduct empirical studies that result in sufficiently generalizable findings. The study by Saleem et al. does not provide enough information about the CRs, the sample population of users (compared to typical VHA users), how the tasks tested match actual and typical VHA work processes, models of VHA clinical situations, or whether the CR content of the study was representative of VHA CR content in general or whether all of these components fit together well.

One important aspect of nursing informatics research focuses on the integration of information technologies into the practice of nursing. The practice of nursing, as defined by the American Nurses Association, is “the diagnosis and treatment of human response to development and disease.”6 There is nothing in the article by Saleem et al.5 that directly addresses the practice of the nurses involved in the study. In fact, short of the presence of nurses as participants in this study, it is unclear from the study how CR integration into nursing practice should progress. To ensure effective integration of CRs into nursing practice, one must carefully examine the practice of nurses in ambulatory care, not simply the presence of nurses in ambulatory care.

The lack of a clear definition of nursing practices seems to underlie one of the key barriers to CR implementation; that is, whose responsibility REALLY is it to ensure that someone attends to the guideline. It is unclear from Saleem’s study how the designation of “N” or “P” clearly delineates the roles and responsibilities of the nurse regarding CR implementation or, significantly, how nurses and physicians really cooperate (or not) in the effective use of the CRs designated by “N” and “P” in this context. Fundamentally, it is not clear from the article by Saleem et al. why some CRs are defined within the purview of nursing and others within the purview of physicians. Application of the previously mentioned theoretical models can facilitate such delineations by anchoring HCI investigations in a larger model or framework. Such anchoring is essential to ensure that any informatics innovation truly supports the professional practice of nursing. More importantly, nurse participants are a scarce resource. The authors of this Editorial assert that the critical thinking and practice scope of nurses should be carefully considered when designing studies of human computer interaction vis-à-vis clinical systems. It would seem prudent to determine whether the nurse should be the one to “hold” the “hot potato” before one conducts studies of HCI related to nursing practice.

Ensuring effective computer system implementations for clinical reminders represents a daunting challenge. We are encouraged and excited to see the rise in systematic attention to human–computer interaction aspects of IT design. We issue a call to action for informatics researchers to optimize the use of available theoretical frameworks and models to guide their usability investigations. We plead that attention to usability be complemented with an awareness that the roles of individuals and the tasks they are responsible for over time within a particular context remain central elements in the investigation of who should hold the ‘hot potato’.