Overcoming challenges to achieving meaningful use: insights from hospitals that successfully received Centers for Medicare and Medicaid Services payments in 2011
- 1Department of Health Services Research, Management and Policy, University of Florida, Gainesville, Florida, USA
- 2Rawls College of Business Administration, Texas Tech University, Lubbock, Texas, USA
- 3Department of Management, University of North Carolina at Greensboro, Greensboro, North Carolina, USA
- 4Department of Global Health Systems and Development, Tulane University, New Orleans, Louisiana, USA
- 5Department of Healthcare Organization and Policy, University of Alabama at Birmingham, Birmingham, Alabama, USA
- Correspondence to Dr Christopher A Harle, Department of Health Services Research, Management and Policy, University of Florida, PO Box 100195, Gainesville, FL,32610 USA;
- Received 31 May 2012
- Accepted 2 September 2012
- Published Online First 22 September 2012
Objective In an effort to understand better the federal electronic health record (EHR) incentive programme's challenges, this study compared hospitals that did and did not receive meaningful use (MU) payments in the programme's first year based on the challenges they anticipated a year before.
Materials and Methods This cross-sectional study used 2010 American Hospital Association survey data and 2011 Centers for Medicare and Medicaid Services data that identify hospitals receiving MU payments. Multivariate regression analysis assessed differences in 2010 anticipated challenges to MU for hospitals that were successful in earning 2011 MU payment compared to hospitals that intended to participate in the programme but were not yet successful.
Results The study sample consisted of 2475 hospitals, 313 of which received MU payments in 2011. Controlling for standard hospital characteristics, hospitals that reported the computerized provider order entry (CPOE) MU criterion as a primary challenge were 18% less likely to receive a 2011 MU payment compared to hospitals that reported other criteria as primary challenges.
Discussion CPOE was the main challenge among hospitals that failed to achieve MU in the first year of the programme. In order to maximize the incentive programme's effectiveness, policymakers, healthcare organizations, and EHR vendors may benefit from increased attention to hospitals’ challenges with CPOE.
Conclusion As the EHR incentive programme matures, policymakers and other stakeholders should consider strategies that maintain the critical elements of MU while adequately supporting hospitals that desire to become MU but are impeded by specific technological, cultural, and organizational adoption and use challenges.
The Health Information Technology for Economic and Clinical Health Act of 2009 aims to reduce healthcare costs and improve quality by rewarding providers that adopt and meaningfully use certified electronic health records (EHR).1 The act's incentive programme addresses the substantial financial barriers to EHR adoption2–5 by providing financial rewards to hospitals for up to 5 years. To accelerate adoption, the programme offers higher bonus payments to those that meet the requirements for ‘meaningful use’ (MU) in earlier years. In 2011, the programme's first year, a reported 605 hospitals received over US$1.1 billion in payments disbursed through Medicare for meeting the MU requirements.6 Thousands of other hospitals registered to participate in the programme but did not attest to achieving MU and therefore did not earn reward payments in the programme's first year.
While the financial barriers to EHR adoption are significant, hospitals also face cultural, organizational and technological challenges.7 ,8 In 2010, before the requirements for MU were finalized, the American Hospital Association (AHA) found that 55% of hospitals intended to participate and earn the 2011 Medicare reward payments. In addition, the AHA asked hospitals to state their anticipated challenges to satisfying the proposed MU requirements.9 Given that a relatively small subset of intending hospitals successfully earned MU reward payments in 2011, a better understanding of the challenges identified by hospitals that did not yet achieve MU can provide insight to policymakers and other Health Information Technology for Economic and Clinical Health Act stakeholders who are interested in ensuring the programme achieves its goals in a timely fashion.
The purpose of this study is to compare the self-reported challenges as stated in 2010 among hospitals that did and did not receive stage one MU reward payments through the Medicare programme in calendar year 2011. Understanding the ways in which these challenges affect MU achievement is essential to ensuring the next wave of hospitals are successful EHR adopters. Identifying the MU challenges that slow EHR adoption will help policymakers direct limited resources in ways that can maximize the overall success of the MU programme.
We drew data from three data sources—the AHA's 2010 EHR adoption survey, the AHA's annual survey of hospitals and Centers for Medicare and Medicaid Services data identifying hospitals that received Medicare MU payments in 2011.6 We limited our study to hospitals potentially eligible to receive Medicare MU incentive payments. Typically, these are non-federal acute care facilities located in the 50 US states. The study sample contains MU payment-eligible hospitals that responded to the AHA EHR adoption survey and either received a 2011 Medicare MU payment or reported in 2010 an intention to participate in the programme but did not attest to MU in 2011, and therefore did not earn the incentive.
Our analysis was designed to assess differences in reported MU challenges for hospitals that were successful in earning MU payment compared to hospitals that intended to participate in the programme but did not. Dichotomous variables representing MU challenges, the dependent variables, were determined by the 2010 AHA EHR adoption survey question that asked hospital to choose two of seven MU criteria that ‘will or would be the most challenging to achieve’. The seven potential challenges were: (1) ‘implement clinical decision support (CDS) rules’; (2) ‘implement computerized provider order entry (CPOE) at specified level of sophistication’; (3) ‘exchange clinical information with other providers’; (4) ‘perform medication reconciliation across settings of care’; (5) ‘give patients access to their data in electronic form’; (6) ‘generate problem lists used codified datasets’; and (7) ‘generate numerator and denominator data for quality reporting directly from EHR’. Bivariate analyses were conducted using χ2 tests of independence to compare the frequencies of each MU challenge for hospitals that received payments and those that did not.
Multivariate logistic regression was used to assess the relationship between each of the challenge variables and MU payment item while controlling for standard hospital characteristics including geographical location (rural or urban), tax status (for-profit or non-profit), number of beds, teaching status (membership in the Council of Teaching Hospitals and Health Systems) and membership in a hospital system or chain. Hospitals were categorized by number of beds using three levels, 1–125 beds, 126–399 beds and 400+ beds. Teaching status and system membership were binary variables. All analyses were conducted in STATA V.10 and statistical significance was considered at the p<0.05 level. Observations with missing data were removed from analyses that included the missing variables. To assist with the interpretation of the statistical models, marginal effects are reported for each coefficient. Marginal effects represent the difference in the probability of an outcome occurring between a given category and the reference group and are more readily comprehended than OR.10
The sample consisted of 2475 hospitals whose characteristics are displayed in table 1. Of these, 313 (12.6%) received an MU payment in 2011, while 2162 indicated their intention to participate in the programme but did not receive payment. A total of 182 sample-eligible hospitals that received a Medicare MU payment in 2011 did not have corresponding responses to the AHA EHR adoption supplement and therefore were not included. These 182 hospitals were more likely to have a for-profit tax status (42.3% vs 19.8% p<0.001).
The frequency of each anticipated challenge to achieving MU (as reported on the 2010 survey) are presented in table 2 and broken out for two groups of hospitals—those receiving MU payments and those not. The challenges reported by hospitals that achieved MU and received the reward payment differed significantly from those that did not on three of seven items. On one hand, hospitals that achieved MU were less likely to indicate that ‘implementing CPOE at specified level of sophistication’ was an anticipated challenge to MU in 2010 (31.6% vs. 49.8%; p<0.001). On the other hand, those that achieved MU were more likely to indicate that ‘giving patients access to their data in electronic form’ (30.9% vs 23.0%; p=0.004) and ‘generating numerator and denominator data for quality reporting directly from EHR’ (37.5% vs 26.8%; p<0.001) were anticipated challenges.
Similar results were found in multivariate analyses that controlled for the standard set of hospital characteristics. These results are presented in table 3. In particular, even after controlling for geographical location, tax status, bed size, teaching status and membership in a hospital system or chain, hospitals that ultimately achieved MU were 18.3% less likely than their counterparts (p<0.001) to have reported that ‘implementing CPOE at specified level of sophistication’ would be a challenge. Moreover, those that achieved MU remained more likely to report that ‘giving patients access to their data in electronic form’ (marginal effect of +7.0%; p=0.017) and ‘generating numerator and denominator data for quality reporting directly from EHR’ (+12.1%; p<0.001) were expected challenges even after controlling for confounders.
The multivariate analysis also showed that several hospital characteristics were associated with anticipated MU challenges. In particular, urban hospitals were more likely than rural hospitals to report that ‘performing medication reconciliation across settings of care’ (+8.7%; p<0.001) and ‘generating problem lists used codified datasets’ (+3.9%; p=0.04) would be challenges to MU attainment. Moreover, for-profit hospitals were more likely to report the CPOE MU criterion as a challenge (+11.2%; p=0.004), while teaching (−29.8%; p<0.001) and system hospitals (−4.4%; p=0.04) were less likely to report CPOE as a challenge. System hospitals were also less likely to report ‘exchanging clinical information with other providers’ (−5.2%; p=0.01) but more likely to report ‘performing medication reconciliation across settings of care’ (+3.6%; p=0.03).
The MU incentive programme was designed to help hospitals overcome financial barriers to EHR adoption and further the goal of widespread EHR use. However, hospitals also face non-financial EHR adoption barriers, and understanding how these barriers relate to MU achievement can help policymakers improve the incentive programme's participation in coming years. Along these lines, the purpose of this study was to compare hospitals that did and did not receive MU payments in 2011 based on the challenges to success they self-identified in 2010.
We found that a large number of hospitals that intended to participate in the programme but did not achieve MU in 2011 may have failed due to the inability to meet the CPOE criterion specified by the MU requirements. In fact, half of all hospitals that did not achieve MU reported that the CPOE criterion will be a primary challenge, and this rate was significantly higher than hospitals that successfully achieved MU.
The CPOE MU criterion is expected to become more stringent in 2014 when stage two MU criteria are implemented. The current stage one MU criterion calls for CPOE implementation so that ‘over 30% of patients with at least one medication in their medication list have at least one medication ordered through CPOE’.1 The proposed stage two criteria aim to increase the CPOE use requirement to 60% and to include laboratory and radiology orders as well as medications.11 If CPOE issues served as a key barrier to the achievement of stage one MU, further raising the bar on CPOE capabilities, without simultaneous improvements in technology usability and demonstrated clinical usefulness, may prevent many hospitals from becoming meaningful users and receiving reward payments in future years.
A common challenge in designing incentive programmes is minimizing the extent to which payments reward already high-performing entities rather than motivate improvements among low performers.12 ,13 While totally eliminating this issue is not necessarily desired or feasible, disallowing hospitals more gradually to overcome their challenges with CPOE may disproportionately reward hospitals that were already using EHR rather than persuading those that had not yet fully adopted EHR to achieve MU. Indeed, further analysis showed hospitals that had at least begun to implement CPOE in 2010 were 27.6% more likely to achieve MU in 2011 than their counterparts with no implementation activity (p<0.001). If the current EHR incentive programme is simply rewarding hospitals that already had the infrastructure in place to meet the MU criteria, its value as an incentive may be suspect. Policymakers must strategically manage the challenges that could prevent hospitals from making the transition to MU.
On one hand, relative to other EHR functions, there is more evidence of CPOE's value in terms of quality and safety improvements.14–17 On the other hand, CPOE is associated with relatively low EHR adoption rates, implementation failures and challenges to consistent use by physicians.18–20 Although policymakers relaxed the overall MU requirements by proposing an extension of stage two implementation to 2014;11 a more effective approach may be to focus on the details of the CPOE criterion because this is where hospitals are disproportionately struggling. Given the complex social, organizational and technological challenges associated with CPOE deployment,21 ,22 especially compared to technically simpler MU criteria (eg, generating quality reports, or giving patients access to their data in electronic form), failing to address the current challenges faced by many hospitals risks attenuating the success of the overall MU programme. A multifaceted solution to these challenges includes strategies to ensure that the MU CPOE criterion is meaningful but attainable and policies that ensure vendor technologies can be efficiently integrated into clinical workflows, provide reporting capabilities that support MU measure calculations, and add clear value to care delivery.
Another interesting result was hospitals that earned MU reward payments reported two challenges at higher rates than those that did not meet the mark. Overcoming those two challenges, ‘giving patients access to their data in electronic form’ and ‘generating numerator and denominator data for quality reporting directly from EHR’, involves mainly technological solutions; whereas successfully implementing CPOE involves surmounting technological, cultural and organizational barriers. Additional analysis (not shown) found that these two challenges were much more likely to be reported by hospitals that did not report CPOE as a primary challenge. These patterns may suggest that challenges in giving patients access to electronic data and generating numerator and denominator reports are primary for hospitals once they have become successful with CPOE but are less likely actually to impede reward payments.
Our study also found that certain hospital characteristics were independently associated with an anticipated challenge with the CPOE MU criterion. Whereas for-profit hospitals were more likely to indicate CPOE as a challenge, teaching hospitals and system hospitals were less likely. CPOE and many associated benefits have been historically more common in academic facilities and system hospitals.14 ,17 ,23–25 Council of Teaching Hospitals and Health Systems-affiliated hospitals may thus have had greater experience with EHR technology, making it easier for them to achieve MU. If so, the incentive programme may also widen the digital divide between those able to overcome vexing CPOE challenges and those that are not. Policymakers should consider ways to support hospitals that struggle with CPOE adoption and use further, paying special attention to non-teaching hospitals, which represent the majority of facilities.
Finally, this study found that system hospitals were less likely to report that ‘exchanging clinical information with other providers’ was a challenge. While not surprising given system hospitals’ integrated nature, the proposed stage two MU criteria potentially introduce a new barrier to this information exchange advantage by requiring that at least 10% of summary of care records be electronically transmitted to a recipient ‘with no organizational affiliation’.11 As the MU programme unfolds, more research is needed on this issue and on the broader challenges associated with coordinated information exchange among unaffiliated providers as well as public health organizations.
The current study is one of the first assessments of the relationship between hospitals’ self-identified MU challenges and their actual MU achievement. However, a limitation of this study was the fact that for-profit hospitals that received MU payments were less likely to have participated in the AHA EHR adoption survey, which reduces the representativeness of our sample. The present study is also limited by its observational nature. The relationships observed between anticipated MU barriers and subsequent MU achievement are thus associations subject to unobserved confounders and are not necessarily causal.
There are additional strengths and weaknesses of the AHA EHR adoption database that should be mentioned. The database is strengthened by the fact that it is produced through a collaboration between the AHA and the Office of the National Coordinator for Health Information Technology and targets the person most knowledgeable about a hospital's health information technology. On the other hand, the survey only asked hospitals about seven MU challenges (a subset of all MU objectives), and hospitals were only asked to identify the two most challenging in no particular order. In future surveys, it would be useful to allow hospitals to rank their challenges and to know whether MU objectives not included in the survey would be judged as more challenging than CPOE.
In conclusion, our analysis suggests that MU achievement in 2011 was related to specific, identifiable challenges that hospitals anticipated in 2010. As the MU programme's subsequent stages evolve, organizational stakeholders and policymakers interested in ensuring the programme's success should carefully consider the nature of these challenges and that financial resource supplements alone may not promote attainment. In particular, the technological, cultural, and organizational barriers to adoption associated with specific EHR applications will need to be overcome.
Contributors CAH: Study conception and design, Centers for Medicare and Medicaid Services data collection, data linking and analysis, paper drafting, editing and revising. TRH: Study conception and design, review and critique of analysis, paper editing and revising. EWF: Study conception and design, paper editing and revising for policy and management content. MLD: Study conception and design, AHA data guidance and analysis review, paper editing and revising. NM: Study conception and design; AHA data collection, data linking and analysis, paper drafting, editing and revising.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.