rss
J Am Med Inform Assoc 13:261-266 doi:10.1197/jamia.M1984
  • The Practice of Informatics
  • Technology Evaluation

Return on Investment for a Computerized Physician Order Entry System

  1. Rainu Kaushal,
  2. Ashish K Jha,
  3. Calvin Franz,
  4. John Glaser,
  5. Kanaka D Shetty,
  6. Tonushree Jaggi,
  7. Blackford Middleton,
  8. Gilad J Kuperman,
  9. Ramin Khorasani,
  10. Milenko Tanasijevic,
  11. David W BatesBrigham and Women's Hospital CPOE Working Group
  1. Affiliations of the authors: Brigham and Women's Hospital and Harvard Medical School, Boston, MA (RK, AKJ, TJ, BM, RKh, MT, DWB); Department of Public Health, Weill Medical College of Cornell University, New York, NY (RK, GJK); Department of Health Policy and Management, Harvard School of Public Health, Boston, MA (AKJ); Eastern Research Group, Inc., Lexington, MA (CF); Department of Medicine, Columbia University Medical Center, New York, NY (KDS); Information Systems, Partners Healthcare System, Boston, MA (JG, BM, DWB)
  1. Correspondence and reprints: Rainu Kaushal, MD, MPH, Division of General Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, 1620 Tremont Street, Boston, MA 02120-1613; e-mail: <rkaushal{at}partners.org>
  • Received 26 September 2005
  • Accepted 1 February 2006

Abstract

Objective Although computerized physician order entry (CPOE) may decrease errors and improve quality, hospital adoption has been slow. The high costs and limited data on financial benefits of CPOE systems are a major barrier to adoption. The authors assessed the costs and financial benefits of the CPOE system at Brigham and Women's Hospital over ten years.

Design Cost and benefit estimates of a hospital CPOE system at Brigham and Women's Hospital (BWH), a 720-adult bed, tertiary care, academic hospital in Boston.

Measurements Institutional experts provided data about the costs of the CPOE system. Benefits were determined from published studies of the BWH CPOE system, interviews with hospital experts, and relevant internal documents. Net overall savings to the institution and operating budget savings were determined. All data are presented as value figures represented in 2002 dollars.

Results Between 1993 and 2002, the BWH spent $11.8 million to develop, implement, and operate CPOE. Over ten years, the system saved BWH $28.5 million for cumulative net savings of $16.7 million and net operating budget savings of $9.5 million given the institutional 80% prospective reimbursement rate. The CPOE system elements that resulted in the greatest cumulative savings were renal dosing guidance, nursing time utilization, specific drug guidance, and adverse drug event prevention. The CPOE system at BWH has resulted in substantial savings, including operating budget savings, to the institution over ten years.

Conclusion Other hospitals may be able to save money and improve patient safety by investing in CPOE systems.

Between 44,000 to 98,000 Americans die each year due to medical errors, and about 1 million people are injured.1 Although there is controversy regarding the accuracy of the mortality estimates, general agreement exists that iatrogenic injuries are common, costly, and often preventable.2 3 4 5 6 7 8

Medications represent the single most common cause of iatrogenic injuries, accounting for nearly 20% of all such events.9 Reducing medical errors requires a multifaceted approach. Computerized physician order entry (CPOE) with clinical decision support systems (CDSS) is a promising intervention to improving medication safety.10 In one study, CPOE reduced the serious medication error rate by 55%.11 Other studies have also demonstrated reductions in medication errors.12 13 14 Most CPOE systems allow physicians to enter medication orders as well as laboratory, admission, radiology, and transfusion orders electronically. When combined with clinical decision support, CPOE improves medication safety, clinical laboratory and radiology testing, medication costs, adoption of critical pathways, and the efficiency of hospital workflow.13 14 15 16 17

Despite growing evidence and mandates to implement CPOE, adoption has been slow, with only an estimated 15% of hospitals having even partially implemented CPOE.18 19 High costs are one important barrier to faster adoption.20 Hospitals must make a large up-front capital investment without clear data on return on investment or confidence in physician use of implemented systems. These costs include the time required of hospital staff, particularly physicians, for training and use of CPOE systems. Further, while hospitals must shoulder the costs of CPOE, benefits accrue to multiple parties including payers, employers, and patients.21

Given the tension between the clinical benefits of CPOE and the high up-front costs, hospitals deciding whether to implement CPOE need to better understand how and when financial benefits of such systems accrue. Brigham and Women's Hospital (BWH), a 720-adult bed, academic, tertiary care medical center in Boston, MA, implemented a home-grown CPOE system in 1993. We assessed the costs and benefits associated with the implementation of the BWH CPOE system over ten years.

Methods

BWH internally developed and implemented a CPOE system in 1993 with CDSS introduced incrementally over the ensuing years as the system was enhanced. We began by calculating the costs and all financial benefits of the BWH CPOE system. Since many CPOE savings are not demonstrable in an organization's operating budget, we then calculated the benefits that are actually reflected in the organization's operating budget such as decreased drug costs.

Cost Data

For BWH, based on internal documents and interviews with the developers of the CPOE system, we determined the capital costs and assigned 60% of the costs to 1992, 20% to 1993, and 20% to 1994. We began to count operational costs starting on January 1, 1993, and running through December 31, 2002. We included costs such as hardware including workstations and printers, software, network, leadership, and training costs. However, we did not include costs such as those for a pharmacy system, medication administration system, or clinical data repository.

Benefit Data

To estimate the benefits from CPOE, we identified each intervention and calculated the associated cost savings. We obtained actual BWH benefit data from relevant published literature, institutional key informants, and internal documents. We depict the method and amount of cost savings for the most financially profitable clinical decision elements in Table 1. We reference relevant literature in Table 1 as well, although much internal data were also used.

Table 1

Cumulative Benefits for CDSS Elements at Brigham and Women's Hospital

The most rigorously studied aspect of the BWH CPOE system is CDSS to reduce adverse drug events.3 15 22 23 Consequently, we calculated many benefits based on the number of averted adverse drug events (ADEs). For example, the BWH system has drug–drug interaction alerts that check and warn for hazardous interactions between drugs.3 We determined cost savings from these alerts by multiplying the number of averted ADEs by the average cost of an ADE ($4,685 in 1997 dollars).24

We estimated other benefits based on decreased drug costs generally through decreased use. For example, a clinical decision support tool to decrease the frequency of ceftriaxone from twice a day to once a day was introduced in 1999, resulting in 80% of orders being switched to daily dosing.25 We calculated cost savings by multiplying the number of saved doses by the cost of each dose. Another important intervention leading to decreased drug costs was early conversion of intravenous to oral medications for patients who were already taking either other oral medications or an oral diet.26 In this case, we estimated the amount of savings by subtracting the costs of the oral medications from the costs of the intravenous medications.

Some interventions saved money by decreasing laboratory test usage such as an alert that warned physicians when redundant laboratory tests were ordered.27 For laboratory savings, we only had access to laboratory charge savings rather than cost savings due to the internal structuring and recording of hospital data. Although the ratio of charges to costs varied for individual laboratory tests, we used a 0.2 charge-to-cost ratio (i.e., a charge of $1 is associated with a cost of $0.20) on the advice of institutional laboratory experts.

Other interventions addressed radiology test overuse and misuse. For example, one intervention improved appropriate ordering of radiology tests.28 We calculated the savings from this intervention by subtracting the costs of performed tests from the costs of canceled inappropriate tests.

Improved workflow and efficiency led to additional cost savings. For example, improved nursing time utilization resulted in staff and resource savings. Similarly there were savings in physician time utilization.

For some interventions, particularly those that did not show up in the operating budget, we were unable to calculate benefits, and while benefits likely exist for many, we excluded them from the analysis. These interventions include advanced chemotherapy decision support that prevented incorrect dose calculations, lack of rescue medications, and inappropriate administration of medications to the wrong patient. We did not include guided dose algorithms that prevent either over- or underdose of medications requiring calculations such as heparin and digoxin. Nor did we include several radiological interventions such as online scheduling, patient preparatory instructions, and allergy tracking. We also did not include pathway order sets, automated cosign and documentation requirements, or automated sign outs. Finally, some interventions such as transfusion guidance were implemented too recently to include in these analyses.

Live Date and End Date

CPOE CDSS interventions at BWH were phased in during the period of analysis. To account for the costs and benefits of each intervention, we used a number of conventions. When an intervention became active (live date) in the middle of a month, we assumed benefits would not start to accrue until the first day of the following month. When we only had data on the year in which an intervention went live, we assumed that benefits would start to accrue at the midpoint of the year (July 1). Many interventions were initially introduced and then significantly enhanced. In these cases, we reported multiple start dates in Table 1.

Operating Budget Analysis

We also performed an analysis to assess benefits that would be included in the hospital's operating budget. Many savings from CPOE are not realized in the operating budget. We analyzed each clinical decision support element to determine whether the resulting savings were observable in the operating budget. For example, cost savings from lower drug costs directly improve the hospital's operating income. Savings attributable to workflow improvements, on the other hand, may not directly reduce operating costs since improvement in efficiency does not necessarily translate into full-time equivalent reduction.

Discounting, Annualization, Constant Dollars

We discounted all costs and benefits at a 7% annual percentage rate in accordance with the recommendations of the U.S. Office of Management and Budget for economic analyses performed for the federal government.29 This represents a societal discount rate rather than a hospital-specific rate. We discounted all costs and benefits on a monthly basis with costs discounted using a “beginning of period” convention, while benefits discounted using an “end-of-period” convention.

In addition to discounted values, we calculated annualized values. Annualization converts the entire stream of discounted costs and benefits into a series of equal annual payments analogous to mortgage payments on a house.

All current dollar values for costs and benefits were converted to a constant dollar basis to adjust for inflation. We used the Bureau of Labor Statistics' Producer Price Index time series for General Medical and Surgical Hospitals to deflate values to a constant 2002 base year.

Prospective Reimbursement

Prospective reimbursement rates affect the amount of hospital savings from CPOE. If a patient's care is not prospectively reimbursed, then savings do not necessarily accrue to the hospital from an avoided ADE or an unnecessary test since the hospital may be reimbursed by the insurance company regardless of whether the utilization was avoidable. For this period of time at BWH, the average prospective reimbursement rate was 80%.

Results

The Brigham and Women's Hospital Experience

All results are present value figures reported in constant 2002 dollars. In 1992, BWH spent approximately $3.7 million in capital costs and between $600,000 to $1.1 million per ensuing year from 1993 to 2002 in operational costs for total costs of $11.8 million to develop, implement, and operate CPOE (Fig. 1). During these 11 years, the CPOE system saved a total of $28.5 million given the 80% prospective reimbursement rate at BWH. This resulted in a net benefit of $16.7 million ($2.2 million annualized). The operating budget benefits totaled $21.3 million for a net cumulative present value of $9.5 million ($1.3 million annualized). As the project length increased, benefits increased since most of the costs were incurred early in the implementation. It took over five years for BWH to realize a net benefit and over seven years for BWH to realize an operating budget benefit from CPOE.

Figure 1

The net cumulative present value of computerized physician order entry (CPOE) at Brigham and Women's Hospital (BWH) from 1992 to 2002 given an 80% prospective reimbursement rate. Six years after the implementation of CPOE, in 1998, BWH began to make a net profit from the CPOE system. This net profit continues to grow steeply. In 1999, 7.5 years after the implementation of CPOE, a financial benefit accrued in the operating budget.

Cumulative Computerized Physician Order Entry Clinical Decision Support System Benefits for Brigham and Women's Hospital

The largest cumulative savings were from renal dosing guidance ($6.3 million), improved nursing time utilization ($6.0 million), specific or expensive drug guidance (human growth hormone, vancomycin, ceftriaxone, ondansetron, and histamine receptor antagonists) ($4.9 million), ADE prevention ($3.7 million), laboratory charge display and redundant laboratory warnings ($1.9 million), and panic laboratory alerting ($1.8 million)3 15 22 23 24 25 27 30 31 32 33 (Table 1). Because many of the clinical decision support elements were not implemented until the latter part of the evaluated time period, the savings were much greater during the latter years.

Renal dosing guidance saved the institution the most money. In this intervention, the system recommends drug-dosing adjustments based on the patient's renal function. In a large study of this intervention, the number of appropriate orders increased from 30% to 51% (p < 0.001) and mean adjusted length of stay decreased from 4.5 days in the control group to 4.3 days in the intervention group (p = 0.009).30 The annual cost savings were most heavily driven by the decreased length of stay.

The BWH CPOE system improved the efficiency of nursing time utilization. The largest efficiency was attributed to the automation of the medication administration record eliminating the need for a handwritten medication administration record. Other cost savings were due to decreased rework of problematic orders to avoid medication errors or ADEs since the CPOE system was performing many of these checks at the time of order creation.

Specific or expensive drug guidance saved the institution $4.9 million during the ten-year study period.15 25 31 An intervention that suggested appropriate indications for initiation and continuation of vancomycin through explicit indications and prompts to discontinue the drug after three days resulted in 32% fewer vancomycin orders. The net effect was $20,042 in annual cost savings ($80,166 in vancomycin savings and $60,124 in replacement drug costs).31 Prior to another intervention, 80% of ceftriaxone orders were written for twice-daily dosing. Providers were prompted to dose daily instead with an 80% switch to once-daily dosing and a cost savings of $175,094 annually.25 An intervention to reduce the use of human growth hormone by suggesting appropriate indications resulted in $320,000 in annual cost savings. Another intervention increased the number of ondansetron orders with a three-times daily frequency instead of four times daily, from 5.9% to 93.5% of orders. This medication was ordered an average of 975 times each year. After adjusting for the average duration of each order, the annual savings for this intervention was demonstrated to be $249,218 in 2002 dollars given an 80% prospective reimbursement rate.15 A final intervention recommended nizatidine and ranitidine as histamine-2 receptor antagonist of choice in compliance with the BWH formulary at the time. In a time-series analysis, nizatidine and ranitidine use increased from 11.7% and 0%, respectively, to 97.9% and 97%, respectively.15

A number of interventions aimed at improving patient safety were demonstrated to decrease preventable ADEs from 4.69 to 3.88 per 1,000 patient days in one study.11 Annual cost savings associated with these interventions were determined by multiplying the reduction in ADEs by the cost of ADEs. These interventions include, among others, default drug dose, frequencies, and routes as well as drug–drug and drug–allergy checks.

Another important intervention of the BWH CPOE system allows critical laboratory results to be automatically detected and the responsible provider paged. This system reduces time to appropriate treatment thereby averting ADEs.33

Three important interventions were introduced too recently to be included in the analyses; however, some data are available on cost savings associated with them. A pilot study has indicated that a transfusion guidance system saved the hospital $1.3 million in 2002 dollars. An intervention that suggests appropriate ordering of Clostridium difficile cytotoxin assays is estimated to save $12,000 per year, while an intervention to improve ordering of digoxin levels is estimated to save $74,000 per year. These three interventions were not included in the study.

Annual Computerized Physician Order Entry Clinical Decision Support System Benefits for Brigham and Women's Hospital

Since the cumulative benefits are dependent on the length of intervention, we also summarized the annual savings in 2002 dollars for each CDSS intervention (Table 2). Again renal dosing guidance led the interventions (2.24 million), followed by ADE prevention, nurse time utilization, and specific or expensive drug guidance.

Table 2

Annual Benefits for CDSS Elements at Brigham and Women's Hospital

Discussion

A CPOE system at a large academic hospital that was implemented about 10 years ago saved the hospital about $2.2 million annually with current savings of $16.7 million per year. The operating budget savings were $9.5 million ($1.3 million annualized). It took over five years for the BWH system to begin accruing a net benefit and over seven years to begin accruing an operating budget benefit.

The level and type of decision support were directly related to the amount of savings the hospital achieved. Renal drug dosing, ADE prevention, and expensive or specific drug guidance were the most financially beneficial interventions. It is important to note that the majority of savings accrued from a relatively small number of interventions. These results suggest that hospitals should consider focusing on these CDSS interventions to increase the chance of financial profitability from their CPOE systems. In addition, hospitals should pay careful attention to the method of workflow integration to save nursing and physician time. Expensive drug guidance is an increasingly important type of decision support given the rapid emergence of new drugs. In this model, we included cost containment of human growth hormone and ondansetron but not other expensive medications.

Of note, we included only BWH CDSS elements for which there were good estimates of cost savings in the model. We were unable to include several BWH CDSS elements in the study due to either their timing of implementation or the lack of reliable cost savings data. Even though many interventions have been implemented at BWH, this CPOE system lacks numerous other highly effective interventions such as LDS Hospital's antibiotic assistant.34 35 Further studies of the potential benefits of specific elements of CDSS are necessary for hospitals to accurately understand the value of CPOE.

In performing these analyses, we assumed that costs and benefits would be equally affected by inflation over time. If the price of medical services is growing more rapidly than general inflation, as is likely, then the discount rate is actually declining over time in real terms. If the discount is overestimated in this way, the model would tend to underestimate benefits rather than costs as we front-loaded costs and back-loaded benefits in our analysis.

To achieve the types of benefits modeled based on BWH data, a hospital must have nearly 100% physician use, well-designed CDSS elements, and effective interfaces among CPOE, pharmacy, laboratory, and medication administration record systems. Some hospitals have overcome large financial barriers to implement CPOE, only to fail to achieve widespread use due to physician resistance.36 37 In addition, the automated knowledge necessary for CDSS elements must be represented in ways that allow it to be readily interchanged between different computer systems. Benefits may vary substantially among different vendor applications based on factors like these. The benefits of increased workflow efficiency are perhaps the most difficult to achieve as they require all these factors along with a quick system, although they are very important given the national shortages of nurses and pharmacists.

This study has several limitations. Ideally, all benefit data would have been collected prospectively rather than retrospectively. Active data collection would have decreased the number of estimates by institutional experts. In addition, we did not include less direct benefits of CPOE such as averted malpractice litigation from fewer ADEs.38 We excluded several decision support elements of the BWH system for which we could not calculate benefits. Our model is purely cost avoidance and does not directly address increased revenue. CPOE systems often result in improved billing, but these savings were not incorporated in our benefit estimates. Nor did we include increased efficiencies for personnel such as pharmacists since reliable institutional estimates were not available.

Of note, we did not include all the costs of knowledge engineering by clinicians and engineers to create and encode clinical information with CDSSs. Information technology staff time is included, but not the time devoted by clinicians and researchers in developing clinical rules. These hidden costs were piecemeal over many years and probably represented a relatively small part of the entire costs. Finally, it is essential to note that the vast majority of implemented CPOE systems are vendor based rather than home grown such as the BWH system. Clearly the benefits of CDSS from vendor systems may be different than those from the BWH system limiting generalizability.

In conclusion, the BWH saved significant money by implementing the CPOE system. Other hospitals may realize even greater benefits, particularly if they have high levels of clinical decision support and rates of prospective reimbursement. While many hospitals need assistance accessing capital to purchase a CPOE system, the financial benefits may help justify the expense. Furthermore, hospitals may not require long-term ongoing financial support, although it will be important for institutions to take a long-term perspective. Patient safety is a critical part of the health care mission and the implementation of CPOE can make health care safer and save money.

Footnotes

  • Most of the individual studies that were aggregated in this report were supported by the Agency for Healthcare Research and Quality, Rockville, MD.

  • Members of the Brigham and Women's Hospital CPOE Working Group include Melissa Christino, William Churchill, Petr Jarolim, Joshua Peterson, Eric Poon, Jeffrey Rothschild, Cynthia Spurr, Carl Stapinski, Jonathan M. Teich, Debra Thomas, and Samuel Wang.

References

Free Sample

This recent issue is free to all users to allow everyone the opportunity to see the full scope and typical content of JAMIA.
View free sample issue >>

Access policy for JAMIA

All content published in JAMIA is deposited with PubMed Central by the publisher with a 12 month embargo. Authors/funders may pay an Open Access fee of $2,000 to make the article free on the JAMIA website and PMC immediately on publication.

All content older than 12 months is freely available on this website.

AMIA members can log in with their JAMIA user name (email address) and password or via the AMIA website.