The economics of health information technology in medication management: a systematic review of economic evaluations
- 1Programs for Assessment of Technology in Health (PATH) Research Institute/St Joseph's Healthcare, Hamilton, Ontario, Canada
- 2Department of Clinical Epidemiology and Biostatistics, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
- 3Centre for Evaluation of Medicines, Hamilton, Ontario, Canada
- Correspondence to Dr Daria O'Reilly, Department of Clinical Epidemiology and Biostatistics, Faculty of Health Sciences, McMaster University, 25 Main Street West, Suite 2000, Hamilton, ON L8P 1H1, Canada;
- Received 11 April 2011
- Accepted 7 September 2011
- Published Online First 7 October 2011
Objective To conduct a systematic review and synthesis of the evidence surrounding the cost-effectiveness of health information technology (HIT) in the medication process.
Materials and methods Peer-reviewed electronic databases and gray literature were searched to identify studies on HIT used to assist in the medication management process. Articles including an economic component were reviewed for further screening. For this review, full cost-effectiveness analyses, cost-utility analyses and cost-benefit analyses, as well as cost analyses, were eligible for inclusion and synthesis.
Results The 31 studies included were heterogeneous with respect to the HIT evaluated, setting, and economic methods used. Thus the data could not be synthesized, and a narrative review was conducted. Most studies evaluated computer decision support systems in hospital settings in the USA, and only five of the studied performed full economic evaluations.
Discussion Most studies merely provided cost data; however, useful economic data involves far more input. A full economic evaluation includes a full enumeration of the costs, synthesized with the outcomes of the intervention.
Conclusion The quality of the economic literature in this area is poor. A few studies found that HIT may offer cost advantages despite their increased acquisition costs. However, given the uncertainty that surrounds the costs and outcomes data, and limited study designs, it is difficult to reach any definitive conclusion as to whether the additional costs and benefits represent value for money. Sophisticated concurrent prospective economic evaluations need to be conducted to address whether HIT interventions in the medication management process are cost-effective.
- Son's name
- medical informatics
- evidence-based medicine
- research methodology
- systematic reviews
- information retrieval
- informatics education
- medical informatics
- library science
- machine learning
- predictive modeling
- statistical learning
- privacy technology
- decision modeling
- public health informatics
Background and significance
The introduction of health information technology (HIT) into the medication management process holds the promise of reducing adverse drug events (ADEs), increasing efficiency of care delivery, improving quality of care, reducing costs, and saving money over the longer term. However, even if these technologies are effective, they are complex and expensive to acquire, implement, and maintain. Assessing the cost versus effectiveness is critical to determining HIT's value and ultimately its adoption. An ideal economic evaluation of these technologies would explicitly measure all direct healthcare costs (eg, capital costs) and direct non-healthcare costs (eg, home care services) as well as indirect costs (eg, productivity losses or gains) that could be affected by an intervention. Additionally, the full enumeration of the total costs needs to be synthesized with the consequences of the intervention.1 A review of the economic literature to determine cost-effectiveness and value for money for such interventions is warranted.
The purpose of this paper is to review and synthesize the evidence on the costs and cost-effectiveness of HIT in the medication process. This review was undertaken as part of a larger study funded by the US Agency for Healthcare Research and Quality to review the evidence on the effectiveness of HIT in all phases of the medication management process.2
Details of the search strategy have been described previously.2 Briefly, electronic databases (eg, Medline, Embase, Business Source Complete) and gray literature were searched. The search strategy comprised controlled vocabulary and keywords to identify papers concerned with specific devices supporting the medication process (eg, electronic medical record (EMR)) published in all languages up to the summer of 2010. Hand searches of bibliographies were also performed.
For inclusion, studies had to meet the following criteria: electronic systems that collect, process, or exchange health information about patients and formal care givers; medication management information technology that was integrated with at least one HIT system that processed patient-specific information and provided advice to the healthcare provider or patient or dealt with transmission or order communication between pharmacist and clinical prescriber. Any article that included an economic component was tagged and underwent further screening. For this review, full and partial economic evaluations were eligible for inclusion. A full economic evaluation is the comparative analysis of alternative courses of action in terms of both costs and consequences, and these were further classified into one of the three categories: (1) cost-effectiveness analysis; (2) cost-utility analysis; and (3) cost-benefit analysis.1 The label partial economic evaluation indicates that the studies are restricted to either costs or outcomes or consider no alternative to the intervention being studies. However, cost analyses can provide useful information on ‘upfront’ costs compared with ‘downstream’ cost avoidance1 and thus were included in this review. Within each of these classifications, articles were further categorized into the setting in which the evaluation took place as well as the type of HIT implemented. A checklist for assessing economic evaluations developed by Drummond et al3 was used to help guide the assessment of the literature.
Of the 35 510 articles identified in the original search, 40 contained some cost information. Of these, 31 were included in this analysis (figure 1). The nine excluded studies either did not include any cost data or did not provide an evaluation of the costs. The majority of studies evaluated computer decision support systems (CDSSs) (n=19), and 65% of the studies were conducted in hospital settings. Most of the reports (n=23, 74%) were from the USA, three (10%) were from Canada, and the rest were in Europe and Israel (tables 1 and 2). Only 16% (n=5) of the studies performed full economic evaluations. Two (7%) of the studies were conducted in pediatric populations, and eight (26%) were concerned with improving the use of, and reduction in, the costs of antibiotic therapy. Owing to the heterogeneity of studies, a synthesis of the data was not possible, and thus a narrative description of the findings is provided in the following sections.
Full economic evaluations
The following section reports the findings of the five economic evaluations that provided information on the incremental costs and the incremental effects an HIT. The HITs included computerized physician order entry (CPOE) (n=2), computerized reminder systems (n=2), and CDSS (n=1) for improving prescribing practices (table 1).
Computerized physician order entry
Two studies evaluated the economic consequences of implementing a CPOE system to an existing EMR in a hospital setting. The first study used both primary data and data from various sources to obtain ADE rates as well as cost data.6 The new system included a menu of medications from the formulary with typical doses, drug-allergy checking, drug–drug interaction, and duplicate drug checking. The analysis included system costs (eg, implementation) and cost savings due to decreased healthcare utilization resulting from reductions in ADEs. It was estimated that the electronic CPOE would avert 261 ADEs over a 10-year time horizon compared with the standard paper ordering approach. Given the incremental cost of the new CPOE of US$3.95 million, the incremental cost-effectiveness ratio (ICER) was US$15 192 per ADE averted.
The second economic evaluation of a CPOE system developed a decision analytic model to estimate the net benefits of the intervention aimed at reducing medication errors.7 Monetary values were assigned to the interventions, efficiency savings, and treatment and health effects of preventable ADEs. In addition, quality-adjusted life year-based monetary values of the preventable ADEs were included in the net benefit analysis. The CPOE had a mean net benefit of US$64.9 million over a 5-year time horizon with the intervention and maintenance costs included in the model. The monetary value of lost health needed to be included in the model for the interventions to have a high probability of producing positive net benefits.
Computerized reminder system
The one hospital-based study measuring the cost-effectiveness of a CDSS reminder system conducted a trial-based economic evaluation. The 1-year, prospective trial evaluated the effect of three reminder systems (ie, computer-generated, telephone, mailed letter) on patient compliance with tetanus vaccination.8 The costs of contacting patients were estimated (eg, physician time). The cost per additional vaccination was US$0.64, US$8.16, US$9.11 for the computer reminder system, telephone reminders, and mailed out letter, respectively, versus usual care. The cost of setting up the computerized reminder system was not included in the analysis.
Primary care setting
Computerized reminder system
A trial-based economic evaluation compared the costs and effects of a multifaceted intervention aimed at improving prescribing of antihypertensive and cholesterol-lowering drugs in primary care.9 The intervention included: (1) educational outreach visit to clinics; (2) audit and feedback on adherence to guidelines; and (3) computerized reminders to physicians during patient consultations. This intervention was compared with passive dissemination of guidelines through a national medical journal. Over the 1-year study period, the ICER was US$570.25 per additional patient started on thiazides rather than another antihypertensive agent. The reduction in drug expenditures did not outweigh the costs of the intervention; however, if the effect was sustained for a second year, the intervention would be expected to lead to savings.
Computer decision support system
A multicentre, pragmatic randomized study determined the cost-effectiveness of a CDSS designed to promote guidelines for the treatment of asthma.10 Twenty physicians (198 asthmatic patients) were randomized to a hand-held CDSS that offered therapeutic recommendations based on guidelines or usual care. Effectiveness was determined by measuring the quality of life through the St Georges Respiratory Questionnaire (SGRQ). Costs were calculated from the consumption of resources over the 1-year study period. From a societal perspective, the intervention dominated standard care (ie, less costly and more effective). From the healthcare payer perspective, the ICER was US$66.42 per percentage point reduction in the SGRQ. The cost of the CDSS was not included in the analysis.
Partial economic evaluations
Most of the economic literature reported the results of partial evaluations (26 of 31, 84%). All of these evaluations took the form of cost analyses whereby the costs and effects of the alternatives were examined separately in the analysis.
Computerized physician order entry
A computerized order set within a CPOE was designed to manage hospitalized pediatric patients with asthma.11 A before–after study evaluated the relationship between computerized order set use and financial outcomes by studying the use of three generally recommended inpatient asthma treatments (ie, systemic corticosteroids, pulse oximetry, and metered-dose inhalers). There was no significant difference in the total inpatient costs among the pre- and post-intervention groups. Hospital charges were US$4381 and US$4616, while the pharmacy charges were US$458 and US$527 in the pre- and post-intervention groups, respectively.
Another pre/post study assessed the introduction of a CPOE system and electronic medication administration record across all inpatient clinical areas.12 More than 450 evidenced-based order sets (eg, drug interaction), designed to meet the needs of all clinical specialties, were available to facilitate and expedite electronic order entry to support best practice. Severity-adjusted total cost per admission for all services did not change significantly in the health system.
A CPOE introduced to help with the management of surgical patients in an academic, multispecialty hospital had no effect on the rate of medication errors.13 However, a redistribution of workload was found leading to personnel changes resulting in a savings of US$485 512. The authors noted that considerable gains in efficiencies will likely result in long-term cost savings and improved quality of care. However, this was an expensive technology to implement (US$3.2 million) and operate (US$2.5 million).
Finally, a randomized controlled trial (RCT) estimated the effects of a CPOE system that displayed various cost information associated with an order with the aim of promoting cost-effective ordering and reduce costs.14 The total charges per admission were significantly less (US$1534) for the intervention teams than for the control teams, with similar differences in all types of charges. If these effects were extrapolated to all medicine service admissions, the projected savings would be US$5.2 million in charges per year. The network hardware costs were approximately US$33 181 per ward, with additional costs for installation and maintenance (not included in the cost savings calculation).
Computer decision support system
Various modifications to a CDSS aimed at improving the use of and reducing the cost of antibiotics have been reported in four separate studies. The evaluation of the first version of the antibiotic consultant was conducted in an academic, tertiary, private hospital. The computer displayed five antibiotic regimens most likely to be effective for an infected patient and suggested an appropriate antibiotic regimen. The average cost for 24 h of antibiotic therapy recommended by the CDSS was US$16.54 per patient less than what was actually prescribed.15 The same CDSS, with additional user options incorporated, was evaluated in two studies that took place in a 12-bed shock/trauma/respiratory intensive care unit. A 7-month pilot study revealed a mean reduction in the cost of antibiotics of US$132.70 less per patient compared with the pre-intervention period.16 Over a 1-year period, the mean cost of antibiotics was US$151 versus US$504 and US$633, while the cost of hospitalization was US$39 017 vs US$52 314 and US$66 522 for the computer regimen followed, regimen overridden, and no CDSS, respectively.17 Finally, an antibiotic-dose monitor was incorporated into the CDSS to check the renal function of patients to identify those who may be receiving excessive dosages of antibiotics.18 The intervention group received fewer mean doses of antibiotics at a lower average cost (US$116.11) than patients during the pre-intervention period (US$133.88). If this reduction in cost is summed for all 4483 patients in the intervention group, this would result in a total decrease of more than US$79 210/year. The costs associated with developing and implementing the CDSS were not included.
Another CDSS for antibiotic prescribing was introduced in a 450-bed community teaching hospital.19 The 5-month study compared patients whose microbiologic data were processed in the usual manner with patients whose data were processed using the software. There was a difference in variable healthcare costs (eg, pharmaceuticals) of US$1988 less per patient in the study group compared with the severity-adjusted control group. Using these adjusted data, the estimated annual cost savings from the intervention was US$3 976 749. If the list price of the CDSS (US$60 357) was subtracted from the expected annual cost savings, the resulting savings (US$3 916 393) would be substantial in the first year.
A third antimicrobial CDSS was evaluated in a 3-month RCT in a 648-bed tertiary care, academic hospital.20 Antimicrobial utilization was managed by an existing management team using the web-based system in the intervention arm and without the system in the control arm. Expenditures for antimicrobial drugs were US$341 891 for the intervention group and US$442 605 for the control group, for a saving of US$100 714 (23%).
An evaluation of another CDSS to support appropriate antibiotic treatment used a cohort study followed by a multicenter (Israel, Germany and Italy), cluster, RCT.21 The trial compared hospital wards using the CDSS with antibiotic monitoring without the CDSS. Total antibiotic costs were US$344 lower per patient for the CDSS, a relative decrease of 48%, the difference originating from lower ecological costs (eg, costs associated with loss of antibiotic efficacy) in intervention wards in Israel and Italy. Direct antibiotic costs, as well as costs incurred by observed adverse events, were similar.
An anti-infective decision support tool, designed for a pediatric population, was introduced in a 26-bed intensive care unit in an academic hospital.22 During the pre-CDSS 6-month period, all patient care orders were handwritten. The study found no difference in hospital costs in the pre-CDSS period (US$38 326.63) compared with the post-CDSS period (US$33 951.71) or on anti-infective costs per patient (US$372.70 in the control group versus US$392.79 in the intervention group).
Chertow et al23 studied the effect of adding a CDSS to an existing CPOE for prescribing drugs to hospitalized patients with renal insufficiency. The authors measured the difference between the intervention and control groups in hospital and pharmacy costs and found that there were no differences (US$6766 vs US$6887 in total costs for the intervention and the control groups, respectively).
CPOE and CDSS
A study conducted in an orthopedic institution assessed the safety and potential cost savings of using a laboratory-based computerized program to manage inpatient warfarin therapy after major joint arthroplasty.24 Consecutive patients having major joint arthroplasty surgery over a 3-year period were compared with a historical cohort undergoing similar procedures in the 18-month period before the program. Financial measures considered were pharmacy and comparative nursing care costs associated with the program. The authors estimated that the potential savings per patient of US$4.62/day were due to a reduction in nursing time (ie, administration time reduced from 15 min per patient to 4 min), for a total annual figure of US$46 910.
The costs associated with the implementation of CPOE with a CDSS over a 10-year period were measured in a 720-adult bed, tertiary care academic hospital.25 Reductions in items such as ADEs, drug costs, and laboratory tests were found, and it was estimated that the system saved the hospital US$35.8 million, even after including the capital and operational costs of US$14.8 million, with cost savings of US$21 million. The authors determined that it took about 6 years for the intervention to be cost saving.
Computerized ADE surveillance system
A computerized ADE surveillance system was used to help identify and prevent specific types of ADEs in hospitalized patients.26 The authors compared the length of stay of patients incurring an ADE with a historical control group of inpatients with no ADEs, and showed that the average length of stay was 20 days for patients with severe ADEs, 13 days for those with moderate ADEs, and 5 days for those with no ADEs. This translated into a cost of US$61 213, US$36 196, and US$10 179 for patients with severe, moderate, and no ADEs, respectively. It is important to acknowledge that the cases were not matched for disease severity and that no direct cost analysis was made of the ADEs prevented by the system compared with before implementation of the system.
A recent publication measured the impact of an ADE alert system on cost and quality outcomes in seven community hospitals within a health network.27 The ADE alerts were triggered in real time, which enabled immediate pharmacy intervention. The results showed a statistically significant decrease in average pharmacy department costs per patient from before to after implementation (US$898 vs US$856, p<0.001). In contrast, the external control group had a significant increase in pharmacy department costs (US$760 vs US$826, p=0.029). If this percentage cost decrease was extrapolated to the control groups' results, this would yield pharmacy department cost savings in excess of US$11.4 million. It was noted that these savings coincided with only modest quality improvements in projected mortality and length of stay. The costs associated with averting ADEs were not measured in this study.
Primary care setting
Computerized physician order entry
Weingart et al28 designed an empirical study to understand the potential benefits of medication safety alerts generated by an e-prescribing system in ambulatory care. Using a modified Delphi technique and data from 1.8 million prescriptions, the authors estimated that e-prescribing alerts possibly averted 133–846 ADEs. An expert panel reviewed a sample of common drug interaction alerts, and estimated the likelihood and severity of ADEs associated with each alert, the likely injury to the patient and the healthcare resource utilization required to address each ADE. The analysis estimated that the cost savings due to the e-prescribing by using third-party-payer and publicly available information was US$451 277 (IQR US$158 054–US$1 134 736).
Computer decision support system
In two separate RCTs, the effect of a CDSS that provided guidelines for the treatment of ischemic heart disease and chronic heart failure29 and patients with asthma or chronic obstructive pulmonary disease were evaluated.30 In both studies, care recommendations were displayed to physicians or pharmacists, or both, and the results were compared with not receiving care recommendations. In the heart disease study, there were no differences between the groups in terms of adherence to guidelines or any clinical or subjective patient outcome, including outpatient, inpatient, or total healthcare costs (physician, US$9,076; pharmacist, US$10 639; physician and pharmacist, US$7639; control, US$10 117). Similarly, the CDSS had no impact on total healthcare costs across groups in the asthma and chronic obstructive pulmonary disease study (physician, US$11 530; pharmacist, US$7681; physician and pharmacist, US$8140; control, US$8353).
McMullin et al published two papers that evaluated the impact of a CDSS that provided evidence-based recommendations during the e-prescribing process on prescription costs for a range of medications used in primary care. A retrospective cohort study using pharmacy claims data found that the average cost for new and refilled prescriptions was US$5.97 lower in the intervention group with 6-month savings of US$4127 per clinician.31 A 6-month extension of this study showed 12-month savings on new prescriptions of US$127 152.32
A cluster-randomized, pragmatic trial assessed the cost and effectiveness of a CDSS based on recommendations of the European Society of Cardiology and other societies for Hypercholesterolemia Management in comparison with usual care for patients with hypercholesterolemia.33 The CDSS included recommendations on treatment, drugs, and follow-up visits according to the patient's cardiovascular risk and low-density lipoprotein goals. The CDSS did not modify effectiveness, but the treatment costs of hypercholesterolemia were markedly different in the two groups: US$254 484 in the control group and US$149 415 in the intervention group.
E-prescribing with CDSS
Ornstein et al34 measured the effect of displaying cost information in an EMR at the time of prescribing by family physicians. The authors found no effect on overall drug costs (mean cost per prescription was US$31.44 and US$31.73) in the control and intervention periods.
Hospital and primary care settings
Computer decision support system
A claims-driven CDSS system was designed as a ‘rule-based artificial intelligence engine’ combined with an automatic message generator that conveyed clinical recommendations and supporting literature to physicians.35 A 12-month randomized study found that charges among those whose recommendations were communicated were US$90.14 per member per month (pmpm) lower, and paid claims were US$78.77 pmpm lower than controls compared with baseline values. The intervention cost was US$1.16 pmpm and was associated with lower paid claims of US$9.34 pmpm, suggesting an eightfold return on investment from the payer perspective. However, this study was not intended as a formal cost-effectiveness analysis.
An extension of this analysis used an additional 2 years of data.36 This evaluation showed a reduction in average total healthcare charges in the study group by US$31.61 pmpm compared with the control group.
Relative to the volume of research evidence that has examined the impact of HIT on the medication management process, the amount of literature evaluating the economic impact of these systems lags far behind.2 Compounding the problem is the fact that the quality of the available economic evidence is poor. For example, most of the studies reviewed here would not be considered full economic evaluations.1 Only 16% of the papers measured the cost per successful patient outcome, and 84% simply provided cost data. Economic evaluation of health services is a comparative analysis of alternative courses of action in terms of both their costs and consequences.1 The goal is to identify which intervention is most efficient. The main categories of costs of healthcare interventions are the costs associated with the use of resources within the health sector, those used by patients and their families, those used in other sectors, and productivity changes (figure 2).37 The cost may also include downstream costs associated with ongoing treatment, or the management of adverse events or side effects of the treatment. The consequences are the relevant outcomes of interest caused by an intervention (either clinically measured or reported by the patient). The outcomes could be expressed in terms of final health outcomes such as gains in health-related quality of life, or in terms of intermediate health outcomes (eg, mm Hg in hypertension). However, in general, one should choose an effectiveness measure relating to a final outcome.38
Very few of the studies included the large cost items such as the purchase of new software (capital outlay) or implementation costs (eg, training costs, maintenance costs). Additionally, the settings where HIT programs had already been introduced had existing technology infrastructure (eg, EMRs) to support the new interventions; this may not be the case in many areas. Additionally, whether the technology is commercial or home-grown (eg, academic health center) will have implications for start-up costs and organizational savvy.
The heterogeneity (eg, cost elements) between studies was so great that combining the studies was not possible. This has meant that, while studies have been broadly grouped according to setting and type of intervention, the review has been presented on a study-by-study basis, rather than as a complete synthesis of the results. This makes the interpretation of the results somewhat complicated and commenting on the cost-effectiveness of HIT for medication management difficult. Many of the studies provided evidence of some reductions in costs in certain areas due to the intervention (eg, reduction in drug costs and hospital length of stay). The assumption is that these changes will likely result in long-term cost savings and improved quality of care. These potential cost savings are speculative and are not conclusive.
Despite the limitations of the literature, the great strength of this review lies in the fact that extensive searches were undertaken and included studies that reported any relevant information on the economics of the impact of HIT on medication management. The systematic review searches were updated in July 2010, and should therefore provide a comprehensive up-to-date review of the evidence available. Furthermore, to the best of our knowledge, this is the first systematic review conducted to combine the available research in this area.
Adoption of newer technologies needs to be based on formal evaluation of whether the additional benefit is worth the additional cost. Given the tension between the benefits of HIT for medication management and the high up-front costs, decision-makers deciding whether to implement these technologies need to better understand how and when financial benefits of such systems accrue. These types of analyses are important for well-informed decision-making. In addition, one needs to bear in mind that the effectiveness of any given system is dependent on the system's design, implementation, the user(s) of the system, and the setting into which the system is being introduced. However, because of the focus of our review (ie, systematic review of economic evaluations), we did not conduct a detailed review of implementation issues; this is left for future research.
In summary, the quality of the economic literature in this area is poor. A few of the studies reviewed found that HIT interventions may offer cost advantages despite their acquisition costs. However, given the uncertainty that surrounds the cost and outcomes data, and limited study designs available in the literature, it is difficult to reach any definitive conclusion as to whether the additional costs and benefits represent value for money. Analyses of the consequences of using health technology, both in terms of costs and benefits, is crucial for decisions on resource allocation. We acknowledge that the use of economic methods in this area is relatively immature (74% of the articles were published since 2001), but some of the groundwork has now been carried out for future work in this field.
DO'R and J-ET hold Ontario Ministry of Health and Long-term Care Career Scientist 2 Awards.
Funding The major evidence report was funded by Agency for Healthcare Research and Quality, US Department of Health and Human Services, Contract No HHSA 290-2007-10060-I.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.