Context.— Adverse drug events [ADEs] are a significant and costly cause of injury during hospitalization.
Objectives.— To evaluate the efficacy of 2 interventions for preventing nonintercepted serious medication errors, defined as those that either resulted in or had potential to result in an ADE and were not intercepted before reaching the patient.
Design.— Before-after comparison between phase 1 [baseline] and phase 2 [after intervention was implemented] and, within phase 2, a randomized comparison between physican computer order entry [POE] and the combination of POE plus a team intervention.
Setting.— Large tertiary care hospital.
Participants.— For the comparison of phase 1 and 2, all patients admitted to a stratified random sample of 6 medical and surgical units in a tertiary care hospital over a 6-month period, and for the randomized comparison during phase 2, all patients admitted to the same units and 2 randomly selected additional units over a subsequent 9-month period.
Interventions.— A physician computer order entry system [POE] for all units and a team-based intervention that included changing the role of pharmacists, implemented for half the units.
Main Outcome Measure.— Nonintercepted serious medication errors.
Results.— Comparing identical units between phases 1 and 2, nonintercepted serious medication errors decreased 55%, from 10.7 events per 1000 patient-days to 4.86 events per 1000 [P=.01]. The decline occurred for all stages of the medication-use process. Preventable ADEs declined 17% from 4.69 to 3.88 [P=.37], while nonintercepted potential ADEs declined 84% from 5.99 to 0.98 per 1000 patient-days [P=.002]. When POE-only was compared with the POE plus team intervention combined, the team intervention conferred no additonal benefit over POE.
Conclusions.— Physician computer order entry decreased the rate of nonintercepted serious medication errors by more than half, although this decrease was larger for potential ADEs than for errors that actually resulted in an ADE.
HOSPITALS, health care delivery systems, and health care providers all aim to provide the safest care possible. However, many injuries occur during hospitalization. Estimates have suggested that 1.3 million injuries may occur in the United States annually.1 Although many hospital injuries are unpredictable and unavoidable, 20% to 70% may be preventable.2-4
Adverse drug events [ADEs] are an important source of injuries. In the Harvard Medical Practice Study,1,5 ADEs accounted for 19% of injuries in hospitalized patients and represented the single most common cause of injury. In addition to their human costs, ADEs are costly to health care systems. Nationally, ADEs occurring after hospitalization have been projected to cost hospitals $2 billion per year, not including malpractice costs or the costs of injuries to patients.6,7 Hospitalizations initiated by ADEs appear to be at least as expensive.8
These data suggest that health care organizations should be working hard to prevent ADEs, and indeed a number of efforts to curb them have begun recently. However, enthusiasm for these efforts has been hampered by a lack of data proving that specific strategies can prevent ADEs.
Accordingly, we conducted an ADE prevention study, not only to evaluate the frequency and types of ADEs9 but also to analyze the associated errors using a multidisciplinary, systems-oriented approach to understand their causes.10,11
We evaluated 2 interventions, the first targeting primarily the ordering stage and the second targeting the administration and dispensing stages. The first intervention was a physician computer order-entry [POE] system, in which physicians wrote all orders online. The second intervention was a "team" intervention, comprising several small interventions developed by teams of nurses, physicians, and pharmacists. This intervention primarily targeted the administration and dispensing of drugs. Our primary end point was to reduce the number of preventable ADEs and nonintercepted potential ADEs, the combination of which we refer to as nonintercepted serious medication errors. As secondary end points, we evaluated the numbers of types of events that we expected each intervention to target, for example, drug allergies for the POE intervention.
All adults admitted to study units at Brigham and Women's Hospital, Boston, Mass, a 726-bed tertiary care hospital, during the study periods were included. Phase 1, collection of baseline data for the units, took place over a 6-month period from February through July 1993,9 while phase 2, the intervention phase, was conducted over a 9-month period between October 1994 and July 1995. Although the original study was conducted at 2 hospitals, only Brigham and Women's data are included herein because staff changes and administrative changes at Massachusetts General Hospital, Boston, resulted in inadequate data collection. This study was approved by the institutional review boards of Brigham and Women's Hospital and the Harvard School of Public Health, Boston.
In phase 1, conducted prior to the implementation of POE, there were 23 adult, nonobstetrical units at Brigham and Women's Hospital. These units were stratified according to whether they were medical or surgical and whether they were intensive care or general care units. Six study units were selected randomly from all units within a stratum using a random number generator. The study units included 2 intensive care units [ICUs] [1 surgical and 1 medical] and 4 general care units [2 medical and 2 surgical]. We implemented the 2 interventions [POE or POE plus team] in these same units for phase 2 and, in addition, implemented the intervention in 2 other randomly selected general care units [1 medical and 1 surgical] to increase power. Thus, for phase 2, all units received POE, while half the units [randomly selected within stratum by unit type and level of care] received the team intervention as well [Figure 1].
The 2 interventions evaluated were POE and POE plus team. Two main comparisons were made. The first assessed the effectiveness of the interventions by comparing phase 1 and phase 2 for the units assessed in both phases. The second compared the effect of the team intervention with POE vs POE-only by comparing units randomized to either arm.
The POE intervention represented a major systems change and included many features that would be expected to reduce errors. As part of the POE application, physicians were provided with a menu of medications from the formulary and default doses and a range of potential doses for each medication. Physicians were required to enter dosage, route, and frequency for all orders. Also, computerization ensured that all orders were legible, including the signatures of the prescribers. Transcription was greatly reduced, although not eliminated, because the medication administration record was still on paper. For a number of medications, relevant laboratory results were displayed on the screen at the time of ordering [eg, potassium levels when furosemide was ordered]. Other features included consequent orders, which are orders that should follow from other orders [eg, suggestions to perform aminoglycoside levels when aminoglycosides were ordered], and limited drug-allergy checking, drug-drug interaction checking, and drug-laboratory checking. This included checking for the most frequent drug allergies, about 80 of the most serious drug-drug interactions,12 and several drug-laboratory combinations [eg, potassium levels in patients receiving potassium]. More comprehensive checking was implemented in 1996.13 Thus, not all features that could reduce the number of ADEs were in place or present in mature form during the study.
The POE application14,15 functions as part of an internally developed information system, Brigham Integrated Computing System, which manages the hospital's administrative, financial, and clinical data.16 All orders are written using this application, primarily by house officers, although fellows and attending physicians also write orders. Approximately 16000 orders are written daily, 40% of them for medications.
The team intervention was a combination of several specific process changes and was implemented in half the study units selected at random. These process changes included changing the role of the pharmacist; distributing a recommended dilutions chart; making available a computerized drip–rate calculation program; standardizing labeling of intravenous bags, tubes, and pumps; and implementing a pharmacy communication log so that the nursing staff could communicate better with the pharmacy staff. Changing the role of the pharmacists involved changing their work flow so that the pharmacists were much more often present on the unit and available for questions. The pharmacists made daily rounds with the team in the study ICU but not in the control ICU.
Primary and Secondary End Points
Our primary end point was the number of nonintercepted serious medication errors [preventable and nonintercepted potential ADEs]. We define preventable ADEs as those resulting from an error or having been preventable by any means currently available.9 Potential ADEs are errors that have potential for harm but do not result in injury. These include errors that are intercepted before injury occurs and nonintercepted potential ADEs, which are errors that by chance resulted in no injury [eg, penicillin given to a patient with a known allergy, but no reaction occurred].9 We excluded intercepted potential ADEs from our primary outcome because these errors serve to demonstrate that the "safety net" is working, and this rate could actually increase as error-prevention systems are fine-tuned.
As secondary outcomes, we evaluated the numbers of errors in each stage and also within specific categories that were targeted by each intervention. For the POE plus team intervention, the main stages targeted were ordering and transcription. For the team intervention, the main stages were the administration and dispensing of drugs. However, we expected some crossover, ie, that ordering errors would also be diminished by certain team interventions [eg, availability of the pharmacist on physician rounds] and that the administration error rate would be decreased by POE because orders would be clearer. Specific categories of errors targeted by POE included wrong dose, known drug allergy, and drug-drug interactions. Specific categories targeted by the team intervention included errors in concentration of intravenous solutions, in calculation, and in labeling of intravenous tubing, pumps, and bags.
We used the following 3 mechanisms for identifying incidents: [1] nurses and pharmacists participating in the study and control units were asked to report incidents to study investigators; [2] a study investigator visited each unit at least twice daily on weekdays and solicited information from nurses, pharmacists, and clerical personnel concerning all actual or potential drug-related incidents; and [3] the study investigator reviewed charts of all patients at least daily on weekdays.
All incidents were evaluated as previously described9 by 2 physician reviewers blinded to the intervention groups. The reviewers classified each incident as an ADE, a potential ADE, or an exclusion. The ADEs and potential ADEs were further classified by severity and by preventability. If an error was present, the reviewer determined the type of error and at what stage the error occurred. Categories of preventability were defined as being definitely preventable, probably preventable, probably not preventable, and definitely not preventable.17 Results were collapsed into preventable [definitely + probably] and not preventable [probably not + definitely not] in the analyses. Categories of severity were defined as fatal, life-threatening, serious, and significant.18 The stages of the process were ordering, transcribing, dispensing, and administering the medications. When there were disagreements about how an event should be classified [eg, 1 reviewer scored it as preventable, but the other did not], the reviewers met and reached consensus. If consensus could not be reached, a third reviewer evaluated the incident. Reliability for these judgments has previously been reported.9 For judgments about whether an incident was an ADE, κ was 0.81 to 0.98; for preventability, κ was 0.92; and for severity κ was 0.32 to 0.37.
For the before-after analysis, paired t tests were used to compare baseline data [phase 1] from the 6 study units with data after the interventions were implemented in the same units [Figure 1]. To compare the effects of the 2 interventions [POE vs POE plus team], we performed 2 additional analyses. First, to evaluate the effect of the team intervention, we compared all phase 2 data for POE plus team with POE-only units, using a multivariate regression model that included level of care [ICU vs general care] and service [medical vs surgical]. Second, to compare POE vs no POE, we analyzed data from both phases when available from all 8 units, adjusting for level of care and service status with a multivariate regression model. All 8 units in phase 2 received the POE intervention, 4 also received the team intervention, and the 6 phase 1 units were controls in this analysis. To account for correlation between phase 1 and phase 2 results for specific units, we used a generalized estimating approach to estimate SEs of the regression coefficients. Because the error rate climbed substantially with sedatives in phase 2, which could have obscured an effect by stage, we performed a secondary post hoc analysis removing all sedative errors and determining how this would affect the results by stage. All analyses were performed using SAS [SAS Institute Inc, Cary, NC].19
Phase 1 included 2491 admissions to 6 units over a 6-month period. Phase 2 included 4220 admissions to 8 units over a 9-month period [Table 1]. Phase 2 patients were older, more often female, and more often minority, although these differences were small.
In paired analyses comparing phase 1 and phase 2 [Table 2], the rate of nonintercepted serious medication errors fell 55%, from 10.7 events per 1000 patient-days to 4.86 events [P=.01]. The preventable ADE rate was 17% lower in phase 2, although this result did not achieve statistical significance [P=.37]. The rate of nonintercepted potential ADEs fell 84% [P=.002], and the rate of intercepted potential ADEs decreased 58% between the 2 phases, although this result was not statistically significant [P=.15]. The rate of nonpreventable ADEs was unchanged at 11.3 events per 1000 patient-days in both phase 1 and phase 2. In unpaired analyses comparing phases 1 and 2 [Table 3], the results were generally similar. When the team intervention was controlled for in the analysis, the results did not change [results available on request]. Contemporaneous analyses comparing phase 2 POE plus team intervention with POE-only units showed no significant differences for any of the event types.
The results by severity show that decreases were seen across all levels of severity for nonintercepted serious medication errors [Table 4], and that the proportions of errors considered life-threatening, serious, and significant remained relatively constant across the 2 phases. However, for preventable ADEs, while the proportion that were life-threatening remained similar, the proportion of ADEs that were classified as serious was higher in phase 2 than in phase 1 [47% vs 22%], and the rate of serious preventable ADEs was about twice as high [1.96 events vs 0.98 events per 1000 patient-days]. Of these serious ADEs, 16 [33%] of 48 were due to sedatives.
When the results were evaluated by steps in the process from ordering to administering, the rate of ordering errors decreased 19% overall [P=.03, Table 5]. The number of transcription errors fell by 84% [P