A COMPUTATIONAL ANALYSIS OF DIFFERENCES IN THERAPY
BETWEEN BENCHMARK AND NON-BENCHMARK HOSPITALS
FOR PATIENTS WITH ACUTE MYOCARDIAL INFARCTIONS
Raphael Bahati
1
, Michael Bauer
2
and Femida Gwadry-Sridhar
1
1
I-THINK Research, Lawson, 801 Commissioners Rd., Suite B3041, London, ON, Canada
2
Department of Computer Science, The University of Western Ontario, London, ON, Canada
Keywords:
Acute myocardial infarction, Predicting health outcomes, Mathematical modeling, Cluster analysis.
Abstract:
Acute Myocardial Infarction (AMI) remains a leading cause of mortality in most industrialized nations. Mor-
tality rates for AMI patients are often used as a measure of the overall effectiveness of care provided by
hospitals. Age, gender, and severity adjusted, the mortality rates within Canada have been shown to vary sig-
nificantly from province to province. Some studies, for example, have shown significant variations between
counties, even when adjacent to each other. In this paper, we present an approach aimed at understanding the
causes of this variability by investigating the extent to which evidence-based therapies and processes within
hospitals might be affecting mortality rates. We use cluster analysis to identify beneficial therapies and pro-
cesses responsible for the improvement in treatment outcomes (as measured in terms of standardized mortality
ratio) in benchmark compared to non-benchmark hospitals.
1 INTRODUCTION
Acute Myocardial Infarction (AMI), commonly
known as a heart attack, is caused by a sudden depri-
vation of blood circulation to parts of the heart mainly
as a result of a blockage of coronary artery. The short-
age of oxygen often causes permanent myocardial
(heart) tissue damage or death. In 1996, 38,000 my-
ocardial infarctions were reported in Canada, the ma-
jority due to coronary artery disease (HSFC, 2010).
Approximately 15% of AMI sufferers died and 23%
were readmitted within the first year post AMI
(CHSSS, 2000). Consequences of coronary artery
disease include morbidity such as angina, conges-
tive heart failure and arrhythmias. These disease re-
lated morbidities often lead to significant disability
and economic impact from diminished productivity
and ongoing health care costs.
The mortality rates for AMI have been shown
to vary significantly from province to province and
within provinces from county to county. The Institute
for Clinical Evaluative Sciences published a Cardio-
vascular Atlas, which compiled the rate of mortality
from AMI by district or county in Ontario, Canada
(Basinski et al., 1999). It showed significant variance
between counties even when adjacent to each other.
In southwestern Ontario, the rate of age and sex ad-
justed cardiovascular mortality per 100,000 popula-
tion greater than age 20 ranged from 368.7 (Middle-
sex County) to 526.5 (Kent County). Another study
by the Canadian Cardiovascular Outcomes Research
Team (CCORT) published a nationwide perspective
of AMI mortality rates from seven provinces, adjust-
ing for age and sex. The adjusted mortality rates
ranged from a low of 10.2% in Alberta to a high of
13% in Saskatchewan (Tu et al., 2003). What this
demonstrates is that age or sex differences did not ac-
count for the variations in the rate of AMI mortal-
ity. Furthermore, the variability in mortality could be
seen when examined at the level of individual hospi-
tals. Severity adjusted, (expected) mortality ranged
between 10 and 17% in a representative group of hos-
pitals in Ontario, Alberta and Newfoundland, but ob-
served mortality had nearly four times as much vari-
ation (between 3.5 and 12.5%). Thus factors such as
age, co-morbid disease and severity of presentation
did not entirely account for the differences in treat-
ment outcomes.
This begs the question as to why we see signif-
icant variability in outcome indicators such as AMI
in-hospital mortality and ICU mortality. AMI is di-
vided into 2 major categories based on electrocar-
254
Bahati R., Bauer M. and Gwadry-Sridhar F..
A COMPUTATIONAL ANALYSIS OF DIFFERENCES IN THERAPY BETWEEN BENCHMARK AND NON-BENCHMARK HOSPITALS FOR PATIENTS
WITH ACUTE MYOCARDIAL INFARCTIONS .
DOI: 10.5220/0003157502540259
In Proceedings of the International Conference on Health Informatics (HEALTHINF-2011), pages 254-259
ISBN: 978-989-8425-34-8
Copyright
c
2011 SCITEPRESS (Science and Technology Publications, Lda.)
diogram (ECG) diagnostic test for detecting heart-
muscle damage: ST elevation and non-ST elevation
myocardial infarction (MI). ST elevation MI is caused
by an acute thrombotic occlusion of a major coronary
vessel while Non-ST elevation MI is often associated
with partial closure of an epicardial vessel or with dif-
fuse coronary artery disease. Effective therapies exist
for both categories of AMI. One possible explanation
for the variability in treatment outcomes is the extent
to which different kinds of therapies available at each
hospital might be affecting mortality rates. Specific
treatment issues may be reflected in the percentage
of patients who receive medications or interventions
known to improve the chance of survival.
In this paper, we investigate the impact of the dif-
ferent kinds of therapies on the mortality rates by
identifying beneficial therapies and processes respon-
sible for improving treatment outcomes in benchmark
versus non-benchmark hospitals. Our focus is on ST
elevation MI and the specific evidence-based thera-
pies to achieve rapid reperfusion of the occluded ves-
sel. The rest of this paper is organized as follows.
We begin in Section 2 with an overview of the data
source used in the analysis, describing patient com-
position within ICUs involved as well as the bench-
mark methodology used to distinguish benchmark
from non-benchmark hospitals. We then describe our
analytical approach for identifying beneficial thera-
pies and processes responsible for improving treat-
ment outcomes in benchmark versus non-benchmark
hospitals in Section 3. We conclude with a summary
of the implications of our study and describe possible
directions for future work in Section 4.
2 DATA SOURCE
The Critical Care Research Network is a network of
ICUs within Ontario established to conduct evidence-
based research within both teaching and commu-
nity hospitals and to facilitate research transfer to
the decision-makers within these settings. The Net-
work has been collecting a Minimum Data Set (MDS)
since January 1995. The MDS currently contains
over 125,000 records from 45 hospitals from across
Canada. The dataset contains hospital and ICU ad-
mission and discharge dates, hospital outcome, ICU
admitting diagnosis, and physiologic data for calcu-
lating an illness severity score on the day of ICU ad-
mission. Every admission to the ICU is recorded.
Acute myocardial infarction is one of the specific di-
agnoses captured in the dataset. Sites collect data on
all ICU admissions with > 90% of records contain-
ing complete data. Strengths of the database include
the APACHE (Acute Physiology And Chronic Health
Evaluation) II score, collected as part of the MDS,
which has been validated as an index of severity and
can be used to adjust for illness severity when com-
paring outcomes between coronary care units. This is
the most widely used method worldwide for risk ad-
justment of ICU patients. Also, the diagnosis has to
be determined during the first 24 hours of ICU admis-
sion and the patient location prior to ICU admission
is recorded. Thus, patients with AMI as a secondary
diagnosis (e.g. post-operative) can be excluded.
2.1 Site and Patient Selection
Sites were included in the analysis if they were a com-
munity hospital (since most teaching hospitals have
separate coronary care units) and at least 10 cases per
year were recorded in the database. Although only
ICUs were included in this study, this represented the
majority of community practice, since only 8 of 28
Critical Care Research Network (CCR-Net) commu-
nity hospitals reported a coronary care unit separate
from the main intensive care unit in our most recent
survey. Patients were included in the analysis if they
had a diagnosis of acute myocardial infarction, were
admitted directly from the emergency department to
the ICU, and were at least 16 years old. The com-
position of ST elevated MI patients within hospitals
and the corresponding demographics are summarized
in Table 1.
2.2 Benchmark Methodology
Objective methodology to identify best practice has
been described in (Weissman et al., 1999) and used in
randomized controlled trials for quality improvement
(Kiefe et al., 2001). This methodology was imple-
mented using risk-adjusted mortality. Thus, the pre-
dicted risk of death is calculated for each patient using
the APACHE II risk prediction model (Knaus et al.,
1985). The average predicted risk of death is then
determined for each ICU and compared to the actual
mortality rate as a ratio (SMR, standardized mortal-
ity ratio), with an adjustment for small sample sizes
by adding 1 to the numerator and denominator. Sites
were then ranked in order of the SMR. Starting with
the highest ranked site, sites were added to the bench-
mark group until at least 10% of the total patient pool
was included. A pooled SMR was generated for the
overall benchmark group of patients. Confidence in-
tervals were then generated according to the method
of Hosmer and Lemeshaw (Hosmer and Lemeshow,
1989) and used to group ICUs into benchmark versus
non-benchmark hospitals.
A COMPUTATIONAL ANALYSIS OF DIFFERENCES IN THERAPY BETWEEN BENCHMARK AND
NON-BENCHMARK HOSPITALS FOR PATIENTS WITH ACUTE MYOCARDIAL INFARCTIONS
255
Table 1: Demographics of ST elevated MI patients. HLOS = Hospital Length of Stay, Values: mean (standard deviation).
Benchmark Hospitals Non Benchmark Hospitals
Hospital ID 1629 3411 4067 1754 1768 1853 3587
Patients 25 25 22 25 25 25 25
Gender[F/M] 5/20 8/17 4/18 4/21 9/16 10/15 6/19
Age
54.52 60.56 55.05 57.32 67.24 60.20 59.48
(10.32) (14.02) (10.96) (11.17) (10.23) (10.21) (10.53)
HLOS 4.08 (1.66) 5.16 (2.87) 7.36 (4.47) 5.04 (3.92) 7.28 (3.66) 4.64 (2.78) 3.88 (1.86)
Drugs 6.36 (1.08) 5.88 (1.79) 5.91 (1.27) 5.68 (0.99) 5.76 (1.51) 6.60 (1.53) 5.68 (1.49)
APACHE II 7.91 (1.66) 9.88 (3.27) 8.62 (4.73) 8.92 (3.26) 8.56 (2.68) 8.28 (3.06) 7.88 (3.38)
3 ANALYSIS
In this section, we describe the approach taken in un-
derstanding the variability in health outcomes of pa-
tients with AMI as a result of being treated at either
benchmark or non-benchmark hospitals. In particular,
we examine how Evidence Based Medicine (EBM)
therapies and processes affected treatment outcomes
as measured in terms of the standardized mortality
ratio. The aim is to identify beneficial therapies by
comparing treatment outcomes in benchmark versus
non-benchmark hospitals.
3.1 Approach
Our analytical approach involved two key steps. We
used the analysis of variance (ANOVA) to test the ef-
fects of the different factors (i.e., therapies and tests)
on the outcome measure (i.e., standardized mortal-
ity ratio). We used ANOVA as a starting point for
identifying therapies of interest (Section 3.2), whose
results were then fed onto a k-means clustering al-
gorithm (Kiefe et al., 2001) for partitioning patients
data (Section 3.3). Thus, instead of building clusters
using all therapies and tests in Tables 2, 3, and 4 (25,
in this case), we only included those that significantly
distinguished benchmark from non-benchmark hospi-
tals. This allowed us to eliminate irrelevant therapies
and tests resulting in greater cluster stability.
The use of clustering in our analysis served two
purposes. First, it allowed us to verify and test the
accuracy of the identified therapies in classifying pa-
tients as having been treated at either a benchmark or
a non-benchmark hospital. Second, it provided a nat-
ural way of partitioning patients into groups based on
treatment characteristics. Consequently, we were then
able to compare characteristics of individual clusters
to determine which therapies and processes were ben-
eficial and which were not.
3.2 Analysis of Variance
A multi-factor analysis of variance (ANOVA) was
used to test the effects of different therapies and tests
on the treatment outcome as determined by whether
a patient was treated at a benchmark versus a non-
benchmark hospital (see Section 2.2). Each factor
(therapy) consisted of two levels (denoted by 1 and 0)
indicating whether or not a particular treatment was
administered to a patient during a hospital visit. From
this analysis, a number of therapies and tests emerged
as significant (at p < 0.05) and are highlighted in Ta-
bles 2, 3, and 4. They include one pharmacologic
therapy (A), four non-pharmacologic therapies (B, C,
D, and E), and five diagnostic tests (F, G, H, I, and J).
Table 2: Pharmacologic Therapies.
Therapy P
ASA 0.36
Reperfusion 0.38
Thrombolytics 0.14
Anti-thrombotic 0.17
Statin 0.41
Beta blocker 0.98
A ACE-Inhibitor 0.04
Statin preprint 0.73
3.3 Cluster Analysis
Having identified the therapies of interest, we then
used cluster analysis to group patients such that each
group (or cluster) consisted of patients that underwent
a combination of treatments with similar character-
istics. In particular, a k-means clustering algorithm
was used to partition patients based on which ther-
apies, from among the therapies identified as signif-
icant in the previous section, they received while in
hospital. Thus, given a set of patients (see, for exam-
ple, Table 5), k-means clustering partitioned the pa-
tients into k groups such that each patient belonged to
HEALTHINF 2011 - International Conference on Health Informatics
256
Table 3: Non Pharmacologic Therapies.
Therapy P
Patients monitored for 48-72 hours 0.40
Patients reassessed at 48-72 hours to remove or continue monitoring 0.34
Patients reassessed with trained personnel observing 0.47
Protocol for early mobilization -
Staged or formal mobilization of patient -
Pharmacist input documented 0.43
B Pharmacist participation in rounds <0.01
C Disease education documented <0.01
D Education material made available for patients and families 0.03
E Discharge planning documented 0.04
Discharge formalized with planning 0.86
Table 4: Key Diagnostic Tests.
Diagnostic Test P
Chest pain unit or rapid triage for chest pain 0.38
F Assessment of infarction size 0.05
G Assessment of residual ischemia pre-discharge <0.01
H Referral made for cardiac catheterization <0.01
I Cardiac Care Network forms in chart <0.01
J Formal guidelines for decision making regarding cardiac catheterization <0.01
a cluster with similar treatments characteristics. Each
patient’s information, in this case, consisted of a set
of binary numbers each corresponding to a therapy in
{A,B,. . . ,J} indicating whether (1) or not (0) a partic-
ular therapy was administered.
Table 5: Sample ST elevated MI patients data.
Treatments and Tests
Patient A B . . . I J
1 1 0 . . . 0 0
2 1 1 . . . 1 1
.
.
.
n 0 0 . . . 0 1
The k value for the algorithm, which determines
the number of groups (or clusters) to be created, was
chosen as follows. We experimented with several
cluster configurations (where 2 k 20) such that
k resulted in the most accurate classification of pa-
tients. Accuracy, in this case, described the patient
composition of each cluster based on whether they
were treated at a benchmark or a non-benchmark hos-
pital. Thus, an accuracy measure of, say, 0.8 meant
that, for every cluster formed, at least 80% of its pa-
tients were treated at the same kind of hospital. As
such, the aim was to group patients into the smallest
possible number of clusters while ensuring the most
accurate grouping of patients based on whether they
were treated a benchmark versus a non-benchmark
hospital. In this approach, five clusters were formed
as shown in Table 6 with cluster 2 being the least ac-
curate (at 73.91%) and clusters 4 and 5 being the most
accurate (at 100%).
Table 6: Clusters Prediction.
% Patients Composition
Cluster Benchmark Non-Benchmark
1 75.51 24.49
2 26.09 73.91
3 92 8
4 0 100
5 0 100
Of particular interest, clusters 4 and 5 (highlighted
in Table 6) had a 100% accuracy in classifying pa-
tients as having been treated at non-benchmark hospi-
tals. Looking more closely at the patient composition
within the clusters reveals some interesting character-
istics as Table 7 shows. For example, of the 25 pa-
tients in cluster 3, 92% were treated at Hospital 1629
while 8% were treated at Hospital 1768. Note also
that, all the patients in cluster 4 were treated at Hospi-
tal 3587, while all the patients in Hospital 1754 ended
up in cluster 5. This suggests, to some extent, a cor-
relation between therapies and hospitals.
Table 8 shows the percentage of patients that un-
A COMPUTATIONAL ANALYSIS OF DIFFERENCES IN THERAPY BETWEEN BENCHMARK AND
NON-BENCHMARK HOSPITALS FOR PATIENTS WITH ACUTE MYOCARDIAL INFARCTIONS
257
Table 7: Percentage patient composition within clusters.
Benchmark Hospitals Non Benchmark Hospitals
Cluster 1629 3411 4067 1754 1768 1853 3587 # Patients
1 0 36.73 38.78 0 22.45 0 2.04 49
2 4.35 15.22 6.52 0 26.09 47.83 0 46
3 92 0 0 0 8 0 0 25
4 0 0 0 0 0 0 100 24
5 0 0 0 89.29 0 10.71 0 28
# Patients 25 25 22 25 25 25 25
Table 8: Percentage of patients that underwent specific therapies and tests within clusters.
Pharm. Non-Pharmacologic Diagnostic Tests
Cluster A B C D E F G H I J Accuracy
1 63.27 2.04 46.94 100 89.8 93.88 55.1 44.9 4.08 100 0.76
3 48 0 60 36 52 92 76 100 100 8 0.92
2 56.52 0 8.70 19.57 6.52 80.43 17.39 93.48 78.26 95.65 0.74
4 29.17 100 29.17 20.83 91.67 91.67 95.83 95.83 70.83 100 1
5 67.86 89.29 0 0 3.57 100 39.29 57.14 39.29 0 1
Figure 1: Comparing therapies and tests between benchmark and non-benchmark clusters.
derwent specific therapy treatments and tests within
each cluster. The clusters are grouped into benchmark
(clusters 1 and 3) versus non-benchmark (clusters 2,
4, and 5) based on the patient composition within in-
dividual clusters (see Table 6). For example, cluster 1
is considered a benchmark cluster since 75.51% of its
patients were treated at benchmark hospitals whereas
cluster 2 is considered a non-benchmark cluster. In
order to compare individual therapies and tests be-
tween benchmark and non-benchmark clusters, we
computed the average percentage of patients that un-
derwent each specific treatment in {A,B,. . . ,J} as
shown in Figure 1.
Since the aim was to identify which therapies
were beneficial in terms of the improvement in treat-
ment outcomes (assuming, of course, that therapies
and tests did not contribute to the worsening of treat-
ment outcome as measured in terms of standardized
mortality ratio), several conclusions can be drawn
from the results. On the one hand, therapy B, which
corresponded to pharmacists participation in rounds
(see Table 3), does not appear to be beneficial as it
was mostly administered to patients treated at non-
benchmark hospitals, which performed worse based
on standardized mortality rates. On the other hand,
we see much larger differences in the percentage of
patients that underwent therapies C, D, and E in fa-
vor of benchmark hospitals. This suggests that such
therapies and tests may have contributed to the im-
provement in treatments outcomes.
HEALTHINF 2011 - International Conference on Health Informatics
258
A t-test comparison of the percentages of patients
that underwent specific therapies between benchmark
and non-benchmark clusters at a 95% confidence
shows that non-pharmacologic therapy C was the only
significant predictor of the variations in health out-
comes (p = 0.03). Worth pointing out, however, is
the fact that not all benchmark hospitals (or non-
benchmark hospitals for that matter) offered the same
kinds of therapies. As such, it could be a combination
of different treatments that may have been responsible
for the overall improvement in treatment outcomes.
Cluster analysis is only the first step in helping us
identify some of these characteristics.
4 CONCLUSIONS
The variability in mortality rates of AMI patients be-
tween hospitals is due to the differences in the kinds
of therapies and tests administered to patients at dif-
ferent hospital locations. In this paper, we have pre-
sented an approach for identifying beneficial thera-
pies and processes responsible for improving treat-
ment outcomes of ST elevation MI patients. To
achieve this, we first used the analysis of variance
(ANOVA) to test the effects of the different therapies
on the outcome measures. This allowed us to iden-
tify therapies of interest, which we then used on a k-
means clustering algorithm to group patients. In this
approach, patients belonging to the same cluster un-
derwent similar therapy treatments while in hospitals
with similar outcomes. Consequently, we were able
to compare treatment characteristics within clusters to
determine which therapies were beneficial based on
the differences in the percentage of patients treated at
benchmark versus non-benchmark clusters.
Several conclusions could be drawn from the anal-
ysis presented in this paper. First, therapy B (phar-
macists participation in rounds) does not appear to
have any benefit in the improvement of treatment
outcomes as it was mostly administered to patients
treated at non-benchmark hospitals, which performed
worse based on the standardized mortality ratio. Sec-
ond, therapy C (disease education documentation)
was the only significant predictor in the overall im-
provement of treatment outcomes in benchmark ver-
sus non-benchmark hospitals.
In our current approach, we have utilized the
APACHE II risk prediction model to determine the
standardized mortality ratio, which we computed by
comparing the predicted risk of death as determined
by the APACHE II score to the actual hospitals’ mor-
tality rates. Future work includes the incorporation
of other outcome indicators such as hospital length of
stay within the cluster analysis. An interesting ques-
tion, which deserves further investigation, is whether
or not patients who leave hospital earlier tend to have
a worse clinical outcome compared to those who stay
in hospital longer. For example, is it possible that hos-
pital length of stay might indirectly be impacting the
variations in mortality rates? This is the focus of our
future investigation.
ACKNOWLEDGEMENTS
This research was funded by the Canadian Institute
of Health Research (CIHR) and the Critical Care Re-
search Network (CCR-Net).
REFERENCES
Basinski, A. S., Naylor, C. D., and Slaughter, P. (1999).
Hospitalization for Cardiovascular Medical Diag-
noses, Cardiovascular Health and Services In Ontario.
In Institute for Clinical Evaluative Services and Heart
and Stroke Foundation (Ontario), Toronto, Canada.
ICES Atlas.
CHSSS (2000). The Changing Face of Heart Disease and
Stroke in Canada. In Canadian Heart and Stroke
Surveillance System, Ottawa, ON, Canada. Health
Canada.
Hosmer, D. W. and Lemeshow, S. (1989). Applied Logistic
Regression. John Wiley & Sons, Inc., New York, NY,
USA.
HSFC (2010). Heart & Stroke Foundation of Canada.
http://www.hsf.ca/research/en/general/home.html.
Kiefe, C. I., Allison, J. J., and Williams, O. D. (2001). Im-
proving quality improvement using achievable bench-
marks for physician feedback: a randomized con-
trolled trial. In JAMA, volume 285, pages 2871–2879,
Ottawa, ON, Canada.
Knaus, W. A. et al. (1985). APACHE II: a severity of dis-
ease classification system. In Critical Care Medicine,
volume 13, pages 818–829.
Tu, J. V. et al. (2003). Outcomes of Acute Myocardial In-
farction in Canada. In Canadian Journal of Cardiol-
ogy, volume 19, pages 893–901.
Weissman, N. W. et al. (1999). Achievable benchmarks
of care: the ABCs of benchmarking. In J Eval Clin
Pract, volume 5, pages 269–281. Health Canada.
A COMPUTATIONAL ANALYSIS OF DIFFERENCES IN THERAPY BETWEEN BENCHMARK AND
NON-BENCHMARK HOSPITALS FOR PATIENTS WITH ACUTE MYOCARDIAL INFARCTIONS
259