www.ohiokepro.com
Vol. 2 No. 5
Spring 2005
Making Sense of the Numbers:
Why Don't They Match?
Why Don't They Match?
By John Dooley, MBA, Health Data Analyst
I
f all Home Health Agency (HHA) figures come from OASIS data,
why don't they match? Why don't OBQI 3-Bar repor ts, Home
Health Compare repor ts, and vendor repor ts always say the
why don't they match? Why don't OBQI 3-Bar repor ts, Home
Health Compare repor ts, and vendor repor ts always say the
same thing? This ar ticle will address this issue. Let's begin by
looking at a par tial 3-Bar Risk-Adjusted Outcome Repor t as well as
an illustration of a Home Health Compare repor t.
looking at a par tial 3-Bar Risk-Adjusted Outcome Repor t as well as
an illustration of a Home Health Compare repor t.
OBQI 3-Bar Report
Requested Current Period: 01/2002 12/2002
Requested Prior Period: 01/2001 12/2001
Actual Current Period: 01/2002 12/2002
Actual Prior Period: 01/2001 12/2001
# Cases: Cur r 402 Prior 374
Number of Cases in Reference Sample: 2,325,615
Requested Prior Period: 01/2001 12/2001
Actual Current Period: 01/2002 12/2002
Actual Prior Period: 01/2001 12/2001
# Cases: Cur r 402 Prior 374
Number of Cases in Reference Sample: 2,325,615
All Patients' Risk-Adjusted Outcome Repor t
End Result Outcomes:
Elig. Cases
Signif.
Current
Adjusted Prior
Improvement in Grooming
63.3% (107)
56.9%
66.7%
56.9%
66.7%
Stabilization in Grooming
89.8% (317)
92.7%
92.8%
92.7%
92.8%
Improvement in Upper
Body Dressing
Body Dressing
53.7% (73)
48.8%
58.2%
48.8%
58.2%
Improvement in Lower
Body Dressing
Body Dressing
59.5% (121)
49.1%
64.6%
49.1%
64.6%
Center for Health Ser vices Research, UCHSC, Denver, CO
** The probability is 5% or less that this difference is due to chance,
and 95% or more that the difference is real.
and 95% or more that the difference is real.
The white bar represents an agency's obser ved outcome rate for
t h e c u r re n t p e r i o d .
The yellow bar represents the agency's outcome rate for the prior
period risk-adjusted to account for differences in case mix
between the 2 time periods.
t h e c u r re n t p e r i o d .
The yellow bar represents the agency's outcome rate for the prior
period risk-adjusted to account for differences in case mix
between the 2 time periods.
The blue bar represents the risk-adjusted national reference and
is based on the agency's predicted outcome rate using the risk
models, which varies from agency to agency and includes an
adjustment factor to compensate for changes in national outcome
performance and/or case mix since the time the risk model
sample was drawn.
is based on the agency's predicted outcome rate using the risk
models, which varies from agency to agency and includes an
adjustment factor to compensate for changes in national outcome
performance and/or case mix since the time the risk model
sample was drawn.
Continued on page 2
Making Sense of the Numbers... (cont. from page 1)
The graph below shows how the data are displayed on the
Home Health Compare Web site, which can be accessed through
the Medicare Web site at www.Medicare.gov.
Home Health Compare Web site, which can be accessed through
the Medicare Web site at www.Medicare.gov.
Percentage of patients who get better at getting dressed
Why is this impor tant?
Most people value being able to take care of themselves...
Why is this impor tant?
Most people value being able to take care of themselves...
THE AVERAGE FOR ALL THE
HOME HEALTH AGENCIES IN THE
UNITED STATES
HOME HEALTH AGENCIES IN THE
UNITED STATES
61%
THE AVERAGE FOR ALL HOME
HEALTH AGENCIES IN THE STATE
OF XXXXX
HEALTH AGENCIES IN THE STATE
OF XXXXX
ABC HOME HEALTH AGENCY
64%
67%
The first yellow bar represents the national
reference rate.
The second yellow bar represents the risk-
adjusted state average for the measure.
The blue bar represents the risk-adjusted
rate for the specific home health agency.
reference rate.
The second yellow bar represents the risk-
adjusted state average for the measure.
The blue bar represents the risk-adjusted
rate for the specific home health agency.
In this example, the agency's rate for improvement in upper body
dressing appears to be 6 percentage points above the national
reference. Note that data are rounded to the nearest whole percent
on Home Health Compare.
dressing appears to be 6 percentage points above the national
reference. Note that data are rounded to the nearest whole percent
on Home Health Compare.
While quality measures and outcomes are computed the same
way, the information will be presented differently for Home Health
Compare.
way, the information will be presented differently for Home Health
Compare.
The difference seen between the agency rate and national reference
on the OBQI repor t will be approximately the same as the difference
between the agency and the national rate on Home Health Compare.
In other words, the relationship between the agency and national
reference will be approximately the same on both repor ts, with
some difference due to rounding on Home Health Compare.
on the OBQI repor t will be approximately the same as the difference
between the agency and the national rate on Home Health Compare.
In other words, the relationship between the agency and national
reference will be approximately the same on both repor ts, with
some difference due to rounding on Home Health Compare.
Outcome rates are computed the same way for Home Health
Compare as they are for agency outcome repor ts. The data come
from OASIS assessments conducted at the star t or resumption of
care and at discharge or transfer. There must be a star t/resumption
of care assessment AND a discharge/transfer during the specified
12-month period in order for an episode of care to be included in
computing an outcome rate.
Compare as they are for agency outcome repor ts. The data come
from OASIS assessments conducted at the star t or resumption of
care and at discharge or transfer. There must be a star t/resumption
of care assessment AND a discharge/transfer during the specified
12-month period in order for an episode of care to be included in
computing an outcome rate.
For Home Health Compare:
The agency value is risk adjusted. It is computed using the
agency's current obser ved rate (what is currently being
repor ted on the OBQI repor t), plus a risk-adjustment factor.
The risk-adjustment factor accounts for the difference in case
mix between the agency and the national population.
The national reference is the obser ved rate for the outcome
for all patients for the specified time period. It is not risk-
adjusted, so it does not change. This was done in order for
consumers to be able to compare performance across agencies
in relation to constant national and state references.
The state reference value is computed using the state's
current obser ved rate, plus a risk-adjustment factor. The
risk-adjustment factor accounts for the difference in case
mix between the state and the national population.
The agency value is risk adjusted. It is computed using the
agency's current obser ved rate (what is currently being
repor ted on the OBQI repor t), plus a risk-adjustment factor.
The risk-adjustment factor accounts for the difference in case
mix between the agency and the national population.
The national reference is the obser ved rate for the outcome
for all patients for the specified time period. It is not risk-
adjusted, so it does not change. This was done in order for
consumers to be able to compare performance across agencies
in relation to constant national and state references.
The state reference value is computed using the state's
current obser ved rate, plus a risk-adjustment factor. The
risk-adjustment factor accounts for the difference in case
mix between the state and the national population.
In calculating the agency percentage or rate:
The numerator includes all episodes that had the outcome, e.g.,
the episodes of care that resulted in improvement in bathing or
resulted in acute care hospitalization.
The denominator includes all eligible episodes, e.g., all those who
had the "potential" to improve in bathing.
If a patient cannot improve fur ther, he or she is not included
in improvement outcomes.
If a patient cannot deteriorate fur ther he or she is not
included in the stabilization outcomes
When patients are transfer red to a facility and there is not
any assessment data to compute the outcome, then they are
not included for that par ticular outcome.
The numerator includes all episodes that had the outcome, e.g.,
the episodes of care that resulted in improvement in bathing or
resulted in acute care hospitalization.
The denominator includes all eligible episodes, e.g., all those who
had the "potential" to improve in bathing.
If a patient cannot improve fur ther, he or she is not included
in improvement outcomes.
If a patient cannot deteriorate fur ther he or she is not
included in the stabilization outcomes
When patients are transfer red to a facility and there is not
any assessment data to compute the outcome, then they are
not included for that par ticular outcome.
As previously stated, outcome rates are computed the same way
for Home Health Compare as they are for agency outcome repor ts.
However, while the same risk-adjustment models are used for
OBQI and Home Health Compare, they differ in how risk-adjustment
is applied.
for Home Health Compare as they are for agency outcome repor ts.
However, while the same risk-adjustment models are used for
OBQI and Home Health Compare, they differ in how risk-adjustment
is applied.
On the OBQI repor t:
The agency's obser ved outcome rate is not risk-adjusted.
The national reference value is based on the agency's
predicted outcome rate using the risk model. Therefore,
each agency's national reference value is different.
The agency's obser ved outcome rate is not risk-adjusted.
The national reference value is based on the agency's
predicted outcome rate using the risk model. Therefore,
each agency's national reference value is different.
Vendor repor ts may differ from OBQI and Home Health Compare
repor ts for several reasons including payer mix, risk adjustment,
and time frames.
OBQI and Home Health Compare repor t only Medicare and
Medicaid data, while vendor repor ts may include all of an
agency's payors.
Measurement Outcome (MO) tags may not necessarily coincide
with those in the OBQI/Home Health Compare repor ts.
OBQI and Home Health Compare repor ts are risk adjusted;
Vendors repor ts are not.
Vendor repor ts also are not limited to rolling 12-month periods.
repor ts for several reasons including payer mix, risk adjustment,
and time frames.
OBQI and Home Health Compare repor t only Medicare and
Medicaid data, while vendor repor ts may include all of an
agency's payors.
Measurement Outcome (MO) tags may not necessarily coincide
with those in the OBQI/Home Health Compare repor ts.
OBQI and Home Health Compare repor ts are risk adjusted;
Vendors repor ts are not.
Vendor repor ts also are not limited to rolling 12-month periods.
Hopefully, you now have a better idea of why data from OBQI, Home
Health Compare, and vendor repor ts do not always agree. If you
have any questions, please contact the Home Health Team at
1-800-385-5080; e-mail: homehealth@ohqio.sdps.org .
Health Compare, and vendor repor ts do not always agree. If you
have any questions, please contact the Home Health Team at
1-800-385-5080; e-mail: homehealth@ohqio.sdps.org .
2
CDC and NIH Respond to Influenza Vaccination Study
A
study in the Archives of Internal Medicine ("Impact of
Influenza Vaccination on Seasonal Mor tality in the US
Elderly Population" by Simonsen et al., Februar y 14, 2005
repor ts that vaccination of the elderly population against
influenza may be less effective in preventing death among the elderly
than previously assumed. This study's findings have caused some
confusion about whether people 65 years old and older should
receive an influenza vaccination.
Influenza Vaccination on Seasonal Mor tality in the US
Elderly Population" by Simonsen et al., Februar y 14, 2005
repor ts that vaccination of the elderly population against
influenza may be less effective in preventing death among the elderly
than previously assumed. This study's findings have caused some
confusion about whether people 65 years old and older should
receive an influenza vaccination.
the vaccine is not as effective in persons aged 65 and older as
we would like it to be, vaccination remains the best way to protect
this population against complications, hospitalization, and death
from influenza.
we would like it to be, vaccination remains the best way to protect
this population against complications, hospitalization, and death
from influenza.
Q: Did previous studies overstate the effectiveness of influenza
immunization at preventing deaths in the elderly?
immunization at preventing deaths in the elderly?
The Centers for Disease Control and Prevention (CDC) and the
National Institutes of Health (NIH) continue to suppor t the Advisor y
Committee on Immunization Practices' (ACIP) recommendation that
people aged 65 and older get vaccinated against influenza each
year. People aged 65 and older are at highest risk for complications,
hospitalizations, and deaths from influenza. Vaccination remains the
best protection from influenza available for people 65 and older and
their loved ones.
National Institutes of Health (NIH) continue to suppor t the Advisor y
Committee on Immunization Practices' (ACIP) recommendation that
people aged 65 and older get vaccinated against influenza each
year. People aged 65 and older are at highest risk for complications,
hospitalizations, and deaths from influenza. Vaccination remains the
best protection from influenza available for people 65 and older and
their loved ones.
Numerous studies have shown that influenza vaccination works--
including to help protect the elderly from serious illness and hospi-
talizations--but the degree to which it works varies from year to
year and can be difficult to measure. For example, influenza seasons
differ each year in length and severity, and the health status of
individuals also matters.
including to help protect the elderly from serious illness and hospi-
talizations--but the degree to which it works varies from year to
year and can be difficult to measure. For example, influenza seasons
differ each year in length and severity, and the health status of
individuals also matters.
A: Many previously published "obser vational studies" suggest a
higher level of influenza vaccine effectiveness against death in the
elderly than indicated in the Simonsen paper. The main strength of
these studies is they include information about individuals (e.g.,
whether they were vaccinated or not, and if they have chronic
medical conditions). However, all obser vational studies can be
affected by cer tain "biases," so it is possible they overestimated the
vaccine's effectiveness at preventing death in the elderly (just as it
is possible that the Simonsen study underestimates the vaccine's
effectiveness). For example, if ver y vulnerable people are less likely
to get vaccinated than the relatively healthy elderly, then this bias can
affect calculated vaccine effectiveness estimates. In this instance,
one would guess that this bias would lead to overestimates of
vaccine effectiveness for preventing deaths. Although the Simonsen
ar ticle suggests the previously published obser vational studies over-
estimate the vaccine's effectiveness, it is possible that both types of
studies might be par tly right but capture the picture incompletely.
higher level of influenza vaccine effectiveness against death in the
elderly than indicated in the Simonsen paper. The main strength of
these studies is they include information about individuals (e.g.,
whether they were vaccinated or not, and if they have chronic
medical conditions). However, all obser vational studies can be
affected by cer tain "biases," so it is possible they overestimated the
vaccine's effectiveness at preventing death in the elderly (just as it
is possible that the Simonsen study underestimates the vaccine's
effectiveness). For example, if ver y vulnerable people are less likely
to get vaccinated than the relatively healthy elderly, then this bias can
affect calculated vaccine effectiveness estimates. In this instance,
one would guess that this bias would lead to overestimates of
vaccine effectiveness for preventing deaths. Although the Simonsen
ar ticle suggests the previously published obser vational studies over-
estimate the vaccine's effectiveness, it is possible that both types of
studies might be par tly right but capture the picture incompletely.
The authors in no way imply that the elderly should not receive influenza vaccine.
Vaccine may prevent fewer deaths
In the current study by Simonsen et al., the authors in no way imply
that the elderly should not receive influenza vaccine. Rather, the
study concludes that the vaccine may prevent fewer deaths among
the elderly than previous studies would have suggested. Therefore,
the authors note that there is room for improvement in influenza
prevention effor ts, including research into developing more effective
vaccines for the elderly and the increased use of medicines to treat
flu. In addition, recently published studies raise the possibility that it
may be beneficial to vaccinate larger numbers of healthy persons,
including children, to prevent transmission of influenza viruses to
high-risk persons such as the elderly.
In the current study by Simonsen et al., the authors in no way imply
that the elderly should not receive influenza vaccine. Rather, the
study concludes that the vaccine may prevent fewer deaths among
the elderly than previous studies would have suggested. Therefore,
the authors note that there is room for improvement in influenza
prevention effor ts, including research into developing more effective
vaccines for the elderly and the increased use of medicines to treat
flu. In addition, recently published studies raise the possibility that it
may be beneficial to vaccinate larger numbers of healthy persons,
including children, to prevent transmission of influenza viruses to
high-risk persons such as the elderly.
Expansion of groups for whom influenza vaccination is recommended
is under discussion by the ACIP and CDC, and is par tly contingent
on adequate vaccine supply in the future.
is under discussion by the ACIP and CDC, and is par tly contingent
on adequate vaccine supply in the future.
Q: Do these findings mean that the vaccine does not work for
people aged 65 and over?
people aged 65 and over?
A: No, the findings do not mean the vaccine is not effective in
protecting people age 65 and over from influenza complications,
hospitalization, and death.
Most impor tantly, since there is no information on which of the
individuals who died were vaccinated or their underlying conditions,
the death and vaccination patterns identified in this study cannot
be directly linked. Apparent associations can be inferred, but may
be misleading or hard to interpret.
The authors looked only at mor tality (i.e., death), not hospitalizations
or other severe outcomes associated with influenza.
The study also lacked the statistical power to rule out a modest
reduction in influenza-related mor tality during 1980-2001 that
would correspond to the vaccine being up to half as effective in
the elderly as it is in younger people.
protecting people age 65 and over from influenza complications,
hospitalization, and death.
Most impor tantly, since there is no information on which of the
individuals who died were vaccinated or their underlying conditions,
the death and vaccination patterns identified in this study cannot
be directly linked. Apparent associations can be inferred, but may
be misleading or hard to interpret.
The authors looked only at mor tality (i.e., death), not hospitalizations
or other severe outcomes associated with influenza.
The study also lacked the statistical power to rule out a modest
reduction in influenza-related mor tality during 1980-2001 that
would correspond to the vaccine being up to half as effective in
the elderly as it is in younger people.
The CDC and ACIP continually review their influenza vaccine
recommendations as well as studies and published research in order
to develop the best recommendations for protecting all Americans
from influenza. This study is a reminder that there is room for
improvement in how we protect the elderly from influenza, and CDC
and NIH encourage research that strengthens the ability to do so.
recommendations as well as studies and published research in order
to develop the best recommendations for protecting all Americans
from influenza. This study is a reminder that there is room for
improvement in how we protect the elderly from influenza, and CDC
and NIH encourage research that strengthens the ability to do so.
The following are some key questions and answers regarding the study:
Q: How do you explain the apparent contradiction between the
results obtained by Simonsen et al. and the results obtained by
previously published studies?
results obtained by Simonsen et al. and the results obtained by
previously published studies?
Q: What is being done to better protect people aged 65 and older
from influenza?
from influenza?
A: The Depar tment of Health and Human Ser vices has recognized
that more needs to be done to protect people aged 65 and over
from influenza, and effor ts are under way to do so. Such steps
include improving vaccination coverage, moving toward "universal"
immunization as the supply allows, developing influenza vaccines
that are more effective in people aged 65 and over, and effectively
using influenza antiviral drugs.
that more needs to be done to protect people aged 65 and over
from influenza, and effor ts are under way to do so. Such steps
include improving vaccination coverage, moving toward "universal"
immunization as the supply allows, developing influenza vaccines
that are more effective in people aged 65 and over, and effectively
using influenza antiviral drugs.
A: Neither previous studies nor the Simonsen paper have all the
information needed to determine influenza vaccine effectiveness in
preventing death among people aged 65 and older. It is possible
that both types of studies (i.e., ecologic and obser vational) might be
par tly right but capture the picture incompletely. While studies indicate
information needed to determine influenza vaccine effectiveness in
preventing death among people aged 65 and older. It is possible
that both types of studies (i.e., ecologic and obser vational) might be
par tly right but capture the picture incompletely. While studies indicate
Q: Will CDC change influenza immunization recommendations for
people aged 65 and over based on the findings in this study?
people aged 65 and over based on the findings in this study?
A: No. CDC will continue to recommend vaccination of people aged
65 and over to protect them from influenza. In setting immunization
65 and over to protect them from influenza. In setting immunization
Continued on page 4
3
CDC and NIH Respond...
(cont. from page 3)
(cont. from page 3)
recommendations, CDC (in coordi-
nation with the Advisor y Committee
on Immunization Practices) considers
the body of scientific evidence
regarding a vaccine's efficacy and
safety, as well as the public health
impact of the disease the vaccine
prevents. Many previously published
"obser vational studies" suggest a
higher level of influenza vaccine
effectiveness against death in the
elderly than indicated in the Simonsen
paper. In light of the larger body of
evidence, and in light of the risks
posed to the elderly by influenza,
CDC will not change influenza
immunization recommendations
for people aged 65 and over.
nation with the Advisor y Committee
on Immunization Practices) considers
the body of scientific evidence
regarding a vaccine's efficacy and
safety, as well as the public health
impact of the disease the vaccine
prevents. Many previously published
"obser vational studies" suggest a
higher level of influenza vaccine
effectiveness against death in the
elderly than indicated in the Simonsen
paper. In light of the larger body of
evidence, and in light of the risks
posed to the elderly by influenza,
CDC will not change influenza
immunization recommendations
for people aged 65 and over.
Save the dates!
CALENDAR OF EVENTS
Regional Work Groups
OBQI Training
April 5, 2005
9:30 a.m. 12:30 p.m.
Sandusky area at
Firelands Regional Medical Center
Call to register at 1-800-385-5080
9:30 a.m. 12:30 p.m.
Sandusky area at
Firelands Regional Medical Center
Call to register at 1-800-385-5080
April 20, 2005
8:15 a.m. 4:30 p.m.
Quest Business Centers,
Columbus, OH
Register online at http://www.ohioke-
pro.com/providers/homehealth/even
ts.asp
8:15 a.m. 4:30 p.m.
Quest Business Centers,
Columbus, OH
Register online at http://www.ohioke-
pro.com/providers/homehealth/even
ts.asp
April 7, 2005
12:00 p.m. 3:00 p.m.
Cambridge area at
Southeast Regional Medical Center
Register online at http://www.ohioke-
pro.com/providers/homehealth/even
ts.asp
12:00 p.m. 3:00 p.m.
Cambridge area at
Southeast Regional Medical Center
Register online at http://www.ohioke-
pro.com/providers/homehealth/even
ts.asp
For fur ther information on the above
events, please contact the Home
Health Team at 1-800-385-5080;
e-mail: homehealth@ohqio.sdps.org
events, please contact the Home
Health Team at 1-800-385-5080;
e-mail: homehealth@ohqio.sdps.org
Ohio KePRO Home Health Team
Executive Editor:
Suzana C. Iveljic, MBA
Suzana C. Iveljic, MBA
Kerri Gilligan, RN
Donna Maynard
Rosalie McGinnis, RN, MS
Ramona Pennell, RN, BA
Chris Titus, MCSE, CNA, MCP
Donna Maynard
Rosalie McGinnis, RN, MS
Ramona Pennell, RN, BA
Chris Titus, MCSE, CNA, MCP
Alice Stollenwerk Petrulis, MD, FACP
Betty Pilous, RN, MHSA, CPHQ
Linda Day, RN, BSN, MBA
Cindy O'Connell, RN, BSN
Eileen Wallenhorst, RN, BSPA
John Dooley, MBA
Betty Pilous, RN, MHSA, CPHQ
Linda Day, RN, BSN, MBA
Cindy O'Connell, RN, BSN
Eileen Wallenhorst, RN, BSPA
John Dooley, MBA
Editor:
Robert A. Feigenbaum, MS
Robert A. Feigenbaum, MS
Please note: Ohio KePRO's
office will be closed on
May 30, 2005, in obser vance
of Memorial Day.
office will be closed on
May 30, 2005, in obser vance
of Memorial Day.
Publication No. 4011-OH-006-3/2005. This material was prepared by Ohio KePRO, the Medicare Quality Improvement
Organization for Ohio, under contract with the Centers for Medicare & Medicaid Ser vices (CMS), an agency of the
U.S. Depar tment of Health and Human Ser vices. The contents presented do not necessarily reflect CMS policy. For
more information, please call 1-800-MEDICARE (1-800-633-4227), or visit CMS's Web site at www.medicare.gov.
Organization for Ohio, under contract with the Centers for Medicare & Medicaid Ser vices (CMS), an agency of the
U.S. Depar tment of Health and Human Ser vices. The contents presented do not necessarily reflect CMS policy. For
more information, please call 1-800-MEDICARE (1-800-633-4227), or visit CMS's Web site at www.medicare.gov.
THE SPRING 2005 ISSUE OF QUALITY OUTCOMES HAS ARRIVED!
THE NEWSLETTER OF THE HOME HEALTH OUTCOME-BASED QUALITY IMPROVEMENT INITIATIVE
THE NEWSLETTER OF THE HOME HEALTH OUTCOME-BASED QUALITY IMPROVEMENT INITIATIVE
Ohio KePRO Provider QIC Line--Your Quality Improvement Connection: 1-800-385-5080 · Medicare Help Line: 1-800-MEDICARE (1-800-633-4227)
Home Health Team e-mail: homehealth@ohqio.sdps.org
Home Health Team e-mail: homehealth@ohqio.sdps.org