Denominators : Comparing routine coverage rates and interpreting data over time
Watch on YouTubeVideo summary
The video discusses methods for validating routine health coverage estimates by comparing them against external survey data, such as the Demographic and Health Survey (DHS). While routine systems often provide national-level estimates for specific regions or districts, these figures can be cross-checked with robust methodologies derived from sample-based surveys that include health indicators. This comparison serves to build confidence in the accuracy of the information within a country's own system; if the routine data closely aligns with survey results, it suggests the internal data is reliable. The speaker notes that this validation process can be performed using various tools, including annual report applications and data visualizers with built-in validation rules.
A critical aspect of data analysis involves examining trends over time to ensure logical consistency between service delivery numbers and population denominators. For instance, when calculating coverage rates for vaccines like BCG, the number of doses administered should never exceed the number of surviving infants in the target age group. If a graph shows vaccination numbers surpassing the population count or if the calculated coverage rate exceeds 100%, it indicates a fundamental error in the data, such as an incorrect population estimate. The video illustrates this with a hypothetical example where erratic fluctuations in the target population and excessive vaccination counts create unrealistic spikes in coverage, signaling that the underlying data requires investigation before any conclusions can be drawn.
To prevent such errors, the speaker emphasizes the importance of carefully reviewing denominator selection to ensure it aligns with local definitions and standard indicator protocols. Indicators are sometimes modified slightly based on specific national procedures, so relying solely on default settings without verification can lead to inaccuracies. Using resources like a data dictionary is highly recommended to clarify how specific indicators should be defined within a particular context. By regularly checking that the chosen denominator matches the operational definition of the indicator, health officials can maintain data integrity and avoid misinterpretations caused by mismatched metrics or flawed population estimates.
Read the full video transcript
Like Okay? Um we can also compare our
routine coverage estimates with survey
data. Okay? So, this is an example for
ANC 1. This is the annual report
application. We're going to show you how
to utilize this, but you could also do
this uh just in any other tool,
validation rules in data visualizer,
etc. Um with the same data.
Okay? And we'll show some examples of
that, I believe, later on. Okay? So,
what this is doing is uh in this case, I
have an ANC 1 coverage value for let's
say for a year, okay?
And I'm comparing it with uh data that's
been calculated using a in this case,
DHS stands for Demographic and Health
Survey. There's different surveys, of
course, that you can implement over time
um that might have health indicators on
them. You could have um other related
nutrition surveys or or other types of
demographic surveys, um
you know, whatever it might be, right?
And you could take the data from that
survey, which obviously they're using a
different method. Often, they're using a
sample uh of people, right? They're not
You have a
typically in your systems, you know,
this is a national level figure of some
kind where you're trying to get an
estimate for your overall country as
well as other regions, districts,
geography within your country as well.
Okay? Um but they wouldn't have that,
but there would be a very kind of robust
methodology typically behind these
surveys as well.
Okay? So, it's often good to compare,
you know, if your routine data is pretty
close to the survey data, it can once
again give you some confidence um about
the the information that you have um in
your own system.
Okay?
We can also look at comparing data over
time, right? Um
we can basically map our uh
denominator
against a service, and we can see for is
that service eclipsing our denominator,
right? So, I'm using coverage rate in my
example, okay? And what I mean here is
uh I provide a number of doses, okay?
But, I have a denominator, in this case,
infants survival, uh survival less than
one, okay? And uh I have BCG doses
given, and I want to make sure that my
BCG doses given isn't more than my
surviving infants, right? If it is, that
means that my number of surviving
infants is not correct, right? If I
overlay one more item here, the
percentage,
if the percentage eclipses 100%, it
tells us the same thing, right? That my
denominator is not correct, right? Cuz
it doesn't make sense to immunize more
people than people actually exist,
okay? So, that's the whole idea behind
this is that you should you can also
look at data over time. And here's an
example,
it's a textbook example. It's it's not a
one that I have access to in real life,
but just to show you
um what it can look like when the data
gets very erratic, okay? Um and it
starts to not make sense, right? Um
you have uh this gray line, which is
your coverage rate, okay? This black
line, which is your population, and
these white dot white points, these uh
empty points here, that reference the
number of doses given, okay?
Now, what we see here is, you know, the
vaccination kind of trend is kind of
going up, and it seems okay. But, then
here we have uh numerous problems,
right?
The target population drops all of a
sudden, okay? And then it increases
drastically in the next year,
okay? The number of vaccinations is more
than our target population, and as a
result, our coverage spikes, and this is
the the axis on the right here
and it's over 100% okay? So there's a
couple different problems starting it
looks like 2007-ish
um
or 2006 okay? And then this spreads to
further years unfortunately.
Um so you know we would have to
investigate this further before we could
use
um any of this data to really make any
assumption about our vaccination
coverage.
Okay? Um and the last point here is to
check your uh
your denominator selection. Okay?
Um so just check that the indicators are
actually using the correct denominator
according to your definition your local
definition of that indicator right? So
this is where something like a data
dictionary will become useful to see how
you define that indicator. So let's say
for example here
in this example like BCG coverage okay?
BCG coverage it should be using um
um survival under one as the
denominator. Sometimes we see a
different denominator used you might
want to check that. Okay? And whether or
not that's correct it's once again
dependent on your local definition.
Okay? As a lot of times these indicators
are modified slightly um based on your
own procedures.