[6] Unadjusted, along some dimensions there are huge
discrepancies in the results, for example: social trust ("do
you think most others can be trusted or you can't be too careful")
ranges from 36% believing that most others can be trusted
in one community to 75% in another; or the percent saying
they signed a petition in the last 12 months ranged from 17%
in one community to 60% in another.
[8] In most cases, the survey area was one county
or a cluster of contiguous counties; some of the community
samples are municipalities and others are entire states.
Most of the community surveys called for proportionate sampling,
that is, no over- or under-sampling of sub-areas or population
groups.
[9] To gauge the extent to which the results of a survey are representative, pollsters commonly use two measures: response
rate and cooperation
rate. Both are expressed as percentages. Response rate refers to the number of completed interviews relative
to the estimated number of
eligible individuals (or households) whose phone numbers
were dialed at least once. [The calculation ignores ineligible
phone numbers such as numbers that are no longer working,
fax machines, business numbers, etc.] For individuals who
were not contacted (due to devices like answering machines
and caller ID or due to non-answered phone numbers), the response
rate estimates the number of eligible respondents.
The
cooperation rate refers to the percentage of completed interviews
out of the number of eligible individuals who were
contacted. The cooperation rate is equal to 100% less the percent of eligible individuals
contacted who refused to complete an interview (refusal
rate). Because the cooperation rate does not include those
who could not be contacted, it is almost always higher than
the response rate. Both are important measures of the quality
of the data and
results, but some researchers worry more about low cooperation
rates because they fear that persons who consciously
refuse to participate
are more likely to hold different survey-relevant views
than those who do. In theory, of course, some of those who
"cannot be contacted" may also be consciously avoiding
being surveyed (through caller ID, etc.). In addition, in
theory those who are hard to contact may also hold different
views from those easier to contact.
The
AAPOR (the American Association of Public Opinion Researchers)
RR2 formula for response rates is: RR2 = I / ((I + R + NC
+ O + e(UH)), where: I = the number of completed interviews;
R = the number of refusals and terminations; NC = the number
of households where the designated respondent was not reached
(and there was no explicit refusal); O = other (health or
language barriers); UH = unknown eligibility / unknown if
household – mostly repeated busy signal or Caller ID block.
The proportion of unknowns estimated to be eligible (e)
was .25. In most samples, there was no geographic or race/ethnicity
screening, so all adults qualified (incidence = 100%).
In
the community surveys where screening occurred (as in the
national survey), incidence was less than 100% – requiring
an adjustment to make the screened and unscreened sample response
rates comparable. The adjustment consisted of multiplying
the sum of the non-response categories in the denominator
of the formula [R, NC, O, e(UH)] by the estimated incidence
and recalculating RR2. The incidence proportion was calculated
as the sum of (the completed interviews plus partial interviews
plus terminates) divided by the sum of (the completed interviews
plus partial interviews plus terminates plus the number of
households screened and determined to be ineligible).
The
Adjusted Cooperation Rate uses the same logic as the RR2 Response
Rate – only it deletes the NC, O, and e(UH) terms from the
denominator.