Towards a new agenda for research on minority bias
Insights gained about the issues already investigated also allow us to clarify
which issues might be given priority next. Ideally, each of the ten initial
recommendations might be transformed into a testable research hypothesis, and
could hence inspire its own piece of evaluation research. Randomised experiments
should provide more definite causal evidence, in particular about the impact of
factors like linguistic arrangements, contact strategies, composition of field teams, or
interviewer payment schemes on the representation of minorities in general surveys.
In the Swiss context, the recent introduction of a full population register opens
important new perspectives for such research, and invites us to take advantage of
register information available on non-respondents, and to describe them in more
detail than possible so far. But there is no single royal avenue to grasp the complex
issue of minority bias. At least three complementary lines of research can be
identified, each requiring a different methodological approach.
First, correlational studies on the relation between different types of survey
procedures and minority bias should be extended to a more comprehensive approach
to compare cumulative data quality across existing surveys. To overcome the rather
artificial distinction between survey non-response, partial response, or arbitrary
responses, it appears wise to look not only at whether minority respondents answer
survey questions, but also at how they answer them. For example, compulsory
surveys or very insistent recruitment procedures could result in pushing minority
respondents into strong “satisficing” modes of survey participation, especially if they
are not accompanied by simultaneous measures to make the survey accessible and
relevant for minorities. It is therefore important to develop indicators of meaningful
survey participation, rather than just formal survey participation.
Second, in a more qualitative line, ethnographic approaches to interviewer
experiences and interviewer-respondent interactions should provide a more finegrained
understanding of the micro-processes by which certain types of respondents
are excluded from survey participation, on the basis of reciprocal expectations,
perceptions, and communicative practices.
Third, simulation studies should provide a more detailed picture of the actual
consequences of minority bias (and hence of different survey arrangements that
produce or reduce such bias) on the accuracy of statistical indicators or models based
on the corresponding survey data. These estimates are particularly needed because
they would locate the debate on the relative cost of different survey options within a
more realistic framework. Rather than wondering how much it costs to get any
indicator of poverty, inequality, vulnerability, and so on, such evidence would put us
in a position to ask how much it costs to get an accurate and precise enough such
indicator.
Against this backdrop, we would anticipate that opening the black box and
engaging with some of the strategies outlined here to improve minority participation
in general social surveys will ultimately not only be cost- but also a gain-factor, even
from a simple “economic” point of view. Hopefully, the ideas and findings presented
in the working group’s first publications will encourage more survey researchers to
engage with the agenda that we have outlined here, enrich it, and push further the
difficult but necessary debate on minorities in general social surveys.
References
Deding M., Fridberg T. and Jakobsen V. (2008). Non-response in a survey
among immigrants in Denmark. Survey Research Methods 2 (3), pp. 107-121.
Feskens R., Hox J. Lensvelt-Mulders G. and Schmeets H. (2006). Collecting
Data among Ethnic Minorities in an International Perspective, Field Methods 18 (3),
pp. 284-304.
Feskens R., Hox J., Lensvelt-Mulders, G. and Schmeets, H. (2007). Nonresponse
among Ethnic Minorities: A Multivariate Analysis. Journal of Official
Statistics, 23(3), 387-408.
Groves, R., M. (2006). Nonreponse rates and nonresponse bias in household
surveys. Public Opinion Quaterly, 70 (5), 646-675.
Laganà, F., Elcheroth, G., Penic, S., Kleiner, B. & Falsel, N. (2011). National
minorities and their representation in Swiss surveys (II): Which practices make a
difference? FORS Working Paper Series, paper 2011-3. Lausanne: FORS
(forthcoming in Quality and Quantity).
Lipps, O., Laganà, F., Pollien, A., & Gianettoni, L. (2011). National minorities
and their representation in Swiss surveys (I): Providing evidence and analysing
causes for their under-representation. FORS Working Paper Series, paper 2011-2.
Lausanne: FORS (forthcoming in M. Méndez Lago & J. Font (Eds.), Surveying ethnic
minorities and immigrant populations: methodological challenges and research
strategies. Amsterdam University Press).
Peytchev, A., Baxter, R. and Carley-Baxter, L. (2009). Not all survey effort is
equal. Reduction of nonresponse bias and nonresponse error. Public Opinion
Quaterly, 73 (4), 785-806.
