Starting point
Since early 2009, a working group in Lausanne has investigated and reflected
on questions regarding representation of national minorities in Swiss surveys.
Composed of social scientists from the then newly established Swiss Centre of
Expertise in the Social Sciences (FORS) and the University of Lausanne’s Research
Centre on Methodology, Inequality and Social Change (MISC), the creation of this
working group was a direct consequence of the new institutional opportunity and
expectation that FORS and its host University should work hand in hand to improve
the quality of nationwide Swiss social surveys. Combining data producer and data
user perspectives, the working group joined a methodological interest in survey
processes with a substantive interest in vulnerable populations and social exclusion.
Combining these interests and approaches we soon arrived at the initial conclusion
that the inclusion/exclusion of minority groups in/from general social surveys might
be one of the most challenging and under-studied issues in contemporary survey
research. Further, to make a concrete contribution to opening this persistent black
box of survey research, the group chose to focus first on one particular type of
minority: foreigners in Switzerland. Strongly correlated (in Switzerland as elsewhere)
with manifold other markers of potential minority status, such as class position,
socio-cultural capital, language, and ethnic identity, the identity inscribed in a
person’s passport thus became our empirical entry into a neglected and sometimes
disconcerting facet of survey research.
Two empirical papers are now available, which describe in detail the
theoretical frameworks and empirical methods used, as well as the findings obtained
by the group (Lipps, Lagana, Pollien & Gianettoni, 2011; Lagana, Elcheroth, Penic,
Kleiner & Fasel, 2011). Rather than repeating these here, the present position paper
pursues two goals: 1) to propose an integrated summary of the main empirical
conclusions from both papers for the busy reader, and 2) to extrapolate, beyond the
strict descriptive results of our analyses, to the more prescriptive outcomes of our
reflections. We put forward a series of concrete recommendations for interested
survey researchers regarding practices that appear, to our eyes, to be the most
promising in dealing with the problem of minority bias in representative survey
research.
These two papers are only meant to be a starting point, and should ideally
encourage and stimulate further contributions to the much wider issue of minorities,
broadly defined, in social surveys. We should therefore first say a few words about
why this is an important – and possibly critical - issue for the future of surveys on
large and heterogeneous populations. After presenting our findings and
recommendations, we conclude by pointing out some promising avenues for future
studies in this emerging field of research.
Addressing minority bias
The issue of minority bias fits within a wider realm of goals and concerns
shared by survey researchers. First of all, the notion of an observed sample as a
representative, unbiased, and sufficiently precise reflection of an underlying
population that is not observed but which constitutes the real interest of a study, lies
at the very heart of survey research. Whenever the relationship between a sample and
its underlying population is not at the core of our attention, then we are not doing
survey research, and we will not want to use statistical inference as a tool of
generalisation from findings.
On the basis of insights gained from studies conducted in other European
countries (Deding, Fridberg & Jakobsen, 2008; Feskens, Hox, Lensvelt-Mulders &
Schmeets, 2006, 2007), we anticipated that the invisible frontier between the
effectively targeted majority and the implicitly relegated minority might be
delimitated by things like speaking (one of) the survey language(s), having material
living circumstances and habits that make someone “reachable” by way of standard
procedures, holding a system of beliefs about the self within society that make survey
questions appear meaningful and oneself capable of answering them (in the eyes of
both the respondent and the interviewer), and so on. To be sure, there is no
deterministic relationship between a nationality inscribed on a passport and any of
these factors, but there are good reasons to anticipate a substantial correlation in
many cases.
Accepting the tacit compromise to leave closed the black box around the
processes by which minorities are excluded (or, sometimes, included) might have the
advantage that it allows circumventing a potentially painful process of redrawing
more narrowly the boundaries of the populations we are actually studying
appropriately, with the means at our disposal. But there is also a cost to such a
position, as it implies a lack of precision in our understanding of actual selection
processes. This lack of knowledge then precludes precise enough understanding of
what “Swiss” (or any other generic label) actually stands for in survey outcomes such
as “X% of the Swiss support policy Y” or “X% of the Swiss live in poverty”. Such lack
of accuracy becomes problematic when similar statements are eventually interpreted
literally (e.g., as a statement on the poverty rate among all Swiss residents), while the
data production process actually involves a more narrowly defined effective reference
population (which, to pursue the example, is in all likelihood at a lower overall risk of
poverty).
Mechanisms of social exclusion in surveys
The gap between all residents of Switzerland and residents that have a fair
chance to be included in a general social survey is not random. This leads to another
type of issue that might draw social scientists’ attention to the issue of minority bias:
the substantive problem of the social mechanisms that produce social exclusion. The
interesting question is to what extent mechanisms that generate non-participation in
social surveys might overlap with mechanisms that impede social participation more
generally. In this perspective, far from being just a technical issue, the study of survey
non-representation can even contribute to a better understanding about how members of certain social categories are prevented from taking part in certain social
activities that are in theory open to everyone.
To spin this idea a little bit further, systematic bias in survey response also
intrigues because it appears to betray a democratic ideal that is frequently projected
onto surveys: one person, one voice. If surveys are to reveal the preferences,
aspirations, or needs of the public as a whole, then every individual’s position has to
be represented equally. Unaccounted systematic differences make survey samples
look more similar to a shareholders general assembly, where votes are weighted by
individual assets, than to the idealised democratic public. The question of why such
distortions sometimes are not a source of concern (in the eyes of interviewers,
researchers, policy-makers, or the general public) is at least as interesting as knowing
why they are in other cases. The tacit acceptance that some categories of people will
remain silent in a survey might precisely be anchored in more or less implicit
conceptions about variable levels of civic legitimacy within the overall public, that is,
beliefs about different levels of entitlement to have one’s preferences, aspirations, or
needs being expressed and taken into account.
Whenever we as survey researchers embrace this tacit acceptance uncritically,
we are at risk of producing findings and theories about social reality that are bounded
to the reality experienced by the majority. Therefore, the substantive concern about
processes that produce social exclusion, and that reproduce it in particular by way of
exclusion from social surveys, goes hand-in-hand with the pragmatic concern to
enhance the representativeness of surveys, not least in order to break societal and
scientific cycles that render certain minorities invisible to the public eye (and leave
the public indifferent to their fate).
