Breaking the bias for better gender data
Generating high quality statistics relies on eliminating gender bias at all stages of the production process. This blog looks at how gender bias occurs in statistics and what the ILO is doing to support efforts to minimize it.
“Break the bias” was the theme of International Women’s Day this year, which focused attention on the persistent biases, stereotypes and discrimination that hold society back from being inclusive and gender equal. When it comes to statistics on the world of work, household surveys remain the best way to obtain unbiased information from a random sample of the population. As such, survey data, the methods used for its collection and the statistics ultimately produced, have a central role to play in breaking gender bias.
There are two pathways through which bias can undermine the utility of survey data for gender-based analysis. First, bias in terms of statistical error relating to how adequately a survey sample corresponds to the population of interest, and how adequately statistical concepts correspond to the subjects, objects, and phenomena they represent. Second, bias in terms of conceptual blind spots and omissions, which relates to who and what is (and is not) designated as eligible for representation and measurement in official statistics.
Gender bias imported through statistical error
Sample surveys aim to generate statistics for a population based on values obtained for a subset of sampled individuals (or other sampled units). Statistical error introduced at one or more stages of a survey’s design, administration or processing can threaten the accuracy of the resulting data.1
Some level of random sampling error occurs whenever a sample of the population, rather than the entire population of interest, is surveyed. Representative sample survey procedures are designed to minimise and control for this.
Non-sampling error – which may be imported at any, or all, stages of the survey process through coverage error, non-response error and measurement error – is less straightforward to address, as it takes several forms:2
Coverage, or frame, error
Unit non-response error
Measurement error
Gender bias can be found in each of these error components, though it is particularly relevant to measurement error. Gender bias can be imported through measurement error originating in instrument design (through problems with construct validity, or the wording or sequencing of questions and response codes), survey mode effects, interviewers’ manner (or demographic characteristics), the interview context (for example, respondent self-censorship in the presence of other household members), or insufficient / absent restrictions on proxy-reporting for relevant topics.4
Efforts to minimize gender bias in these different sources of statistical error have led to a number of key insights. Methods have evolved to identify, quantify, and classify measurement error at the survey design stage. These include qualitative or “formative” studies, cognitive interviewing, randomized experiments, repeated measurement studies and record-check studies. In some contexts, a move to conversational interviewing has been shown to reduce biases originating in misalignment between respondent comprehension and question intent.
Bias due to blind spots or omissions
As recent works such as Data Feminism and Invisible Women explore, stereotypes depicting distinct social roles, and related spheres of activity, for men and women continue to bias what gets measured, counted, and made visible in statistics.
The effects of such bias have been long noted in the field of labour statistics and have been the subject of extensive corrective efforts. Examples include the undercounting of women engaged in work historically coded as “men’s work” (for instance, labour migration5), and the omission of productive activities historically coded as “women’s work”, such as home-based employment6 and unpaid care and domestic work. Although vital and valuable (consider the costs to outsource it), most unpaid care and domestic work continues to be excluded from standard measures of economic production, such as gross domestic product.7
These measurement issues and others – together with the scarcity of sex-disaggregated and gender‑relevant data – prompted the establishment of the field of gender statistics in the mid-1980s. Since then, there have been growing efforts to identify and address sources of gender bias in survey statistics. However, as noted by the Commission on the Status of Women at its most recent session earlier this year, work remains to be done in traditional areas of statistics as well as in emerging topics, such as gender data relevant to addressing climate change.8 9
The ILO’s contribution to breaking the gender bias in statistics
A major advance in overcoming the gender bias in work, employment and labour statistics occurred in 2013, when the highest decision-making body on international labour statistics – the International Conference of Labour Statisticians – adopted a new and expansive reference concept of “work”. This marked an important ‘first,’ and it signified a major step change in the measurement of unpaid productive activities. One important outcome has been the potential for greatly improved analysis of gender-based inequalities in labour force participation, employment characteristics, the division of paid and unpaid labour, and total working time. The ILO is supporting improvements in traditional methods for measuring such gender-related concerns. This includes piloting new light time-use modules for attachment to labour force surveys in low- and middle-income countries, to generate statistics on unpaid care and domestic work when a dedicated time-use survey is not feasible. The ILO is also piloting new labour force survey questions to produce more and better data and statistics on women and men undertaking informal work, the source of paid work for more than 60 per cent of employed people worldwide.
These two pilot projects are linked by their shared goal to reduce gender biases in measures of work. Disproportionate responsibility for the provision of unpaid care and domestic work directly impacts women’s capacity and opportunity to engage in paid work – whether formal or informal.