---
title: Bias Intervention Dataset (BID) Overview
url: https://www.emergentmind.com/topics/bias-intervention-dataset-bid
type: topic
---

# Bias Intervention Dataset (BID) Overview

Searching arXiv for recent and relevant papers on “bias intervention dataset” and closely related benchmark/dataset-bias work.
Bias Intervention Dataset (BID) denotes a class of research resources in which bias-relevant variables are deliberately manipulated, paired, removed, or balanced so that bias can be probed under controlled conditions rather than inferred only from unconstrained observational data. In its most explicit recent use, BID is the name of a manually constructed benchmark for causal analysis of how social bias states affect faithfulness hallucinations in large language models [2508.07753]. In broader data-centric usage, the same idea appears in controlled retrieval benchmarks such as Grep-BiasIR, paired stereotype-evaluation resources such as INDIAN-BHED, and debiased training subsets produced by black-box intervention on historical data [2201.07754][2309.08573][2102.03054].

## 1. Conceptual scope and historical framing

A BID is defined less by modality than by intervention structure. The central design principle is that the bias-relevant factor is isolated while other properties are held fixed as much as possible. In the LLM setting, this takes the form of explicit bias interventions on a bias state variable \(B\), with confounders \(Z\) treated as separate causes of hallucination \(H\) [2508.07753]. In retrieval, it takes the form of gender-neutral queries paired with documents that are identical except for gendered wording, so that differences in ranking can be attributed more confidently to gendered wording rather than topical content [2201.07754].

This intervention-centered view is adjacent to, but distinct from, the older literature on dataset bias. “A Deeper Look at Dataset Bias” distinguishes capture bias, category/label bias, and negative bias, and studies how these biases persist under BOWsift and DeCAF representations across a cross-dataset testbed over twelve datasets [1505.01257]. That paper does **not** introduce a new dataset in the sense of a new collection of images; instead, it uses and extends a cross-dataset benchmark/testbed and proposes a new cross-dataset measure,
$$
CD = \frac{1}{1 + \exp\{-(\text{Self} - \text{Mean Others})/100\}},
$$
with values above \(0.5\) indicating the presence of bias [1505.01257]. This suggests a broader genealogy in which BID is best understood as a controlled benchmark or data-construction paradigm for diagnosing, measuring, and sometimes mitigating bias.

## 2. Core intervention designs

Across the literature, BID-style resources recur in a small number of intervention patterns. Some resources vary wording while preserving relevance or semantics; some construct matched templates under alternative bias states; some remove datapoints judged to be bias-inducing; and some rebalance data through augmentation or undersampling. The common objective is not merely to detect biased outputs, but to create conditions under which causal or quasi-causal comparisons are possible [2201.07754][2309.08573][2508.07753][2102.03054][2205.01811].

| Resource | Domain | Controlled intervention |
|---|---|---|
| Grep-BiasIR | Information retrieval | 118 bias-sensitive queries; each query has 6 associated documents: relevant/non-relevant \(\times\) female/male/neutral |
| INDIAN-BHED | LLM stereotype evaluation | Two English sentences per example: stereotypical and anti-stereotypical |
| BID | LLM hallucination analysis | Pro-stereotype, Anti-stereotype, and Non-stereotype versions of matched templates |
| Debiased training subset | Binary classification on historical data | Remove top-ranked biased datapoints and retrain on the remaining subset |
| UTKFace augmentation study | Face-attribute classification | Undersampling, geometric transformations, VAE generation, and StarGAN generation |

The most controlled designs have a counterfactual flavor. Grep-BiasIR rewrites each base document into female, male, and neutral versions while preserving meaning except for gender-indicating language [2201.07754]. BID for hallucination analysis requires that an intervention satisfy effectiveness, precision, and consistency, so that the intended bias state is forced, only the social-attribute elements corresponding to bias change, and the Pro / Anti / Non interventions involve equivalent levels of textual modification [2508.07753]. By contrast, the historical-data intervention of “Removing biased data to improve fairness and accuracy” treats the original dataset \(\mathcal{D}\) itself as the intervention target: biased records are removed rather than relabeled or edited because the source of bias may be label bias or selection bias [2102.03054].

## 3. Retrieval and paired-sentence benchmark instantiations

Grep-BiasIR is a thoroughly-audited dataset for investigating gender representation bias in information retrieval results, and it is particularly clear as a BID because it is built to let researchers probe, measure, and eventually mitigate gender representation bias in ranking systems under tightly controlled conditions [2201.07754]. It contains **118 bias-sensitive queries** and **708 documents total**, organized so that each query has one relevant and one non-relevant base document, each rewritten into female, male, and neutral variants. The queries are grouped into **7 gender-related stereotypical concepts**—Appearance, Child Care, Cognitive Capabilities, Domestic Work, Career, Physical Capabilities, and Sex and Relationship—with reported counts of **14**, **14**, **12**, **15**, **20**, **19**, and **23** queries respectively. Query lengths range from **2 to 9 words**. Male indicators are **man, men, male, father, dad, paternal, he, him**; female indicators are **woman, women, female, mother, mom, maternal, she, her**; neutral indicators are **people, person, partner, parent, parental, you, their, them**. Two post-doctoral researchers independently reviewed the quality of each query and document and the expected stereotype of each query, with the final dataset including only high-quality items and assigning male or female stereotype labels only under full agreement [2201.07754].

The design objective in Grep-BiasIR is specific: for a gender-neutral query, a fair system should not systematically privilege documents written with male indicators over otherwise equivalent female or neutral documents [2201.07754]. The dataset therefore supports search engine behavior analysis, studies of gender bias in human relevance judgments, and human query generation analysis. Its value lies in holding the information need constant while varying only the gender cues in relevant and non-relevant documents.

INDIAN-BHED, described in the paper as **Indian Bias Evaluation Dataset**, is a paired-sentence benchmark for measuring stereotypical bias in LLMs in the Indian context [2309.08573]. It contains **228 English examples** total, with **123 religion-related examples** and **105 caste-related examples**, and follows the same paired-sentence structure as CrowS-Pairs: one sentence expresses a stereotypical association and the other expresses an anti-stereotypical association. The benchmark focuses on India-centric axes—especially **Brahmin vs Dalit** for caste and **Hindu vs Muslim** for religion—while also comparing against Western-centric categories using filtered CrowS-Pairs subsets of **386 race-related sentences** and **159 gender-related sentences** [2309.08573]. Its methodological significance is comparative: it tests whether models prefer stereotypical associations over anti-stereotypical ones and whether models behave differently in India-centric versus Western-centric settings.

## 4. Causal BID for faithfulness hallucination in large language models

The most explicit formalization of BID appears in “Exploring Causal Effect of Social Bias on Faithfulness Hallucinations in Large Language Models,” where BID is a manually constructed benchmark designed specifically to support causal analysis of how social bias states affect faithfulness hallucinations in LLMs [2508.07753]. The paper formulates an SCM with bias state \(B\), hallucination state \(H\), and confounders \(Z\), with causal structure
$$
Z \rightarrow B,\qquad Z \rightarrow H,\qquad B \rightarrow H.
$$
Bias intervention cuts the incoming causal influence into \(B\), so that changes in hallucination can be attributed to the manipulated bias state rather than to \(Z\).

BID uses three bias states—**Pro-stereotype**, **Anti-stereotype**, and **Non-stereotype**—and covers **five social bias categories**: **Gender**, **Religion**, **Socioeconomic Status (SES)**, **Age**, and **Disability** [2508.07753]. The dataset has **11,032 entries total**, distributed as **1,594** for Gender, **1,784** for Religion, **3,436** for SES, **3,190** for Age, and **1,840** for Disability. Templates contain at least **two individuals with configurable social attributes** and at least **one additional person without social attributes**, allowing the same underlying scenario to be observed under different bias states while preserving the rest of the context.

The paper organizes the data into pairwise comparisons—**Non–Pro**, **Non–Anti**, and **Pro–Anti**—where each pair differs by only one social attribute [2508.07753]. The Individual Causal Effect is defined for paired binary outcomes, for example
$$
ICE^{\text{Pro-Anti}} = H\mid_{do(B=\text{Pro})} - H\mid_{do(B=\text{Anti})},
$$
so that \(ICE \in \{-1,0,1\}\). Because the outcomes are paired and binary, the paper uses **McNemar’s test**, and it defines **Unified Causal Significance (UCS)** as a signed version of the McNemar statistic:
$$
UCS^{\text{Pro-Anti}} = \text{sign}\!\left(\sum_{i=1}^{n} ICE_i\right) X.
$$

Empirically, across **seven mainstream LLMs**—Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.2, Gemma-2-9b-it, Llama-3-8B-Instruct, Llama-3.2-3B-Instruct, GPT-4o-mini, and GPT-3.5-turbo—the hallucination rate follows the same ordering:
$$
\text{Pro-stereotype} < \text{Non-stereotype} < \text{Anti-stereotype}.
$$
Out of **105 pairwise model–bias comparisons**, **85** show significant causal effects; **34 out of 35** Non vs. Anti comparisons are significant, **19 out of 35** Non vs. Pro comparisons are significant, and **32 out of 35** Pro vs. Anti comparisons are significant [2508.07753]. The paper further distinguishes **unfairness hallucination** from **common hallucination** and reports that social bias has a significant causal effect only on unfairness hallucinations, not on common hallucinations. In the confidence analysis, the reported ordering is
$$
\text{Correct} > \text{Unfairness hallucination} > \text{Common hallucination},
$$
which implies that unfairness hallucinations are harder to detect using confidence or logit-based heuristics [2508.07753].

## 5. Data-centric intervention pipelines and benchmark protocols

A different BID formulation appears in “Removing biased data to improve fairness and accuracy,” where the intervention is a black-box pre-processing procedure that starts with a historical training dataset \(\mathcal{D}\), detects biased training points, removes those points to form a debiased dataset, and trains the final classifier on this debiased subset [2102.03054]. The learning setting is binary classification with a sensitive attribute \(S\), and fairness is evaluated using **individual discrimination** and **statistical disparity**. Individual discrimination is defined as
$$
\text{individual discrimination} = \frac{|\text{discriminatory pairs}|}{|\text{similar pairs}|},
$$
with similar pairs generated synthetically under two settings: threshold \(=0\), where individuals are identical on all non-sensitive features, and threshold \(>0\), where categorical non-sensitive features match exactly and numerical non-sensitive features are within a 10% normalized range [2102.03054]. The method ranks training points by influence on discriminatory predictions, iteratively removes the highest-ranked points, retrains, and returns the subset just before discrimination increases again, that is, a local minimum. In experiments on **6 real-world datasets** and **8 experimental settings**, the method achieves **0% individual discrimination** for all datasets in the threshold \(=0\) least-discrimination setting, and reported averages for the threshold \(>0\) setting are about **0.65%** discrimination with **91%** accuracy in the least-discrimination setting [2102.03054].

In computer vision, “Assessing Dataset Bias in Computer Vision” studies bias from uneven class distributions in **gender**, **age**, and **ethnicity** using a subset of **UTKFace** as the source dataset and evaluating several dataset-level interventions: **undersampling**, **geometric transformations**, **variational autoencoders (VAEs)**, and **StarGAN** [2205.01811]. The UTKFace subset is split stratified into **60% training**, **20% validation**, and **20% test**. For classifier training, the paper uses a pretrained **InceptionV3** with replaced top layers, batch size **64**, **25 epochs**, **SGD**, learning rate **0.0001**, and momentum **0.9**. The best reported UTKFace test-set accuracies are **91.75%** for gender with StarGAN, **91.30%** for age with geometric transformations, and **87.20%** for ethnicity with StarGAN, with the best performing models also exhibiting a uniform performance across the classes within each attribute [2205.01811]. On external datasets, the best reported LFWA+ results are **91.0%** for gender with StarGAN, **82.2%** for age with Geometric, and **74.1%** for ethnicity with StarGAN; on CelebA, the best reported results are **83.3%** for gender with StarGAN and **74.5%** for age with StarGAN [2205.01811]. Here the BID logic is balancing and augmentation rather than paired textual intervention.

The cross-dataset benchmark of “A Deeper Look at Dataset Bias” supplies a complementary protocol-level perspective [1505.01257]. It evaluates BOWsift, DeCAF6, and DeCAF7 on a **sparse set** of **105 ImageNet classes** aligned across **12 datasets** and a **dense set** of **40 shared classes** across Bing, Caltech256, ImageNet, and SUN. DeCAF features often improve within-dataset accuracy and cross-dataset recognition, but they do **not** eliminate dataset bias; in some cases their increased discriminative power makes dataset-specific differences more pronounced [1505.01257]. The paper evaluates **Unbias**, **Landmark**, **Subspace Alignment (SA)**, **Domain Adaptation Machine (DAM)**, **reshape + SA**, **reshape + DAM**, and **self-labelling**, and finds that common debiasing or adaptation methods are not uniformly effective under DeCAF. Within a BID framework, this result is important because it separates intervention construction from intervention success: a benchmark can be well designed even when standard debiasing methods fail on it.

## 6. Limitations, misconceptions, and research significance

A recurrent misconception is that BID names a single universally accepted dataset. The literature instead shows both explicit and retrospective usage. BID is an explicit dataset title in the hallucination study [2508.07753], while Grep-BiasIR is described as best understood as a bias intervention dataset for information retrieval [2201.07754], and the debiased subset of the black-box fairness paper is the output of a bias-intervention dataset construction procedure rather than a named public benchmark [2102.03054]. A plausible implication is that BID should be treated as a family of controlled data resources rather than a single canonical artifact.

A second misconception is that better models or better features automatically solve bias. The dataset-bias analysis of DeCAF explicitly reports that deep features do **not** eliminate dataset bias and may even make dataset identity easier to predict [1505.01257]. The hallucination study likewise reports that overall model performance is not a reliable proxy for bias-induced hallucination sensitivity: models with relatively low hallucination rates can still show strong causal bias effects [2508.07753]. BID-style evaluation is therefore complementary to standard performance evaluation, not a substitute for it.

A third misconception is that any data intervention is a debiasing intervention. The computer-vision study reports that **StarGAN** usually gives the best overall accuracy and most balanced performance, **Geometric augmentation** is often nearly as good and much faster, **Undersampling** often hurts majority-class performance, and **VAE-generated images** perform the worst and can even worsen bias [2205.01811]. Similarly, the black-box fairness paper removes datapoints rather than editing labels because the source of bias is unknown and may involve both label bias and selection bias [2102.03054]. BID work is therefore as much about intervention validity as about intervention existence.

The limitations of BID resources are also explicit. Grep-BiasIR is limited to **male/female** gender indications, may reflect **Western social norms**, and is described by its authors as **small-scale**, even though they emphasize that it is, to their knowledge, the **first dataset of its kind**, grounded in a stereotype framework and enriched with stereotype meta-annotations [2201.07754]. The LLM BID is template-structured, manually constructed, centered on a QA task form, and limited to **five bias categories**, which may constrain naturalness and scope [2508.07753]. INDIAN-BHED is in **English** and focuses principally on caste and religion in India, although that focus is also its central contribution to geographically and culturally inclusive bias benchmarking [2309.08573].

The significance of BID lies in what these controlled resources make measurable. They support search engine behavior analysis, studies of human relevance judgments, query-generation analysis, causal analysis of hallucination, fairness-oriented preprocessing of historical data, and comparative evaluation of augmentation and domain-adaptation strategies [2201.07754][2508.07753][2102.03054][2205.01811]. In that sense, BID is both a dataset concept and an experimental logic: it converts bias from a diffuse property of model outputs into an object of intervention, comparison, and, in some settings, causal inference.

Source: https://www.emergentmind.com/topics/bias-intervention-dataset-bid