BADF: Bias Association Discovery Framework

• The paper introduces a multi-stage pipeline that extracts, scores, and filters demographic identity–concept pairs from LLM-generated narratives.
• It employs statistical tests (chi-squared with p<0.05) and cosine similarity (threshold 0.63) to verify significant non-definitional bias associations.
• Empirical results reveal hundreds of associations across gender, race, and religion, highlighting biases beyond predefined identity–concept templates.

Bias Association Discovery Framework (BADF) denotes a systematic approach for extracting both known and previously unrecognized associations between demographic identities and descriptive concepts from open-ended LLM outputs. In its explicit formulation, BADF was introduced as a pipeline that generates open-ended stories, extracts salient descriptive concepts about demographic identities, scores which concepts are significantly and distinctively associated with each identity, and filters out concepts that are merely definitional or exclusive facts rather than model-inferred bias (Pan et al., 2 Aug 2025). In adjacent literature, several methods are described in “BADF-style” terms because they likewise seek to discover which groups, contexts, latent factors, or causal pathways are associated with discrimination, favoritism, systematic error, or representational harm (Qureshi et al., 2016).

1. Origin and conceptual scope

BADF was proposed to address a limitation of standard LLM bias evaluations: most prior methods test predefined identity–concept pairs or fixed templates, which are useful for confirming known stereotypes but cannot discover unexpected or emerging forms of representational harm. The BADF formulation instead targets free-form generation, especially story generation in real-world contexts, on the premise that bias often appears as subtle narrative regularities rather than as isolated stereotype completions (Pan et al., 2 Aug 2025).

The framework is therefore centered on association discovery rather than only association verification. Its core object is the relation between a demographic identity and a descriptive concept as it emerges from open-ended generation. The paper’s motivating examples include portrayals such as groups being repeatedly depicted as anxious, determined, devout, supportive, or professional. The stated purpose is to surface both known and previously unrecognized associations and to distinguish model-inferred patterns from literal identity facts (Pan et al., 2 Aug 2025).

Within the broader research landscape, the expression “BADF-style” is used more generally for frameworks that do not merely ask whether a model is biased, but also ask which groups or contexts are associated with that bias. In this wider sense, BADF-like work spans causal discrimination discovery, unsupervised subgroup discovery in vision, segmentation auditing without target labels, recommendation embedding audits, semantic bias naming, and fairness-relevant causal path recovery (Qureshi et al., 2016).

2. Pipeline structure and statistical criteria

The BADF pipeline in open-ended generation has three main stages: association extraction, significant association identification, and bias association identification. Before these stages, it begins with open-ended generation under multiple settings: Single-character base, Two-character base, Balanced-valence two-character generation, Negative two-character generation, and Open-box generation using patchscope-style intervention. The base prompts pair demographic identities with locations so that the model produces short stories in realistic settings (Pan et al., 2 Aug 2025).

In the extraction stage, Qwen3-32B is used as the core LLM for all stages except concept unification. For each generated story, the model extracts the most essential, explicit, and defining characteristics of each character while avoiding scene-specific details, one-off actions, vague generalities, and unsupported inferences. The pipeline then applies post-hoc self-refinement, decomposition of compound concepts into finer-grained concepts, and a second post-hoc check for conflations or ambiguities. To merge semantically similar concepts, it uses sentence-transformer embeddings and cosine similarity, with a cosine similarity threshold of 0.63 for merging (Pan et al., 2 Aug 2025).

Significant association identification combines a distinctiveness score with a chi-squared test of independence. For target demographic identity $A$ and concept $Y$, the paper defines

$$S(Y, A) = \frac{N_A}{N_A}\big(n_A(Y)-n_{\min}(Y)\big).$$

The paper states that $S(Y,A)\in[0,1]$ and that if $n_{\min}(Y)\ge n_A(Y)$, then $S(Y,A)=0$; it also notes that the textual presentation of the formula is somewhat compressed or possibly formatting-truncated, but that the intended use is a normalized distinctiveness measure based on the gap between $A$’s frequency and the minimum frequency among other identities. Statistical significance is then tested with a $\chi^2$ test of independence within each location category, with null hypothesis “concept distribution is independent of identity,” and the association is considered significant if $p<0.05$ (Pan et al., 2 Aug 2025).

The retention criterion is explicit:

$S(Y,A) > 0 \quad \text{and} \quad p < 0.05.$

A concept is therefore kept only if it is both distinctive for an identity and statistically significant. BADF then applies a final filtering stage to remove concepts that are universally and unquestionably exclusive to the identity itself. This exclusivity check is performed with another LLM prompt, and a concept is retained as a bias association only if it is significantly associated, distinctive, and not merely definitional or exclusive (Pan et al., 2 Aug 2025).

3. Empirical coverage and findings in open-ended generation

The BADF study covers 3 demographic categories—Gender, Race, and Religions—and 10 location categories spanning 87 real-world locations: Education, Sports, Healthcare, Workplace, Art/leisure, Technology, Media, Economics, Law/policy, and Environment. In the two-character setting, the reported story counts are 8,700 for Gender, 10,440 for Race, and 10,440 for Religions. The models used for generation are Llama-3.2-11B-Vision-Instruct, Llama-3.2-3B-Instruct, and Qwen3-8B, while Qwen3-32B is used for the BADF pipeline itself (Pan et al., 2 Aug 2025).

Manual evaluation is reported for sampled outputs from the LLM-assisted stages. The sample sizes are 50 generations for concept extraction, decomposition, and unification, and 100 significantly associated concepts for bias association filtering. The reported metrics are precision, recall, decomposition accuracy, homogeneity, completeness, V-measure, and exclusivity accuracy, with values 0.93, 0.98, 0.97, 1.00, 0.89, 0.94, and 98%, respectively (Pan et al., 2 Aug 2025).

The main findings are qualitative as well as statistical. BADF is reported to find hundreds of associations per identity across settings, with two-character generation usually yielding more associations than single-character generation, especially for gender and race. The paper highlights less-obvious associations such as Black ↔ successful entrepreneur, Asian ↔ struggles to communicate, Asian ↔ medical professional, Middle-East ↔ bridges cultural divides, Christian ↔ devout, and Buddhism ↔ mindfulness / meditation, alongside gendered emotional associations linking female identities to anxiety or resilience and male identities to support or strategic behavior (Pan et al., 2 Aug 2025).

Prompt design changes the discovered association landscape. The Negative prompt produces the most associations, Balanced-valence is intermediate, and the base prompt produces the fewest. The reported trend is toward more negative, conflict-laden, and emotionally charged concepts in the negative setting. The paper also reports that open-box and black-box generation surface different sets of biases: open-box tends toward more occupational and collaborative roles and more social connection concepts, while black-box tends toward more emotional and spiritual variety and stronger surface-level religious associations in some cases. Cross-model comparisons indicate that Qwen3-8B generally produces the highest number of bias associations, and that scaling from Llama3.2-3B to Llama3.2-11B does not uniformly reduce them (Pan et al., 2 Aug 2025).

4. BADF-style variants across adjacent research areas

Several adjacent papers are explicitly framed as BADF-style methods because they discover associations between sensitive attributes, contexts, or latent structures and measurable failures or harms.

Setting	Core mechanism	Discovered association
Causal discrimination discovery	Propensity score analysis and regression trees (Qureshi et al., 2016)	discrimination/favoritism patterns after filtering confounding
Medical image segmentation	Reverse Classification Accuracy and group-wise estimated DSC gaps (Gaggion et al., 2023)	hidden bias associations between protected attributes and segmentation quality
Deep visual recognition	Hierarchical clustering of embeddings and Grad-CAM (Krishnakumar et al., 2021)	low-performing subpopulations and spurious visual cues
Visually grounded slice discovery	CAM-grounded DOMINO, FACTS, and Bias-to-Text (Marani et al., 2024)	semantically meaningful unknown subgroups and spurious attributes
Recommendation embeddings	bias directions, EAA/GEAA/DEAA, R-RIPA, classification, PCA (Beattie et al., 2023)	sensitive attribute association bias in latent factor representations
Semantic bias naming in classifiers	captioning, keyword extraction, text embeddings, similarity ranking (Ciranni et al., 2024)	natural-language “names” for learned shortcuts
Fairness-aware causal discovery	LLM-guided BFS with active learning and dynamic scoring (Zanna et al., 13 Jun 2025)	direct, indirect, and spurious fairness-relevant paths

These frameworks differ in what they treat as the relevant association object. In open-ended LLM generation, the association is identity–concept co-occurrence in narrative outputs. In causal discrimination reasoning, the association is a causally adjusted effect of group membership on decisions, quantified by propensity score weighting and summarized by regression-tree rules. In medical segmentation, the association is a protected-attribute-linked performance gap inferred from estimated segmentation quality without target masks. In computer vision slice discovery, the association is between systematic model failure and latent subgroup structure in the model’s own embedding space. In recommendation auditing, the association is the geometric entanglement of sensitive attributes in latent factor entity vectors. In semantic bias naming, the association is between learned model behavior and textually describable shortcut concepts (Qureshi et al., 2016).

A plausible implication is that BADF is best understood not as a single algorithm but as a research pattern: discover candidate bias-linked structures, score or estimate their strength, and convert them into interpretable outputs such as concepts, rules, slices, directions, or causal paths. The BADF label is explicit only in the open-ended LLM framework, but the broader literature repeatedly uses it as a lens for methods that expose otherwise hidden bias associations (Ciranni et al., 2024).

5. Causality, fairness, and interpretation

A central distinction in BADF-oriented work is between associational discovery and causal discrimination reasoning. The causal discrimination framework based on propensity score analysis treats the protected group as the treatment and the unprotected group as the control, estimates

$Y$0

reweights unprotected tuples with

$Y$1

and defines the causal discrimination score

$Y$2

Here, $Y$3 indicates that observed disparity is mostly explained by confounders, $Y$4 indicates causal discrimination, and $Y$5 indicates causal favoritism. Regression trees are then trained over these scores to expose contexts of high discrimination or favoritism (Qureshi et al., 2016).

A more recent causal-discovery formulation targets fairness-critical pathways directly. It classifies paths from sensitive attributes to outcomes into Direct paths, Indirect paths, and Spurious paths, and evaluates not only global graph structure but also fairness path recovery. The paper reports effect decomposition

$Y$6

and a normalized bias measure

$Y$7

Its explicit warning is that naive LLM-based causal discovery can over-attribute bias by inventing sensitive paths, while purely statistical causal discovery can under-detect bias by missing mediated discrimination (Zanna et al., 13 Jun 2025).

At the same time, BADF-oriented association discovery should not be equated with empirical fairness. A conceptual intervention in language-model auditing distinguishes association bias from empirical fairness and reports no obvious correlation between representational bias metrics and fairness metrics across widely used LLMs. In that formulation, association bias concerns representation-level geometry or association strength, while empirical fairness concerns downstream group-wise performance differences such as $Y$8 and $Y$9. The paper’s warning is interpretive: discovering a biased association does not by itself establish unequal performance, and reducing association bias does not necessarily improve fairness (Cabello et al., 2023).

This distinction also appears in BADF-style work outside language. The medical segmentation framework operationalizes fairness as a difference in group-wise performance and explicitly treats estimated subgroup gaps in $$S(Y, A) = \frac{N_A}{N_A}\big(n_A(Y)-n_{\min}(Y)\big).$$0 as a performance gap proxy, not as fairness measured as an abstract ethical quantity. The sign of

$$S(Y, A) = \frac{N_A}{N_A}\big(n_A(Y)-n_{\min}(Y)\big).$$1

is used to infer which group is favored, but the method’s stated goal is to discover hidden bias associations between protected attributes and performance rather than to prove or eliminate bias in a broader normative sense (Gaggion et al., 2023).

6. Limitations, misconceptions, and open problems

BADF inherits the limitations of the discovery setting in which it is instantiated. In open-ended LLM generation, the framework may miss subtle harms or misattribute concepts if extraction fails; it relies on LLM-assisted extraction and filtering, and some discovered bias associations may reflect prompt artifacts rather than stable model behavior. The paper further emphasizes that extracted associations should not be interpreted as truths about real people, because they reflect model behavior rather than actual demographic properties, and may themselves be harmful if misused or decontextualized (Pan et al., 2 Aug 2025).

Across BADF-style variants, “unsupervised” often has a narrower meaning than “label-free overall.” In medical image segmentation, the method is unsupervised for the target population, but it still requires a labeled reference database, and its performance depends on registration quality and reference similarity. In recommender auditing, the framework is designed mainly for binary attributes, PCA-based direction construction can be fragile when pairings are poor, R-RIPA depends strongly on direction quality, and PCA visualization is qualitative rather than dispositive. In visually grounded discovery, gains depend on whether the base model and the CAM method provide informative heatmaps; if the failure mode is not visually localizable, improvements may be smaller. In LLM-guided causal discovery, the stated limitations include hyperparameter sensitivity, computational cost, token limits, metadata quality, reproducibility challenges, and the risk that the LLM itself encodes stereotypes (Gaggion et al., 2023).

A common misconception is that removing a sensitive feature from training input eliminates the corresponding bias association. The recommendation case study explicitly reports persistent gender association structure even when user gender is removed as a model feature during training, with significant DEAA remaining in both with-gender and without-gender models. Another misconception is that subgroup discovery alone proves a fairness violation. UDIS is explicit that it targets bias manifested as failure modes—spurious correlations that reduce test accuracy—and not every discovered failure mode is necessarily a bias in the strict social or fairness sense (Beattie et al., 2023).

Open problems identified in BADF-oriented literature include metric validity, causal interpretation, multilingual and cross-cultural generalization, intersectionality, task dependence, and the distinction between normative and empirical notions of bias and fairness. This suggests that BADF is most robust when treated as a discovery framework that reveals model-associated structures for further analysis, rather than as a standalone certificate of fairness or unfairness (Cabello et al., 2023).