Human-in-the-Loop Embedding Refitting

• The paper introduces an interactive framework that refits word embeddings using quadratic constraints to correct biases and enhance semantic structure.
• It employs efficient closed-form update rules and adaptive freezing to rapidly converge embedding adjustments, requiring only 20–50 iterations on large vocabularies.
• Empirical results show significant bias reduction (WEAT score from 1.02 to 0.45) and improved classifier accuracy (from 78.3% to 82.7%) after human-guided refinements.

Human-in-the-Loop Word Embedding Refitting refers to a class of interactive post-processing techniques for real-valued word embeddings in which a human agent iteratively steers the geometry of the space to address undesirable semantic artifacts—such as bias or incoherence—by imposing direct constraints (attraction or repulsion) on specific word sets or pairs. These approaches are situated at the intersection of post-hoc embedding correction, interactive machine learning, and bias mitigation. They formalize human knowledge as quadratic constraints in a global convex objective solved over the embedding space and are agnostic to the initial embedding algorithm (e.g., word2vec, GloVe).

1. Mathematical Foundations of Interactive Refitting

Let $V = \{1,\ldots, N\}$ denote a vocabulary and let $\hat q_i \in \mathbb{R}^d$ denote the original embedding for word $i$. The goal is to produce a refitted embedding $q_i$ for each word $i$ such that $q_i$ stays close to $\hat q_i$ but also satisfies a set of human-imposed constraints. The general quadratic objective function for interactive refitting is

$$L(\{q_i\}) = \sum_{i=1}^N \alpha_i \|q_i - \hat q_i\|_2^2 + \sum_{(i,j)\in E} \beta_{ij} \|q_i - q_j\|_2^2 + \sum_{k=1}^K \gamma_k \sum_{i<j;~i,j\in S_k} \|q_i - q_j\|_2^2$$

where:

• $\alpha_i > 0$ regulates fidelity to the pretrained embedding.
• $E \subset V\times V$ is a set of static lexical constraints (e.g., synonymy), each with weight $\beta_{ij} \geq 0$.
• $S_1, \ldots, S_K \subset V$ are user-selected word groups to be “pulled together,” each with group-similarity strength $\gamma_k\geq 0$.
• Words or pairs can also be “pushed apart” (antonym-repel), and this is typically handled by negative or margin-based terms in the loss.

The form above unifies the retrofitting objective (from lexical resources) (Powell et al., 2020), the human-selected tight clusters (“set pull”), and per-pair/antonym repulsion (Yuan et al., 2019). On each user interaction, additional terms are appended or updated in this objective, making it inherently modular and extensible.

2. Human Interaction to Constraint Induction

Human-in-the-loop refitting systems expose users to the current geometry of the embedding space in one or more modalities:

• Ranked lists of nearest neighbors for a given term by cosine similarity.
• Low-dimensional visualizations (PCA, t-SNE, force-directed graphs) of neighborhood structure.

Users interact by:

• Selecting subsets of words to be considered semantically similar (grouping for set pull).
• Marking pairs to move closer (must-link) or farther apart (cannot-link/antonym-repel).
• Adjusting the strength parameter ($\gamma_k$, $\beta_{ij}$) for each constraint via UI sliders.

User actions are translated into new or updated quadratic penalties; for example, selecting $\{$"woman", "female", "she"$\}$ as a group with $\gamma = 0.5$ triggers an attraction constraint that minimizes the pairwise distances among these words (Powell et al., 2020). Systems typically accumulate the evolving user constraint set in a configuration structure (e.g., JSON), supporting live undo/redo, strength adjustment, and previews of embedding changes (Powell et al., 2021).

3. Optimization Algorithms and Computational Properties

The optimization induced by the loss $L$ is a positive-definite quadratic program with a unique global solution. Efficient closed-form coordinate update rules can be derived for each embedding vector $q_i$:

$$q_i \gets \frac{ \alpha_i \, \hat q_i + \sum_{j:(i,j)\in E} \beta_{ij}\, q_j + \sum_{k: i \in S_k} \gamma_k \sum_{j\in S_k,\,j\neq i} q_j } { \alpha_i + \sum_{j:(i,j)\in E}\beta_{ij} + \sum_{k: i \in S_k} \gamma_k(|S_k|-1) }$$

Iterations are repeated synchronously or in a block-parallel fashion until all updated vectors change by less than a set threshold in $\ell_2$ norm (e.g., $10^{-6}$). Empirically, convergence is fast—on the order of 20–50 iterations for vocabularies of size $|\mathit{V}| \approx 50$K (Powell et al., 2020).

For very large vocabularies, only a subgraph of embeddings within an $\ell$-hop neighborhood of the changed words is updated (“adaptive freezing”), improving computational efficiency and avoiding unnecessary drift elsewhere in the embedding space (Powell et al., 2020). For cross-lingual settings, group constraints and must/cannot-link annotations induce analogous update rules, typically solved with simple gradient descent restricted to the annotated subspace (Yuan et al., 2019).

4. User Interface Workflows and Interaction Patterns

The canonical interaction loop comprises four phases: Search → Navigate → Visualize → Adjust (Powell et al., 2021). After querying and inspecting, users impose corrective constraints, which are immediately reflected in the embedding space and visualizations:

• Nearest neighbor lists for affected terms are refreshed.
• Clustering structures in 2D projections (e.g., PCA) update in real-time.
• Mean pairwise distances within user groups are previewed as the constraint strength $\gamma_k$ changes.

Critical interface affordances include drag-and-drop group formation, sortable/filterable neighbor tables, group strength sliders, “undo/redo,” and “reset to original” controls, as well as compact visualizations of the global or neighborhood embedding structure (Powell et al., 2020).

5. Empirical Effects: Bias Mitigation and Semantic Realignment

Interactive refitting has demonstrated significant impacts on mitigating social biases and repairing semantic incoherence in pretrained embeddings (Powell et al., 2020, Powell et al., 2021). In bias-oriented case studies:

• Gender-relevant vectors “she,” “female,” “woman,” “women” can be pulled together as a group, reducing their mean $\ell_2$ spread from 0.78 (original) to 0.33 (after set-refitting), compared to 0.42 via pairwise-only correction.
• In a gender-neutral pronoun resolution task, downstream classifier accuracy improved from 78.3% (original) to 82.7% (after interactive refitting).
• The Word Embedding Association Test (WEAT) bias score $D$ can decrease from 1.02 to 0.45 following human-aided set refitting.

Tabular illustration of a science-domain semantic realignment (Powell et al., 2021):

Term	$\cos_{\mathrm{before}}$	$\cos_{\mathrm{after}}$
science	0.5929	0.8663
astronomy	0.5644	0.8379
biophysics	0.5623	0.8399
biology	0.5387	0.8428
biochemistry	0.5305	0.8389

This demonstrates a strong contraction of the discipline cluster, with $\approx$0.29 mean increase in pairwise similarity after only one targeted round of interactive refitting.

6. Extensions, Limitations, and Best Practices

Limitations include the high cognitive load placed on human users when constructing many or large groups, the risk of overfitting small groups with excessive constraint weights ($\gamma_k$), and potentially global drift in unrelated regions of the embedding space. Preventive guidelines recommend:

• Limit group size ($\leq$10 words).
• Restrict $\gamma_k$ to $[0,1]$ and provide preview for average group pairwise distance.
• Visualize the global embedding drift via PCA for top frequent tokens.

Extensions include antonym repulsion (negative- or margin-based penalties for pairs to be separated), crowd-sourced curation of group constraints, and adaptive freezing for scalable subspace updating (Powell et al., 2020, Yuan et al., 2019).

Recommended practices for practical deployment:

• Expose fine-grained controls (strength sliders, undo).
• Use bias-detection heuristics to suggest candidate groups.
• Batch constraints and annotate high-salience words first (especially for downstream performance impact) (Yuan et al., 2019).
• Monitor marginal gains and stop refinement on plateau.

7. Impact and Cross-Domain Applicability

Interactive refitting enables practitioners and domain experts to rapidly discover and address semantic quality and bias issues in word embeddings post hoc—without retraining on the underlying corpus (Powell et al., 2020, Powell et al., 2021). This methodology is agnostic to language and domain, operating effectively on monolingual, cross-lingual, general, and specialized vocabularies. In low-resource settings, selective human annotations—prioritized by classifier-driven salience—yield substantial gains in downstream task accuracy after relatively few pairwise judgments (Yuan et al., 2019).

A plausible implication is that the core design—tight human loop, constraint-driven optimization, and efficient online updates—can generalize not only to traditional word-level embeddings but also to other dense representation spaces, including entity, phrase, or cross-modal embeddings, wherever modular injection of expert knowledge is required to repair data-driven artifacts.