--- title: Exemplar Partitioning (EP) Overview url: https://www.emergentmind.com/topics/exemplar-partitioning-ep type: topic --- # Exemplar Partitioning (EP) Overview Exemplar Partitioning (EP) denotes a class of unsupervised methods that construct hard, non-parametric partitions of high-dimensional data by selecting a subset of observed data points, called exemplars, to serve as anchors for “Voronoi” regions. Each data point is assigned to its nearest exemplar according to a task-appropriate geometry or similarity. EP is notably leveraged for interpretable feature discovery in deep neural activations, including mechanistic analysis of large language models, and for exemplar-based clustering with flexible, nonparametric priors on cluster structure. EP produces directly comparable, anchor-based dictionaries for analysis and intervention, requires no gradient optimization, and, when applied to model activations, achieves comparable interpretability and causal utility to sparse autoencoders at orders-of-magnitude lower computational cost [2605.14347][1206.3294]. ## 1. Mathematical Construction and Algorithmic Specification EP partitions the input space $\mathbb{R}^d$ through a sequential, distance-thresholded leader clustering. Consider a stream of activation vectors $a \in \mathbb{R}^d$, where each $a$ denotes the (possibly normalized) activations at a particular model layer and token position. EP proceeds as follows: 1. **Centering & Normalization:** The data mean $\mu \in \mathbb{R}^d$ is computed on a calibration set. Each activation is mapped to the unit sphere: $$\phi(a) = \frac{a - \mu}{\|a - \mu\|_2}$$ so that the dictionary is formed over normalized directions. 2. **Leader Clustering:** For each normalized activation $u = \phi(a)$ streamed in, compute its minimum Euclidean distance from the current exemplar set $E$. If $\min_{e \in E} \|u-e\|_2 > \tau$ (where $\tau$ is a distance threshold), create a new exemplar $u$; otherwise assign $u$ to the closest existing $e^*$. In concise pseudocode: ```python Input: stream A, center μ, threshold τ Initialize E = {} for a in A: u = (a - μ) / ||a - μ||_2 if E is empty: add u to E else: e* = argmin_{e in E} ||u - e||_2 if ||u - e*||_2 > τ: add u to E else: assign u to e* return E ``` 3. **Voronoi Dictionary:** Each $e \in E$ defines a region $V_e = \{a : \|\phi(a)-e\|_2 \le \tau\}$; the collection $\{V_e : e \in E\}$ partition the unit sphere into encoder-determined cells. 4. **Threshold Calibration:** The cluster threshold $\tau$ is set as the $p$-th percentile of pairwise distances over $M$ calibration activations. Reporting the percentile $p$ (e.g., $p_1$ for the $1^\text{st}$ percentile) normalizes cluster resolution across models, layers, and datasets. **Emergent dictionary size** is determined by activation geometry at fixed $\tau$, with the process terminating at batch “saturation”—when a full batch yields no new exemplars. ## 2. Probabilistic and Prior-Driven Extensions EP is generalized by coupling with nonparametric priors on partitions, notably Dirichlet process (DP) or Pitman–Yor priors, as a framework for flexible exemplar-based clustering [1206.3294]: - Let $N$ data points $X = \{x_1,\dots,x_N\}$. Each cluster is defined by its exemplar, and each point is assigned to one exemplar. - The generative model is: $$ P(C, E, X|\alpha, G_0) = P(C|\alpha) P(E|C) P(X|C,E,G_0) $$ where $P(C|\alpha)$ is the partition prior (e.g., DP), $P(E|C)$ enforces one exemplar per non-empty cluster, and $P(X|C,E,G_0)$ specifies emission from exemplars. - The DP prior allows the number of clusters to be determined adaptively, $$ P(C|\alpha) = \frac{\Gamma(\alpha)}{\Gamma(N+\alpha)}\,\alpha^K \prod_{k=1}^K (N_k-1)! $$ with $\alpha > 0$ the concentration parameter, $K$ clusters and cluster sizes $N_k$. - *MAP inference* is solved via max-product belief propagation on a structured factor graph over the $N^2$ assignment variables, with computational cost $O(TN^2 \log N)$ for $T$ rounds of message passing and $O(N^2)$ space [1206.3294]. - Flexible priors $P(C) \propto \prod_k w(N_k)$ encode different cluster-size behaviors, e.g., Pitman–Yor or power-law. ## 3. Mechanistic Interpretability in Model Activations EP directly supports mechanistic interpretability in deep models by constructing feature dictionaries aligned to observed activation geometry [2605.14347]: - **Exemplar Anchoring:** Each region anchor is a true observed activation. Thus, dictionaries constructed from the same stream are directly comparable across layers, training checkpoints, or model variants. - **Cross-Checkpoint Comparison:** Matching exemplars across model checkpoints (e.g., base vs. instruction-tuned) via Hungarian algorithm and cosine similarity reveals which activation directions persist across fine-tuning. For example, in Gemma-2-2B, only a small fraction of high-cosine matches ($\cos \ge 0.7$) survived across checkpoints, implying that most activation geometry is re-anchored by fine-tuning. - **Intervention Experiments:** Projecting activations off an exemplar associated with a specific behavior (such as refusal in instruction-tuned LLMs) can causally suppress that behavior (e.g., baseline refusal 0.98 drops to 0.02 upon ablation of the corresponding region anchor, a difference $\Delta = -0.96$). - **Quantitative Feature Alignment:** EP regions exhibit partial overlap with sparse autoencoder (SAE) features: $\sim20\%$ of EP regions match an SAE feature at $F_1 > 0.5$, with mean $F_1 \approx 0.34$ at $p_{10}$. Conversely, only $0.3\%$ of SAE features match an EP region at the same threshold, with higher coverage at finer percentile granularity. - **One-Hot Probe Accuracy & AUROC:** Encoding activations into EP one-hot sparse codes preserves $\sim98\%$ of linear probe accuracy compared to using raw activations. For latent concept detection (AxBench), EP at $p_1$ achieves mean AUROC $0.881$, exceeding standard SAE ($0.755$) and closely approaching label-supervised SAE-A ($0.911$). - **Out-of-Distribution Signal:** The nearest-exemplar distance serves as a free measure of distributional shift; activations on random or under-represented inputs display significantly greater mean distance to nearest anchor than in-distribution samples. ## 4. Comparative Analysis: EP vs Sparse Autoencoders and Other Methods EP and sparse autoencoders (SAEs) impose fundamentally distinct geometric constraints on learned representations: | Feature | Exemplar Partitioning (EP) | Sparse Autoencoders (SAE) | |-------------------------|------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------| | Partition geometry | Hard Voronoi cells (unit sphere, L2) | Linear coding, soft selection | | Anchor type | Observed activations | Learned weights | | Dictionary size | Emergent, dictated by threshold and data geometry | Prespecified, e.g., $K=16$k | | Compute requirements | Single-stream, zero backward passes (~$10^5$–$10^7$ activations) | Millions–billions of tokens, backpropagation, many gradient steps | | Inter-dictionary match | Direct comparison across layers, models, or checkpoints | No inherent cross-model alignment | | Shared coverage | $\sim20\%$ of EP regions match an SAE feature at $F_1 > 0.5$ | SAE features more fragmented, less likely to coincide with EP at coarse resolution | | Interpretability | Region cause directly intervenable (projection off exemplar collapses associated features) | Demands indirect or aggregate intervention | EP efficiently yields dictionaries with $K = 200$ to $20{,}000$ with no learned parameters, compared to SAE's resource-intensive requirement for explicit gradient optimization and fixed basis size. This suggests EP is $\sim10^3\times$ more token-efficient for unsupervised feature generation at comparable interpretability [2605.14347]. ## 5. Computational and Statistical Properties - **Streaming and Online Construction:** EP dictionaries are constructed in a single forward pass, suitable for streaming and online adaptations. - **Emergent Resolution and Stopping Condition:** Dictionary growth halts after a batch produces no new exemplars; size reflects intrinsic density and dispersion of activation space as parametrized via $\tau$. - **Prior-Driven Clustering:** By incorporating flexible priors (DP, Pitman–Yor, etc.) over partitions, one can bias cluster numbers and size profiles appropriate to the task [1206.3294]. The parameter $\alpha$ in DP controls expected number of regions, and the prior can avoid pathologies of vanilla affinity propagation (such as poor modeling of heterogeneous cluster-size distributions). - **Computational Complexity:** For prior-based EP with affinity propagation and max-product message passing, main computational costs are $O(T N^2 \log N)$ time ($T$ iterations), $O(N^2)$ space. ## 6. Practical Considerations and Example Use Cases - **Activation Geometry and Model Analysis:** EP is now foundational in activation-space analysis of LLMs, supporting direct, cross-comparable, and interpretable region dictionaries for feature tracing, intervention, and model-family studies [2605.14347]. - **Cluster-Size Control and Priors:** In applications where knowledge of the cluster-size profile is available or desired, prior-based EP can be controlled by $\alpha$ (or other hyperparameters), tuned via likelihood or empirical Bayes. - **Image Segmentation:** When applied to image superpixel graphs, DP-EP yields segmentations that reflect true underlying structure (e.g., avoids oversegmentation seen in unregularized methods). - **Resource Efficiency:** As an unsupervised dictionary discovery method, EP has become especially attractive where compute and data limitations preclude large-scale gradient optimization, while offering high accuracy and direct interpretability with minimal cost. ## 7. Impact and Future Directions EP bridges the gap between interpretable dictionary learning and scalable, resource-efficient partitioning of high-dimensional model activations and general data. Its anchoring in observed activations makes cross-layer, cross-model, and cross-checkpoint comparisons tractable. Prior-based formulations provide powerful flexibility in shaping the solution space and statistical properties of the clusters, enabling a wide range of applications beyond interpretability, including unsupervised representation learning, anomaly detection, and clustering with domain-informed structure [2605.14347][1206.3294]. Continued research seeks to further improve the scalability of inference in prior-driven EP, expand the interpretability and intervention toolkit enabled by region-anchored dictionaries, and formalize the conditions under which Voronoi/cone and linear/subspace-based features yield convergent or divergent decompositions of latent space. Source: https://www.emergentmind.com/topics/exemplar-partitioning-ep