Feature Activation Coverage (FAC)

• Feature Activation Coverage (FAC) is a model-aware measure that quantifies data diversity by evaluating the activation of latent, task-relevant features rather than superficial text variations.
• It leverages a sparse autoencoder to derive an interpretable feature space, guiding a two-step synthesis process that efficiently generates data to cover missing internal features.
• FAC demonstrates strong empirical correlations with enhanced performance across tasks like toxicity detection and instruction following, and facilitates cross-model transfer of learned features.

Feature Activation Coverage (FAC) is a model-aware notion of data diversity for LLMs that measures whether a synthetic dataset activates the internal, task-relevant features of an LLM rather than merely exhibiting surface-level lexical or semantic variation. It was introduced in the context of post-training data synthesis, where the central claim is that standard text-based diversity metrics such as Distinct-n, n-gram entropy, POS diversity, embedding cosine distance, and semantic entropy often provide weak signals for the latent behavioral concepts that determine downstream performance. FAC instead evaluates diversity in an interpretable sparse feature space derived from model activations, and it serves both as a metric and as the organizing principle of an overview pipeline designed to generate data that covers features missing from a seed dataset (Li et al., 11 Feb 2026).

1. Conceptual basis

FAC is motivated by a mismatch between text diversity and task-relevant diversity. Prior metrics largely measure variation “in the text itself,” but two prompts can look linguistically diverse while covering the same latent behavior, and two nearly identical prompts can activate very different internal features. FAC is designed to address this by asking whether a dataset covers the model’s own task-relevant feature space rather than whether it varies at the surface form level (Li et al., 11 Feb 2026).

The relevant unit in FAC is not the token, sentence, or embedding as such, but an interpretable feature discovered inside the LLM. A feature is considered meaningful insofar as it corresponds to a latent behavioral concept that can be identified from activation patterns. This suggests that FAC operationalizes diversity as coverage over internal behavioral structure rather than over observable textual heterogeneity.

The paper situates FAC in a data-centric optimization framework for LLM post-training. Its practical takeaway is that a relatively small amount of carefully synthesized, feature-covered data can outperform much larger generic synthetic corpora—sometimes by orders of magnitude in sample efficiency—if synthesis is guided by the right internal features rather than surface text diversity (Li et al., 11 Feb 2026).

2. Sparse feature space and formal definition

FAC is defined in a sparse autoencoder (SAE) feature space. Given an LLM activation vector $x \in \mathbb{R}^d$, the SAE encoder produces a sparse code

$z = \sigma(xW) \in \mathbb{R}^k,$

and reconstructs

$\hat{x} = zW,$

with $W \in \mathbb{R}^{d \times k}$ and $k \gg d$. The SAE objective is

$$L_{\text{SAE}} = \lVert x - \hat{x} \rVert_2^2 + \lambda \lVert z \rVert_1.$$

In the reported implementation, a Top-K SAE keeps only the $K$ largest activations to encourage interpretability (Li et al., 11 Feb 2026).

For a sequence $X=(x_1,\dots,x_T)$, token-level SAE activations $Z(X)$ are max-pooled over positions after the chat-template prefix:

$g_i(X)=\max_{t \ge t_0} Z_i(X,t).$

This yields a fixed-length feature vector $z = \sigma(xW) \in \mathbb{R}^k,$0. A feature is treated as activated by a sample if its pooled activation exceeds a threshold $z = \sigma(xW) \in \mathbb{R}^k,$1:

$z = \sigma(xW) \in \mathbb{R}^k,$2

The task-relevant feature set $z = \sigma(xW) \in \mathbb{R}^k,$3 is identified by human/LLM annotation of the top activating text spans for each SAE feature. For a target distribution $z = \sigma(xW) \in \mathbb{R}^k,$4, estimated from an anchor corpus, and a synthetic distribution $z = \sigma(xW) \in \mathbb{R}^k,$5, FAC is defined through the active task-relevant feature sets

$z = \sigma(xW) \in \mathbb{R}^k,$6

$z = \sigma(xW) \in \mathbb{R}^k,$7

and the coverage ratio

$z = \sigma(xW) \in \mathbb{R}^k,$8

The missing features are

$z = \sigma(xW) \in \mathbb{R}^k,$9

In practice, FAC is computed by extracting SAE activations for a large anchor set and for the synthetic set, thresholding activations with $\hat{x} = zW,$0, and checking which task-relevant features appear at least once. The threshold is consequential: the paper studies $\hat{x} = zW,$1, reports that moderate thresholds improve quality by filtering weak or noisy activations, and notes that overly large thresholds make the feature set too sparse and hurt performance (Li et al., 11 Feb 2026).

3. FAC Synthesis

FAC is paired with a generation framework named FAC Synthesis. Its objective is to synthesize data that explicitly activates the missing feature set $\hat{x} = zW,$2. The pipeline has two stages.

First, for each missing feature $\hat{x} = zW,$3, the method constructs a contrastive pair: a positive example that strongly activates the feature and a negative example that activates it weakly. This is done by prompting a generator with a feature description $\hat{x} = zW,$4, sampling candidates, and scoring them by SAE activation $\hat{x} = zW,$5. Second, the method builds a contrastive prompt $\hat{x} = zW,$6, samples additional candidates, filters them by whether $\hat{x} = zW,$7, and keeps top-ranked samples. Aggregating over all missing features yields the final synthetic dataset

$\hat{x} = zW,$8

This two-step process is presented as important because it lowers uncertainty in the synthetic set. The paper argues, via a PAC-Bayesian analysis, that reducing the entropy or variability of the generated data helps reduce sampling error. In that account, contrastive guidance makes the generator more likely to express the desired feature reliably rather than producing noisy off-target samples (Li et al., 11 Feb 2026).

Ablation results reinforce the role of the synthesis design. Increasing the proportion of covered missing features monotonically improves performance, whereas simply increasing the sample count with fixed feature coverage yields much smaller gains. The two-step synthesis procedure also consistently achieves higher FAC than one-step prompting and better downstream results. This supports the paper’s claim that task-relevant feature coverage, not merely raw data volume or surface diversity, drives post-training gains (Li et al., 11 Feb 2026).

4. Generalization, correlation with performance, and data efficiency

The theoretical argument for FAC connects feature coverage to generalization. The paper derives an upper bound on the post-training generalization error and then argues that the distribution gap can be controlled in SAE feature space by reducing the divergence of $\hat{x} = zW,$9 from $W \in \mathbb{R}^{d \times k}$0 over $W \in \mathbb{R}^{d \times k}$1 and $W \in \mathbb{R}^{d \times k}$2. The stated intuition is that increasing feature coverage reduces the mismatch between the task distribution and the synthetic distribution (Li et al., 11 Feb 2026).

Empirically, FAC is reported to be strongly linked to downstream performance across four tasks: toxicity detection, reward modeling, behavior steering, and instruction following. On toxicity detection, FAC correlates with AUPRC at Pearson $W \in \mathbb{R}^{d \times k}$3 and Spearman $W \in \mathbb{R}^{d \times k}$4; similar strong correlations are reported on the other three tasks. Figure-level comparisons show that methods with higher FAC tend to perform better, whereas standard text diversity metrics correlate weakly or inconsistently with performance (Li et al., 11 Feb 2026).

The paper also reports a data-efficiency score,

$W \in \mathbb{R}^{d \times k}$5

in another ablation context, emphasizing efficiency. FAC itself, however, is the feature coverage ratio rather than an efficiency metric. A plausible implication is that FAC and DES play different analytical roles: the former measures coverage over task-relevant latent features, while the latter summarizes efficiency relative to trainable scale.

5. Shared feature spaces and cross-model transfer

A notable result is the identification of a shared, interpretable SAE feature space across the model families LLaMA-3.1-8B-Instruct, Mistral-7B-Instruct, and Qwen2-7B-Instruct. Synthetic data built from features discovered in one model transfers to the others and improves performance across model families. In toxicity detection, training on shared synthetic data improves all three backbones, and the paper reports a weak-to-strong transfer effect: features extracted from the weaker LLaMA model can still improve the stronger Qwen model, sometimes more than Qwen’s own features (Li et al., 11 Feb 2026).

This result is significant because it suggests that the discovered SAE features capture transferable task structure rather than merely model-specific quirks. The paper presents this as evidence that FAC is not restricted to idiosyncratic internal codes of a single backbone. This suggests a broader use of feature-space-guided synthesis as a cross-model knowledge transfer mechanism in post-training regimes.

The cross-model finding also constrains how FAC should be interpreted. FAC is not simply a score over arbitrary latent coordinates; it depends on a feature basis that is sufficiently stable and interpretable to support transfer. The strength of transfer reported across LLaMA, Mistral, and Qwen is therefore central to the claim that FAC is practically actionable rather than only descriptive (Li et al., 11 Feb 2026).

6. Relation to adjacent activation-based notions and limitations

FAC should be distinguished from several adjacent constructs that also operate at the level of internal activations but solve different problems. In metric learning, Similar Feature Activation Map (SFAM) assigns channel-wise contribution importance scores to explain similarity between two image embeddings and then visualizes weighted spatial activations. The paper introducing SFAM explicitly states that it does not define a metric named Feature Activation Coverage and does not formulate a coverage ratio over activated channels or regions; it is a feature-level attribution method rather than a coverage metric (Liao et al., 2 Jun 2025).

In deep neural network testing, DeepFeature shifts attention from neurons to feature maps and introduces Feature map Attack Score (FAS) and Feature map Vulnerability Score (FVS). It likewise does not define FAC. Its operational unit is the vulnerable feature map, and its prioritization mechanism is ranking-based rather than threshold-based. The reported distinction is that it uses feature-map sensitivity to perturbation rather than a proportion of task-relevant features covered by data (Huang et al., 2023).

In Android GUI testing, CovAgent is described as FAC-like because it turns previously unreachable activities or features into reachable ones by inferring activation conditions and generating dynamic instrumentation scripts. The paper explicitly states that it does not introduce the term “Feature Activation Coverage” as a formal metric name. Its notion of coverage is activity coverage, together with class, method, and line coverage, with activity coverage functioning as a proxy for feature reachability (Minn et al., 29 Jan 2026).

The FAC formulation for LLMs also comes with explicit caveats. The method depends on the quality of the SAE and on feature interpretation. Sophisticated reasoning features may be distributed across multiple SAE layers, making them harder to capture with a single sparse feature basis. FAC is only as good as the chosen threshold and the human/LLM feature labeling process. The paper further notes that the method can be misused to amplify harmful content because it explicitly targets latent safety-related features, and it recommends filtering and human oversight (Li et al., 11 Feb 2026).