MAEM Maximally-Activating Example Metamodels ⏬

• MAEMs are specific metamodels that map internal system directions to examples designed to elicit strong internal responses, optimizing the generative representation and activation objectives.
• They integrate generative representations, activation objectives, and surrogate modeling, enabling customization and exploration of activations within subject systems for various applications such as language models, spiking neural networks, and safety-critical systems.
• Examples include generating text that activates specific concepts in language models, maximizing activation for neuron behaviors in spiking networks, and selecting safety-critical scenarios in autonomous vehicle testing.

Maximally-Activating Example Metamodels (MAEMs) are metamodels that map an internal direction, activation target, or behavioral objective to examples designed to produce a strong response in a subject system. In the most explicit formulation, a MAEM receives a residual-stream direction $v$ from a frozen LLM and generates text $x$ such that some subject-model activation $h_t(x)$ has high cosine similarity with $v$ (Schamphelaere et al., 5 Nov 2026). More broadly, MAEMs combine generative representations, activation objectives, surrogate modeling, and example-selection procedures to produce examples that do more than replicate an existing artifact: they support customization, exploration, interpretation, validation, and potentially innovation.

1. Conceptual foundations and terminology

The term metamodel has several related uses. In design theory, a metamodel is “a language to define another language and thus what it can represent.” In engineering, it contains sufficient information to generate a range of related models, or a family of designs. Such systems are also called configurators, toolkits, or codesign platforms; in Thingiverse, they are called customizers (Kyriakou et al., 2017).

A conventional model represents one completed artifact. A metamodel represents a generative space. In the Thingiverse setting, a model is a description of the surface geometry of one object, whereas a metamodel contains:

1. design parameters controlling variation;
2. a parameterized model or template;
3. an interface through which users manipulate parameters;
4. implicitly, a compiler or generation process that converts parameter values into a model.

The corresponding workflow is:

$$\text{user assigns parameter values} \rightarrow \text{template is populated} \rightarrow \text{compiler generates model} \rightarrow \text{interface displays model}.$$

A MAEM extends this generative conception from design artifacts to activation-oriented examples. Its output is not merely an explanation, reconstruction, or instance selected from a fixed corpus. It is an example intended to activate a target property in the system being studied. The target may be a residual-stream direction, an SAE feature, a steering vector, a LoRA payload, a safety-related behavior, a neuron, or a scalar output of an expensive simulator.

This distinguishes three processes of knowledge reuse. Reuse for replication applies an existing solution without broadly integrating it with other knowledge. Reuse for innovation modifies, recombines, or extends prior knowledge to produce novelty. Reuse for customization manipulates parameters of a metamodel to produce a model fulfilling personal or local requirements (Kyriakou et al., 2017). MAEMs are most closely related to reuse for customization and reuse for innovation: they expose a space of possible examples rather than a single finished exemplar.

The term maximally activating is stronger than highly reusable. A metamodel can produce many descendants without those descendants being useful, diverse, or semantically informative. In Thingiverse, users reported that customizers could produce “slightly different useless custom variations” and make search difficult. Consequently, activation must be distinguished from raw reuse, popularity, or derivative count.

2. Formal activation objectives

For a frozen subject LLM $M$, let $v\in\mathbb{R}^d$ be a supplied residual-stream direction, $x$ a generated text span, and $h_t(x)\in\mathbb{R}^d$ the subject-model residual activation at position $t$. A MAEM policy $\pi_\theta$ defines a conditional distribution:

$x\sim\pi_\theta(\,\cdot\mid v).$

The central activation-inversion score is:

$$s(v,x)=\max_{1\leq t\leq |x|} \cos\!\bigl(v,h_t(x)\bigr),$$

where

$\cos(a,b)=\frac{a^\top b}{\|a\|_2\|b\|_2}.$

The MAEM objective is therefore to produce an external example whose forward pass reaches a target internal direction. The generated text is scored by running the frozen subject model without conditioning injection. Conditioning is used only to provide the MAEM with the target direction.

In the Qwen3.6-27B experiments, the MAEM is initialized from the subject model, and $v$ is injected after the MAEM’s second transformer block using norm-matched addition:

$r'_m=r_m+\frac{\|r_m\|_2}{\|v\|_2}v.$

The resulting generated text is then processed by the unmodified subject model. Training therefore uses a label-free reward supplied by the subject model itself:

$R(v,x)=s(v,x).$

The training reward is computed over the last 16 generated tokens and includes a length penalty:

$$R_{\mathrm{total}}(v,x) = R(v,x) - 2.5\times10^{-4}\max(0,|x|-8).$$

A centered evaluation score can remove a shared mean component from residual activations:

$$s_\mu(v,x)= \max_t \cos\!\left(v-\mu,h_t(x)-\mu\right),$$

where $\mu$ is the archived training-time mean residual activation. Reinforcement learning uses the uncentered cosine, whereas centered cosine is used in many held-out inversion comparisons.

MAEM objectives can also be defined for other systems. For a neuron in a spiking neural network, the most-exciting image is:

$$\mathrm{MEI}_i = \arg\max_{x\in\mathrm{StimulusSpace}} \mathrm{Activation}_i(x).$$

In the spiking-network setting, activation is measured by the number or frequency of spikes over a prescribed interval. The optimization is performed in a generator latent space:

$z_i^\star=\arg\max_{z\in\mathcal Z}A_i(G(z)),$

with generated stimulus $x_i^\star=G(z_i^\star)$ (Pospelov et al., 2023).

For expensive simulation systems, a probabilistic metamodel can approximate:

$\widetilde{\mathcal M}:x\mapsto p(y\mid x),$

where $x$ is a parameterized scenario and $y$ is an output or safety metric. An acquisition function then selects examples that are critical, uncertain, near a behavioral boundary, or otherwise informative (Winkelmann et al., 2021). These systems provide the surrogate-model and sequential-design components of a MAEM methodology, although they do not themselves define maximally activating examples.

3. Architectures and generation mechanisms

Conditional generative MAEMs

The principal MAEM implementation uses a full-parameter fine-tune initialized from Qwen3.6-27B. It is trained in two stages. Supervised fine-tuning uses paired directions and associated texts, including corpus activations, SAE encoder directions, and SAE decoder directions. This establishes a conditional generator before activation-based reinforcement learning.

The second stage uses label-free RL. For each direction, the policy generates multiple rollouts, computes subject-model activation alignment, centers advantages within the rollout group, and updates the policy using the CISPO policy loss. The RL pool contains 941,132 directions, consisting of corpus activations, SAE encoder directions, and SAE decoder directions. Training uses eight rollouts for each of 2,048 directions per update, 16,384 rollouts per step, temperature $1.0$, 8–192 generated tokens, 300 RL steps, and no KL regularization (Schamphelaere et al., 5 Nov 2026).

This architecture amortizes activation maximization. Instead of scanning a corpus or solving a new optimization problem for every direction, the MAEM learns a reusable conditional mapping from directions to examples. The output remains verifiable because it can be passed through the subject model and scored directly.

Generator-constrained activation maximization

For spiking neural networks, MANGO performs gradient-free activation maximization in the latent spaces of SN-GAN and VQ-VAE generators. The SNN is treated as a black-box evaluator because its spike function is nondifferentiable and its temporal dynamics complicate direct optimization. PROTES represents a probability distribution over discretized latent coordinates in Tensor Train form and adaptively concentrates sampling around high-scoring configurations (Pospelov et al., 2023).

A dense $d$-dimensional tensor requires $\prod_{k=1}^d n_k$ entries, whereas a Tensor Train representation requires approximately:

$\sum_{k=1}^{d}n_kr_{k-1}r_k$

parameters. For uniform mode size $n$ and rank $r$, the representation has approximately $O(dnr^2)$ parameters rather than $O(n^d)$. The TT representation is therefore a computational surrogate for the activation landscape or for the sampling distribution; it is not necessarily a semantic metamodel of the neuron.

Probabilistic surrogate metamodels

For automated-vehicle scenario testing, Gaussian processes, deep Gaussian processes, extra-trees, and Bayesian neural networks approximate expensive system evaluations. The metamodel predicts a mean $\mu(x)$ and uncertainty $\sigma(x)$, often through a predictive distribution:

$$y\mid x,\widetilde{\mathcal M} \sim \mathcal N\bigl(\mu(x),\sigma^2(x)\bigr).$$

The associated acquisition functions include boundary identification:

$$a_{\mathrm{BI}}(x) = 1-2\left|P_{\mathrm c}(x)-0.5\right|,$$

importance sampling:

$a_{\mathrm{IS}}(x)=P_{\mathrm c}(x)p(x),$

and uniform exploration:

$a_{\mathrm{ES}}(x)=1.$

The method begins with Sobol sampling, executes concrete scenarios, trains a metamodel, constructs a sampling distribution from the acquisition function, and iterates. Flexible models such as BNNs are useful for broad exploration of complex behavior, whereas GPs are more conservative and reliable for targeted validation (Winkelmann et al., 2021).

EML-type and rational metamodels

A separate architectural direction uses the EML operator:

$\mathrm{eml}(x,y)=\exp(x)-\ln(y).$

The related activation

$F(z)=\mathrm{eml}(z,1/z)=e^z+\ln z$

can, under suitable branch restrictions and nonzero denominators, recover $e^z$ and then $\ln z$ through rational combinations of $F(z)$, $F(2z)$, and $$F(3z). The proposed Möbius layer replaces affine transformations with ratios of affine forms:$$ z_j(x)= \frac{a_j^\top x+b_j}{c_j^\top x+d_j}, \qquad h_j=F(z_j).

$This provides a compact analytic design space for metamodels, but the architecture’s optimization properties have not been established. Exact symbolic expressivity does not imply trainability, numerical stability, semantic usefulness, or reliable activation maximization [2609.11210]. ## 4. Training, search, and validation A MAEM pipeline generally contains four stages: target specification, conditional generation or surrogate fitting, candidate search, and subject-system validation. The target may be a residual direction, a neuron, an SAE feature, a probe direction, a safety threshold, or a scalar activation statistic. For an LLM activation direction $u$, a point target can be defined as:$

J(x)= \mathbb E_{s\sim\mathbb P_D} \left[g(A_\ell(x,s))\right],

$where$A_\ell(x,s)$is a pooled answer activation and$g$maps it to a scalar. For a linear target$g(a)=u^\top a$:$

J(x)= u^\top\mathbb E_s[A_\ell(x,s)].

$$A context–answer metamodel forecasts this quantity from context activations before the first answer token. It can predict the conditional mean$$\mu_\ell(x)$or the rollout-induced distribution$P_\ell^x$$[2611.06837]. The principal point predictors are linear ridge and MLP models; distribution predictors are Gaussian and conditional-flow models. At 500,000 training prompts, the reported pooled$$R^2$values were: | Model | Representation |$R^2$$| |---|---|---:| | Ridge | Final context token | 0.7844 | | MLP | 32-bin context trace | 0.8739 | | MLP | Full context trace | 0.8808 | | Flow | 32-bin context trace | 0.8517 | | Flow | Full context trace | 0.8599 | These results establish forecasting rather than optimization. A MAEM search layer must rank or generate candidates using the forecast and then validate selected candidates with actual rollouts. Candidate acquisition can use predicted-mean ranking, uncertainty-aware upper confidence bounds, expected improvement, diversity penalties, or Pareto objectives. For a direction$$u$and Gaussian forecast covariance$\widehat\Sigma(x)$:$

\operatorname{UCB}_\beta(x)

u^\top\widehat\mu(x) + \beta\sqrt{u^\top\widehat\Sigma(x)u}.

$For a finite candidate pool, direct ranking is the safest initial procedure. Other candidate spaces include prompt templates, demonstration subsets, evolutionary mutations, retrieval neighborhoods, constrained language-model generation, and continuous prompt embeddings. Continuous optimization risks producing off-manifold embeddings without valid textual realizations and therefore requires projection and forward-pass validation. The final stage is validation on the original subject system. A candidate is not accepted merely because a surrogate predicts a high score. It must be executed, rescored, and compared with its forecast. Active-learning correction adds validated candidates to the training or calibration data, particularly when optimization enters regions unlike the original training distribution. ## 5. Empirical evidence and applications ### Residual-stream interpretation In the Qwen3.6-27B experiments, MAEM best-of-1 activation inversion achieved centered cosine $0.759$, compared with$0.572$for corpus search using 10 million tokens and$0.612$for NLA best-of-2. MAEM best-of-8 reached$0.822$, and best-of-64 reached$0.850$$[2611.08465]. A single MAEM sample exceeded the corpus-search result on 92$$0.42$ centered cosine. The model generalized beyond its English web-text training setting. It outperformed 10-million-token search in the target domain’s own corpus on 12 of 21 unseen language and domain arms, including Chinese, Czech, Russian, Arabic, Mandarin, Japanese, Python, JavaScript, C, Rust, OpenWebMath, and arXiv. It was inconclusive on Greek and Shell, while corpus search was better on Hindi, Thai, Go, Haskell, SQL, Lean, and Isabelle. MAEMs also recovered concepts behind steering vectors, payloads of rank-1 LoRA backdoors, and some model-workspace content. For 334 AxBench concepts, MAEM achieved 95.8% identification, compared with 92.8% for NLA, 85.3% for J-lens, and 61.4% for corpus search. For 119 BiPO persona and behavior vectors, MAEM achieved 83.7%, while NLA achieved 85.7%. ### SAE feature activation On 512 held-out features of a 131k-feature SAE, best-of-8 MAEM examples fired 89% of features above threshold, reached at least half the corpus peak for 78%, exceeded the 10-million-token corpus peak for 31%, and exceeded the corpus peak for 30% under the per-feature expected-best statistic. Best-of-64 firing rose to approximately 94%. These results show that cosine alignment is related to feature activation, but alignment does not guarantee a complete semantic interpretation. Features may be polysemantic, positional, formatting-sensitive, or dependent on structural context unavailable from a direction alone. ### Spiking-neural-network activation maximization MANGO’s TT-based methods were approximately two to four times faster than the Nevergrad benchmark and performed as well as or approximately 10–20% better than the best non-gradient baseline in target activation. SN-GAN produced higher-activation and more interpretable MEIs than VQ-VAE. The gap was approximately 50% in some early layers and narrowed to approximately 5% in deeper layers [2401.10748]. The experiments also found that highly selective neurons can emerge in early layers, while deeper layers often exhibit greater diversity of high-activation stimuli. These results support the use of generator-constrained, gradient-free search when the target system is nondifferentiable or available only as a black-box evaluator. ### Open design and customization Thingiverse provides an empirical precursor to MAEMs. In a dataset of 24,173 designs from 8,079 designers, metamodels were reused more often than ordinary models, while generated models were less likely to receive further reuse. Metamodels created by more experienced designers exhibited amplified reuse [1702.08072]. The results support a distinction between a generative example and one of its outputs. A generated object embodies specific parameter choices and is therefore often valuable primarily to its creator. The metamodel remains abstract, parameterized, and reusable by users with different needs. Metamodels can also be extended or recombined, producing new families of models. ### Safety and scenario testing Probabilistic metamodels for automated-vehicle testing demonstrate how acquisition strategies can select informative or safety-critical examples from continuous scenario spaces. BNNs achieved the highest global likelihood in three of four principal scenarios and generally achieved the best overall RMSE with increasing data volume. GPs and DGPs often achieved comparable or better RMSE despite lower likelihoods, while extra-trees provided comparatively good calibration [2110.02892]. Targeted sampling often mattered more for task-specific performance than surrogate choice. GP, DGP, and BNN models reached precisions at 90% recall that increased the observed frequency of critical cases by two or more orders of magnitude in all scenarios. These results motivate a MAEM design in which the acquisition function, rather than the surrogate alone, determines which examples are generated and validated. ## 6. Limitations, controversies, and research directions A high activation score is not equivalent to a correct semantic interpretation. An example may activate a direction through an incidental, positional, formatting-related, or degenerate pathway. A MAEM can therefore be a grounded exemplifier without being a complete semantic decoder. The central technical limitation is surrogate exploitation. A search procedure can discover candidates that score highly under a forecasting model but fail when evaluated by the subject system. This is especially likely under distribution shift, because optimization deliberately moves toward the tails of the candidate distribution. Validation with the original system, active-learning correction, ensembles, uncertainty penalties, and candidate-distribution constraints are therefore necessary. Uncertainty estimates are not uniformly reliable. Context–answer distribution metamodels captured rollout variability but were generally overdispersed, especially out of distribution. In cross-model experiments, mean forecasts transferred reasonably well while variance forecasts were poor. A model can therefore rank candidates accurately on average while being unreliable about rare activation excursions. Diversity is another unresolved issue. The MAEM objective rewards a highly aligned example but does not inherently reward coverage of the full set of contexts that activate a direction. MAEM samples had a Vendi score of approximately 2.2, compared with 2.9 for corpus top-eight examples, 3.4 for an LLM writing from corpus windows, and 2.0 for NLA outputs [2611.08465]. Low diversity limits recall and can cause a MAEM to repeatedly express one narrow facet of a feature. The direction itself may underdetermine the desired example. Failures were concentrated in rare features, whitespace-position features, formatted-digit features, repeated-trigram or induction features, list and table features, template-position features, and cross-document-context features. These features can be reachable in the corpus while remaining difficult to activate from the direction alone. Generator constraints introduce a related limitation. In SNN activation maximization, the optimizer can only produce stimuli in the support of SN-GAN or VQ-VAE. A low activation may indicate either weak neuronal selectivity or an inability of the generator to express the relevant stimulus. In EML-type architectures, domain restrictions, branch cuts, singular denominators, overflow, underflow, and complex intermediate values create analogous constraints. Formal expressivity also does not imply practical expressivity. EML can represent a broad exp-log-style function language, but reported depth-three symbolic-regression experiments had recovery rates from 0% to 100%, depending on architecture and training protocol. Thus:$

\text{representational capacity} \neq \text{reachable capacity under training}.

$$Future MAEM research is likely to require explicit objectives for diversity, semantic validity, robustness, coverage, and causal behavioral effect. A formal set-valued MAEM could represent a distribution of high-response examples rather than a single output:$$

\mathcal E_v

\left{ x: x\sim q_v,\; s(v,x)\geq\tau_v \right}, $$ captures variation among valid activating contexts. Candidate selection could jointly optimize activation, uncertainty, novelty, and validity. For example, a constrained acquisition function could combine predicted activation with uncertainty and distribution-shift penalties.

A complete evaluation should therefore report activation fidelity, top-$k$ recall, activation regret, calibration, diversity, coverage, robustness under perturbations, false-positive and false-negative rates, semantic or behavioral validity, and computational cost. It should also distinguish whether the target is average activation, a reliable activation increase, a rare high-activation rollout, an activation probability, or high variability.

MAEMs are consequently best understood as a family of grounded, generative, and potentially probabilistic metamodels. Their defining property is not merely that they produce examples associated with a concept, but that the generated examples can be executed on the target system and verified to induce the intended activation. This connects open design metamodels, active-learning surrogates, gradient-free activation maximization, analytic operator architectures, context–answer activation forecasting, and mechanistic interpretability within a common framework: the construction of generative examples whose downstream effects are measurable, optimizable, and subject to direct validation.