---
title: Non-Trainable Modification (NTM) in Neural Models
url: https://www.emergentmind.com/topics/non-trainable-modification-ntm
type: topic
---

# Non-Trainable Modification (NTM) in Neural Models

Searching arXiv for papers on "Non-Trainable Modification" and related NTM usages to ground the article in current literature.
Non-Trainable Modification (NTM) denotes a class of model designs in which a consequential part of the computational structure is fixed rather than optimized, so that learning or control is concentrated in the remaining trainable components. In the literature considered here, the label is used both for a graph-guided neural architecture for stochastic differential games and for a broader inference-time behavior-steering paradigm in large language models; related work in neural rendering and graph representation learning operationalizes the same principle through non-trainable images, non-parametric projection, and non-trainable spectral clustering [2509.12484] [2502.11096] [2305.10579] [2211.14425].

## 1. Terminological scope and acronym overload

The label “NTM” is not unique across recent arXiv literature. In some work it explicitly denotes **Non-Trainable Modification**; in other work it denotes unrelated constructs.

| Paper | Meaning of “NTM” | Domain |
|---|---|---|
| [2509.12484] | Non-Trainable Modification | Nash equilibria in stochastic differential games on graphs |
| [2502.11096] | Non-Trainable Modification | Inference-time behavior modification in MoE LLMs |
| [2305.10579] | Non-trainable representation | Neural radiance fields |
| [2211.14425] | Non-trainable clusters / patches | Graph representation learning |
| [2605.08078] | Normalizing Trajectory Models | Few-step image generation |
| [2510.03174] | Neural Topic Models (“NTMs”) | Topic modeling |

This usage pattern suggests that “NTM” must be interpreted from domain context rather than treated as a standardized acronym. In the non-trainable-modification sense, the common thread is not the elimination of learning, but the insertion of fixed structure—graph masks, projections, images, or cluster assignments—into an otherwise learnable pipeline.

## 2. Core design principle: fixed structure with selective learning

The defining mechanism of NTM is the use of non-trainable components that encode prior structure directly into the model. In the graph-based architecture of stochastic differential games, this takes the form of **graph-guided sparsification**: a weight \( W_{pr,q}^{(k)} \) is present, and trainable, **if and only if** \( v_q \) is a neighbor of \( v_p \); otherwise it is fixed, often zero, and non-trainable. The forward propagation rule is local:
$$
z^{(k+1)}_p = \sum_{r=1}^{M} g_{pr}^{(k)} \odot \sigma \Big(
W_{pr,p}^{(k)} z_p^{(k)}
+
\sum_{q: v_q \in \mathcal{N}_G(v_p)} W_{pr,q}^{(k)} z_q^{(k)} + h^{(k)}_{pr}
\Big) + b^{(k)}_p.
$$
The resulting parameter count scales with the edge set \(|E|\) rather than \(N^2\), which is especially relevant for large sparse graphs [2509.12484].

In MultiPlaneNeRF, the non-trainable element is the scene representation itself: a set of fixed 2D images \(I_1,\dots,I_n\) from known viewpoints. For a 3D point \(p=(x,y,z)\), the model projects \(p\) onto each image using camera intrinsics and extrinsics, samples interpolated RGB values, and concatenates color with image-plane coordinates:
$$
Z_{(x, y, z)} = [I_1[\mathbf{z}_{I_1}], \mathbf{z}_{I_1}, \ldots, I_n[\mathbf{z}_{I_n}], \mathbf{z}_{I_n}] \in \mathbb{R}^{5n}.
$$
The projection is purely geometric, not parametrized, and the images are completely static; only a very shallow decoder is trained [2305.10579].

In PatchGT, the fixed component is the graph partition. Patch construction is performed by spectral clustering without any trainable parameters. The normalized Laplacian
$$
\mathbf{L} = \mathbf{I} - \mathbf{D}^{-1/2}\mathbf{A}\mathbf{D}^{-1/2}
$$
is eigendecomposed, the number of clusters is determined by a spectral gap threshold \(\gamma\), and k-means is applied to the first \(k\) eigenvectors to obtain patches \(\mathcal{P}=\{C_1,\ldots,C_k\}\). GNN layers then operate on node and patch representations, while a Transformer produces the graph-level representation [2211.14425].

A common misconception is that a non-trainable modification implies a non-learned model. The cited systems do not remove learning; they relocate it. MultiPlaneNeRF retains a trainable decoder, PatchGT retains trainable GNN and Transformer blocks, and the graph NTM retains trainable weights on graph-supported connections.

## 3. Representative instantiations across domains

In **stochastic differential games on graphs**, NTM is a neural architecture for computing Nash equilibria in graph-structured multi-agent systems. It is incorporated into Direct Parameterization and Deep BSDE, producing sparse variants denoted **NTM-DP** and **NTM-DBSDE**. The architecture mirrors local interaction structure, allows information to spread across \(K\)-hop neighborhoods as depth increases, and is intended to improve interpretability, stability, and computational efficiency in large sparse settings [2509.12484].

In **neural rendering**, MultiPlaneNeRF replaces trainable voxel or plane features with unmodified input images and known camera poses. The primary scene representation is therefore explicit and non-trainable, while a small MLP decoder learns to aggregate projected features into radiance and density. The same decoder can be reused on new objects by replacing only the images and camera metadata, without additional training [2305.10579].

In **graph representation learning**, PatchGT learns from non-trainable graph patches rather than from nodes directly. After spectral clustering, node embeddings are pooled inside each patch, a coarse graph is constructed, and Transformer multi-head attention aggregates patch-level information into a graph representation. The design combines spectral information with GNN and Transformer components, and the paper contrasts this with hierarchical trainable clustering methods such as DiffPool and MinCutPool [2211.14425].

These instantiations are heterogeneous at the application level, but they share a methodological pattern: a fixed structural transform is inserted upstream of the trainable network. A plausible implication is that NTM is best viewed as a structural prior encoded directly in model topology or representation rather than as a single architecture family.

## 4. Inference-time behavior steering in large language models

In large language models, NTM appears as a **training-free intervention paradigm** rather than as a fixed network family. The Mixture-of-Tunable-Experts (MoTE) extends the Mixture-of-Experts architecture by overriding routing decisions during inference. In the DeepSeek-R1 setting analyzed in the paper, the model has **58 MoE layers**, each with **256 routed experts** and one always-on shared expert, for **14,848 routed experts** in total; only the **top-8 routed experts** are activated per token. MoTE suppresses a targeted expert by setting its activation weight to zero and renormalizing the others, or stimulates it by forcing it into the top-\(k\) set with maximal weight [2502.11096].

The identification of behaviorally specific experts is performed by **functional Token Resonance Imaging (fTRI)**. The procedure constructs analytical prompt templates, records expert activations, computes classwise averages, and forms differential activation maps for target behaviors such as refusal responses. Using this procedure, the paper reports that switching off the **top 10 most refusal-relevant experts**, which amount to **0.07%** of the routed experts, yields a **52% refusal reduction** on sensitive reference prompts **without performance degradation on MT-Bench**. Random expert deactivation produces a smaller effect with increased noise, and forced activation of the same experts increases refusal rates. The same method is also used to identify experts associated with English reasoning, and suppressing them causes about **10%** of English reasoning prompts to switch to Chinese [2502.11096].

This usage broadens the meaning of NTM beyond representational design. Here the non-trainable modification is not a fixed input encoding or graph mask, but a reversible, per-inference intervention on already specialized expert subnetworks.

## 5. Theoretical properties: invariance, expressiveness, and approximation

The graph-based NTM architecture is accompanied by a universal approximation result for static games on graphs. Under the assumption that the joint best-response operator \(A(x,a)\) is a contraction and each coordinate best response can be approximated by a shallow neural network using only local information, the NTM architecture can approximate the Nash equilibrium feedback map. The paper gives the bound
$$
\| \phi^{\mathrm{NTM}}_{i,K,M,G}(x) - \widehat{\phi}_i(x) \|_\infty
\leq
\frac{\delta}{1-\rho} + \rho^{K-1}\|\widehat{\phi}(x)\|_\infty,
$$
where \(\rho\) is the contraction parameter, \(K\) is depth, and \(M\) is the channel count. The stated interpretation is that the error decays exponentially in \(K\) [2509.12484].

PatchGT develops a different theoretical profile. The non-trainable spectral clustering procedure is proved to be **permutation invariant**: if node order is permuted, the Laplacian eigenvalues are unchanged, eigenvectors are permuted up to the usual ambiguities, and k-means yields the same patching up to renumbering. The paper further argues that trainable clustering methods inherited from 1-WL-type GNNs cannot distinguish graphs that 1-WL cannot, whereas PatchGT with spectral clusters and a suitable GNN–Transformer stack can distinguish certain graphs beyond that regime. The method is therefore presented as both more expressive than 1-WL-type GNNs and better able to address information bottlenecks in graphs [2211.14425].

The theoretical emphasis differs across these works. Graph NTM formalizes locality-aligned approximation of equilibrium maps; PatchGT formalizes invariance and expressiveness induced by non-trainable partitioning. In both cases, the non-trainable component is not an empirical convenience alone but part of the argument for what the architecture can represent.

## 6. Empirical profile, practical consequences, and recurring points of confusion

Empirically, non-trainable modifications are presented as parameter-efficient alternatives to fully trainable representations. MultiPlaneNeRF reports results comparable to state-of-the-art models for synthesizing new views and emphasizes generalization: the decoder can be trained on many objects and then reused on unseen objects by replacing only the 2D images and camera poses. The paper also states that the MultiPlane decoder can serve as a component in large generative models like GANs, and the supplied details report competitive FID/KID values for MultiPlaneGAN, with fewer parameters than other hybrid methods [2305.10579].

PatchGT is reported to save computation, improve model performance, and provide interpretability to predictions. The reduction from node-level attention to patch-level attention changes the Transformer cost from \(O(|V|^2)\) to \(O(k^2)\) when \(k \ll |V|\). The paper also notes that spectral decomposition is cubic in graph size, while emphasizing that approximation methods and partial computation can mitigate this and that the computation is parallelizable [2211.14425].

For graph-structured stochastic differential games, NTM-based solvers are reported to achieve performance comparable to fully trainable counterparts while improving computational efficiency. On supervised learning tasks for known Nash equilibria, NTM matches FNN accuracy with far fewer trainable weights; the supplied details state that it can require less than half the trainable parameters of an FNN at reasonable depths and is notably more stable than Chebyshev GCN and standard spectral GCNs in policy-function fitting [2509.12484].

A persistent source of confusion is terminological. Outside the non-trainable-modification literature, **NTM** also names **Normalizing Trajectory Models**, where each reverse step in few-step generation is modeled as a conditional normalizing flow with exact likelihood training [2605.08078]. In topic modeling, **NTMs** refers to **neural topic models** such as ETM, DecTM, TSCTM, CombinedTM, NSTM, and ECRTM; one recent paper contrasts these with long-context LLM-based topic generation and argues that “a majority of NTMs are outdated” in that domain-specific sense [2510.03174]. This terminological collision makes explicit acronym expansion essential in technical writing.

Taken together, the literature portrays Non-Trainable Modification as a design strategy that fixes representation, routing, or connectivity in a way that reflects known geometry, topology, or modular specialization. The recurrent reported benefits are parameter reduction, interpretability, stability, and, in some settings, zero-shot or plug-and-play generalization. The recurrent constraint is equally clear: performance depends on whether the imposed fixed structure is well aligned with the task, whether that structure is graph locality, camera geometry, spectral organization, or expert specialization.

Source: https://www.emergentmind.com/topics/non-trainable-modification-ntm