Non-Trainable Modification (NTM) in Neural Models

• Non-Trainable Modification is a design paradigm that fixes part of a model’s structure to reduce parameter count while channeling learning to selected modules.
• NTM approaches employ fixed graph masks, static scene representations, and non-trainable clustering to improve computational efficiency and model stability.
• Empirical results show NTM architectures achieve competitive performance with fewer trainable parameters and enhanced interpretability.

Searching arXiv for papers on "Non-Trainable Modification" and related NTM usages to ground the article in current literature. Non-Trainable Modification (NTM) denotes a class of model designs in which a consequential part of the computational structure is fixed rather than optimized, so that learning or control is concentrated in the remaining trainable components. In the literature considered here, the label is used both for a graph-guided neural architecture for stochastic differential games and for a broader inference-time behavior-steering paradigm in LLMs; related work in neural rendering and graph representation learning operationalizes the same principle through non-trainable images, non-parametric projection, and non-trainable spectral clustering (Hu et al., 15 Sep 2025, Dahlke et al., 16 Feb 2025, Zimny et al., 2023, Gao et al., 2022).

1. Terminological scope and acronym overload

The label “NTM” is not unique across recent arXiv literature. In some work it explicitly denotes Non-Trainable Modification; in other work it denotes unrelated constructs.

Paper	Meaning of “NTM”	Domain
(Hu et al., 15 Sep 2025)	Non-Trainable Modification	Nash equilibria in stochastic differential games on graphs
(Dahlke et al., 16 Feb 2025)	Non-Trainable Modification	Inference-time behavior modification in MoE LLMs
(Zimny et al., 2023)	Non-trainable representation	Neural radiance fields
(Gao et al., 2022)	Non-trainable clusters / patches	Graph representation learning
(Gu et al., 8 May 2026)	Normalizing Trajectory Models	Few-step image generation
(Xu et al., 3 Oct 2025)	Neural Topic Models (“NTMs”)	Topic modeling

This usage pattern suggests that “NTM” must be interpreted from domain context rather than treated as a standardized acronym. In the non-trainable-modification sense, the common thread is not the elimination of learning, but the insertion of fixed structure—graph masks, projections, images, or cluster assignments—into an otherwise learnable pipeline.

2. Core design principle: fixed structure with selective learning

The defining mechanism of NTM is the use of non-trainable components that encode prior structure directly into the model. In the graph-based architecture of stochastic differential games, this takes the form of graph-guided sparsification: a weight $W_{pr,q}^{(k)}$ is present, and trainable, if and only if $v_q$ is a neighbor of $v_p$; otherwise it is fixed, often zero, and non-trainable. The forward propagation rule is local:

$$z^{(k+1)}_p = \sum_{r=1}^{M} g_{pr}^{(k)} \odot \sigma \Big( W_{pr,p}^{(k)} z_p^{(k)} + \sum_{q: v_q \in \mathcal{N}_G(v_p)} W_{pr,q}^{(k)} z_q^{(k)} + h^{(k)}_{pr} \Big) + b^{(k)}_p.$$

The resulting parameter count scales with the edge set $|E|$ rather than $N^2$, which is especially relevant for large sparse graphs (Hu et al., 15 Sep 2025).

In MultiPlaneNeRF, the non-trainable element is the scene representation itself: a set of fixed 2D images $I_1,\dots,I_n$ from known viewpoints. For a 3D point $p=(x,y,z)$, the model projects $p$ onto each image using camera intrinsics and extrinsics, samples interpolated RGB values, and concatenates color with image-plane coordinates:

$$Z_{(x, y, z)} = [I_1[\mathbf{z}_{I_1}], \mathbf{z}_{I_1}, \ldots, I_n[\mathbf{z}_{I_n}], \mathbf{z}_{I_n}] \in \mathbb{R}^{5n}.$$

The projection is purely geometric, not parametrized, and the images are completely static; only a very shallow decoder is trained (Zimny et al., 2023).

In PatchGT, the fixed component is the graph partition. Patch construction is performed by spectral clustering without any trainable parameters. The normalized Laplacian

$v_q$0

is eigendecomposed, the number of clusters is determined by a spectral gap threshold $v_q$1, and k-means is applied to the first $v_q$2 eigenvectors to obtain patches $v_q$3. GNN layers then operate on node and patch representations, while a Transformer produces the graph-level representation (Gao et al., 2022).

A common misconception is that a non-trainable modification implies a non-learned model. The cited systems do not remove learning; they relocate it. MultiPlaneNeRF retains a trainable decoder, PatchGT retains trainable GNN and Transformer blocks, and the graph NTM retains trainable weights on graph-supported connections.

3. Representative instantiations across domains

In stochastic differential games on graphs, NTM is a neural architecture for computing Nash equilibria in graph-structured multi-agent systems. It is incorporated into Direct Parameterization and Deep BSDE, producing sparse variants denoted NTM-DP and NTM-DBSDE. The architecture mirrors local interaction structure, allows information to spread across $v_q$4-hop neighborhoods as depth increases, and is intended to improve interpretability, stability, and computational efficiency in large sparse settings (Hu et al., 15 Sep 2025).

In neural rendering, MultiPlaneNeRF replaces trainable voxel or plane features with unmodified input images and known camera poses. The primary scene representation is therefore explicit and non-trainable, while a small MLP decoder learns to aggregate projected features into radiance and density. The same decoder can be reused on new objects by replacing only the images and camera metadata, without additional training (Zimny et al., 2023).

In graph representation learning, PatchGT learns from non-trainable graph patches rather than from nodes directly. After spectral clustering, node embeddings are pooled inside each patch, a coarse graph is constructed, and Transformer multi-head attention aggregates patch-level information into a graph representation. The design combines spectral information with GNN and Transformer components, and the paper contrasts this with hierarchical trainable clustering methods such as DiffPool and MinCutPool (Gao et al., 2022).

These instantiations are heterogeneous at the application level, but they share a methodological pattern: a fixed structural transform is inserted upstream of the trainable network. A plausible implication is that NTM is best viewed as a structural prior encoded directly in model topology or representation rather than as a single architecture family.

4. Inference-time behavior steering in LLMs

In LLMs, NTM appears as a training-free intervention paradigm rather than as a fixed network family. The Mixture-of-Tunable-Experts (MoTE) extends the Mixture-of-Experts architecture by overriding routing decisions during inference. In the DeepSeek-R1 setting analyzed in the paper, the model has 58 MoE layers, each with 256 routed experts and one always-on shared expert, for 14,848 routed experts in total; only the top-8 routed experts are activated per token. MoTE suppresses a targeted expert by setting its activation weight to zero and renormalizing the others, or stimulates it by forcing it into the top-$v_q$5 set with maximal weight (Dahlke et al., 16 Feb 2025).

The identification of behaviorally specific experts is performed by functional Token Resonance Imaging (fTRI). The procedure constructs analytical prompt templates, records expert activations, computes classwise averages, and forms differential activation maps for target behaviors such as refusal responses. Using this procedure, the paper reports that switching off the top 10 most refusal-relevant experts, which amount to 0.07% of the routed experts, yields a 52% refusal reduction on sensitive reference prompts without performance degradation on MT-Bench. Random expert deactivation produces a smaller effect with increased noise, and forced activation of the same experts increases refusal rates. The same method is also used to identify experts associated with English reasoning, and suppressing them causes about 10% of English reasoning prompts to switch to Chinese (Dahlke et al., 16 Feb 2025).

This usage broadens the meaning of NTM beyond representational design. Here the non-trainable modification is not a fixed input encoding or graph mask, but a reversible, per-inference intervention on already specialized expert subnetworks.

5. Theoretical properties: invariance, expressiveness, and approximation

The graph-based NTM architecture is accompanied by a universal approximation result for static games on graphs. Under the assumption that the joint best-response operator $v_q$6 is a contraction and each coordinate best response can be approximated by a shallow neural network using only local information, the NTM architecture can approximate the Nash equilibrium feedback map. The paper gives the bound

$v_q$7

where $v_q$8 is the contraction parameter, $v_q$9 is depth, and $v_p$0 is the channel count. The stated interpretation is that the error decays exponentially in $v_p$1 (Hu et al., 15 Sep 2025).

PatchGT develops a different theoretical profile. The non-trainable spectral clustering procedure is proved to be permutation invariant: if node order is permuted, the Laplacian eigenvalues are unchanged, eigenvectors are permuted up to the usual ambiguities, and k-means yields the same patching up to renumbering. The paper further argues that trainable clustering methods inherited from 1-WL-type GNNs cannot distinguish graphs that 1-WL cannot, whereas PatchGT with spectral clusters and a suitable GNN–Transformer stack can distinguish certain graphs beyond that regime. The method is therefore presented as both more expressive than 1-WL-type GNNs and better able to address information bottlenecks in graphs (Gao et al., 2022).

The theoretical emphasis differs across these works. Graph NTM formalizes locality-aligned approximation of equilibrium maps; PatchGT formalizes invariance and expressiveness induced by non-trainable partitioning. In both cases, the non-trainable component is not an empirical convenience alone but part of the argument for what the architecture can represent.

6. Empirical profile, practical consequences, and recurring points of confusion

Empirically, non-trainable modifications are presented as parameter-efficient alternatives to fully trainable representations. MultiPlaneNeRF reports results comparable to state-of-the-art models for synthesizing new views and emphasizes generalization: the decoder can be trained on many objects and then reused on unseen objects by replacing only the 2D images and camera poses. The paper also states that the MultiPlane decoder can serve as a component in large generative models like GANs, and the supplied details report competitive FID/KID values for MultiPlaneGAN, with fewer parameters than other hybrid methods (Zimny et al., 2023).

PatchGT is reported to save computation, improve model performance, and provide interpretability to predictions. The reduction from node-level attention to patch-level attention changes the Transformer cost from $v_p$2 to $v_p$3 when $v_p$4. The paper also notes that spectral decomposition is cubic in graph size, while emphasizing that approximation methods and partial computation can mitigate this and that the computation is parallelizable (Gao et al., 2022).

For graph-structured stochastic differential games, NTM-based solvers are reported to achieve performance comparable to fully trainable counterparts while improving computational efficiency. On supervised learning tasks for known Nash equilibria, NTM matches FNN accuracy with far fewer trainable weights; the supplied details state that it can require less than half the trainable parameters of an FNN at reasonable depths and is notably more stable than Chebyshev GCN and standard spectral GCNs in policy-function fitting (Hu et al., 15 Sep 2025).

A persistent source of confusion is terminological. Outside the non-trainable-modification literature, NTM also names Normalizing Trajectory Models, where each reverse step in few-step generation is modeled as a conditional normalizing flow with exact likelihood training (Gu et al., 8 May 2026). In topic modeling, NTMs refers to neural topic models such as ETM, DecTM, TSCTM, CombinedTM, NSTM, and ECRTM; one paper contrasts these with long-context LLM-based topic generation and argues that “a majority of NTMs are outdated” in that domain-specific sense (Xu et al., 3 Oct 2025). This terminological collision makes explicit acronym expansion essential in technical writing.

Taken together, the literature portrays Non-Trainable Modification as a design strategy that fixes representation, routing, or connectivity in a way that reflects known geometry, topology, or modular specialization. The recurrent reported benefits are parameter reduction, interpretability, stability, and, in some settings, zero-shot or plug-and-play generalization. The recurrent constraint is equally clear: performance depends on whether the imposed fixed structure is well aligned with the task, whether that structure is graph locality, camera geometry, spectral organization, or expert specialization.