---
title: Context-Conditioned Neural Operators
url: https://www.emergentmind.com/topics/context-conditioned-neural-operator
type: topic
---

# Context-Conditioned Neural Operators

A context-conditioned neural operator is a neural-operator formulation in which the effective operator applied at inference is not fixed solely by global weights, but is selected, modulated, or inferred from auxiliary context such as sparse input-output observations, demonstration pairs, short trajectories, multimodal measurement sets, or even an input density. In standard neural-operator training, one typically learns a map such as \( \mathcal{G}: a \mapsto u \) from fully observed input fields to output fields. Context-conditioned variants instead cast operator learning as conditional inference in function space, so that prediction depends on both the query input and a task-specific context \(C\). Recent formulations include sparse conditional PDE surrogates, prompt-based in-context operator learners, hypernetwork-conditioned conservative solvers, conditioned neural fields on irregular multimodal observations, and operator-theoretic treatments of probabilistic conditioning itself [2606.22946; 2304.07993; 2605.05488; 2605.06873; 2505.17838].

## 1. Conceptual scope and formal definitions

The strongest modern formulation appears in "Neural Operator Processes for Probabilistic Operator Learning under Partial Observations" [2606.22946]. There, the operator-learning problem is defined on a spatial domain \(\Omega \subset \mathbb{R}^{d_x}\) with
\[
\mathcal{G}: a \mapsto u,
\]
but each task is observed only through a sparse context set
\[
C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},
\]
together with a query set
\[
Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.
\]
The deterministic objective is
\[
\hat U_Q = f_\theta(Q,C),
\]
and the probabilistic objective is
\[
p_\theta(U_Q \mid Q, C).
\]
This reformulation is decisive: the model is no longer asked merely to map a dense \(a(\cdot)\) to a dense \(u(\cdot)\), but to infer a task-specific operator instance from partial joint observations of \((x,a,u)\).

Prompt-based in-context operator learning gives a second formalization. ICON represents a task by a prompt of demonstrations \(\{(\text{condition}^j,\text{QoI}^j)\}_{j=1}^J\), a question condition, and query coordinates, and predicts the question QoI without weight updates. The operator is therefore specified implicitly by the demonstrations rather than by an explicit parameter vector [2304.07993]. GICON makes the same move for graph spatiotemporal systems, conditioning prediction of \(\mathbf{u}_{t+\Delta t}\) on support examples \(\{(\mathbf{u}^{(j)}_{t_j-\tau+1:t_j}, \mathbf{u}^{(j)}_{t_j+\Delta t})\}_{j=1}^k\) [2603.12725].

A third formulation conditions the operator itself at the parameter level. HFluxNO uses a context window
\[
U^{n-k+1:n}=(u^{n-k+1},\dots,u^n)
\]
to produce a context vector \(c=\mathcal{E}(U^{n-k+1:n})\), then generates target-network parameters
\[
\Theta = H(c),
\]
and advances the state by a Flux Neural Operator instantiated by \(\Theta\) [2605.05488]. Here the context does not merely modulate activations; it generates the numerical flux operator used by the solver.

A useful abstraction is that context-conditioned neural operators realize a family of context-indexed operators \(\mathcal{G}_{\theta,C}\), where \(C\) may be a sparse observation set, a prompt of demonstrations, a trajectory prefix, or another function-valued object. This suggests that the unifying object is not any particular backbone such as FNO, transformer, or state-space model, but the conditionalization of operator inference itself.

## 2. Conditioning interfaces and architectural families

The principal architectural question is how the context enters the operator. Recent work exhibits several distinct interfaces.

In NOP, sparse sets are first converted into a context-conditioned representation field \(R_C(x)\), then fused with query-side inputs and decoded by an FNO-style backbone [2606.22946]. One pathway is a convolutional pooled summary,
\[
R_C(x) = \mathrm{SetConv}_\phi(C; x),
\]
with a learned kernel \(w_\phi\); another is query-aligned attention,
\[
r_{\mathrm{att}}(x) = \sum_{i=1}^{n_C}\alpha_i(x)\,h_i^{(c)},
\]
which preserves local context-query geometry. The paper’s main architectural claim is that this bridge between sparse set conditioning and dense operator decoding is the central design challenge.

Prompt-based transformer models use a different interface. ICON encodes demonstrations and query information as key-value tokens and performs context aggregation by encoder-decoder attention, while GICON interleaves example keys and values in a sequence
\[
[\mathbf{k}^{(1)}, \mathbf{v}^{(1)}, \ldots, \mathbf{k}^{(k)}, \mathbf{v}^{(k)}, \mathbf{k}_{\text{query}}],
\]
then alternates graph message passing within each item and transformer attention across examples for each node [2304.07993; 2603.12725]. GICON’s example-aware positional encoding is specifically designed so that models trained with \(k \le 5\) can still be evaluated with up to \(100\) examples [2603.12725].

Hypernetwork conditioning is exemplified by HFluxNO. After encoding the context trajectory with a recurrent ViT, the hypernetwork generates \(W_{\text{lift};\Theta}\), \(W_{\text{proj};\Theta}\), and operator kernels \(k^{(\ell)}_\Theta\) for the target Flux NO [2605.05488]. This is a stronger form of context conditioning than latent concatenation: the context alters the operator weights themselves.

Conditioned neural fields occupy a related but distinct position. OmniField defines
\[
\hat y_m(\mathbf{x},t \mid C) = \mathcal{D}_{\omega,m}\!\left( \mathcal{P}_\psi\big(\gamma(\mathbf{x}),\eta(t),\mathcal{E}_\phi(C;\mathbf{x},t)\big) \right),
\]
with modality masks, multimodal crosstalk blocks, and iterative cross-modal refinement [2511.02205]. The paper treats this as a conditioned neural field rather than a classical fixed-grid neural operator, though it explicitly allows an operator interpretation from partial multimodal observation sets to continuous target fields.

The literature also distinguishes stronger from weaker forms of conditioning. HNO introduces input-dependent multiplicative gates
\[
z^{n+1} = \xi^n \odot \mathcal{K}(h^n,z^n),
\]
but the long filters \(h^n\) are functions of position, not of the particular sample; the paper therefore characterizes HNO as instance-modulated rather than fully dynamic-filter-conditioned [2306.16524]. WNO conditions directly on the input field \(a\) through a wavelet-domain operator layer
\[
\left(\mathcal{K}(\phi)*v_j\right)(x)=\mathcal{W}^{-1}\!\left(R_\phi\cdot \mathcal{W}(v_j)\right)(x),
\]
yet it does not introduce a separate context encoder or explicit per-instance operator generation [2205.02191]. These cases are relevant because they show that “uses context” and “is context-conditioned” are not always equivalent in the stronger architectural sense.

| Family | Context object | Conditioning path |
|---|---|---|
| NOP | Sparse joint \((x,a,u)\) observations | SetConv, attention, or fused representation field |
| ICON / GICON | Demonstration pairs | Transformer attention over prompted examples |
| HFluxNO | Short trajectory prefix | Hypernetwork-generated Flux NO parameters |
| OmniField | Partial multimodal observation sets | Query-local encoder plus iterative cross-modal refinement |
| HNO | Current latent/input features | Shared long convolution with instance-dependent gates |
| WNO | Input field | Direct field-conditioned wavelet operator |

A further extension is CHOP, which keeps ICON frozen and wraps it in explicit prompt-side and prediction-side operators \(F\) and \(G\),
\[
(\mathcal{C},x^*) \xrightarrow{F} (\mathcal{C}',x^{*\prime}) \xrightarrow{\text{ICON}} \hat y' \xrightarrow{G} \hat y^*.
\]
This makes context conditioning partly external to the neural backbone: the prompt itself is transformed so that the frozen model sees an induced operator \(T'\) that is easier to solve [2606.12318].

## 3. Probabilistic and continuous extensions

Probabilistic operator learning under partial observations is developed most explicitly in NOP. The probabilistic model introduces a latent variable \(z\in\mathbb{R}^{d_z}\) with conditional prior
\[
p_\theta(z \mid C)=\mathcal{N}\!\big(z;\mu_p(r_C), \mathrm{diag}(\sigma_p^2(r_C))\big),
\]
approximate posterior \(q_\phi(z\mid C,U_Q)\), and conditionally factorized Gaussian likelihood for \(U_Q\) [2606.22946]. Training uses the ELBO with linear \(\beta\)-scheduling. A key empirical claim is architectural rather than purely statistical: input-stage latent injection is substantially more stable than decoder-wide FiLM-style modulation, and uncertainty mechanisms work best when they complement rather than overwrite the local geometric pathway.

Conditioned continuous fields push the same idea beyond fixed discretizations. OmniField is explicitly designed for irregular, sparse, noisy multimodal observations and arbitrary subsets of modalities. Context is encoded as a query-local, permutation-invariant summary, and outputs are queried continuously over \((\mathbf{x},t)\) with Gaussian Fourier features and modality-specific decoders [2511.02205]. This suggests that context-conditioned operator learning can be formulated without committing to a mesh-aligned output tensor.

The notion can be generalized even further: "One Operator for Many Densities" [2605.06873] recasts probabilistic conditioning itself as an operator-learning problem. For a joint density \(\rho(x,y)\), the kernel conditioning operator is
\[
G^\star:\rho \mapsto \kappa_\rho,\qquad \kappa_\rho(x,y)=\rho(x\mid y),
\]
while the in-context version is
\[
\Psi^\star:(\rho,y)\mapsto \rho(\cdot\mid y).
\]
On suitable classes
\[
\mathcal X_\delta = \left\{\rho\in \mathcal P(D\times E): \inf_{y\in E} m_\rho(y)\ge \delta\right\},
\]
the paper proves continuity and then uniform approximation by neural operators on compact subsets. This extends the topic beyond PDE surrogates: the “context” may itself be a joint density, and the target operator is the conditioning map.

## 4. In-context learning as operator inference

A major branch of the field interprets context conditioning as forward-pass adaptation rather than explicit parameter estimation. ICON is the basic example: during one forward pass, the model learns an operator from demos and applies it to a new question without gradient-based adaptation [2304.07993]. GICON strengthens this formulation for irregular spatial domains, adding graph message passing for geometric generalization and example-aware positional encoding for cardinality generalization [2603.12725].

The strongest theoretical account is provided by "Continuum Transformers Perform In-Context Learning by Operator Gradient Descent" [2505.17838]. There, a continuum transformer processes a context window
\[
Z_0=
\begin{pmatrix}
f^{(1)} & \cdots & f^{(n)} & f^{(n+1)}\\
u^{(1)} & \cdots & u^{(n)} & 0
\end{pmatrix},
\]
and is shown to implement gradient descent over an operator RKHS for the in-context least-squares objective
\[
L(O)=\sum_{i=1}^n \|u^{(i)}-Of^{(i)}\|_{\mathcal X}^2.
\]
The resulting update has the form
\[
O_{\ell+1} = O_\ell + r_\ell' \sum_{i=1}^n \kappa(f^{(i)},\cdot)\big(u^{(i)}-O_\ell f^{(i)}\big),
\]
and, in the infinite-depth limit under a Gaussian operator prior matching the attention kernel, the predictor converges to the BLUP/Bayes-optimal predictor. This does not merely show that transformers can use context; it formalizes context-conditioned operator learning as iterative inference in operator space.

CHOP modifies this picture by arguing that in-context operator learners may still fail on out-of-distribution operator tasks, and that adaptation can therefore occur not only within the network but also through explicit prompt and output transformations [2606.12318]. In conservation-law and mean-field-control experiments, CHOP composes frozen ICON with interpretable operators such as cyclic shifts, affine value normalization, mass projection, and residual-transfer corrections. This suggests a broader viewpoint in which context-conditioned neural operators include both neural inference mechanisms and explicit operator-level harnesses around them.

## 5. Empirical regimes and application domains

The empirical literature consistently shows that context conditioning is most beneficial when the operator must be inferred from sparse, irregular, or task-varying information rather than read off from a fully observed dense input.

In sparse conditional PDE learning, NOP reports that context budgets of \(16\)–\(64\) points on Burgers, \(32\)–\(256\) on Darcy, and \(64\)–\(256\) on Navier–Stokes can approach or surpass dense-grid FNO baselines. Deterministic relative \(L^2\) errors are: Burgers dense FNO \(0.0050\), DNOP \(0.0058\), DANOP \(0.0047\); Darcy dense FNO \(0.0254\), DNOP \(0.0242\), DANOP \(0.0226\); Navier–Stokes dense FNO \(0.0343\), DNOP \(0.0323\), DANOP \(0.0326\) [2606.22946]. The design lesson is geometric: pooled convolutional summaries work well in smooth periodic regimes, whereas query-aligned attention becomes essential in non-periodic, boundary-sensitive settings.

Alternative nonlocal backbones support similar conclusions. HNO replaces Fourier or attention mixing with implicit long convolutions and multiplicative gating, reporting stronger relative \(L_2\) performance than FNO on diffusion-reaction and Navier–Stokes benchmarks; for example, on NS1 (\(\nu=10^{-3},T=50\)), HNO achieves \(0.0069\) versus FNO-2D \(0.0128\) and FNO-3D \(0.0086\) [2306.16524]. This does not by itself establish strong context conditioning, but it shows that global receptive field and instance-modulated weighting can matter materially for operator learning.

Conservation laws motivate explicit context-generated operators. HFluxNO outperforms DPOT and DISCO on cubic conservation laws, shallow water, and viscous Burgers, and remains strongest in unseen-flux experiments such as sine-flux dynamics, where context must identify the governing law from a short trajectory prefix [2605.05488]. This is a paradigmatic case of in-context operator adaptation inside a structure-preserving numerical solver.

Real-world irregular spatiotemporal systems have produced two distinct lines of evidence. OmniField reports that it consistently outperforms eight multimodal spatiotemporal baselines and gives a \(22.4\%\) average relative error reduction across its benchmarks, while remaining robust under heavy simulated sensor noise [2511.02205]. GICON, under matched training steps and data, shows that in-context operator learning can outperform classical single-operator learning on more complex air-quality prediction tasks, generalize across Chinese regions, and scale from few examples to \(100\) at inference [2603.12725]. Together these results indicate that context conditioning becomes most useful when the deployment setting includes variable observation support, domain shifts, or operator diversity.

The paradigm has also been exported beyond PDE forecasting. In linear propagator models for transient price impact, ICON is pretrained on families of price-impact operators and then prompted with \(M=5\) example trajectory pairs to infer an unseen operator instance. In-distribution relative \(\ell^2\) errors are \(0.0053 \pm 0.0045\) for exponential kernels, \(0.0045 \pm 0.0024\) for non-singular power-law kernels, and \(0.0052 \pm 0.0036\) for singular power-law kernels; when the inferred operator is embedded in optimal execution, relative objective-value errors are \(5.80\times 10^{-8}\), \(6.91\times 10^{-7}\), and \(4.55\times 10^{-7}\) for the respective kernel families [2501.15106].

Industrial digital twins provide a different application profile. CCSS-IX models wastewater dynamics with interpretable locally linear state-space experts whose couplings are modulated by context via \(A_k(y_t)\), \(B_k(y_t)\), \(E_k(y_t)\), and \(d_k(y_t)\), then adds a conformal runtime layer that can accept, abstain, reopen, or return a falsifying temporal witness [2605.19826]. On Avedøre, the static structured ensemble lies within \(0.78\%\) RMSE of the black-box reference and the adaptive version within \(1.08\%\); the calibrated reopen rule cuts aggregate two-plant regret by \(43.6\%\) at unsafe-action cost weight \(4\), and event-aligned witnesses prevent \(93\) of \(187\) false-safe N\(_2\)O approvals [2605.19826]. Although this is not a classical PDE operator benchmark, it demonstrates that context-conditioned dynamical operators can be used as certified decision-support simulators.

## 6. Limitations, misconceptions, and open problems

A common misconception is that any neural operator with input dependence is already context-conditioned in the strong sense. The literature is more discriminating. HNO provides instance-dependent gates but not sample-generated filters; WNO conditions on input fields through wavelet operators but does not introduce a separate context encoder or dynamic operator generation [2306.16524; 2205.02191]. By contrast, NOP, ICON, GICON, and HFluxNO explicitly condition on demonstrations, sparse observations, or trajectory prefixes in ways that alter the effective operator instance.

Another misconception is that context conditioning uniformly dominates single-operator learning. GICON’s controlled comparison shows a more nuanced pattern: for simpler operators such as \(\Delta t=1\) and \(4\) hour air-quality prediction, classical single-operator learning often performs better, whereas multi-operator in-context learning gains advantage on harder \(\Delta t=12\), \(24\), and out-of-distribution \(48\) hour tasks [2603.12725]. This suggests that context conditioning pays for itself when operator variability or task complexity is high enough.

Robustness and transfer remain unresolved. NOP reports nearly perfect coarse-to-fine transfer on Burgers, but sharp degradation for Darcy and Navier–Stokes under zero-shot super-resolution; increasing context density proportionally does not fix the issue [2606.22946]. The paper interprets this cautiously, suggesting the limitation may arise from the interaction between sparse conditional representations and the fixed FNO decoder. Similarly, CHOP shows that explicit harness operators can help dramatically, but also gives a failure case: shared value normalization harms \(\rho\)-parameter MFC tasks because input and target live in structurally different spaces [2606.12318]. Context-conditioning mechanisms therefore remain sensitive to operator geometry and to the compatibility of chosen transformations.

Probabilistic extensions carry their own caveats. NOP shows that uncertainty improves when latent variables complement local geometry rather than overwrite it, and that decoder-wide FiLM-style modulation can severely degrade prior-conditioned behavior [2606.22946]. On the theoretical side, amortized conditioning by neural operators requires regularity and positivity assumptions. "One Operator for Many Densities" proves continuity of conditioning on classes \(\mathcal X_\delta\) with \(\inf_y m_\rho(y)\ge \delta\), but also shows that conditioning cannot be continuously extended to the full space of probability measures under weak convergence [2605.06873]. This indicates that the universal-operator viewpoint is mathematically powerful but not assumption-free.

Finally, scale remains an open problem for prompt-based models. ICON explicitly notes that its experiments are relatively small, that operator families are limited, and that larger families may require more demos, larger networks, and more compute [2304.07993]. A plausible implication is that the field’s central challenge is no longer to show that context-conditioned neural operators exist, but to determine which conditioning interfaces scale, which transfer across geometries and discretizations, and which preserve enough structure to support uncertainty quantification and downstream control.

Taken together, the literature establishes context-conditioned neural operators as a distinct research program rather than a minor variant of standard operator learning. Its defining move is to treat the operator itself as an inference target conditioned by context. The main unresolved issues are structural: how to preserve geometry, how to handle operator diversity without collapsing to memorized sensor layouts, how to separate global uncertainty from local evidence, and how to retain solver structure or interpretability when the operator must adapt online.

Source: https://www.emergentmind.com/topics/context-conditioned-neural-operator