Context-Conditioned Neural Operators

• Context-Conditioned Neural Operators are models that adjust their effective operator using auxiliary context such as sparse observations, demonstration pairs, or trajectory prefixes.
• They employ diverse architectures—including set convolution, transformer attention, and hypernetwork conditioning—to integrate context information into operator inference.
• Empirical results indicate that context adaptation enhances performance in complex tasks like PDE surrogates and irregular real-world spatiotemporal systems.

A context-conditioned neural operator is a neural-operator formulation in which the effective operator applied at inference is not fixed solely by global weights, but is selected, modulated, or inferred from auxiliary context such as sparse input-output observations, demonstration pairs, short trajectories, multimodal measurement sets, or even an input density. In standard neural-operator training, one typically learns a map such as $\mathcal{G}: a \mapsto u$ from fully observed input fields to output fields. Context-conditioned variants instead cast operator learning as conditional inference in function space, so that prediction depends on both the query input and a task-specific context $C$. Recent formulations include sparse conditional PDE surrogates, prompt-based in-context operator learners, hypernetwork-conditioned conservative solvers, conditioned neural fields on irregular multimodal observations, and operator-theoretic treatments of probabilistic conditioning itself (Lara-Rangel et al., 22 Jun 2026, Yang et al., 2023, Kim et al., 6 May 2026, Tsimpos et al., 7 May 2026, Mishra et al., 23 May 2025).

1. Conceptual scope and formal definitions

The strongest modern formulation appears in "Neural Operator Processes for Probabilistic Operator Learning under Partial Observations" (Lara-Rangel et al., 22 Jun 2026). There, the operator-learning problem is defined on a spatial domain $\Omega \subset \mathbb{R}^{d_x}$ with

$\mathcal{G}: a \mapsto u,$

but each task is observed only through a sparse context set

$$C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},$$

together with a query set

$Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.$

The deterministic objective is

$\hat U_Q = f_\theta(Q,C),$

and the probabilistic objective is

$p_\theta(U_Q \mid Q, C).$

This reformulation is decisive: the model is no longer asked merely to map a dense $a(\cdot)$ to a dense $u(\cdot)$, but to infer a task-specific operator instance from partial joint observations of $C$0.

Prompt-based in-context operator learning gives a second formalization. ICON represents a task by a prompt of demonstrations $C$1, a question condition, and query coordinates, and predicts the question QoI without weight updates. The operator is therefore specified implicitly by the demonstrations rather than by an explicit parameter vector (Yang et al., 2023). GICON makes the same move for graph spatiotemporal systems, conditioning prediction of $C$2 on support examples $C$3 (Wu et al., 13 Mar 2026).

A third formulation conditions the operator itself at the parameter level. HFluxNO uses a context window

$C$4

to produce a context vector $C$5, then generates target-network parameters

$C$6

and advances the state by a Flux Neural Operator instantiated by $C$7 (Kim et al., 6 May 2026). Here the context does not merely modulate activations; it generates the numerical flux operator used by the solver.

A useful abstraction is that context-conditioned neural operators realize a family of context-indexed operators $C$8, where $C$9 may be a sparse observation set, a prompt of demonstrations, a trajectory prefix, or another function-valued object. This suggests that the unifying object is not any particular backbone such as FNO, transformer, or state-space model, but the conditionalization of operator inference itself.

2. Conditioning interfaces and architectural families

The principal architectural question is how the context enters the operator. Recent work exhibits several distinct interfaces.

In NOP, sparse sets are first converted into a context-conditioned representation field $\Omega \subset \mathbb{R}^{d_x}$0, then fused with query-side inputs and decoded by an FNO-style backbone (Lara-Rangel et al., 22 Jun 2026). One pathway is a convolutional pooled summary,

$\Omega \subset \mathbb{R}^{d_x}$1

with a learned kernel $\Omega \subset \mathbb{R}^{d_x}$2; another is query-aligned attention,

$\Omega \subset \mathbb{R}^{d_x}$3

which preserves local context-query geometry. The paper’s main architectural claim is that this bridge between sparse set conditioning and dense operator decoding is the central design challenge.

Prompt-based transformer models use a different interface. ICON encodes demonstrations and query information as key-value tokens and performs context aggregation by encoder-decoder attention, while GICON interleaves example keys and values in a sequence

$\Omega \subset \mathbb{R}^{d_x}$4

then alternates graph message passing within each item and transformer attention across examples for each node (Yang et al., 2023, Wu et al., 13 Mar 2026). GICON’s example-aware positional encoding is specifically designed so that models trained with $\Omega \subset \mathbb{R}^{d_x}$5 can still be evaluated with up to $\Omega \subset \mathbb{R}^{d_x}$6 examples (Wu et al., 13 Mar 2026).

Hypernetwork conditioning is exemplified by HFluxNO. After encoding the context trajectory with a recurrent ViT, the hypernetwork generates $\Omega \subset \mathbb{R}^{d_x}$7, $\Omega \subset \mathbb{R}^{d_x}$8, and operator kernels $\Omega \subset \mathbb{R}^{d_x}$9 for the target Flux NO (Kim et al., 6 May 2026). This is a stronger form of context conditioning than latent concatenation: the context alters the operator weights themselves.

Conditioned neural fields occupy a related but distinct position. OmniField defines

$\mathcal{G}: a \mapsto u,$0

with modality masks, multimodal crosstalk blocks, and iterative cross-modal refinement (Valencia et al., 4 Nov 2025). The paper treats this as a conditioned neural field rather than a classical fixed-grid neural operator, though it explicitly allows an operator interpretation from partial multimodal observation sets to continuous target fields.

The literature also distinguishes stronger from weaker forms of conditioning. HNO introduces input-dependent multiplicative gates

$\mathcal{G}: a \mapsto u,$1

but the long filters $\mathcal{G}: a \mapsto u,$2 are functions of position, not of the particular sample; the paper therefore characterizes HNO as instance-modulated rather than fully dynamic-filter-conditioned (Patil et al., 2023). WNO conditions directly on the input field $\mathcal{G}: a \mapsto u,$3 through a wavelet-domain operator layer

$\mathcal{G}: a \mapsto u,$4

yet it does not introduce a separate context encoder or explicit per-instance operator generation (Tripura et al., 2022). These cases are relevant because they show that “uses context” and “is context-conditioned” are not always equivalent in the stronger architectural sense.

Family	Context object	Conditioning path
NOP	Sparse joint $\mathcal{G}: a \mapsto u,$5 observations	SetConv, attention, or fused representation field
ICON / GICON	Demonstration pairs	Transformer attention over prompted examples
HFluxNO	Short trajectory prefix	Hypernetwork-generated Flux NO parameters
OmniField	Partial multimodal observation sets	Query-local encoder plus iterative cross-modal refinement
HNO	Current latent/input features	Shared long convolution with instance-dependent gates
WNO	Input field	Direct field-conditioned wavelet operator

A further extension is CHOP, which keeps ICON frozen and wraps it in explicit prompt-side and prediction-side operators $\mathcal{G}: a \mapsto u,$6 and $\mathcal{G}: a \mapsto u,$7,

$\mathcal{G}: a \mapsto u,$8

This makes context conditioning partly external to the neural backbone: the prompt itself is transformed so that the frozen model sees an induced operator $\mathcal{G}: a \mapsto u,$9 that is easier to solve (Yang et al., 10 Jun 2026).

3. Probabilistic and continuous extensions

Probabilistic operator learning under partial observations is developed most explicitly in NOP. The probabilistic model introduces a latent variable $$C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},$$0 with conditional prior

$$C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},$$1

approximate posterior $$C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},$$2, and conditionally factorized Gaussian likelihood for $$C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},$$3 (Lara-Rangel et al., 22 Jun 2026). Training uses the ELBO with linear $$C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},$$4-scheduling. A key empirical claim is architectural rather than purely statistical: input-stage latent injection is substantially more stable than decoder-wide FiLM-style modulation, and uncertainty mechanisms work best when they complement rather than overwrite the local geometric pathway.

Conditioned continuous fields push the same idea beyond fixed discretizations. OmniField is explicitly designed for irregular, sparse, noisy multimodal observations and arbitrary subsets of modalities. Context is encoded as a query-local, permutation-invariant summary, and outputs are queried continuously over $$C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},$$5 with Gaussian Fourier features and modality-specific decoders (Valencia et al., 4 Nov 2025). This suggests that context-conditioned operator learning can be formulated without committing to a mesh-aligned output tensor.

The notion can be generalized even further: "One Operator for Many Densities" (Tsimpos et al., 7 May 2026) recasts probabilistic conditioning itself as an operator-learning problem. For a joint density $$C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},$$6, the kernel conditioning operator is

$$C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},$$7

while the in-context version is

$$C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},$$8

On suitable classes

$$C = \{(x_i^{(c)}, a_i^{(c)}, u_i^{(c)})\}_{i=1}^{n_C},$$9

the paper proves continuity and then uniform approximation by neural operators on compact subsets. This extends the topic beyond PDE surrogates: the “context” may itself be a joint density, and the target operator is the conditioning map.

4. In-context learning as operator inference

A major branch of the field interprets context conditioning as forward-pass adaptation rather than explicit parameter estimation. ICON is the basic example: during one forward pass, the model learns an operator from demos and applies it to a new question without gradient-based adaptation (Yang et al., 2023). GICON strengthens this formulation for irregular spatial domains, adding graph message passing for geometric generalization and example-aware positional encoding for cardinality generalization (Wu et al., 13 Mar 2026).

The strongest theoretical account is provided by "Continuum Transformers Perform In-Context Learning by Operator Gradient Descent" (Mishra et al., 23 May 2025). There, a continuum transformer processes a context window

$Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.$0

and is shown to implement gradient descent over an operator RKHS for the in-context least-squares objective

$Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.$1

The resulting update has the form

$Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.$2

and, in the infinite-depth limit under a Gaussian operator prior matching the attention kernel, the predictor converges to the BLUP/Bayes-optimal predictor. This does not merely show that transformers can use context; it formalizes context-conditioned operator learning as iterative inference in operator space.

CHOP modifies this picture by arguing that in-context operator learners may still fail on out-of-distribution operator tasks, and that adaptation can therefore occur not only within the network but also through explicit prompt and output transformations (Yang et al., 10 Jun 2026). In conservation-law and mean-field-control experiments, CHOP composes frozen ICON with interpretable operators such as cyclic shifts, affine value normalization, mass projection, and residual-transfer corrections. This suggests a broader viewpoint in which context-conditioned neural operators include both neural inference mechanisms and explicit operator-level harnesses around them.

5. Empirical regimes and application domains

The empirical literature consistently shows that context conditioning is most beneficial when the operator must be inferred from sparse, irregular, or task-varying information rather than read off from a fully observed dense input.

In sparse conditional PDE learning, NOP reports that context budgets of $Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.$3–$Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.$4 points on Burgers, $Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.$5–$Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.$6 on Darcy, and $Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.$7–$Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.$8 on Navier–Stokes can approach or surpass dense-grid FNO baselines. Deterministic relative $Q = \{(x_j^{(q)}, a_j^{(q)})\}_{j=1}^{n_Q}.$9 errors are: Burgers dense FNO $\hat U_Q = f_\theta(Q,C),$0, DNOP $\hat U_Q = f_\theta(Q,C),$1, DANOP $\hat U_Q = f_\theta(Q,C),$2; Darcy dense FNO $\hat U_Q = f_\theta(Q,C),$3, DNOP $\hat U_Q = f_\theta(Q,C),$4, DANOP $\hat U_Q = f_\theta(Q,C),$5; Navier–Stokes dense FNO $\hat U_Q = f_\theta(Q,C),$6, DNOP $\hat U_Q = f_\theta(Q,C),$7, DANOP $\hat U_Q = f_\theta(Q,C),$8 (Lara-Rangel et al., 22 Jun 2026). The design lesson is geometric: pooled convolutional summaries work well in smooth periodic regimes, whereas query-aligned attention becomes essential in non-periodic, boundary-sensitive settings.

Alternative nonlocal backbones support similar conclusions. HNO replaces Fourier or attention mixing with implicit long convolutions and multiplicative gating, reporting stronger relative $\hat U_Q = f_\theta(Q,C),$9 performance than FNO on diffusion-reaction and Navier–Stokes benchmarks; for example, on NS1 ($p_\theta(U_Q \mid Q, C).$0), HNO achieves $p_\theta(U_Q \mid Q, C).$1 versus FNO-2D $p_\theta(U_Q \mid Q, C).$2 and FNO-3D $p_\theta(U_Q \mid Q, C).$3 (Patil et al., 2023). This does not by itself establish strong context conditioning, but it shows that global receptive field and instance-modulated weighting can matter materially for operator learning.

Conservation laws motivate explicit context-generated operators. HFluxNO outperforms DPOT and DISCO on cubic conservation laws, shallow water, and viscous Burgers, and remains strongest in unseen-flux experiments such as sine-flux dynamics, where context must identify the governing law from a short trajectory prefix (Kim et al., 6 May 2026). This is a paradigmatic case of in-context operator adaptation inside a structure-preserving numerical solver.

Real-world irregular spatiotemporal systems have produced two distinct lines of evidence. OmniField reports that it consistently outperforms eight multimodal spatiotemporal baselines and gives a $p_\theta(U_Q \mid Q, C).$4 average relative error reduction across its benchmarks, while remaining robust under heavy simulated sensor noise (Valencia et al., 4 Nov 2025). GICON, under matched training steps and data, shows that in-context operator learning can outperform classical single-operator learning on more complex air-quality prediction tasks, generalize across Chinese regions, and scale from few examples to $p_\theta(U_Q \mid Q, C).$5 at inference (Wu et al., 13 Mar 2026). Together these results indicate that context conditioning becomes most useful when the deployment setting includes variable observation support, domain shifts, or operator diversity.

The paradigm has also been exported beyond PDE forecasting. In linear propagator models for transient price impact, ICON is pretrained on families of price-impact operators and then prompted with $p_\theta(U_Q \mid Q, C).$6 example trajectory pairs to infer an unseen operator instance. In-distribution relative $p_\theta(U_Q \mid Q, C).$7 errors are $p_\theta(U_Q \mid Q, C).$8 for exponential kernels, $p_\theta(U_Q \mid Q, C).$9 for non-singular power-law kernels, and $a(\cdot)$0 for singular power-law kernels; when the inferred operator is embedded in optimal execution, relative objective-value errors are $a(\cdot)$1, $a(\cdot)$2, and $a(\cdot)$3 for the respective kernel families (Meng et al., 25 Jan 2025).

Industrial digital twins provide a different application profile. CCSS-IX models wastewater dynamics with interpretable locally linear state-space experts whose couplings are modulated by context via $a(\cdot)$4, $a(\cdot)$5, $a(\cdot)$6, and $a(\cdot)$7, then adds a conformal runtime layer that can accept, abstain, reopen, or return a falsifying temporal witness (Simethy et al., 19 May 2026). On Avedøre, the static structured ensemble lies within $a(\cdot)$8 RMSE of the black-box reference and the adaptive version within $a(\cdot)$9; the calibrated reopen rule cuts aggregate two-plant regret by $u(\cdot)$0 at unsafe-action cost weight $u(\cdot)$1, and event-aligned witnesses prevent $u(\cdot)$2 of $u(\cdot)$3 false-safe N$u(\cdot)$4O approvals (Simethy et al., 19 May 2026). Although this is not a classical PDE operator benchmark, it demonstrates that context-conditioned dynamical operators can be used as certified decision-support simulators.

6. Limitations, misconceptions, and open problems

A common misconception is that any neural operator with input dependence is already context-conditioned in the strong sense. The literature is more discriminating. HNO provides instance-dependent gates but not sample-generated filters; WNO conditions on input fields through wavelet operators but does not introduce a separate context encoder or dynamic operator generation (Patil et al., 2023, Tripura et al., 2022). By contrast, NOP, ICON, GICON, and HFluxNO explicitly condition on demonstrations, sparse observations, or trajectory prefixes in ways that alter the effective operator instance.

Another misconception is that context conditioning uniformly dominates single-operator learning. GICON’s controlled comparison shows a more nuanced pattern: for simpler operators such as $u(\cdot)$5 and $u(\cdot)$6 hour air-quality prediction, classical single-operator learning often performs better, whereas multi-operator in-context learning gains advantage on harder $u(\cdot)$7, $u(\cdot)$8, and out-of-distribution $u(\cdot)$9 hour tasks (Wu et al., 13 Mar 2026). This suggests that context conditioning pays for itself when operator variability or task complexity is high enough.

Robustness and transfer remain unresolved. NOP reports nearly perfect coarse-to-fine transfer on Burgers, but sharp degradation for Darcy and Navier–Stokes under zero-shot super-resolution; increasing context density proportionally does not fix the issue (Lara-Rangel et al., 22 Jun 2026). The paper interprets this cautiously, suggesting the limitation may arise from the interaction between sparse conditional representations and the fixed FNO decoder. Similarly, CHOP shows that explicit harness operators can help dramatically, but also gives a failure case: shared value normalization harms $C$00-parameter MFC tasks because input and target live in structurally different spaces (Yang et al., 10 Jun 2026). Context-conditioning mechanisms therefore remain sensitive to operator geometry and to the compatibility of chosen transformations.

Probabilistic extensions carry their own caveats. NOP shows that uncertainty improves when latent variables complement local geometry rather than overwrite it, and that decoder-wide FiLM-style modulation can severely degrade prior-conditioned behavior (Lara-Rangel et al., 22 Jun 2026). On the theoretical side, amortized conditioning by neural operators requires regularity and positivity assumptions. "One Operator for Many Densities" proves continuity of conditioning on classes $C$01 with $C$02, but also shows that conditioning cannot be continuously extended to the full space of probability measures under weak convergence (Tsimpos et al., 7 May 2026). This indicates that the universal-operator viewpoint is mathematically powerful but not assumption-free.

Finally, scale remains an open problem for prompt-based models. ICON explicitly notes that its experiments are relatively small, that operator families are limited, and that larger families may require more demos, larger networks, and more compute (Yang et al., 2023). A plausible implication is that the field’s central challenge is no longer to show that context-conditioned neural operators exist, but to determine which conditioning interfaces scale, which transfer across geometries and discretizations, and which preserve enough structure to support uncertainty quantification and downstream control.

Taken together, the literature establishes context-conditioned neural operators as a distinct research program rather than a minor variant of standard operator learning. Its defining move is to treat the operator itself as an inference target conditioned by context. The main unresolved issues are structural: how to preserve geometry, how to handle operator diversity without collapsing to memorized sensor layouts, how to separate global uncertainty from local evidence, and how to retain solver structure or interpretability when the operator must adapt online.