Higher-Order KoPE: Kernel-Operator Semantics

• Higher-Order KoPE is a probabilistic framework that unifies Markov-kernel and linear operator semantics by reinterpreting linearity as sampling.
• It employs a two-level calculus where the MK language handles non-linear probabilistic computation and the LL language supports higher-order linear logic.
• The framework introduces a dedicated 'sample' construct that ensures single sampling with controlled reuse, enabling precise categorical interpretations.

Higher-Order KoPE is most naturally understood as a higher-order probabilistic language that places Markov-kernel semantics and linear-operator semantics in a single formal framework. The formulation most directly aligned with this idea is the two-level calculus of “A Higher-Order Language for Markov Kernels and Linear Operators,” which combines a first-order kernel-centric language with a higher-order linear language and connects them through a dedicated sampling construct. Its conceptual shift is a resource interpretation of linear logic in which the managed resource is sampling rather than variable use, so that the linear arrow $A \multimap B$ is read as “by sampling from $A$ once I get $B$” (Amorim, 2022).

1. Semantic problem and conceptual interpretation

The framework starts from a tension internal to probabilistic programming semantics. One tradition interprets programs by Markov kernels, which are the standard model for probabilistic computation in a call-by-value style. Another interprets programs by linear operators on spaces of distributions, which are central in linear-logic-based semantics and support algebraic reasoning about stochastic processes, inference, and ergodic behavior. The two traditions have different strengths: kernel semantics handles probabilistic computation naturally, including continuous distributions and higher-order functions via tools such as quasi-Borel spaces, whereas linear-operator semantics supports elegant reasoning but is constrained by linearity and by difficulties with the usual exponential modality ! for sampling and continuous probability (Amorim, 2022).

Higher-Order KoPE, in this sense, is not a kernel-only language and not an operator-only language. Its core thesis is that probabilistic computation should be organized by a resource interpretation of linear logic where the resource being kept track of is sampling. This reorients linearity: a term is linear not because its variable must be syntactically used once, but because the underlying probabilistic resource is sampled once. A common misconception is to read the system as a direct import of ordinary linear logic into probabilistic programming; the framework instead reinterprets linearity as sampling linearity (Amorim, 2022).

2. Two-level calculus

The calculus is split into two languages, each with its own typing discipline and semantic target. MK is a Markov-kernel language, while LL is a linear higher-order language. A bridge syntax transports values computed in one language into the other.

Language	Role	Types
MK	Markov-kernel, first-order, non-linear	$\tau ::= 1 \mid \tau \times \tau$
LL	higher-order, linear	$$\underline{\tau} ::= 1 \mid \underline{\tau} \multimap \underline{\tau} \mid \underline{\tau} \otimes \underline{\tau}$$

MK is described as the internal language of a Markov category. Its terms include variables, unit, let, pairing, projections, and primitives $f(M)$. A representative typing rule is

$\inferrule[Let]{\Gamma \vdash M : \tau_1 \quad \Gamma, x : \tau_1 \vdash N : \tau}{\Gamma \vdash let\ x\ M\ N : \tau}.$

LL is a simply typed linear lambda calculus. Its terms include variables, unit, abstraction, application, tensoring, and tensor-let. Representative rules are

$\inferrule[Abstraction]{\Gamma, x : \tau_1 \vdash t : \tau_2}{\Gamma \vdash \lambda x.\, t : \tau_1 \multimap \tau_2}$

and

$\inferrule[Application]{\Gamma_1 \vdash t : \tau_1 \multimap \tau_2 \quad \Gamma_2 \vdash u : \tau_1}{\Gamma_1, \Gamma_2 \vdash t\,u : \tau_2}.$

The division of labor is exact. MK supplies non-linear probabilistic computation in kernel form; LL supplies higher-order structure and linear-operator interpretation. The bridge is therefore not auxiliary syntax but the mechanism that makes the two semantics jointly programmable (Amorim, 2022).

3. The sample construct and sampling linearity

The main innovation is the mixed-language construct

$sample\ t_1,\dots,t_n\ x_1,\dots,x_n\ M.$

Its intended behavior is sequential: first evaluate LL programs $A$0, then obtain sampled or produced objects, bind them to MK variables $A$1, and continue with the MK program $A$2. Its typing rule is given as

$A$3

This construct expresses the resource-sensitive reading of linear logic directly. A sampled result becomes a reusable MK variable, but the underlying distribution is sampled only once. The paper emphasizes the example

$A$4

which is deterministic precisely because the coin is sampled once and then compared with itself. It also gives the formation of perfectly correlated pairs,

$A$5

and the discarding of a sampled value,

$A$6

These examples clarify a common misunderstanding. The point is not merely that LL computes distributions and MK consumes them; rather, the bridge enforces a specific operational discipline in which reuse of the sampled value is separated from resampling of the source distribution. That distinction is the semantic content of sampling linearity (Amorim, 2022).

4. Categorical semantics

The semantics uses two categorical settings. MK terms are interpreted in Markov categories, including examples such as $A$7 for discrete probability and $A$8 for measurable spaces and Markov kernels. A Markov category is a semicartesian symmetric monoidal category in which each object has copy/delete structure,

$A$9

LL terms are interpreted in symmetric monoidal closed categories (SMCCs), with tensor $B$0, linear implication $B$1, evaluation $B$2, and currying $B$3.

The bridge between the levels is a functor

$B$4

from a Markov-category semantics $B$5 to a linear-logic model $B$6. This functor must be at least lax monoidal, with structure maps

$B$7

These maps are what make the sample rule interpretable: the LL side may produce several distributions $B$8, while the MK continuation expects a joint input $B$9. The semantic clause is

$\tau ::= 1 \mid \tau \times \tau$0

The semantic sequence is therefore explicit: construct LL-side distributions, combine them with $\tau ::= 1 \mid \tau \times \tau$1, translate the MK continuation using $\tau ::= 1 \mid \tau \times \tau$2, and compose. This is the higher-order bridge in precise categorical form (Amorim, 2022).

5. Equational theory and concrete models

The paper proves several equations expected of a compositional denotational semantics. A central equation shows compatibility of sample with MK composition:

$\tau ::= 1 \mid \tau \times \tau$3

Another equation is

$\tau ::= 1 \mid \tau \times \tau$4

which states that sampling a distribution and returning the sampled value unchanged is semantically equivalent to the original distribution. The framework also proves substitution for LL and a compositionality theorem expressing denotational soundness under substitution and composition (Amorim, 2022).

The framework is instantiated in both discrete and continuous settings. In the discrete case it uses probabilistic coherence spaces $\tau ::= 1 \mid \tau \times \tau$5 and constructs

$\tau ::= 1 \mid \tau \times \tau$6

which is actually strong monoidal, with

$\tau ::= 1 \mid \tau \times \tau$7

In the continuous case it uses regularly ordered Banach spaces $\tau ::= 1 \mid \tau \times \tau$8 and constructs

$\tau ::= 1 \mid \tau \times \tau$9

where $$\underline{\tau} ::= 1 \mid \underline{\tau} \multimap \underline{\tau} \mid \underline{\tau} \otimes \underline{\tau}$$0 maps a measurable space to signed measures on it and a kernel $$\underline{\tau} ::= 1 \mid \underline{\tau} \multimap \underline{\tau} \mid \underline{\tau} \otimes \underline{\tau}$$1 to the linear operator

$$\underline{\tau} ::= 1 \mid \underline{\tau} \multimap \underline{\tau} \mid \underline{\tau} \otimes \underline{\tau}$$2

Here the functor is lax monoidal but not strong monoidal, reflecting the fact that not every joint distribution decomposes as a tensor of marginals. This explains why MK syntax is still needed for genuine correlations. The bridge thus unifies the two semantic traditions without erasing their structural differences (Amorim, 2022).

6. Extensions, significance, and neighboring higher-order frameworks

The framework is presented as extending beyond probability to commutative effects via monoidal monads, with a generic commutativity equation of the form

$$\underline{\tau} ::= 1 \mid \underline{\tau} \multimap \underline{\tau} \mid \underline{\tau} \otimes \underline{\tau}$$3

Within probability proper, its significance is stated in three parts: it unifies two semantic traditions, it gives a higher-order probabilistic language, and it provides a principled explanation of sampling in which sampling rather than variable usage is the tracked resource (Amorim, 2022).

The phrase “higher-order” also appears in adjacent but distinct research programs. Open Higher-Order Logic interprets formulas as predicates over open rather than closed objects, so that continuity, differentiability, and monotonicity can be expressed following the structure of the underlying program (Lago et al., 2022). Coinductive higher-order constrained Horn clauses instead provide a greatest-model semantics suitable for reducing higher-order recursion scheme equivalence to logical solvability over a complete and decidable theory of trees (Jochems, 2021). This suggests that Higher-Order KoPE belongs specifically to the semantic unification of probabilistic programming by kernels and operators, rather than to higher-order open logical relations or higher-order verification.

A plausible implication is that the enduring value of Higher-Order KoPE lies in its precision about where non-linearity enters probabilistic computation. LL provides the higher-order linear world of operators; MK provides the non-linear world of sampled values and correlated computation; and the sample construct is the exact interface through which one becomes the other.