Universal Decision Learner (UDL)

• Universal Decision Learner (UDL) is a categorical framework that composes left Kan extensions for candidate generation with right Kan extensions for global consistency.
• It unifies diverse decision-making paradigms—planning, reinforcement learning, causal intervention, and game-theoretic equilibrium—under a common semantic pattern.
• By leveraging universal constructions, UDL formalizes local-to-global extension with fixed-point consistency, facilitating abstraction and invariant decision modeling.

Universal Decision Learner (UDL) denotes a categorical formulation of decision semantics in which partially specified local decision data are extended from observed contexts to new contexts by composing two universal constructions, a left Kan extension and a right Kan extension. In its canonical form, a UDL is defined by

$UDL_J(F)=Ran_J(Lan_JF),$

with the intended reading

$$\text{local decision data} \xrightarrow{\;Lan\;} \text{rolled-out candidates} \xrightarrow{\;Ran\;} \text{globally consistent decisions}.$$

Within this framework, left Kan extensions express rollout, aggregation, and candidate generation, whereas right Kan extensions express consistency, constraint satisfaction, and fixed-point semantics. The construction is presented not as a single algorithm but as a common semantic pattern underlying planning, reinforcement learning, causal intervention, online learning, and game-theoretic equilibrium (Mahadevan, 29 May 2026).

1. Formal definition and conceptual scope

The defining object of the framework is a pair of functors

$J:D\to C,\qquad F:D\to E,$

where $D$ is the category of observed or locally accessible contexts, $C$ is the larger category of global decision contexts, and $F$ assigns local decision semantics to the observed contexts. A decision context category $C$ is described as a category whose objects are contexts at which decisions, values, policies, interventions, or predictions may be evaluated, and whose morphisms encode admissible transitions, refinements, observations, continuations, or information-preserving maps between contexts (Mahadevan, 29 May 2026).

The formal definition states:

$UDL_J(F)=Ran_J(Lan_JF),$

whenever the displayed Kan extensions exist. More generally, a UDL is any decision semantics obtained by composing left and right Kan extensions along problem-specific inclusions of local context into global context (Mahadevan, 29 May 2026). The ambient category $E$ is assumed to have the limits and colimits needed for the Kan extensions under discussion.

This formulation makes the term “universal” precise in a categorical sense. It does not assert that all decision problems share the same computational procedure. Rather, it asserts that many decision formalisms can be described as instances of one semantic problem: canonically extend local behavioral data, then characterize the globally coherent extensions (Mahadevan, 29 May 2026). A recurrent misunderstanding is to identify UDL with a particular planning algorithm, reinforcement-learning architecture, or equilibrium solver. The framework explicitly rejects that identification.

2. Kan extensions as decision semantics

The left Kan extension $Lan_JF:C\to E$ gives the forward or expansive side of the semantics. Pointwise, it is

$$\text{local decision data} \xrightarrow{\;Lan\;} \text{rolled-out candidates} \xrightarrow{\;Ran\;} \text{globally consistent decisions}.$$0

This construction gathers all observed ways of mapping into a new context $$\text{local decision data} \xrightarrow{\;Lan\;} \text{rolled-out candidates} \xrightarrow{\;Ran\;} \text{globally consistent decisions}.$$1 and aggregates the corresponding local decision data by a colimit. The associated interpretation is rollout, interpolation, forward propagation, aggregation over all ways to reach a context, or search over candidate explanations (Mahadevan, 29 May 2026).

The right Kan extension $$\text{local decision data} \xrightarrow{\;Lan\;} \text{rolled-out candidates} \xrightarrow{\;Ran\;} \text{globally consistent decisions}.$$2 gives the backward or constraining side:

$$\text{local decision data} \xrightarrow{\;Lan\;} \text{rolled-out candidates} \xrightarrow{\;Ran\;} \text{globally consistent decisions}.$$3

Here the value at $$\text{local decision data} \xrightarrow{\;Lan\;} \text{rolled-out candidates} \xrightarrow{\;Ran\;} \text{globally consistent decisions}.$$4 is computed by enforcing compatibility with all observed consequences, continuations, and restrictions. This is the natural locus for Bellman consistency, feasibility constraints, posterior compatibility, equilibrium conditions, and other fixed-point-like requirements (Mahadevan, 29 May 2026).

The paper develops a reward-enriched left-Kan instance in the max-plus semiring

$$\text{local decision data} \xrightarrow{\;Lan\;} \text{rolled-out candidates} \xrightarrow{\;Ran\;} \text{globally consistent decisions}.$$5

for which

$$\text{local decision data} \xrightarrow{\;Lan\;} \text{rolled-out candidates} \xrightarrow{\;Ran\;} \text{globally consistent decisions}.$$6

This yields the familiar dynamic-programming recurrence

$$\text{local decision data} \xrightarrow{\;Lan\;} \text{rolled-out candidates} \xrightarrow{\;Ran\;} \text{globally consistent decisions}.$$7

which is identified as a left-Kan-style planning recurrence (Mahadevan, 29 May 2026).

Reinforcement learning appears as a right-Kan instance. The Bellman operator is written

$$\text{local decision data} \xrightarrow{\;Lan\;} \text{rolled-out candidates} \xrightarrow{\;Ran\;} \text{globally consistent decisions}.$$8

and the optimal value function satisfies

$$\text{local decision data} \xrightarrow{\;Lan\;} \text{rolled-out candidates} \xrightarrow{\;Ran\;} \text{globally consistent decisions}.$$9

In the supplementary development, a discounted Markov decision process is modeled coalgebraically as

$J:D\to C,\qquad F:D\to E,$0

and the value function is identified with a right Kan extension,

$J:D\to C,\qquad F:D\to E,$1

Value iteration, policy iteration, Monte Carlo, TD learning, and Q-learning are then interpreted as computational approximations to the same universal object (Mahadevan, 29 May 2026).

The same pattern is extended to causal interventions, online regret, and equilibrium. In online learning, histories

$J:D\to C,\qquad F:D\to E,$2

serve as contexts; cumulative loss is treated as left Kan aggregation along a trajectory, whereas no-regret learning is treated as approximate right Kan consistency. In games, Nash equilibrium is described as the finite-game instance of a right-Kan fixed point, namely a global strategy profile compatible with all local continuation and response constraints (Mahadevan, 29 May 2026).

3. Universal properties, behavioral equivalence, and minimal abstraction

The categorical force of the framework lies in the universal properties of Kan extensions. If $J:D\to C,\qquad F:D\to E,$3 exists, then for every global decision model $J:D\to C,\qquad F:D\to E,$4 there is a natural bijection

$J:D\to C,\qquad F:D\to E,$5

Thus $J:D\to C,\qquad F:D\to E,$6 is initial among global decision models receiving the local data. Dually, if $J:D\to C,\qquad F:D\to E,$7 exists, then

$J:D\to C,\qquad F:D\to E,$8

so $J:D\to C,\qquad F:D\to E,$9 is terminal among global decision models whose restrictions are compatible with the local data (Mahadevan, 29 May 2026).

From these two results the framework derives the UDL comparison principle. Let $D$0 and suppose $D$1 exists. Then any globally defined decision model $D$2 whose restriction is compatible with the rolled-out local semantics admits a canonical comparison map

$D$3

When this comparison is an isomorphism, $D$4 computes the same decision semantics as the Universal Decision Learner (Mahadevan, 29 May 2026). The significance of this statement is semantic rather than algorithmic: different computational procedures may realize the same UDL semantics if they satisfy the same universal comparison property.

The framework then defines several notions of invariance. A property is Kan-invariant if it depends only on the induced extension $D$5, where $D$6 is either $D$7, $D$8, or a UDL composite, and not on the particular syntactic presentation of $D$9. Two decision models are Kan bisimilar when, after equivalence of their context categories, their induced decision semantics are naturally isomorphic:

$C$0

The distinction between left Kan bisimulation and right Kan bisimulation tracks whether one is preserving forward aggregate behavior or fixed-point consistency semantics (Mahadevan, 29 May 2026).

Abstraction is formulated by quotienting contexts that are semantically indistinguishable. For a fixed decision model with induced semantics $C$1, the paper defines Kan equivalence by

$C$2

If the relevant decision structure descends along the quotient, then there is a quotient

$C$3

that identifies exactly Kan-equivalent contexts, and any quotient preserving the induced semantics factors uniquely through $C$4 (Mahadevan, 29 May 2026). The paper explicitly notes that ordinary MDP bisimulation is a special case of equality of right Kan semantics.

4. Relation to Universal Decision Models

An important antecedent is the formalism of Universal Decision Models (UDMs), which supplies a broad category-theoretic grammar for decision problems but does not itself provide the Kan-extension-based learner semantics of UDL. A UDM is defined as a category $C$5 whose decision objects are tuples of the form

$C$6

where $C$7 is a finite universe of elements, $C$8 is a probability space of exogenous uncertainty, $C$9 is the decision space for element $F$0, $F$1 is the measurable structure on $F$2, and $F$3 is the information field available to $F$4 (Mahadevan, 2021).

The UDM category contains decision objects, observation objects, and solution objects. Observation generation is represented by maps

$F$5

with

$F$6

when the observations generate the information field of $F$7. Solution objects are those in which the decision equations admit a unique fixed-point solution,

$F$8

The framework interprets information integration through products and limits, decision solvability as a fixed-point property, and hierarchical abstraction through quotients, coproducts, colimits, and bisimulation morphisms (Mahadevan, 2021).

Bisimulation is the central abstraction mechanism. A UDM morphism

$F$9

is represented by surjections on elements and joint decision spaces, and the quotient information field of an equivalence class is

$C$0

For MDPs the induced homomorphism condition is

$C$1

$C$2

while for predictive state representations the paper gives

$C$3

The result is a very broad umbrella encompassing structural causal models, dynamical systems, MDPs, semi-MDPs, PSRs, multiplayer games, and Witsenhausen-style intrinsic models (Mahadevan, 2021).

In relation to UDL, the UDM formalism is best understood as a structural target rather than a complete learner. The paper itself is explicit that it does not build a learning algorithm in the modern ML sense. Instead, it suggests that a UDL-like system would need to infer the decision object, its information fields, the correct abstraction or quotient structure, and the appropriate fixed-point or equilibrium solver. On this reading, UDM supplies the ontology of universal decision making, while UDL supplies a canonical extension semantics for local-to-global decision inference (Mahadevan, 2021).

5. Relation to universal learning under general stochastic processes

A second, conceptually distinct use of “universal” arises in learning theory. “Learning Whenever Learning is Possible: Universal Learning under General Stochastic Processes” studies universally consistent function learning under arbitrary stochastic processes and introduces the notion of an optimistically universal learner (Hanneke, 2017). This is not the same notion as the categorical UDL, but it addresses a closely related decision-theoretic question: can one construct a learner that succeeds whenever success is possible at all?

The paper considers three protocols. In inductive learning, a predictor is frozen after $C$4 labeled examples and evaluated by the long-run average loss

$C$5

In self-adaptive learning, the learner may observe future unlabeled inputs and update before each prediction:

$C$6

In online learning, labels are revealed after each prediction and the objective is

$C$7

The major theorem is that an optimistically universal self-adaptive learning rule exists, whereas no optimistically universal inductive learning rule exists when the instance space $C$8 is uncountable Polish (Hanneke, 2017).

For inductive and self-adaptive learning, the learnable processes are characterized exactly by the tail condition

$C$9

where

$UDL_J(F)=Ran_J(Lan_JF),$0

Equivalently, the asymptotic empirical frequency functional is a continuous submeasure. The paper also identifies a necessary condition for online learning,

$UDL_J(F)=Ran_J(Lan_JF),$1

for every disjoint sequence $UDL_J(F)=Ran_J(Lan_JF),$2, and leaves open whether this is sufficient in general uncountable spaces (Hanneke, 2017).

This line of work clarifies a common ambiguity around the word “universal.” In the categorical UDL literature, universality refers to universal constructions and comparison properties. In the stochastic-process literature, it refers to a learner that is guaranteed to work whenever learning is possible. The two notions are related by a shared decision-theoretic ambition, but they are formally different.

6. Sequential decision models and UDL-style implementations

The paper “Unified token representations for sequential decision models” does not use the term “Universal Decision Learner” explicitly, but it can be interpreted as a decision-learning framework component (Tian et al., 24 Oct 2025). Its central contribution is a Unified Token Representation (UTR) that compresses the standard Decision Transformer token triplet at each timestep,

$UDL_J(F)=Ran_J(Lan_JF),$3

into a single token. The motivation is that Decision Transformer-style models expand a trajectory of length $UDL_J(F)=Ran_J(Lan_JF),$4 into a token sequence of length $UDL_J(F)=Ran_J(Lan_JF),$5, creating redundant tokenization and quadratic attention cost (Tian et al., 24 Oct 2025).

The construction proceeds by embedding the return-to-go,

$UDL_J(F)=Ran_J(Lan_JF),$6

using a shifted action

$UDL_J(F)=Ran_J(Lan_JF),$7

and forming the unified input

$UDL_J(F)=Ran_J(Lan_JF),$8

This is fused by

$UDL_J(F)=Ran_J(Lan_JF),$9

followed by normalization

$E$0

The resulting input sequence has shape $E$1. The paper emphasizes that this restores the original trajectory length, preserves causal alignment via the shifted-action trick, and reduces attention complexity from

$E$2

(Tian et al., 24 Oct 2025).

Two variants are introduced. UDT is a Unified Decision Transformer that keeps the transformer decoder or MetaFormer-like backbone while replacing the DT token triplet with the unified token. UDC is a Unified Decision Conv that uses the same unified tokenization with a gated depthwise convolutional decision module:

$E$3

$E$4

The design is explicitly described as a pure gated CNN: no self-attention, no state-space model, and only causal depthwise separable convolution plus gating (Tian et al., 24 Oct 2025).

The theoretical argument is a Rademacher-complexity comparison between merged and separated tokenizations. Under the simplified assumptions that each token component has covariance trace at most $E$5, pairwise correlation at most $E$6, $E$7, and the same sample size $E$8 and weight norm budget $E$9, the unified representation

$Lan_JF:C\to E$0

and the separated representation

$Lan_JF:C\to E$1

satisfy

$Lan_JF:C\to E$2

$Lan_JF:C\to E$3

and therefore

$Lan_JF:C\to E$4

The paper interprets this as a tighter generalization bound for the merged representation (Tian et al., 24 Oct 2025).

Empirically, the models are evaluated on D4RL tasks from MuJoCo locomotion—Hopper, HalfCheetah, Walker2d, and Ant, across medium, medium-replay, medium-expert, and expert settings—and on AntMaze umaze and umaze-diverse, against DT, DC, and DMamba. The paper reports, for example, Hopper-medium: DT 67.6, DC 94.5, DMamba 83.5, DUT 79.4, DUC 86.5; AntMaze umaze: DT 69.8, DC 76, DMamba 79, DUT 71, DUC 80; and AntMaze umaze-diverse: DT 70.3, DC 66, DMamba 80, DUT 65, DUC 78. It emphasizes that the unified-token models are competitive and sometimes better, though not uniformly dominant on every task (Tian et al., 24 Oct 2025).

On hopper-medium, evaluated on a single NVIDIA RTX A6000, the reported efficiency numbers are:

Model	Time (s)	FLOPs (B) / Params (M)
DT	6.12	9.46 / 2.63
DUT	5.78	3.09 / 2.63
DC	4.93	6.14 / 1.99
DUC	3.45	1.54 / 1.46

The reported reductions are 67.34% FLOPs and 5.56% wall-clock time for DUT relative to DT, and 74.92% FLOPs, 30.02% time, and 26.63% parameters for DUC relative to DC. The paper notes that modern GPUs are highly parallel, so lower FLOPs do not always translate linearly to lower latency, especially for smaller models; I/O latency and memory bandwidth can dominate. It also states that UTR-based models are more scalable, lower latency, lower memory footprint, and better suited for real-time and resource-constrained deployment (Tian et al., 24 Oct 2025).

As a UDL-style interpretation, the paper argues for a representation principle that is model-agnostic, works with both transformer and convolutional backbones, unifies standard DT tokenization, and simplifies the sequential decision-learning interface. This suggests a plausible role for UTR as a universal input representation within a broader UDL-like architecture, with UDT and UDC as two concrete instantiations (Tian et al., 24 Oct 2025).

7. Limits, misconceptions, and open directions

Several limits are explicit in the current literature. First, the central claim of the UDL framework is not that every decision problem has the same algorithm; it is that many decision formalisms instantiate the same universal problem of extending local data canonically and then characterizing globally coherent extensions (Mahadevan, 29 May 2026). Treating UDL as an off-the-shelf universal solver therefore misstates the scope of the proposal.

Second, the UDM framework is foundational rather than algorithmically complete. It provides decision objects, observation objects, solution objects, bisimulation morphisms, quotient structures, and a topological reduction procedure for computing minimal objects, but it does not provide a concrete end-to-end training algorithm, loss functions, stochastic gradient updates, or sample-complexity guarantees for learning the structure (Mahadevan, 2021). The paper’s minimal-object algorithm operates by building the topology from the induced preorder, collapsing equivalence classes, removing down beat points and up beat points, and iterating until no beat points remain; the resulting core is unique up to homeomorphism (Mahadevan, 2021).

Third, the stochastic-process theory of optimistic universality remains incomplete in the online setting. The existence of an optimistically universal self-adaptive learner is established, and an optimistically universal inductive learner is ruled out for uncountable Polish instance spaces, but the general existence of an optimistically universal online learner remains open, as does the sufficiency of the identified online condition in general uncountable spaces (Hanneke, 2017).

Fourth, the emerging sequential decision-learning implementations remain partial. The unified-token models are competitive and sometimes better, but not uniformly dominant. In particular, DMamba performs best on AntMaze umaze-diverse, while UDC remains competitive with a much simpler architecture (Tian et al., 24 Oct 2025). The strongest defensible conclusion is therefore modular rather than final: present work supports a layered picture in which categorical semantics supplies a canonical notion of extension and abstraction, structural formalisms such as UDM supply the ontology of decision objects and solvability, universal-learning theory supplies a stringent criterion for learning under weak assumptions, and efficient sequence models supply concrete approximation mechanisms.

A plausible implication is that future UDL-like systems would combine structure learning, abstraction learning, solver selection, and efficient sequential representations rather than collapsing all of these tasks into a single monolithic method.