Categorical Formulation of Supervised Learning

• Categorical formulation of supervised learning is a framework that uses category theory to define and relate parameters, data, and residuals through objects, morphisms, and functors.
• It establishes a duality via the Gauss-Markov adjunction, where functors map parameter translations to data corrections and vice versa in a structured, reversible manner.
• This approach enhances explicability and auditability of learning systems by providing a formal, denotational semantics that clarifies convergence and update mechanisms.

The categorical formulation of supervised learning, as developed in recent research, refers to the rigorous modeling of supervised learning processes and structures using the formal apparatus of category theory. This approach provides enhanced clarity regarding the relationships among core learning elements—such as parameters, data, and residuals—and establishes an explicit correspondence between model components and categorical constructs (objects, morphisms, functors, adjunctions), with implications for the interpretability and explicability of learning systems.

1. Categorical Modeling of Supervised Learning

The framework defines two concrete categories tailored to the structure of multiple linear regression—one of the foundational forms of supervised learning:

• Parameter Category ($\mathbf{Prm}$):
  • Objects: Parameter vectors $a \in \mathbb{R}^m$.
  • Morphisms: Translations $+\delta a: a \to a+\delta a$; that is, every morphism is simply the addition of a parameter increment (invertible, vector addition).
• Data Category ($\mathbf{Data}$):
  • Objects: Data/output vectors $y \in \mathbb{R}^n$.
  • Morphisms: Translations $+\delta y: y \to y+\delta y$; again, morphisms are additions of data increments.

While the two categories are structurally analogous—both reflecting the groupoid of vector addition—they are distinguished by their semantic roles (parameters vs. data), with differing dimensions ($m$ vs. $n$) and functions mediated by functors.

Between these categories, the model introduces an adjoint pair of functors:

• Forward Functor ($\mathcal{F} : \mathbf{Prm} \to \mathbf{Data}$):

$$\mathcal{F}(a) = X a + b,\qquad \mathcal{F}(+\delta a) = +X\delta a$$

It maps parameters to data (the classical regression prediction map).

• Gauss-Markov Functor ($\mathcal{G} : \mathbf{Data} \to \mathbf{Prm}$):

$$\mathcal{G}(y) = G y,\qquad \mathcal{G}(+\delta y) = + G \delta y$$

where $G = (X^\top X)^{-1} X^\top$ is the standard Moore–Penrose pseudo-inverse, yielding the ordinary least squares estimator from data.

2. The Gauss-Markov Adjunction and Its Significance

At the core of this modeling is the Gauss-Markov Adjunction (GMA)—an explicit categorical adjunction between the parameter and data categories as realized by $\mathcal{F}$ and $\mathcal{G}$:

$$\Phi_{\mathrm{GM}}: \operatorname{Hom}_{\mathbf{Data}}(\mathcal{F}a, y) \cong \operatorname{Hom}_{\mathbf{Prm}}(a, \mathcal{G}y)$$

This formalizes a duality: every morphism moving data ($y$) relative to a prediction ($\mathcal{F}a$), i.e., a residual adjustment, corresponds uniquely to a morphism moving parameters ($a$) relative to the OLS solution ($\mathcal{G}y$), i.e., a parameter update. The unit and counit of this adjunction are explicitly constructed:

• Unit: $\mu_a = +Gb$
• Counit: $\varepsilon_y = +(I-P)y - b$, where $P = XG$ is the projection onto the column space of $X$

This structure encapsulates the bidirectional interplay between changes in parameter space and the resulting residuals in data space.

3. Information Flow: Duality of Parameter and Residual Variations

The categorical construction provides a rigorous correspondence between:

• Residuals ($r = y - Xa - b$) in data space: Morphisms representing corrections needed in outputs for a fixed parameter vector.
• Parameter increments ($\delta a$) in parameter space: Morphisms representing adjustments to parameters, which, through the regression map, affect the outputs.

Given the adjunction, the action of mapping a data residual into the parameter category via

$$+\delta \alpha = \mathcal{G}(+\delta r) \circ \mu_a$$

establishes that parameter updates encode the information carried by residuals, and vice versa, with the relationships made explicit using the functors and their natural transformations.

Additionally, the preservation of limits by the right adjoint ($\mathcal{G}$) ensures that:

• Sequences of parameter updates under gradient descent converge (in the categorical sense) to the OLS solution $a^* = Gy$,
• Corresponding sequences of residuals converge to the minimum residual $r^* = (I-P)y$.

This reflects the convergence properties of learning algorithms within the categorical semantics.

4. Denotational Semantics and Explicability

The approach is situated as an instance of extended denotational semantics, known from theoretical computer science for mapping program syntax to formal mathematical structures (e.g., as in typed lambda calculi). Here, the components of supervised learning—data, parameters, residuals, and their transformations—are interpreted as categorical objects, morphisms, and functors, with adjunctions providing semantic correspondence.

This abstraction forms a semantically grounded foundation for explicability in AI, as required in contemporary AI ethics and governance. By denotationally modeling the structure and transformation of learning systems at a categorical level, explanations are rendered intelligible above the level of code or numerical procedure, supporting system auditability and conceptual clarity.

5. Mathematical Formulations and Diagrams

Key mathematical expressions in this framework include:

• Functors:

$$\mathcal{F}(a) = X a + b,\quad \mathcal{G}(y) = G y$$

• Adjunction:

$$\Phi_{\mathrm{GM}}:\ +\delta r \in \operatorname{Hom}_{\mathbf{Data}}(\mathcal{F}a, y) \longleftrightarrow +\delta \alpha = G (b + \delta r) \in \operatorname{Hom}_{\mathbf{Prm}}(a, \mathcal{G}y)$$

• Limit preservation (gradient descent convergence):

$$a^* = \lim_{\longleftarrow} a_i = G y,\quad r^* = (I - P) y$$

$$\mathcal{G}\Big( \lim_{\longleftarrow} r_i \Big) = \lim_{\longleftarrow} \mathcal{G}(r_i)$$

Commutative diagrams in the paper make the interrelations explicit by tracking the flow of information between categories along data and parameter axes.

6. Structural Table

Aspect	Data Side ($\mathbf{Data}$)	Parameter Side ($\mathbf{Prm}$)	Categorical Link
Objects	Output vectors $y \in \mathbb{R}^n$	Parameters $a \in \mathbb{R}^m$	Functors $\mathcal{F}, \mathcal{G}$
Morphisms	Residual translations $+\delta y$	Parameter updates $+\delta a$	Adjoint correspondence
OLS/Min-Residual	Minimum residual $(I-P)y$	OLS estimator $Gy$	$\mathcal{G}$ preserves limits
Dual Flow	Data residual implies param update	Update in parameters reflected in data	Adjunction/morphism

7. Implications and Ongoing Directions

By modeling supervised learning categorically, this framework provides:

• Explicit description of dual relationships between model components;
• The capacity to track, explain, and reason about parameter updates and residual corrections at a structural level;
• The basis for generalized semantic modeling of AI systems, beyond linear regression, and potential extension to more complex or hierarchical learning systems;
• A principled formalism supporting explicability, auditability, and high-level interpretability, aligning with emerging guidelines in responsible AI.

This approach marks a transition from syntactic and algorithmic explanations of learning to structural-semantic ones, with potential impact for both the mathematical foundations and the societal acceptance of AI systems.