Gauss-Markov Adjunction in Supervised Learning

• Gauss-Markov Adjunction is a categorical framework that defines the duality between parameters and residuals in supervised learning through an adjunction of functors.
• It employs functors to map parameter vectors to data vectors, structuring ordinary least squares estimation and ensuring limit preservation.
• This framework enhances explicability in AI by providing compositional semantics that transparently link parameter updates to residual corrections.

The Gauss-Markov Adjunction is a categorical framework that structurally formalizes the duality between parameters and residuals in supervised learning, with a particular focus on the setting of multiple linear regression. Grounded in category theory, this approach clarifies the compositional semantics of supervised learning models by representing parameters, data, and their inter-relationships as categories and functors linked through an adjunction. The framework establishes a new instance of extended denotational semantics—traditionally applied to programming language theory—for the explication and interpretability of machine learning systems, aiming to provide a theoretical foundation for Explicability as an AI principle.

1. Categorical Semantics of Supervised Learning

A categorical semantics framework is constructed by identifying two concrete categories:

• Parameter category ($\mathbf{Prm}$): Objects are parameter vectors in $\mathbb{R}^m$; morphisms are vector translations $+\delta a : a \to a + \delta a$.
• Data category ($\mathbf{Data}$): Objects are data vectors in $\mathbb{R}^n$; morphisms are translations $+\delta y : y \to y + \delta y$.

Two core functors implement the model structure:

• The forward functor $\mathcal{F}: \mathbf{Prm} \to \mathbf{Data}$ defines model application as $a \mapsto Xa + b$ (affine transformation).
• The regression (Gauss-Markov) functor $\mathcal{G}: \mathbf{Data} \to \mathbf{Prm}$ defines the estimator as $y \mapsto G y$, where $G$ is the left Moore-Penrose pseudo-inverse.

This structure encodes the passage from abstract parameter variation to its effect on data (model fit), and conversely, the inference of parameters from observed data.

2. The Gauss-Markov Adjunction Structure

The Gauss-Markov Adjunction is established by exhibiting an adjoint pair of functors $(\mathcal{F} \dashv \mathcal{G})$ and a natural isomorphism: $$\Phi_{\rm GM} : {\rm Hom}_{\mathbf{Data}}(\mathcal{F} a, y) \cong {\rm Hom}_{\mathbf{Prm}}(a, \mathcal{G} y)$$ For fixed $(a, y)$, a morphism (translation) from $\mathcal{F} a$ to $y$ in data space—interpreted as a residual—naturally corresponds to a morphism from $a$ to $\mathcal{G} y$ in parameter space—interpreted as a parameter update.

Explicitly, the residual $\delta r = y - X a - b$ is mapped to the parameter shift $\delta \alpha = G y - a$, with the relation

$\delta r = X \delta\alpha + (I-P) y - b$

where $P = X G$ is the projection onto the column space of $X$. This expresses a bijective correspondence between residuals and parameter corrections, clarifying the dual flow of information in prediction and estimation.

The corresponding diagrammatic commutation in category theory renders this correspondence explicit; see, for example, diagram (diag-01) in the source, where arrows and commutative diagrams encode the transitions between parameter updates and residuals.

3. Categorical Foundation for Ordinary Least Squares

The framework demonstrates that the ordinary least squares (OLS) estimator arises as a consequence of the adjunction’s limit preservation properties. In category theory, right adjoint functors preserve limits. Gradient descent iterations for minimizing residuals generate a cone in data space converging to the minimum residual $r_\infty$. Because the regression functor $\mathcal{G}$ is a right adjoint, it preserves this limit: $$\mathcal{G}\left( \lim_{\longleftarrow} r_i \right) \cong \lim_{\longleftarrow} \mathcal{G}(r_i)$$ Thus, the OLS estimator $a^* = G y = \lim_{\longleftarrow} a_i$ is categorically linked to attaining the minimal residual $r_\perp = (I-P)y$ by the functorial mapping $\mathcal{G}(y - r_\perp) = a^*$. This provides a structural explanation for the uniqueness and construction of the OLS estimator within the categorical system.

4. Extended Denotational Semantics in Supervised Learning

This abstract framework positions the Gauss-Markov Adjunction as a case of extended denotational semantics for supervised learning. In denotational semantics, programs are mapped to mathematical objects in a way that preserves structural and compositional properties. Analogously, by assigning categorical meaning to matrices, vectors, parameter updates, and learning processes, the Gauss-Markov Adjunction gives a high-level semantic account of supervised learning that is independent of low-level implementation details.

This extended semantics provides a rigorous mathematical language for structuring explanations and interpretations of learning systems. It generalizes classical denotational semantics, which focused on computation over symbolic domains, to encompass the data-driven, real-valued computation of learning models.

5. Interplay Between Residuals and Parameters

Within the categorical semantics, residuals and parameters form a dual pair, connected via the adjunction. Residuals (as morphisms in $\mathbf{Data}$) and parameter updates (as morphisms in $\mathbf{Prm}$) are related by $\Phi_{\rm GM}$, ensuring that every residual corresponds uniquely to a parameter adjustment, and vice versa. The natural transformations and commutative diagrams in the framework formalize this interplay, which is otherwise hidden in the standard algebraic presentation of regression.

This structuring brings transparency to how model corrections propagate, and shows how learning dynamics—a sequence of adjustments to minimize residuals—correspond to trajectories in parameter space, as mediated by the adjoint functors.

6. Applications and Significance for Interpretability

The Gauss-Markov Adjunction framework facilitates several key applications and implications:

• Semantic Modeling of Deep Learning Networks: The categorical lens highlights residuals as first-class citizens, unifying the interpretation of classical regression and modern architecture designs featuring residual connections (such as ResNets and Transformers).
• Compositional Understanding: Category-theoretic semantics enables modular and hierarchical analysis of learning architectures, supporting formal specification and reasoning about machine learning systems.
• Foundation for Explicability: By recasting the mechanics of supervised learning in terms of categorical adjunctions, this methodology provides structural transparency and explainability, directly responding to demands for Explicability in AI ethics and policy.
• Generalization Potential: The construction is positioned to extend beyond linear models, offering a pathway for the semantic analysis of complex non-linear and hierarchical machine learning models using functorial and adjunction principles derived from category theory.

7. Summary Table

Concept	Categorical Realization	Significance
Parameter, Data Categories	$\mathbf{Prm}$, $\mathbf{Data}$	Organize parameter and data spaces as categories
Functors $\mathcal{F},\mathcal{G}$	$\mathcal{F}a = X a + b$, $\mathcal{G}y = G y$	Map between model and estimator as structure-preserving functors
Gauss-Markov Adjunction	$$\Phi_{\rm GM}: {\rm Hom}_{\mathbf{Data}}(\mathcal{F} a, y) \cong {\rm Hom}_{\mathbf{Prm}}(a, \mathcal{G} y)$$	Formalizes correspondence between residuals and parameter updates
Right Adjoint and Limits	$\mathcal{G}(\lim r_i) \cong \lim \mathcal{G}(r_i)$	Links minimization of residuals to parameter convergence
Extended Denotational Semantics	Categorical mapping of model components and learning processes	Supports AI explainability and rigor
Applications	Neural architectures, modular design, interpretability	Enables explainable, decomposable, and auditable AI systems

---

In conclusion, the Gauss-Markov Adjunction provides a rigorous categorical semantics for supervised learning, elucidating the dual roles of parameters and residuals, and supplying a compositional, limit-preserving architecture that underpins both classical regression and modern machine learning models. This semantic structuring serves as a principled foundation for explicable and interpretable AI, opening avenues for systematic analysis and explanation in both theoretical and practical frameworks.