Causal Manifold Fairness (CMF)

• Causal Manifold Fairness (CMF) is a framework that defines fairness as invariant manifold geometry in latent space using causal interventions.
• It employs autoencoder models with metric tensor and curvature constraints to align representations across sensitive attribute interventions.
• The approach optimizes task performance while penalizing geometric discrepancies, clearly quantifying the fairness-utility trade-off.

Causal Manifold Fairness (CMF) is a framework for representation learning in which fairness is defined and enforced at the level of manifold geometry in latent space, taking explicit account of the causal effects of sensitive attributes on the data-generating process. Rather than treating group membership as a simple shift or perturbation of data distributions, CMF posits and operationalizes the causal warping of the data manifold itself. By constraining the local Riemannian geometry—quantified via metric tensors and curvature—of autoencoder representations to remain invariant across counterfactual interventions on sensitive attributes, CMF enables geometric invariance that translates into downstream counterfactual fairness, while also explicitly quantifying the fairness-utility trade-off via geometric metrics (Rathore, 6 Jan 2026).

1. Latent Manifolds and Riemannian Geometry in Representation Learning

CMF is fundamentally built upon autoencoder-style models, with encoder $f_\theta$ mapping input data $x$ to a latent variable $z$ and decoder (generator) $g_\theta$ reconstructing the input from $z$. Given the decoder $g$, which is assumed to be a smooth map from latent space $\mathcal{Z} \subseteq \mathbb{R}^{d_z}$ to data space $\mathcal{X} \subseteq \mathbb{R}^{d_x}$, its image forms a differentiable manifold $\mathcal{M} \subset \mathbb{R}^{d_x}$.

The geometry of this manifold is determined by how $g$ transforms local neighborhoods in $\mathcal{Z}$: the metric tensor $G(z) \in \mathbb{R}^{d_z \times d_z}$ at $z$ is defined as the pullback of the Euclidean metric from $\mathcal{X}$:

$$g_{ij}(z) = \langle \frac{\partial g(z)}{\partial z_i}, \frac{\partial g(z)}{\partial z_j} \rangle_2 = [J_D(z)]_{:,i}^\top [J_D(z)]_{:,j}$$

where $J_D(z) = \frac{\partial g(z)}{\partial z}$ is the decoder Jacobian. The squared length in the data space for an infinitesimal tangent vector $\delta z$ in latent space is then approximated by $\delta z^\top G(z) \delta z$.

Curvature information, encoding second-order geometric structure, is given by the output-wise Hessians $$H_k(z) = \frac{\partial^2 g_k(z)}{\partial z^2} \in \mathbb{R}^{d_z \times d_z}$$ for $k=1,...,d_x$. This decomposition provides a means to capture "bending" and "twisting" of the manifold under variations in $z$, and is essential for the geometric invariances targeted by CMF.

2. Causal Modeling and Counterfactual Structure

CMF introduces a structural causal model (SCM) with the tuple $$\mathcal{M} = \langle U, V = \{A, X, Y\}, F \rangle$$, where $U$ denotes latent intrinsic variables, $A \in \{0,1\}$ is the sensitive attribute (e.g., gender), $X = f(U, A)$ denotes observed features, and $Y = h(U)$ denotes the target. The essential postulate is that the sensitive attribute $A$ causally "warps" the generative process $X = f(U, A)$, thereby affecting the geometry of the observed manifold.

Counterfactual interventions, $do(A = s)$, correspond to replacing $A$'s value in the generative process and obtaining a counterfactual sample $X_{cf} = f(U, s)$. Passing $X_{cf}$ through the encoder yields counterfactual latent variables $z_{cf} = f_\theta(X_{cf})$. The local geometry at $z_{cf}$, as captured by metric $G(z_{cf})$ and Hessians $H_k(z_{cf})$, is required to match the geometry at $z$ under the original attribute value, for all $s, s'$:

$$G(z | do(s)) = G(z | do(s')), \quad H_k(z | do(s)) = H_k(z | do(s'))$$

for $k = 1,...,d_x$. This enforces invariance of geometric structure to counterfactual manipulations of $A$.

3. Objective Functions and Geometric Regularization

CMF integrates geometric fairness directly into the training objective by imposing penalties on both metric and curvature discrepancies induced by $A$. The total objective is:

$$\mathcal{L}(\theta) = \mathcal{L}_{task} + \lambda_J \mathcal{L}_J + \lambda_H \mathcal{L}_H$$

where:

• $\mathcal{L}_{task}$ comprises utility-driven losses: reconstruction loss $$\mathcal{L}_{rec} = \mathbb{E}_x[\|x - g(f(x))\|^2]$$ and prediction loss $\mathcal{L}_{pred}$ (cross-entropy or regression on $Y$ from $z$).
• $\mathcal{L}_J$ is the Jacobian (metric) penalty:

$$\mathcal{L}_J = \mathbb{E}_z\left[ \| J_D(z) - J_D(z') \|_F^2 \right],$$

aligning first-order geometry.

• $\mathcal{L}_H$ is the Hessian (curvature) penalty:

$$\mathcal{L}_H = \mathbb{E}_z\left[ \sum_{k=1}^{d_x} \| H_k(z) - H_k(z') \|_F^2 \right],$$

enforcing invariance of second-order structure.

Hyperparameters $\lambda_J, \lambda_H$ determine the trade-off: increasing these reduces geometric bias (fairness violation) at the potential cost of utility (higher reconstruction/prediction loss). The fairness-utility trade-off is quantifiable via geometric errors and task metrics (Rathore, 6 Jan 2026).

4. Theoretical Guarantees and Interpretations

The central theoretical proposition of CMF is a geometric isometry guarantee under perfect alignment: if, for all $z$ and $s, s'$,

$$J_D(z|do(s)) \equiv J_D(z|do(s')), \quad H_k(z|do(s)) \equiv H_k(z|do(s')),$$

then the decoder $g_\theta$ is locally an isometry between the manifolds parameterized by the intervention on $A$. Consequently, data points that differ only in $A$ are mapped to regions of latent space exhibiting identical local metric and curvature, enabling any predictor on $z$ to inherit counterfactual fairness.

A Taylor-expansion argument further bounds the disparity in predicted outcomes under $do(A = s)$ versus $do(A = s')$ by the residual task loss and higher-order terms in $g$'s derivatives, conditional on the fairness penalties being minimized. In practice, the framework yields a continuous fairness-utility trade-off curve as geometric regularization is increased.

5. Empirical Evaluation and Results

The CMF approach is validated on a synthetic SCM comprising a “warped Swiss roll,” where $U \sim \text{Uniform}(0, 4\pi)$, $A \in \{0,1\}$, $w = 1 + 0.5A$, and $X = [(U \cdot w) \cos U, (U \cdot w) \sin U, 0]$. This construction yields a data manifold whose tightness or twist varies with $A$, exemplifying geometric warping due to the sensitive attribute.

Autoencoder architectures comprise 3-layer MLP encoders and decoders with ELU activations, implemented with smoothness sufficient for metric and curvature computations. Jacobians and Hessians are obtained via PyTorch autograd. The following metrics are used:

• Utility: classification accuracy on $Y$, reconstruction MSE,
• Fairness: $$\text{MetricErr} = \mathbb{E}[\|G(z) - G(z_{cf})\|_F]$$, $$\text{CurvatureErr} = \mathbb{E}[\sum_k \|H_k(z) - H_k(z_{cf})\|_F]$$.

Representative results for $\lambda_J = \lambda_H = 1.0$ are:

Model	Acc (↑)	MSE (↓)	MetricErr (↓)	CurvErr (↓)
Baseline AE	1.000	0.070	16.39	4.32
CMF (ours)	0.995	0.754	0.018	0.046

The baseline achieves perfect reconstruction but at the expense of high geometric error, effectively learning separate manifolds for each group. CMF, by contrast, produces nearly perfect task performance while dramatically reducing metric and curvature error, signifying near-perfect geometric invariance. As the regularization coefficients increase, geometric errors tend toward zero while MSE increases, quantifying the fairness-utility trade-off. Ablation experiments confirm that setting $\lambda_H \to 0$ enforces only first-order fairness (small MetricErr but large CurvErr), while increasing $\lambda_H$ reduces both errors at the cost of greater reconstruction error.

6. Illustrative Example and Algorithmic Workflow

A canonical toy example involves a scalar latent $z$ and a scalar output $x = g(z)$, with group-specific decoders: $g_0(z) = z^2$ for $A = 0$, $g_1(z) = 2z^2$ for $A = 1$. Jacobians and Hessians differ between groups (e.g., $J_0 = 2z$, $J_1 = 4z$; $H_0 = 2$, $H_1 = 4$), yielding nonzero metric and curvature errors. CMF optimizes for a common decoder $g(z) = c z^2$ with $c$ chosen to jointly minimize geometric penalties, balancing the two worlds according to the regularization parameters. In higher dimensions, this optimization is performed via gradient descent.

A representative training loop is as follows:

for each minibatch {x_i, A_i, Y_i}: z_i = f_theta(x_i) xhat_i = g_theta(z_i) draw s' != A_i x_cf_i = f(U_i, s') z_cf_i = f_theta(x_cf_i) J_i = Jac(g, z_i) J_cf_i = Jac(g, z_cf_i) H_i_k = Hess(g_k, z_i) # for each output k H_cf_i_k = Hess(g_k, z_cf_i) # for each output k L_task = BCE(Y_i,decode→Y) + norm(x_i - xhat_i)**2 L_J = Frobenius(J_i - J_cf_i) L_H = sum_k Frobenius(H_i_k - H_cf_i_k) L_geo = lambda_J * L_J + lambda_H * L_H L = L_task + L_geo update theta by gradient descent on L

This workflow realizes the end-to-end enforcement of geometric invariance under causal interventions, establishing the local isometry required for counterfactual fairness in learned representations (Rathore, 6 Jan 2026).