Log-Odds Algebra in Statistical Modeling

• Log-odds algebra is a framework that converts bounded probability data into an unbounded log-space, restoring additivity and linearity for robust statistical modeling.
• It exploits the transformation of odds to log-odds to enable additive computations in logistic regression and additive probabilistic models, mitigating distortions near probability boundaries.
• The approach underpins practical applications in interpretable machine learning and Bayesian updating by providing bias-free feature attributions and exact confidence intervals in inference.

Log-odds algebra provides a precise algebraic and statistical framework for working with probabilities, odds, and odds ratios, crucial in logistic regression, additive probabilistic models, and linearization of bounded probability data. This framework exploits the transformation of two-sided bounded probabilities to unbounded log-odds space, where additivity and linearity are restored, enabling bias-free computation in both statistical inference and interpretable machine learning contexts (Martínez, 24 Apr 2025, Rembold, 2019, Dervovic et al., 2022).

1. Foundations: Log-Odds, Odds, and Odds Ratios

Given a binary response variable $y\in\{0,1\}$ and explanatory variables $x$ or $X=(x_1,\dots,x_N)\in\{0,1\}^N$, the log-odds (logit), odds, and odds ratios are foundational objects:

• Logit function:

$$\mathrm{logit}\,p(x) = \log\frac{p(y=1|x)}{p(y=0|x)} = B_0+B_1x$$

for univariate binary $x$ (Martínez, 24 Apr 2025).

• Odds:

$O(x) = \frac{p(y=1|x)}{p(y=0|x)}$

• Log-odds (logit):

$\ell(x) = \log O(x) = B_0+B_1\,x$

• Odds-ratio for $x$:

$$\mathrm{OR} = \frac{O(1)}{O(0)} = \exp\left[\ell(1) - \ell(0)\right] = \exp(B_1)$$

For multivariate binary explanatory variables $X\in\{0,1\}^N$, the additivity in log-odds persists:

$$\ell(X) = \log\frac{p(y=1|X)}{p(y=0|X)} = B_0 + \sum_{n=1}^N B_n x_n$$

The general event $E\in\{0,1\}^N$ indexes the $2^N$ possible variable combinations (referred to as "events") (Martínez, 24 Apr 2025).

2. Algebraic Structure: Additivity and Multiplicativity

Log-odds algebra exhibits additivity in log-odds space and multiplicativity in odds space. For any two events $E_r$ and $E_t$ in $\{0, 1\}^N$:

• General Odds Ratio:

$$\mathrm{OR}(E_r \to E_t) = \frac{O(E_t)}{O(E_r)} = \exp\left[\mathbf{B}^T(E_t-E_r)\right]$$

• Group Odds Ratio (relative to "All Zeros" event):

$$\mathrm{G}(E_t) = \frac{O(E_t)}{O(E^{(0)})} = \exp\left[\mathbf{B}^T E_t\right] = \prod_{n=1}^N \exp(B_n)^{E_t[n]}$$

• Log-odds difference (additive):

$$\log\mathrm{OR}(E_r \to E_t) = \mathbf{B}^T(E_t-E_r) = \sum_{n=1}^N B_n [E_t[n] - E_r[n]]$$

• Odds product (multiplicative):

$$\mathrm{OR}(E_r \to E_t) = \prod_{n=1}^N [e^{B_n}]^{E_t[n] - E_r[n]} = \prod_{n:E_t[n]>E_r[n]} e^{B_n}$$

If only one variable changes, the odds ratio reduces to $\exp(B_n)$ ("Basic Odds Ratio" corollary) (Martínez, 24 Apr 2025).

3. Linearization of Probability Space and Weight (W)

Probabilities $p\in(0,1)$ are bounded and non-additive under arithmetic addition. The log-odds transformation (or "Weight", $W$) maps $p$ to the real line, providing an unbounded additive scale:

• Weight (W):

$W = \log_{10}\left(\frac{p}{1-p}\right)$

• Inverse:

$p = \frac{10^W}{1+10^W}$

• Additivity in W-space:

Combining independent weights $W_1$, $W_2$ yields $W_{\text{combined}} = W_1 + W_2$.

• Bayesian updating in W-space:

$W_{\text{post}} = W_{\text{pre}} + I$

where $I$ is "impact" (difference of group means in $W$) (Rembold, 2019).

Means, standard deviations, and inference are algebraically exact in $W$-space, avoiding the bounded-interval bias of $p$-space.

4. Application in Logistic Regression and General Additive Models

Log-odds algebra provides essential structure in logistic regression and additive models:

• Multivariable Logistic Regression: Odds for any event $E$ (with $E\in\{0,1\}^N$) are $O(E) = \exp[B_0+\mathbf{B}^T E]$.
• Group Odds-Ratio: For moving from all-zeros ($E^{(0)}$) to an event $E$, $$\mathrm{G}(E) = \exp[\mathbf{B}^T E] = \prod_{n=1}^N \exp(B_n)^{E[n]}$$ (Martínez, 24 Apr 2025).
• Linear Additive Models (LAM): Instead of a nonlinear sigmoid, LAMs use a clipped linear approximation for the probability unit:

$$\hat{y}_{\text{LAM}}(\mathbf{x}) = \left[0,1\right]\left(\frac{1}{2}+\sum_{i=0}^d \frac{\beta_i}{2\alpha^*} f_i(x_i)\right)$$

enabling feature-to-probability attributions that are globally linear and directly interpretable (Dervovic et al., 2022).

5. Statistical Implications and Computation

Key statistical implications include:

• Linearity Restored: Log-odds mapping transforms two-sided bounded data to an unbounded, linear domain, critical for hypothesis testing, computation of means/SDs, and Bayesian inference (Rembold, 2019).
• Confidence Intervals: Means and standard deviations computed in $W$-space, shifted by $\pm s_W$ about the mean $\overline{W}$, yield correct confidence bands on remapping to the $p$-scale.
• Avoids Distortion: Arithmetic in $p$-space introduces distortion as probabilities approach the boundaries; $W$-space arithmetic does not.
• Normality: Under mild regularity, distributions in $W$-space may be closer to normal than in $p$-space, improving the fidelity of parametric tests (Rembold, 2019).

6. Extensions, Special Cases, and Limitations

The log-odds algebraic structure is contingent on specific model assumptions:

• No Interactions: Formulas hold for models linear in parameters with no interaction terms. Introducing interactions ($B_{12}\,x_1x_2$) breaks additivity in the log-odds (Martínez, 24 Apr 2025).
• Binary Variables: Results as stated assume binary predictors. Continuous extensions are not covered in these formulations.
• Reference Event: The "all-zeros" event is a convenient, but not necessary, reference; ratios can be taken between any pair of events.
• Worked Examples: Explicit numerical cases in $N=3$ dimensions with specified $B_n$ demonstrate both single and group odds ratios in practice (Martínez, 24 Apr 2025).

For non-probability or "untidy" data, a multistep procedure using candidate transforms, standardization, and error function mapping to probability is proposed, with analyses then performed in $W$-space. Observed $p=0$ or $1$ must be nudged inward to avoid infinities (Rembold, 2019).

7. Interpretability and Model Attribution

Linearization via log-odds or weights has significant consequences for interpretability in modern machine learning:

• Attribution in Log-Odds vs Probability Space: In standard models, feature attributions occur in log-odds space, with probability changes being nonlinearly dependent on the baseline. In LAMs, attributions are strictly additive in output probability, enabling transparent explanations (Dervovic et al., 2022).
• Expert-Advice Combination: Algorithms such as SubscaleHedge combine subscale risks in probability space using multiplicative weights adapted from log-odds algebra, admitting transparent online adjustment and interpretability (Dervovic et al., 2022).

A plausible implication is that log-odds algebra provides the theoretical infrastructure behind both statistical rigor and explainability in additive binary classification and certain ensemble settings. It underpins quantitatively exact, bias-free computation and consistent feature attributions across statistical and machine learning domains.