Bayesian change-plane regression

Abstract: Change-plane regression identifies subpopulations through an interpretable linear threshold rule, but likelihood-based inference for the hard-threshold boundary is nonregular: objectives are non-smooth, the boundary is weakly identified under no heterogeneity, and standard large-sample approximations are fragile. We develop a new Bayesian inferential framework based on a probit-gated working likelihood -- a computationally regular surrogate that is deliberately misspecified for any fixed smoothing scale. For fixed smoothing, posterior summaries are therefore interpreted for a well-defined smoothed pseudo-true target; inference for the hard-threshold target is recovered only in a vanishing-smoothing regime, where approximation bias is governed by a boundary-margin condition on the covariate distribution. The resulting theory adapts misspecified Bernstein--von Mises arguments to Bayesian change-plane regression and makes explicit the triangular-array trade-off created by sending the smoothing scale to zero: sharper gates worsen the derivative bounds needed for Gaussian approximation, while approximation bias decreases according to the local amount of covariate mass near the boundary. Building on the resulting joint posterior, we further propose a decision-theoretic reporting protocol that separates evidence for clinically meaningful heterogeneity from the reporting of a subgroup boundary, with boundary uncertainty propagated to the covariate level through posterior membership probabilities. Simulations show favorable accuracy and uncertainty quantification of our new methods relative to the frequentist counterpart, and an application to a randomized lifestyle-intervention trial further demonstrates the utility of Bayesian change-plane regression in understanding treatment effect heterogeneity.

• The paper introduces a Bayesian framework that regularizes subgroup boundaries via a probit-gated likelihood, improving inference in nonregular settings.
• The methodology leverages efficient MCMC techniques including an elliptical slice sampler and global-local shrinkage prior to ensure stable, high-dimensional estimation.
• Extensive simulations and theoretical analysis demonstrate superior bias, coverage, and uncertainty quantification compared to classical frequentist approaches.

Bayesian Change-Plane Regression: A Principled Framework for Subgroup Analysis

Introduction

The paper "Bayesian change-plane regression" (2604.23851) advances the methodology of effect heterogeneity analysis by formulating a Bayesian inferential framework for change-plane regression models. Subgroup analysis is critical in assessing individualized treatment effects, but robustly identifying subpopulations via interpretable, data-adaptive boundaries is challenging due to the nonregularity and nonsmoothness of hard-threshold boundaries. Classical frequentist approaches often depend on heuristic smoothing, tuning parameters, and plug-in boundary estimates, leading to uncertainty quantification failures and sensitivity to model and computational choices. This work introduces a Bayesian approach leveraging a probit-gated working likelihood that regularizes the boundary, provides computational tractability, and cleanly separates regularization from scientific interpretation via sensitivity analysis.

Bayesian Formulation of Change-Plane Regression

Change-plane regression partitions the population into two subgroups through a linear decision rule $\mathbbm{1}\{Z^\top\theta \ge 0\}$, enabling the treatment effect to differ between subpopulations. While this yields a clinically communicable boundary, likelihood-based inference for the indicator boundary is nonregular: the objective is discontinuous in $\theta$, and standard large-sample techniques may break down, especially under weak or null heterogeneity.

The Bayesian framework constructs a smoothed working likelihood by replacing the hard threshold with a probit gate, $\pi_{\theta, \tau}(z) = \Phi(z^\top\theta / \tau)$, introducing a tunable smoothing scale $\tau$. The joint posterior is then characterized for $(\beta, \gamma, \theta, \sigma^2)$, with the boundary supported on the hemisphere $\mathbb{S}_+^{q-1}$. The smoothing parameter $\tau$ defines the level of approximation to the underlying hard-threshold regime, and posterior summaries at fixed $\tau$ target the Kullback-Leibler pseudo-true parameter under the probit-gated model.

For posterior simulation, the model employs an augmentation scheme with latent Gaussian variables, leading to standard conjugate updates for $(\beta, \gamma, \sigma^2)$ and an efficient, tuning-free great-circle elliptical slice sampler for $\theta$. This maintains the geometric constraint $\theta$0 and overcomes mixing issues endemic to Metropolis updates in high dimensions.

Asymptotic Theory and Regularization–Bias Tradeoff

Theoretical contributions of the paper encompass both fixed and vanishing smoothing regimes. For fixed $\theta$1, standard misspecified posterior theory (Bernstein–von Mises) applies: the posterior contracts at $\theta$2 around the pseudo-true parameter, and is asymptotically normal. This justifies Bayesian credible sets as quantifying uncertainty for the smoothed target, not the true boundary parameter.

In the vanishing $\theta$3 regime, the nonregularity of the hard-threshold target is addressed via a bias–variance decomposition. The posterior contracts at rate $\theta$4, with $\theta$5 the margin exponent capturing the covariate density at the boundary. Posterior credible sets thus shrink to the hard-threshold target if posterior bias $\theta$6 is $\theta$7, which only occurs for favorable boundary geometry (large $\theta$8). For typical continuous-score settings, $\theta$9, so the bias decays linearly in $\pi_{\theta, \tau}(z) = \Phi(z^\top\theta / \tau)$0; removal of the bias term via polynomial schedules is not generally attainable, and posterior summaries must then be interpreted as targeting a regularized version of the decision rule.

Figure 1: Bias and coverage of the treatment effect contrast $\pi_{\theta, \tau}(z) = \Phi(z^\top\theta / \tau)$1 across DGPs 1--4. Top: replicate-level estimation error; Bottom: empirical coverage of 95% intervals for $\pi_{\theta, \tau}(z) = \Phi(z^\top\theta / \tau)$2.

Sparsity and High-Dimensional Extensions

In high-dimensional settings, irrelevant or collinear covariates can obscure the estimation of subgroup boundaries. The framework is extended by introducing a global–local shrinkage (horseshoe) prior on an unconstrained coefficient vector $\pi_{\theta, \tau}(z) = \Phi(z^\top\theta / \tau)$3 that parameterizes the boundary direction $\pi_{\theta, \tau}(z) = \Phi(z^\top\theta / \tau)$4. This penalizes inactive modifiers, enabling automatic subgroup detection without sacrificing the interpretability of the change-plane rule. Efficient MCMC is achieved via elliptical slice sampling in the horseshoe-augmented parameterization, propagating uncertainty and promoting sparsity.

Decision-Theoretic Reporting and Practical Protocol

A significant utility of the Bayesian joint posterior is its capacity for coherent, decision-theoretic reporting. The paper introduces a protocol that separates evidence for clinically meaningful heterogeneity from the reporting of a learned boundary. The choice to present a boundary is made only when the posterior probability that the effect contrast exceeds a clinically relevant threshold exceeds a specified loss-calibrated threshold. This approach addresses the weak-identification regime—when there is little or no treatment heterogeneity, the likelihood on $\pi_{\theta, \tau}(z) = \Phi(z^\top\theta / \tau)$5 is flat and posterior summaries are driven by the prior. The protocol prevents spurious boundary claims and ensures that uncertainty in $\pi_{\theta, \tau}(z) = \Phi(z^\top\theta / \tau)$6 is meaningfully reflected in subgroup assignment probabilities $\pi_{\theta, \tau}(z) = \Phi(z^\top\theta / \tau)$7, rather than forced into brittle hard classifications.

Empirical Evaluation and Simulation Results

Extensive simulations compare Bayesian change-plane regression (with both parametric and BART baselines and optional horseshoe sparsity) to frequentist counterparts. The Bayesian procedures provide:

• Superior bias and coverage for treatment effect contrasts $\pi_{\theta, \tau}(z) = \Phi(z^\top\theta / \tau)$8, especially under model misspecification;
• More accurate subgroup boundary recovery, measured via angular error and misclassification rate;
• Robust uncertainty quantification, with credible sets calibrated even in high-dimensional scenarios;
• Stable handling of irrelevant covariates via sparsity priors.

Frequentist plug-in approaches, by contrast, exhibit anti-conservative uncertainty, increased bias, and poor region coverage under both correct and misspecified models.

In the analysis of the PREMIER clinical trial, the Bayesian framework with the reporting protocol highlights stable boundary learning and quantifies uncertainty in subgroup membership and treatment effect contrast. Sensitivity analyses show substantive conclusions are robust to moderate smoothing but may be sensitive in the regime of extreme sharpness (small $\pi_{\theta, \tau}(z) = \Phi(z^\top\theta / \tau)$9), reflecting the geometry-dependence articulated in the theory.

Figures and Interpretation

Figure 1 displayed above shows the bias and coverage behavior across multiple data-generating processes, providing a diagnostic for the stability of subgroup identification and effect contrast estimation.

Figure 1: The boxplots summarize replicate-level estimation error and coverage achieved by Bayesian and frequentist estimators under various scenarios, validating the joint posterior's quantification of uncertainty.

Implications and Future Directions

Practically, the Bayesian change-plane framework enables transparent regularization, interpretable uncertainty quantification, and principled reporting in subgroup analysis—capabilities crucial for regulatory science and precision medicine. Theoretically, it bridges the gap between regularized estimation and nonregular target identification, offering explicit characterization of when hard-threshold inference is justified.

Future work can extend the augmentation and inference principles:

• To generalized outcomes, including survival and longitudinal settings, supported by preliminary theory and applications in recent literature;
• To semiparametric and nonparametric baselines (e.g., BART for robustness to baseline misspecification);
• To functional or tensor-valued outcomes and more complex boundary formations (see, e.g., [Guan et al., 2025]);
• To developing computational diagnostics and sharpened bias-correction techniques in the regime of extreme nonregularity.

Conclusion

The Bayesian change-plane regression approach offers a unified, robust methodology for detecting and quantifying treatment effect heterogeneity. By explicitly regularizing the subgroup boundary and aligning inference with clinically meaningful decision rules, it resolves both the computational and inferential fragility encountered in prior work. Its modular design invites extensions into modern high-dimensional and complex-data settings while maintaining interpretability and principled uncertainty quantification—qualities essential for reliable causal analysis and individualized decision making.