Bayesian Neighborhood Adaptation (BNA)

• Bayesian Neighborhood Adaptation (BNA) is a suite of probabilistic frameworks that adaptively selects optimal neighborhoods in graphs, k-NN, SOMs, and GNNs.
• It leverages hierarchical Bayesian modeling and blockwise posterior factorization to quantify uncertainty and enhance computational efficiency over traditional methods.
• Empirical results demonstrate that BNA reduces computation time and improves accuracy in applications such as gene network analysis and graph-based learning.

Bayesian Neighborhood Adaptation (BNA) is a suite of probabilistic inference frameworks and algorithms developed for adaptive neighborhood selection in network modeling, nearest neighbor methods, self-organizing maps, and graph neural networks. BNA approaches leverage hierarchical Bayesian modeling and blockwise posterior factorization to enable efficient inference of local network structure, optimal neighbor counts, or neighborhood radii, frequently yielding major computational and statistical advantages over traditional non-adaptive or grid-search-based methods (Kaufmann et al., 2015, Nuti, 2017, Regmi et al., 5 Feb 2026, Fillion et al., 2022).

1. Theoretical Foundations and Model Classes

BNA encompasses several problem domains unified by their focus on neighborhood inference:

• Graphical Models / Markov Random Fields: Here, the "neighborhood" is the Markov blanket of a set of query variables, and BNA estimates the induced sub-network via a blockwise factorization of the Gaussian MRF posterior (Kaufmann et al., 2015).
• Bayesian Nearest Neighbors: BNA for k-NN reframes the choice of neighborhood size $k$ as an online change-point detection problem along the distance-ordered data sequence, modeling local data coherence as a random variable with an analytic posterior (Nuti, 2017).
• Self-Organizing Maps (SOMs): BNA treats the neighborhood (topological radius) as a continuous latent variable, controlled by variational Bayesian inference and coupled to model uncertainty (Fillion et al., 2022).
• Graph Neural Networks (GNNs): BNA views the neighborhood scope (number of aggregation hops) as a random variable inferred jointly with network parameters, governed by a stick-breaking beta process (Regmi et al., 5 Feb 2026).

All approaches exploit the conditional independence of neighborhoods given local data properties, replacing exhaustive or ad hoc hyperparameter sweeps with principled probabilistic inference.

2. Bayesian Formulation and Priors

Markov Random Field Subnetwork Estimation

Let $X\in\mathbb{R}^{(p+q)\times n}$ with sample covariance $S = XX^\top$, modeled under a zero-mean Gaussian MRF with $S \sim \mathcal{W}_{p+q}(n, \Sigma)$. The precision $\Sigma^{-1} = W$ is decomposed as

$$S = \begin{bmatrix} S_{11} & S_{12} \ S_{21} & S_{22} \end{bmatrix},\quad W = \begin{bmatrix} W_{11} & W_{12} \ W_{21} & W_{22} \end{bmatrix}$$

with $W_{12}$ encoding dependencies from query ($p$) to neighbor ($q$) variables.

• Priors: $W\sim\mathcal{W}_{p+q}(p+q+1,I)$ (Wishart), and each $X\in\mathbb{R}^{(p+q)\times n}$0 has a Laplace prior as a scale mixture of Gaussians: $X\in\mathbb{R}^{(p+q)\times n}$1, $X\in\mathbb{R}^{(p+q)\times n}$2.
• Compound Priors allow for sparse, edge-selective neighborhoods.

Bayesian k-Nearest Neighbors

For inputs $X\in\mathbb{R}^{(p+q)\times n}$3 and a query $X\in\mathbb{R}^{(p+q)\times n}$4, distances define an ordered sequence. Local data are assumed i.i.d. within adaptive segments, determined by change-point processes with hazard $X\in\mathbb{R}^{(p+q)\times n}$5.

• Conjugate Priors: Exponential family models are exploited for analytic updates, enabling closed-form edge marginal likelihoods.

Bayesian Neighborhood Adaptation for SOMs and GNNs

• SOMs: Model parameter $X\in\mathbb{R}^{(p+q)\times n}$6 (neighborhood radius) is tied to the variational posterior’s scale, inferred by optimizing the ELBO, enforcing topological organization via the KL divergence between data and model posteriors (Fillion et al., 2022).
• GNNs: The number of hops is endowed with a beta process prior:

$X\in\mathbb{R}^{(p+q)\times n}$7

Layer-wise masks $X\in\mathbb{R}^{(p+q)\times n}$8 determine which hops are active.

3. Inference Algorithms and Computational Complexity

Blockwise Factorization in Graphical Models

Posterior factorization under the blockwise Wishart law enables focusing inference exclusively on $X\in\mathbb{R}^{(p+q)\times n}$9 given $S = XX^\top$0, conditional on (independent of) the remainder:

$S = XX^\top$1

Sampling targets:

• $S = XX^\top$2: via inverse Gaussian.
• $S = XX^\top$3: blockwise Gaussian update, vectorized.
• $S = XX^\top$4: matrix-GIG law.

Per-iteration cost: $S = XX^\top$5. By comparison, full-network methods require $S = XX^\top$6 (Kaufmann et al., 2015).

Exact Recursion for Bayesian k-NN

The posterior over neighborhood size $S = XX^\top$7 is obtained analytically via a dynamic programming recursion over run-lengths:

$S = XX^\top$8

with normalization and exponential-family predictive updates for each $S = XX^\top$9 value. Complexity is $S \sim \mathcal{W}_{p+q}(n, \Sigma)$0 in the worst case, but practical heuristics truncate low-probability run-lengths yielding effective rates of $S \sim \mathcal{W}_{p+q}(n, \Sigma)$1 (Nuti, 2017).

Variational Inference for BNA in SOMs and GNNs

• SOMs: The negative ELBO $S \sim \mathcal{W}_{p+q}(n, \Sigma)$2 is minimized by SGD or Adam, updating neuron prototypes and $S \sim \mathcal{W}_{p+q}(n, \Sigma)$3 jointly, with gradients exploiting the soft-neighborhood posterior around the best matching unit (Fillion et al., 2022).
• GNNs: A truncated variational posterior over $S \sim \mathcal{W}_{p+q}(n, \Sigma)$4 and $S \sim \mathcal{W}_{p+q}(n, \Sigma)$5 is adopted, with Monte Carlo estimation of gradients for optimization. The ELBO combines supervised likelihood and KL terms for both mask and beta distributions (Regmi et al., 5 Feb 2026).

4. Empirical Validation and Expressivity

Graphical Models and Markov Blankets

Empirical results for synthetic data (e.g., $S \sim \mathcal{W}_{p+q}(n, \Sigma)$6, $S \sim \mathcal{W}_{p+q}(n, \Sigma)$7) indicate that BNA achieves higher F-scores, fewer false positives, and faster mixing (lower autocorrelation in MCMC traces) compared to full-network Bayesian approaches. In real datasets (e.g., colon cancer: $S \sim \mathcal{W}_{p+q}(n, \Sigma)$8 clinical, $S \sim \mathcal{W}_{p+q}(n, \Sigma)$9 gene variables), BNA with copula augmentation recovered biologically meaningful gene-clinical subnetworks in a fraction of the time required by full-network methods (2 hours vs 120 hours) (Kaufmann et al., 2015).

Bayesian k-NN

On Ripley’s dataset and UCI regression problems, BNA achieves classification and average error rates close to or better than MCMC Bayesian k-NN, but with orders-of-magnitude reductions in computational time (milliseconds per query vs hours). Posterior uncertainty over $\Sigma^{-1} = W$0 enables both robustness and principled outlier detection (Nuti, 2017).

SOMs and Topological Ordering

Dynamic SOM with Bayesian radius adaptation (VDSOM) demonstrates lower mean distortion, faster adaptation to distributional shifts (e.g., “moons $\Sigma^{-1} = W$1 circles”), and robustness across a range of elasticity parameters compared to alternatives such as DSOM. The neighborhood radius $\Sigma^{-1} = W$2 is automatically adjusted in response to nonstationary data (Fillion et al., 2022).

GNNs

BNA for GNNs yields improvements in node classification accuracy and uncertainty calibration (lower ECE) across diverse datasets (homophilic: Cora, Citeseer; heterophilic: Chameleon, Texas). The inferred receptive field (scope) aligns with domain knowledge (e.g., a 3-hop effective neighborhood for protein-protein interaction networks maximizes AUROC) (Regmi et al., 5 Feb 2026). Table excerpt:

Method	Cora	Chameleon
ResGCN	86.2	66.0
Ours+ResGCN	86.8	64.3
GCNII	87.5	59.0
Ours+GCNII	87.3	57.4

5. Practical Guidance and Implementation

• In Markov blanket estimation, $\Sigma^{-1} = W$3 (query set size) should remain modest ($\Sigma^{-1} = W$4) for MGIG efficiency, and the Laplace scale $\Sigma^{-1} = W$5 can be chosen via cross-validation or empirical Bayes. For mixed data, Gaussian-copula augmentation is recommended (Kaufmann et al., 2015).
• Bayesian k-NN requires only feature scaling and hazard rate tuning (to control mean $\Sigma^{-1} = W$6). Run-length recursion is robust to truncation, and applicable for binary, categorical, or Gaussian data via corresponding exponential-family updates (Nuti, 2017).
• In SOMs, the neighborhood radius is “soft” and variationally controlled, removing any need for hand-tuned cooling schedules. Stochastic gradient updates permit large-scale training (Fillion et al., 2022).
• For GNNs, BNA is compatible with standard architectures (GCN, ResGCN, GAT, JKNet, GCNII, ACM-GCN+), incurs minor computational overhead, and provides uncertainty calibration for the receptive field. The default $\Sigma^{-1} = W$7 samples suffices for accurate ELBO estimation in practice (Regmi et al., 5 Feb 2026).

6. Extensions, Limitations, and Research Directions

• BNA in GNNs is currently limited by the truncation depth of the variational posterior; unbiased “Russian roulette” samplers could in principle remove this bound.
• Integration with more advanced GNNs or graph transformers is plausible due to the generality of the masking and beta process structure.
• In all domains, BNA enables uncertainty quantification over the neighborhood itself, contributing to better-calibrated predictions.
• The blockwise and variational decompositions exploited by BNA suggest that further scalability gains may be achievable via distributed or memory-efficient implementations.

7. Relation to Broader Bayesian and Network Literature

BNA synthesizes blockwise inference, nonparametric priors (beta processes, change-point models), and variational Bayesian principles to address the challenge of local model selection within large and complex networks. By replacing externally imposed choices (e.g., neighborhood size, number of hops, neighborhood radius) with statistically inferred latent variables, BNA frameworks provide both computational and statistical improvements for structure learning, nonparametric prediction, and unsupervised network modeling (Kaufmann et al., 2015, Nuti, 2017, Fillion et al., 2022, Regmi et al., 5 Feb 2026).