Distance-Dependent Indian Buffet Process

• The dd-IBP is a non-exchangeable Bayesian nonparametric model that extends the Indian buffet process by incorporating explicit distance metrics to capture spatio-temporal and covariate-based feature dependencies.
• It employs a generative process where each data point draws new features based on a normalized proximity matrix and inherits features through distance-dependent connections.
• The model has demonstrated practical benefits in applications such as Alzheimer’s MRI classification and EEG reconstruction, despite inference challenges due to its complex sampling-based approach.

The distance-dependent Indian buffet process (dd-IBP) is a non-exchangeable Bayesian nonparametric prior for infinite latent feature models. It extends the Indian buffet process (IBP) by encouraging data points proximate in an explicit distance metric to share more latent features, thereby capturing spatio-temporal and covariate-based dependencies that violate the exchangeability assumption inherent in the classical IBP. The dd-IBP offers a flexible framework in which the pairwise structure of data determines both feature allocation and sharing, with recoverability of the standard IBP in the limiting case of uniform proximity.

1. Generative Framework and Model Specification

The dd-IBP defines a prior over an $N \times \infty$ binary matrix $Z=(z_{ik})$ representing feature inclusion for $N$ objects (or "customers") and countably infinite latent features ("dishes"). The generative process proceeds as follows (Gershman et al., 2011):

1. New-dish ownership: Each customer $i$ draws $\lambda_i \sim \operatorname{Poisson}(\alpha/h_i)$ and owns $\mathcal{K}_i$ new features, where $h_i = \sum_{j=1}^N f(d_{ij})$ is a normalization of proximity weights and $\alpha$ is the mass parameter.
2. Distance-dependent connections: For every existing feature $k$, customer $i$ chooses a customer $j$ to "link" with probability $a_{ij} = f(d_{ij})/h_i$, where $f$ is a non-increasing decay function mapping distances $d_{ij}$ to affinities.
3. Dish inheritance: For feature $k$ with owner $c^*_k$, customer $i$ sets $z_{ik} = 1$ if and only if a directed path exists from $i$ to $c^*_k$ in the corresponding connectivity graph defined by the $c_{ik}$ link assignments.

This yields a binary matrix $Z$ as a deterministic function $\phi(c^*, C)$ of ownership and connectivity assignments. The joint prior is: $$P(Z \mid D, \alpha, f) = \sum_{(c^*,C):\,\phi(c^*,C)=Z} P(c^* \mid \alpha) P(C \mid c^*, D, f)$$ with Poisson and multivariate multinomial factors for new-dish ownership and connection assignments, respectively.

The dd-IBP reduces to the original IBP under a sequential distance structure ($d_{ij} = \infty$ for $j > i$) and constant decay $f(d) \equiv 1$.

2. Distance Metrics, Decay Functions, and Affinity Structure

The proximity structure is encoded by a user-supplied symmetric (or optionally asymmetric) distance matrix $D=(d_{ij})$, with typical choices:

• Temporal ($d_{ij} = |i-j|$ or $d_{ij} = \infty$ for $j>i$)
• Spatial (Euclidean/geodesic in $\mathbb{R}^p$)
• Covariate-based (arbitrary non-negative dissimilarities)

A decay kernel $f: \mathbb{R}^+ \cup \{\infty\} \rightarrow [0,1]$ then translates these distances to unnormalized affinities, with normalization $h_i = \sum_j f(d_{ij})$ yielding the normalized proximity matrix $a_{ij}$. Standard forms include:

• Exponential: $f(d) = \exp(-\beta d)$
• Logistic: $f(d) = 1/(1 + \exp(\beta d - \nu))$
• Window: $f(d) = \mathbf{1}[d<\nu]$
• Constant: $f(d) \equiv 1$

These choices enable flexible modeling of smooth or blockwise structure in latent feature sharing.

3. Theoretical Feature-Sharing Properties

Let $R_i = \sum_k z_{ik}$ denote the total number of features for object $i$ and $R_{ij} = \sum_k z_{ik} z_{jk}$ the number shared by $i$ and $j$. Define the reachability indicator $\mathcal{L}_{in}$ as 1 if $i$ can reach the owner $n$ of a feature, 0 otherwise.

Under finite $\alpha$: $$R_i \sim \operatorname{Poisson}\left(\alpha \sum_{n=1}^N h_n^{-1} P(\mathcal{L}_{in} = 1)\right),\qquad R_{ij} \sim \operatorname{Poisson}\left(\alpha \sum_{n=1}^N h_n^{-1} P(\mathcal{L}_{in} = 1, \mathcal{L}_{jn} = 1)\right)$$ As $\alpha \to \infty$, the normalized ratios $\frac{R_i}{\alpha}$ and $\frac{R_{ij}}{\alpha}$ converge in distribution to the expected reachabilities under the proximity structure.

A salient feature is that the fraction of features shared between $i$ and $j$ varies smoothly with their pairwise distance, in contrast to the IBP, where $R_i \sim \operatorname{Poisson}(\alpha)$ and $$R_{ij} \sim \operatorname{Poisson}(\tfrac12 \alpha)$$ for all $(i,j)$, enforcing universal exchangeability in the infinite feature limit.

4. Inference Methodology

Inference in the dd-IBP is typically performed via Markov chain Monte Carlo. In a generic latent-feature model with $$X_i | z_{i\cdot}, \theta \sim P(x_i \mid z_{i\cdot}, \theta)$$, the posterior over $(C, c^*, \alpha, \theta)$ is proportional to the product of the likelihood, dd-IBP prior factors, and hyperpriors.

Key Gibbs/MH steps include:

• Hyperparameter $\alpha$: Conjugate gamma update if $P(\alpha)$ is gamma.
• Feature assignments for owned dishes: For each customer $i$ and active feature $k$, sample new connection $c_{ik}$ proportional to $a_{ij} P(x_i \mid Z^{(i,k \leftarrow j)}, \theta)$.
• Owner allocations (dish birth and death): Metropolis proposals for new Poisson draws and feature reallocations, with acceptance given by likelihood ratio.
• Model parameters ($\theta$): Updated by Gibbs or Metropolis–Hastings as appropriate; for the linear-Gaussian likelihood ($X = ZW + \varepsilon$), the marginal likelihood can be computed in closed form after integrating out the weight matrix $W$.

The joint prior and latent variable dependencies render marginalization of $Z$ intractable, requiring explicit sampling of the connectivity graph $(C, c^*)$.

5. Relation to the IBP and Other Non-Exchangeable Extensions

The dd-IBP strictly generalizes the IBP. It recovers the IBP in the sequential, uniform-affinity case. The standard IBP imposes exchangeability, ensuring that all pairs share $\tfrac12$ of their features as $\alpha \to \infty$. The dd-IBP allows arbitrary sharing patterns dictated by the distance metric and decay function.

Comparison with the attraction Indian buffet distribution (AIBD) (Warr et al., 2021) highlights several distinctions:

Property	dd-IBP	AIBD
Marginal on $Z$	Intractable, requires sampling $(C, c^*)$	Explicit, tractable PMF
Feature counts	Dependent on $h_i$, typically $\geq$ IBP	Exactly as in the IBP
Exchangeability	Only under special proximity and kernel choices	Recovers IBP for $\tau = 0$ or uniform $f$
Asymmetric D	Supported	Requires symmetric distance matrix

A key distinction is that the dd-IBP’s feature introduction process is itself distance-dependent (via $h_i$), while AIBD preserves the IBP’s new-feature marginal and imposes dependence only in the sharing of existing features.

6. Practical Impact and Applications

The dd-IBP is most appropriate in settings where latent structure is expected to follow non-exchangeable, locally correlated patterns. Notable case studies include:

• Alzheimer’s MRI classification: Age-sequential distance, exponential decay $f(d) = \exp(-\beta d)$, resulting in improved AUC for feature-driven classifiers compared to IBP, the dependent IBP, and hierarchical beta processes.
• EEG missing-data reconstruction: Temporal distance from timestamps, with the dd-IBP linear-Gaussian model yielding lower squared imputation error than baseline priors (Gershman et al., 2011).

The model augments classical infinite latent feature models by introducing flexible, distance-aware structure, with applications in longitudinal studies, spatial genomics, and similar domains.

7. Limitations and Computational Considerations

Inference with the dd-IBP can be computationally intensive, as the marginal over $Z$ is intractable, necessitating sampling of the full connectivity graph. Mixing can be slow due to dependencies among assignments. AIBD offers a tractable alternative in cases where symmetry is present and maintenance of IBP-like feature marginals is desired (Warr et al., 2021). Nonetheless, the dd-IBP offers unique modeling flexibility for hierarchical and asymmetric dependency structures not accessible to exchangeable or permutation-invariant priors.

A plausible implication is that, in large-scale applications where interpretability of distance effects and computational efficiency are equally important, tradeoffs between dd-IBP and alternatives such as AIBD or other dependent feature allocation models must be carefully considered.