--- title: Decomposition Sampling (DECOMP) url: https://www.emergentmind.com/topics/decomposition-sampling-decomp type: topic --- # Decomposition Sampling (DECOMP) Decomposition Sampling (DECOMP) refers to a class of sampling, algorithmic, and pre-processing frameworks unified by the principle of breaking up a complex object—sample space, model, signal, or combinatorial pattern—into sub-components amenable to independent or conditional sampling. The unifying goal of DECOMP methods is to exploit structured decompositions (spatial, combinatorial, algebraic, or class-based) to accelerate sampling, eliminate bias, enforce causality, or reduce computational/memory complexity. Methodologies are domain-specific and include subspace overlaps, motif factorizations, sequential window decompositions, tensor-network factor sampling, and class-conditioned region selection. ## 1. Conceptual Foundations and Variants of Decomposition Sampling DECOMP frameworks are defined by their partitioning or decomposing of the space of interest—be it probability measures, graphs, time series, tensors, or image regions—so that sampling can be carried out within or across components, often in parallel or with improved efficiency. Prominent DECOMP designs have appeared in: - Monte Carlo and Markov chain Monte Carlo (MCMC) as parallelization via overlapping covers, supporting independent subchain sampling and subsequent recombination [1402.2828]. - Graph motif sampling/counting, where a target motif $H$ is optimally decomposed into collections of odd cycles $O_k$ and stars $S_p$, with sampling routines tailored per component [2107.06582]. - Signal processing, via stepwise or causal decomposition schemes in hybrid time series forecasting, preventing future-data leakage by stepwise, window-contingent decompositions [2309.10658]. - Tensor methods, through sampling-based factor selection within alternating least squares, informed by leverage scores derived from the current tensor decomposition [2210.03828]. - Bayesian graphical models, where decomposable graphs are explored using Markov chains on junction trees, and permissible multi-edge updates are classified by explicit decomposition partitions of the clique structure [2209.02008]. - Active learning in dense prediction tasks, where pseudo-label-driven class-based decomposition of candidate annotation regions ensures diversity and minor-class coverage [2512.07606]. ## 2. Algorithmic Structures and Mathematical Formulations Core algorithmic elements are specific to the domain but recurrent aspects include: ### Markov Chain & Parallelization Frameworks In parallel MCMC (Metropolis–Hastings), DECOMP constructs a “linked cover” $X = \cup_{i=1}^K X_i$ with overlapping regions. Separate MCMC chains are run on $\widetilde\pi_i(x) = \pi(x)\mathbf{1}_{X_i}(x)/Z_i$ per subset, followed by downsampling and merging to recover samples from the target $\pi$. The method guarantees unbiased estimates under correct normalization, and enables near-ideal speedup proportional to $K$ [1402.2828]. ### Motif Sampling in Graphs For motif $H$ and graph $G$, the canonical decomposition $D^*(H)$ consists of odd cycles and stars obtained via edge fractional covering. DECOMP samples one representative subgraph per component (using StarSampler, CycleSampler) and tests for a successful embedding. The expected query cost is controlled by the decomposition-cost metric, which depends on component counts $o_k$, $s_p$ and target count $h$ [2107.06582]. ### Stepwise/Sequential Decomposition in Signals For a time series $S = \{S_1,\ldots,S_N\}$, the fully stepwise decomposition-based (FSDB) sampling method grows an explanatory window and applies a decomposition operator (e.g., SSA or VMD) at each time, extracting inputs and targets without reference to future data. This strictly preserves real-time causality, avoiding optimistic bias in downstream machine learning models [2309.10658]. ### Leverage-Score-Based Sampling in Tensor Networks ALS-based tensor decomposition leverages leverage score distributions over the design matrices to sample constraints when solving least-squares subproblems at sublinear cost. Each factor update in the network is performed using a small, adaptively sampled batch whose rows are selected according to the exact leverage scores, ensuring $(1+\epsilon)$-approximation error [2210.03828]. ### Partition-Aware Junction Tree Proposals in Graphical Models In decomposable-graph inference, DECOMP partitions permissible edge updates into compatible sets (neighboring cliques or leaf cliques) enabling simultaneous and independent (parallel) MCMC moves within a junction tree. This retains decomposability while increasing mixing relative to single-move proposals [2209.02008]. ### Class-Wise Region Sampling in Active Learning For dense prediction, DECOMP decomposes each sample’s region set according to pseudo-predicted classes, samples class-specific regions weighted by class-level predictive uncertainty, and thus ensures that hard and underrepresented classes are prioritized for annotation [2512.07606]. ## 3. Application Domains and Method Integration ### Time Series Forecasting FSDB sampling has been successfully deployed for water-level forecasting. Integrating FSDB with SSA or VMD produces $K$ components per window, on which $K$ regressors are trained, and their outputs are aggregated to recover the signal forecast. This avoids information leakage and systemic overfitting observed in previous (ODB, KN, SDB) schemes. Empirical results on Chinese river basins demonstrated Nash-Sutcliffe Efficiency (NSE) gains of 1.1–28.8% over previous state-of-the-art samplers [2309.10658]. ### Subgraph Counting and Motif Sampling DECOMP algorithms enable sublinear-time uniform sampling/counting for arbitrary graph motifs. For motifs with favorable decompositions (many star components), polynomial improvements in sample complexity are possible—especially in sparse or low-arboricity graphs. Tight lower bounds, matching algorithmic upper complexity, are established for decompositions containing odd cycles [2107.06582]. ### Tensor Decompositions DECOMP achieves input-sublinear per-iteration complexity in ALS for CP and tensor-ring decompositions, using exact leverage scores on the matricized network’s design matrix. Empirical feature extraction experiments show DECOMP-CP and DECOMP-TR outperform classical deterministic and randomized ALS methods in runtime while maintaining comparably low relative errors and high classification accuracy [2210.03828]. ### Parallel MCMC and Bayesian Graphical Models Within MCMC, DECOMP enables independent parallel simulation across overlapping subregions, with unbiased expectation estimation and accelerated decorrelation, as demonstrated in both synthetic and real calibration problems, including particle marginal MH for stochastic volatility [1402.2828]. In graphical model structure learning, the parallelDG Python package implements DECOMP-based partitioned parallel MCMC over high-dimensional junction trees, outperforming prior art in mixing and ROC accuracy [2209.02008]. ### Active Learning for Dense Prediction Region-based DECOMP AL provides state-of-the-art annotation efficiency in ROI classification, 2D, and 3D segmentation. By decomposing regions by class and proportionally allocating annotation budget, it achieves superior coverage of minority and hard classes. For BRACS, Cityscapes, and KiTS23, DECOMP reached 95% of full-annotation performance with 40%, 2.4%, and 0.15% of the annotation budget respectively—substantially surpassing random, uncertainty, and clustering-based baselines [2512.07606]. ## 4. Theoretical Guarantees and Performance Analysis DECOMP frameworks generally preserve the target distribution (or estimation objective) provided that overlap correction, sufficient sample size, or proper window selection is enforced. - Parallel MCMC: Guarantees unbiasedness and TV-convergence as long as block-weights converge and subchain kernels are irreducible/aperiodic [1402.2828]. - Graph motifs: Delivers uniform random samples at a cost scaling with the maximum per-component sampling cost and target count, with optimality characterized by decomposition structure [2107.06582]. - Stepwise signal decomposition: Empirically shown to avoid over-optimistic validation metrics, eliminate future-data bias, and outperform less-causal alternatives [2309.10658]. - Tensor networks: (1+$\epsilon$)-relative error is achieved with high probability with a sample size proportional to the rank [2210.03828]. - Active learning: DECOMP’s annotation efficiency and class-coverage are tied to the decomposition’s ability to reflect class distribution and model uncertainty; ablation studies confirm distinct contributions from both image and region selection phases [2512.07606]. ## 5. Implementation Considerations and Practical Guidelines - Cover and overlap selection are fundamental; for MCMC, quantile-based splitting informed by pilot runs is recommended, with overlap sizes tuned to ensure all regions are well-sampled [1402.2828]. - In time-series prediction, lag $m$ must reflect system memory, and decomposition parameters (window length, mode count, penalty) are chosen via hyperparameter search on held-out data [2309.10658]. - For motif-sampling, computing exact counts of stars/cycles and optimal decompositions of $H$ is necessary for optimality; in practice, this automation is feasible for moderate-size motifs [2107.06582]. - In tensorized settings, the TN must admit fast contraction and affordable Gram inverses; leverage-score sampling is then implemented using sequential conditional evaluations on the TN graph [2210.03828]. - For active learning, class confidence thresholds are robust; efficient implementation leverages integral images and avoids high-dimensional region feature indexing [2512.07606]. - Graphical model samplers exploit parallel execution across partitions; empirical performance degrades mildly with increasing dimensionality but is mitigated by adjusting prior penalties and skeleton-update frequencies [2209.02008]. ## 6. Extensions, Open Problems, and Comparative Evaluation - For graph motif DECOMP, open problems include (i) removal of suboptimal polylogarithmic factors, (ii) achieving tight lower bounds for star-dominated decompositions, and (iii) extending to non-canonical motif covers [2107.06582]. - In parallel MCMC, the efficacy depends on block choice and the dimensionality regime; high-dimensional spaces or those with complex geometry may challenge naive covers [1402.2828]. - Weaknesses of decomposition-based signal forecasting arise in highly non-stationary scenarios, where ML models without decomposition may outperform all hybrids [2309.10658]. - In AL, region-level DECOMP outcompetes feature-clustering and uncertainty-only strategies especially in minority and poorly predicted classes, but total annotation gain is bounded by the accuracy of pseudo-labels and class granularity [2512.07606]. | Application | Core Decomposition Mechanism | Key Theoretical Guarantee | |-----------------------|----------------------------------|-------------------------------------| | MCMC Parallelization | Overlapping subspace covers | TV-convergent, unbiased expectations| | Motif Sampling | Star/cycle LP-motif decomposition| Query cost tied to decomp-cost | | Signal Forecasting | Stepwise sliding window decomps | Strict causal/no-leakage learning | | Tensor ALS | Leverage-score constrained samps | Sublinear, $(1+\epsilon)$ error | | Graphical Models | Clique-partition parallelism | Correct, fast-mixing junction tree | | Active Learning | Class-conditioned region sets | Optimal budget-diversity tradeoff | ## 7. Summary and Impact Decomposition Sampling methods leverage structural decomposability to improve sampling fidelity, computational efficiency, and downstream statistical or learning performance. By aligning the sampling methodology with inherent substructure—be it topological, algebraic, sequential, or semantic—DECOMP enables rigorous guarantees and it drives advances in high-dimensional inference, combinatorial optimization, time series prediction, tensor learning, and annotation-efficient active learning. Open directions involve dynamic or recursive decompositions, extension to latent-variable models, and formal analysis of performance gains in non-canonical or adversarial settings [1402.2828, 2107.06582, 2210.03828, 2309.10658, 2209.02008, 2512.07606]. Source: https://www.emergentmind.com/topics/decomposition-sampling-decomp