Cluster-Based Filter-Refine Algorithms

• The paper introduces a two-phase framework that decouples efficient filtering of high-density clusters from the refinement of ambiguous or noisy points.
• The algorithm leverages local structures by first removing low-confidence instances and then reassigning pruned points using nearest neighbor or centroid-based methods.
• Empirical benchmarks across density-based, tree-based, and deep representation domains demonstrate state-of-the-art robustness and computational efficiency.

Cluster-based filter-and-refine algorithms constitute a versatile class of methods that explicitly separate the process of identifying high-confidence or high-density structures (“filter”) from the subsequent assignment or refinement of ambiguous, noisy, or low-confidence instances (“refine”). This approach has been developed independently across multiple subfields—density-based clustering, large-scale data filtering, sequential Monte Carlo inference, and neural network interpretability—with rigorous algorithmic procedures, complexity analyses, and empirical benchmarks. Central to this paradigm is the exploitation of local or cluster-wise structure for efficient reduction of complexity and robust handling of noise, outliers, or multimodality.

1. Principle and Algorithmic Framework

The core principle of cluster-based filter-and-refine methods is the decomposition of clustering or filtering into two phases:

1. Filter Phase: Apply a computationally tractable procedure to identify core, high-density, or high-purity clusters and/or to prune or mark low-density, outlier, or ambiguous points.
2. Refine Phase: Assign or re-integrate pruned points, ambiguous items, or low-confidence areas using cluster assignments, centroids, affinity structure, or probabilistic inference.

This scheme appears under various guises, including density-based data clustering (SACA (Bilehsavar et al., 23 Aug 2025), BD-DBSCAN (Zhao, 2024)), hierarchical and tree-based data selection (TBDFiltering (Busa-Fekete et al., 29 Jan 2026)), particle filter partitioning (Structured Filtering (Granade et al., 2016)), and deep representation summarization (CF-CAM (He et al., 31 Mar 2025)). The framework is agnostic to the structure of the input data—it is equally applicable to vector spaces, similarity graphs, hierarchical trees, or deep feature tensors.

2. Density- and Graph-Based Instantiations

Selective Attention-Based Clustering Algorithm (SACA)

SACA operationalizes the filter-and-refine approach on $\mathbb{R}^n$ data as follows (Bilehsavar et al., 23 Aug 2025):

• Filter: Compute nearest-neighbor distances, remove statistical outliers via the Modified Z-score, and determine a global threshold $T$ derived from robust statistics on these distances. For each point, count the number of neighbors within $T$; prune points with counts below the selectivity coefficient $C$.
• Refine: Form preliminary clusters via connected components in the induced core-point graph. Reintegrate pruned points by assigning each to its nearest core point or, optionally, to the nearest cluster centroid.

This two-pass method achieves robust performance without parameter tuning in most practical settings and adapts the neighborhood radius to the data distribution, overcoming limitations of DBSCAN’s fixed $\varepsilon$ (Bilehsavar et al., 23 Aug 2025).

Block-Diagonal Guided DBSCAN (BD-DBSCAN)

BD-DBSCAN extends DBSCAN by embedding cluster-based filter-and-refine in high-dimensional affinity graphs (Zhao, 2024):

• Filter: Employ a DBSCAN-style density traversal on a similarity graph to yield initial high-density clusters (seeds) and a coarse block-diagonal point ordering.
• Refine: Optimize a global permutation to approximate a block-diagonal structure (by gradient descent in the space of doubly-stochastic matrices, projecting back to permutations via the Hungarian algorithm), then recursively split diagonal blocks (if intra-block and off-block affinities fail a specified criterion) to further purify and resolve boundaries.

Empirical evaluation demonstrates that this two-stage process achieves state-of-the-art accuracy and mutual information on challenging, high-dimensional datasets, showing particular robustness to non-convex and variable-density structures (Zhao, 2024).

3. Hierarchical and Adaptive Tree-Based Filters

TBDFiltering

TBDFiltering applies a cluster-based filter-and-refine framework to document quality curation for large-scale LLM training (Busa-Fekete et al., 29 Jan 2026):

• Filter: Construct a hierarchical clustering tree $\mathcal{T}$ over embedded documents; iteratively sample leaves of active nodes, using a limited number of expensive quality oracle queries. Clusters are “kept” or “discarded” based on empirical means crossing tunable $\alpha, \beta$ thresholds; uncertain clusters recurse to children.
• Refine: Upon termination, propagate cluster-level “keep” or “discard” decisions directly to constituent documents; no further per-document inference is needed.

This approach provably minimizes query complexity under cluster purity assumptions and yields significant improvements in LLM downstream performance while issuing far fewer expensive LLM-based quality prompts compared to classifier-based filters (Busa-Fekete et al., 29 Jan 2026).

4. Sequential Monte Carlo and Bayesian Inference

Structured Filtering

Structured Filtering addresses multimodal posterior approximation in sequential Monte Carlo by integrating a filter-and-refine strategy at each Bayesian update step (Granade et al., 2016):

• Filter: Following each new measurement, update particle weights across all filter-nodes; propagate these changes through a structure graph that encodes alternative cluster (mode) hypotheses.
• Refine: When the effective sample size (ESS) of a filter-node falls below threshold, resample or split the node into clusters (via weighted $k$-means). New clusters are resampled and introduced as children under a mixture/decision node, retaining hypotheses over cluster counts. Pruning rules (champion, floor, only-child, single-child) control expansion and shrinkage of the structure graph.

The splitting and selection process automatically adapts to the true number of posterior modes, outperforming unstructured SMC in multimodal settings (e.g., randomized gap estimation, phase estimation in quantum systems) (Granade et al., 2016).

5. Deep Representation and Interpretability

CF-CAM: Cluster Filter Class Activation Mapping

CF-CAM exemplifies the filter-and-refine model over convolutional network channel activations (He et al., 31 Mar 2025):

• Filter: Separate dominant channels by $L_2$-norm thresholding; for the remainder, use DBSCAN to cluster semantically related channels and discard noise-prone ones.
• Refine: For each cluster, smooth gradients (which serve as saliency weights) using Gaussian filtering within-channel clusters, mitigating sensitivity to noise. Generate the final class activation map by softmax-weighted fusion of filtered channels.

The method achieves superior trade-offs in faithfulness, robustness, and efficiency versus prior gradient-based and gradient-free CAMs, with rigorous ablation showing distinct gains from channel clustering and filtering (He et al., 31 Mar 2025).

Algorithm	Filter Step	Refine Step	Domain
SACA (Bilehsavar et al., 23 Aug 2025)	Remove low-density/outlier points via $T$	Assign pruned points by NN or centroid	Euclidean clustering
BD-DBSCAN (Zhao, 2024)	Density-driven traversal and block-order	Permutation learning and split-and-refine	Similarity graphs/high-dim clustering
TBDFiltering (Busa-Fekete et al., 29 Jan 2026)	Top-down sample/query clusters	Propagate keep/discard to leaves	Hierarchical, LLM data curation
Structured Filter (Granade et al., 2016)	Bayes update/pruning on SMC tree	Cluster splitting/resampling and model selection	Bayesian/Sampling inference
CF-CAM (He et al., 31 Mar 2025)	L2 and DBSCAN-based channel selection	Gaussian gradient filtering, channel re-weighting	CNN model interpretability

6. Analysis of Empirical Performance and Complexity

Algorithmic complexity varies by domain but adheres to a common pattern: the filter step is designed for computational efficiency (e.g., $T$0 with trees or $T$1 with complete pairwise distances), often leveraging data structure (trees, graphs, KD-trees) or redundancy (channel similarity). The refine step, while potentially more costly in the worst case, operates on a reduced or structured subset (core points, cluster blocks, pruned leaves). Notable outcomes include:

• SACA operates in $T$2 but can be accelerated to $T$3 with approximate neighborhoods (Bilehsavar et al., 23 Aug 2025).
• BD-DBSCAN achieves comparable or faster runtime than spectral and block-diagonal graph learners, with provable optimal block recovery under ideal separation (Zhao, 2024).
• TBDFiltering reduces LLM oracle queries by an order of magnitude (<10% of data queried), dominating classifier-based approaches in sample efficiency and final downstream model quality (Busa-Fekete et al., 29 Jan 2026).
• CF-CAM attains substantial runtime reductions (712 ms per sample vs. 8.7 s for Ablation-CAM) while maintaining interpretability and robustness, and clearly surpasses prior art in faithfulness/robustness trade-offs (He et al., 31 Mar 2025).

7. Algorithmic Variations and Domain Adaptations

Cluster-based filter-and-refine algorithms accommodate substantial flexibility:

• Parameter-free or single-parameter operation (SACA, BD-DBSCAN), minimizing the need for expert tuning.
• Adaptation to quality, density, or multimodality structure, both in classical (DBSCAN, SMC) and modern large-scale (LLM curation, deep feature) settings.
• Domain-specific tailoring: e.g., TBDFiltering’s tree-based adaptive querying is aligned to document corpus scale and oracle cost; CF-CAM’s integration of DBSCAN addresses fragility in neural gradient interpretability.

A plausible implication is that the filter-and-refine paradigm is particularly suited to scenarios with intrinsic heterogeneity, uncertainty, or label/query costs. Empirically, these approaches demonstrate superior or state-of-the-art performance across tasks involving noise, density variation, hierarchical structure, and interpretability.

---

References:

• (Bilehsavar et al., 23 Aug 2025) (SACA: Selective Attention-Based Clustering Algorithm)
• (Zhao, 2024) (Block-Diagonal Guided DBSCAN Clustering)
• (Busa-Fekete et al., 29 Jan 2026) (TBDFiltering: Sample-Efficient Tree-Based Data Filtering)
• (Granade et al., 2016) (Structured Filtering)
• (He et al., 31 Mar 2025) (CF-CAM: Cluster Filter Class Activation Mapping for Reliable Gradient-Based Interpretability)