Bidirectional Semantic Filtering

• Bidirectional semantic filtering is a framework that couples discrete semantic states with continuous properties to enable robust filtering and dynamic adaptation.
• Key methodologies include probabilistic semantic-parameter coupling, mutual entailment in vision-language models, and adversarial defense through semantic mapping.
• These techniques enhance uncertainty quantification and efficiency in applications like vision-language verification, online adaptive systems, and weakly supervised temporal localization.

Bidirectional semantic filtering refers to a family of model architectures and algorithmic strategies that exploit bidirectional coupling between semantic representations and non-semantic (often continuous or high-dimensional) information. This allows for robust filtering, fusion, uncertainty quantification, and adaptation in dynamic, weakly supervised, or adversarial settings. Research threads employing bidirectional semantic filtering include probabilistic semantic-parameter inference, vision-language answer verification, adversarial robustness, and weakly supervised temporal localization. Theoretical and practical advances combine forward and backward information flow, often leveraging moment-matching, mutual entailment, or cross-supervision.

1. Bidirectional Semantic Filtering: General Principles

In bidirectional semantic filtering, models are structured to maintain an explicit, mutual dependency between discrete semantic states and associated continuous or structured properties. This bidirectionality can refer to two distinct but complementary mechanisms:

• Semantic-parameter coupling: The joint filtering of semantic classes and their associated properties, allowing updates of parameters to influence class posteriors and vice versa.
• Forward-backward consistency constraints: Requiring system outputs on perturbed (e.g., augmented or adversarial) inputs to be consistent with outputs on the originals, applied in both directions.

Mechanisms for bidirectional filtering include jointly formulated posteriors (e.g., Dirichlet-NormalGamma families), mutual entailment checks between model answers, and cross-supervised losses between original and context-modified samples. These approaches support improved robustness, uncertainty estimation, and semantic fidelity in diverse inference settings (Greiff et al., 14 Jan 2026, Wienholt et al., 10 Oct 2025, Li et al., 2023, Bao et al., 2018).

2. Probabilistic Filtering with Semantic-Parameter Coupling

A formal instantiation of bidirectional semantic filtering is the semantic map parameter association framework introduced in "Dynamic Association of Semantics and Parameter Estimates by Filtering" (Greiff et al., 14 Jan 2026). Here, the latent state at each spatial point is modeled as

• $w$: Class weight vector (Dirichlet-distributed, $p(s=i)=w_i$ over closed set of $K$ semantic classes),
• $(m_i, \tau_i)$: Class-conditional continuous parameter mean and precision (NormalGamma-distributed for each class $i$, dimension $J$).

The prior is a product of Dirichlet and NormalGamma factors: $$p(\theta;P) = Dir(w|a) \prod_{i=1}^K NG(m_i, \tau_i|\mu_i, \lambda_i, \alpha_i, \beta_i)$$, supporting a closed set of semantic classes with parameterized property distributions.

The posterior after observing a property measurement $y$ is a $K$-term mixture:

$$p(\theta|y) = \frac{1}{M} \sum_{j=1}^K u_j \, Dir(w|a+e_j) \, c_j^* NG^*(m_j, \tau_j) \prod_{i\neq j} NG(m_i, \tau_i)$$

composing semantic evidence (Dirichlet increments) and property updates.

The sequential filtering algorithm (Algorithm 1) involves:

• Time-prediction (exponential forgetting of hyperparameters towards $p(s=i)=w_i$0),
• Measurement update: Dirichlet increment on semantic observation, $p(s=i)=w_i$1-mixture posterior with moment matching on continuous observations,
• Bayesian moment matching (BMM) to project the mixture posterior back into the conjugate family.

This construct "bidirectionally" fuses semantic and parametric knowledge for each update step, ensuring that semantic evidence can influence parameter estimates (and vice versa). Efficient recursive updates result, with computational complexity scaling linearly in the joint space dimension ($p(s=i)=w_i$2). In driving domain experiments (tire friction on dynamically evolving surfaces), dynamic BMM enables online adaptation to time-varying conditions, outperforming static or uni-directional filtering (Greiff et al., 14 Jan 2026).

3. Bidirectional Semantic Filtering for Vision-Language Hallucination Suppression

In neuro-symbolic and VLM applications, bidirectional semantic filtering is realized as mutual entailment-based answer clustering combined with entropy-based filtering (Wienholt et al., 10 Oct 2025). The approach is as follows:

1. Sample high-temperature answers: For an input $p(s=i)=w_i$3, generate $p(s=i)=w_i$4 model outputs $p(s=i)=w_i$5.
2. Bidirectional clustering: For each pair $p(s=i)=w_i$6, perform mutual entailment checks (model-prompted: $p(s=i)=w_i$7 and $p(s=i)=w_i$8). Form disjoint answer clusters $p(s=i)=w_i$9 such that each within-cluster pair is bidirectionally entailed.
3. Discrete Semantic Entropy (DSE): Compute $K$0 and $K$1.
4. Filtering: Reject question $K$2 if $K$3 (typical $K$4 or $K$5). Only non-rejected instances are returned.

Empirically, this yields statistically significant accuracy gains on medical VQA datasets: for instance, GPT-4o accuracy rises from 51.7% baseline to 76.3% at $K$6, covering 47.3% of the questions (Wienholt et al., 10 Oct 2025).

Bidirectionality ensures that only semantically consistent, mutually entailed answers are clustered. High DSE identifies prompts with semantic inconsistency (often hallucinated or unreliable outputs) for filtration. Limitations include the method's inability to filter confidently hallucinated answers and dependence on the quality of model entailment judgments.

4. Adversarial Defense via Bidirectional Semantic Mapping

Bidirectional semantic filtering is foundational to adversarial defense in the Featurized Bidirectional GAN (FBGAN) (Bao et al., 2018). The architecture comprises:

• Encoder $K$7: Projects input $K$8 into low-dimensional, disentangled "semantic codes" $K$9 using adversarially trained convolutional nets.
• Generator $(m_i, \tau_i)$0: Decodes codes $(m_i, \tau_i)$1 to image space.
• Discriminator $(m_i, \tau_i)$2: Evaluates joint likelihoods; encourages alignment between $(m_i, \tau_i)$3 (data→latent) and $(m_i, \tau_i)$4 (latent→data).
• Mutual Information regularization: Promotes semantic, interpretable representations in $(m_i, \tau_i)$5.

At inference, adversarial examples are "filtered" through $(m_i, \tau_i)$6, which maps all inputs to the semantic manifold learned on clean data. Optional code-regularization (e.g., quantization/clip) further projects $(m_i, \tau_i)$7 to a canonical value. Downstream classifiers process only the reconstructed outputs.

Quantitative defenses are strong: on MNIST under $(m_i, \tau_i)$8-bounded attacks ($(m_i, \tau_i)$9, PGD/FGSM), accuracy after FBGAN filtering remains above 85–90%, versus $i$01% for standard classifiers (Bao et al., 2018). Ablation confirms mutual information components are required for this semantic structure and rejection of non-semantic noise.

5. Cross-supervised Bidirectional Semantic Consistency for Weak Supervision

In weakly-supervised temporal action localization (WTAL), bidirectional semantic filtering is embodied as a bidirectional semantic consistency constraint (Bi-SCC) (Li et al., 2023):

• Temporal context augmentation: Breaks action-context correlations by inter- and intra-video context swaps to produce context-perturbed feature sequences.
• Semantic consistency constraint: KL divergence between background-suppressed T-CAMs (temporal class activation maps) of original and augmented samples.
• Bidirectionality (Bi-SCC): Cross-supervision between original and augmented branches, coupled with comprehensive T-CAM generation via max pooling over $i$1 intra-augmented views. This ensures that each branch must suppress co-scene activation (contextual false positives) while maintaining coverage (completeness of action segment predictions).
• Integration: The Bi-SCC module, auxiliary losses, and MIL-based classification losses aggregate for joint optimization.

On THUMOS14 and ActivityNet benchmarks, Bi-SCC yields 1–2% mAP improvements over state of the art at all IoU thresholds. Bi-SCC outperforms one-way consistency constraints, striking a balance between co-scene suppression and action completeness (Li et al., 2023).

6. Computational Aspects and Practical Considerations

Bidirectional semantic filtering frameworks are typically constructed to be computationally tractable:

• The probabilistic filtering algorithm (Greiff et al., 14 Jan 2026) requires only $i$2 operations per update, as all moment matching, mixture formation, and hyperparameter updates are closed-form and expressible in terms of element-wise arithmetic (no matrix inversion or large-scale optimization).
• Vision-language filtering (Wienholt et al., 10 Oct 2025) leverages black-box API calls and parallelization. The main cost arises from bidirectional entailment checking—$i$3 calls for $i$4 samples—estimating total cost per question at $i$50.72$i$6\sim$6 seconds per question, parallelizable.
• FBGAN training (Bao et al., 2018) is unsupervised and standard GAN-style, with mutual-information regularization entailing minimal computational overhead. Test-time inference is a single pass through encoder and generator.
• Bi-SCC in WTAL (Li et al., 2023) adds only feature-level augmentations and a second (EMA) teacher per branch, requiring negligible runtime overhead and no extra inference-time processing.

These frameworks are compatible with real-time and interactive deployments, particularly for dynamic or safety-critical domains (driving, radiology, adversarial defense).

7. Limitations, Open Issues, and Validity Scope

Limitations and trade-offs include:

• Scope of filtering: DSE-based methods measure semantic consistency, not factual correctness; models may return confidently wrong answers that bypass filtering (Wienholt et al., 10 Oct 2025).
• Model introspection: Entailment and semantic assignment depend on the reliability of current models' internal judgments; performance may degrade with less semantically consistent generative models.
• Coverage-accuracy trade-off: Stricter filtering (DSE, Bi-SCC) increases answer or localization precision at the cost of recall/coverage.
• Model-specificity: Bidirectional mapping, e.g. in FBGAN (Bao et al., 2018), only filters non-semantic components learnable in the chosen manifold; "semantic" is defined relative to training data and model architecture.

A plausible implication is that hybrid, multi-pronged bidirectional semantic filtering—combining semantic clustering, manifold projection, and cross-context consistency—can yield robust, tractable, and generalizable solutions across a range of machine learning domains, provided sufficient modeling of semantic structure and appropriate calibration to application context. Further studies on theoretical guarantees, calibration under distribution shift, and integration with calibration and risk-aware modules are ongoing directions.