SynCB: A Synergy Concept-Based Model with Dynamic Routing Between Concepts and Complementary Neural Branches

Abstract: Concept-based (CB) models provide interpretability and support test-time human intervention, while standard neural networks (NN) offer strong task performance but little transparency. Prior work has explored hybrid formulations that integrate concepts and additional representations to improve accuracy, often at the cost of human interventions. We introduce the \emph{Synergy Concept-Based Model (SynCB)} framework, that combines a CB branch with a complementary neural branch, and a trainable routing module that dynamically selects which branch to use for each input. Unlike prior models, which fuse residual and concept-based predictions, SynCB keeps the two branches distinct and coordinates them through the routing module. Moreover, both branches are learned jointly, allowing information sharing between the complementary neural branch and CB branches through their common backbone. To improve responsiveness to interventions, we further introduce a test-time intervention policy and a corresponding loss. Across five datasets and CB benchmarks, SynCB consistently achieves higher task accuracy while remaining more responsive to human interventions, surpassing the full neural baseline by up to 3.9 percentage points and exceeding the strongest competitor in intervention performance by up to 6.43 percentage points.

• The paper introduces SynCB, a hybrid model that dynamically routes input between a concept branch and a DNN branch to balance interpretability and accuracy.
• It employs joint gradient flow and a composite loss function to optimize both concept fidelity and task performance, improving results on noisy and incomplete datasets.
• The model incorporates Uncertainty Sample Intervention to focus corrections on uncertain concepts, boosting task accuracy with fewer human interventions.

Synergy Concept-Based Modeling with Dynamic Routing: An Expert Perspective

Motivation and Context

Concept Bottleneck Models (CBMs) have been a cornerstone in interpretable neural architectures, providing prediction pathways grounded in human-understandable concepts and permitting test-time interventions. However, CBMs are inherently restricted by the completeness and expressiveness of the concept set; their performance degrades when the concepts are noisy, incomplete, or insufficiently capture the task-relevant information. Hybrid extensions have attempted to combine CBMs with traditional deep neural networks (DNNs), but fusion typically dilutes both interpretability and the efficacy of intervention.

The paper introduces the Synergy Concept-Based Model (SynCB), designed to address these limitations by leveraging the complementary strengths of CBMs and end-to-end DNNs via joint representation learning and dynamic routing. The central thesis is that task accuracy and responsiveness to human intervention need not be a zero-sum tradeoff; explicit routing and joint optimization allow models to specialize dynamically without sacrificing interpretability or task performance.

Architectural Overview

The SynCB framework instantiates two distinct predictive branches—a concept-based branch (CB) and an end-to-end DNN branch—both sharing a common feature extraction backbone. During training, both branches receive the same latent representation $h$; gradients are propagated from both branches to the backbone, ensuring that the feature space accommodates the needs of both symbolic and sub-symbolic reasoning.

A trainable routing module determines, per-input, which branch should produce the final prediction. The routing function $R_\phi(h)$ is parameterized as a small neural network, trained to assign inputs to the CB branch when its predictions are correct (prioritizing interpretability and intervenability), and to the DNN branch otherwise (maximizing task accuracy). Notably, during training, all samples propagate through both branches, circumventing premature specialization and ensuring robust gradient flow.

Intervention Strategies

Classical intervention approaches employ random concept intervention (RCI), whereby selected concepts across all test samples are corrected by a human. The paper argues this is not commensurate with realistic annotation budgets or human-in-the-loop deployment scenarios. SynCB introduces Uncertainty Sample Intervention (USI): only the most uncertain samples (quantified via the levels of ambiguity in concept predictions) are selected for full concept intervention. Uncertainty intervals for each concept are adaptively set based on concept probability distributions; samples are ranked by count of uncertain concepts.

This policy offers budget-efficient and high-yield intervention, demonstrated empirically to afford sharper gains in task accuracy with fewer interventions, especially for datasets with manageable concept sets.

Training Procedure and Losses

SynCB employs a composite loss function:

• Concept loss via binary cross-entropy between predicted and ground-truth concepts.
• Task loss as a weighted sum (typically equal weighting) of cross-entropy losses from both CB and DNN branches.
• Routing loss (binary cross-entropy) trains the routing module to favor CB branch for correct predictions.
• Intervention loss encourages the CB branch's task predictor to leverage ground-truth concepts in its embedding space, enhancing intervenability.

Weights are set via empirical validation, but analysis reveals training is not highly sensitive to their precise values given balanced optimization.

Empirical Results

SynCB is evaluated across five datasets—CUB, AWA, CIFAR10, and their incomplete variants. Two instantiations are presented: SynCBM uses CBM as the CB branch; SynCEM uses Concept Embedding Models (CEM).

Key quantitative findings:

• Task accuracy: SynCB (both variants) consistently improves over pure CBM and CEM baselines, surpassing DNN by up to 3.9 p.p. on incomplete concept sets and narrowing the accuracy gap (within 1.4 p.p.) even on concept-sparse benchmarks.
• Concept accuracy: SynCB models achieve high-fidelity concept predictions, matching or exceeding state-of-the-art intervenable models, demonstrating that task improvements are not driven by shortcut learning.
• Intervenability: SynCB retains strong responsiveness to interventions, with SynCEM nearly matching MixCEM (specialized for concept interventions), while HP-CBM (hybrid residual) performs poorly in this regime.
• Routing behavior: For most datasets, the routing module favors the CB branch for the majority of samples, but dynamically re-allocates to the DNN branch when the CB branch underperforms, particularly when concept sets are noisy or incomplete.
• USI versus RCI: USI interventions exhibit a steeper and higher plateau in task accuracy for a fixed annotation budget compared to RCI, especially when the number of concepts is moderate.

Ablation and Component Analysis

Ablations reveal that joint gradient flow from both branches is critical; removing it from either branch impairs concept or task accuracy. The intervention loss stabilizes performance under interventions. Early routing (i.e., using routing module during training) slightly degrades task accuracy by enforcing premature specialization. Maintaining routing exclusively for test-time ensures maximal utilization of shared representations.

Implications and Future Directions

SynCB demonstrates that joint symbolic and subsymbolic reasoning, coordinated via dynamic routing, allows interpretable models to approach or exceed the performance of pure DNNs while retaining their intervenability. The framework is agnostic to the precise concept-based model used, facilitating extensibility to novel concept formulations (e.g., graph-based, probabilistic, or causal concept graphs).

Practically, SynCB is well-suited for high-stakes domains requiring human oversight and correction, such as medical imaging or scientific discovery. The dynamic routing mechanism could be further enriched with explainable selection strategies or deferred intervention logic (see e.g., Deferring CBMs (Pugnana et al., 20 Mar 2025)). The USI policy is adaptable to other forms of uncertainty estimation, potentially benefiting from calibration or probabilistic embedding mechanisms.

Theoretically, the joint training and routing paradigm opens avenues for analyzing the mutual information and synergy between symbolic and subsymbolic pathways, and for exploring the causal structure underlying concept relationships (cf. Causal Concept Graph Models).

Conclusion

The SynCB framework operationalizes the synergy between concept-based models and DNNs via joint training and a routing module that specializes prediction pathways per-sample. It achieves state-of-the-art accuracy and maintains strong responsiveness to concept interventions, addressing the long-standing tradeoff between interpretability and task performance. The USI intervention policy further enhances annotation efficiency for human-in-the-loop deployments. SynCB establishes a robust foundation for future research in hybrid interpretable modeling, including extensions to richer concept representations and adaptive intervention logic.