Cascaded Binary Prediction Head (CBPH)

• CBPH is a modular framework that decomposes multi-class or multi-label prediction tasks into cascaded binary subproblems for clearer decision boundaries.
• It employs sequential or parallel binary classifiers to mitigate class imbalance, as demonstrated in applications like dementia detection and incremental scene graph generation.
• Empirical results indicate significant improvements in metrics such as Macro-F1 and predicate prediction, highlighting CBPH’s robustness in both imbalanced and continual learning environments.

A Cascaded Binary Prediction Head (CBPH) is a modular prediction architecture in which a multi-class or multi-label classification is decomposed into a cascade or collection of independent binary classifiers, typically arranged either sequentially or in parallel. Each classifier in the cascade solves a distinct binary subproblem, yielding improved robustness to class imbalance, enhanced interpretability, and superior performance on imbalanced or incremental prediction tasks. CBPH frameworks have been adopted in deep LLMs for sequential medical diagnosis from spontaneous speech (Liu et al., 26 May 2025), as well as in continual 3D scene graph generation for incremental relation prediction in graphs (Qi et al., 13 Jun 2026). The following sections delineate CBPH formulations, mathematical objectives, training/inference protocols, practical implementation notes, and empirically validated advantages in each domain.

1. Structural Design and Architectural Variants

CBPH architectures vary by task requirements but share core patterns: a backbone feature extractor (typically a Transformer or PointNet module), followed by a set of MLP-based binary classifiers (heads). In sequential-cascade variants—such as three-way dementia detection—each input is routed through a binary decision chain:

• The first head distinguishes class groupings (e.g., “Healthy Control” vs. “Non-Healthy Control”).
• If classified as positive, a second head further discriminates between remaining classes (e.g., “MCI” vs. “Dementia”).

In incremental multitask settings—such as 3D Semantic Scene Graph Generation—CBPH is realized as a collection of per-class binary heads. At each incremental learning stage (task), new binary heads are appended for newly introduced classes, while existing heads for prior classes are frozen, ensuring knowledge retention.

Use Case	Backbone	CBPH Structure
Dementia Detection	Transformer PLM	2-stage sequential binary (cascade)
Incremental SGG	PointNet + Transformer	Parallel binary heads, expanded per task

Pause encoding (for speech disfluencies) and feature alignment modules (Spatial-guided Feature Adapter) are incorporated within the backbone depending on the domain (Liu et al., 26 May 2025, Qi et al., 13 Jun 2026).

2. Mathematical Formulation

The mathematical treatment of CBPH centers on formulating each head as a scalar-output MLP and defining robust composition rules to recover global class probabilities.

Sequential Cascade (Dementia Detection) (Liu et al., 26 May 2025):

Given backbone feature $\varphi(x) \in \mathbb{R}^{d}$, each head $i$ computes

$$h_i(x) = W_i \varphi(x) + b_i, \quad p_i(x) = \sigma(h_i(x))$$

where $\sigma(t) = 1/(1+e^{-t})$ is the standard sigmoid.

• $p_1(x)$: Probability of Non-HC (vs. HC)
• $p_2(x)$: Conditional probability of Dementia (vs. MCI) given Non-HC

The joint probabilities for 3-way classification: $$\begin{align*} P(\mathrm{HC}\mid x) & = 1 - p_1(x) \ P(\mathrm{MCI}\mid x) & = p_1(x) \cdot [1 - p_2(x)] \ P(\mathrm{Dementia}\mid x) & = p_1(x) \cdot p_2(x) \end{align*}$$

Parallel Cascade (Incremental SGG) (Qi et al., 13 Jun 2026):

For edge embedding $\mathbf{E}_{(i,j)}^{\mathrm{last}}\in\mathbb{R}^d$, each class $c$ has a distinct head $h_c$ (two-layer MLP), producing a logit $i$0. The final CBPH logit vector at task $i$1: $i$2 Each entry is sigmoid-activated and thresholded for classification.

3. Objective Functions and Losses

Dementia Detection, Sequential Cascade (Liu et al., 26 May 2025):

For training, binary cross-entropy loss is applied to each stage: $i$3 with final objective

$i$4

where $i$5 labels Non-HC, $i$6 labels Dementia, and regularization is $i$7.

Incremental SGG, Parallel Cascade (Qi et al., 13 Jun 2026):

At each incremental task, the loss includes:

• Focal loss for new heads:

$i$8

• $i$9 logit distillation for old, frozen heads:

$$h_i(x) = W_i \varphi(x) + b_i, \quad p_i(x) = \sigma(h_i(x))$$0

• Full objective at task $$h_i(x) = W_i \varphi(x) + b_i, \quad p_i(x) = \sigma(h_i(x))$$1:

$$h_i(x) = W_i \varphi(x) + b_i, \quad p_i(x) = \sigma(h_i(x))$$2

where $$h_i(x) = W_i \varphi(x) + b_i, \quad p_i(x) = \sigma(h_i(x))$$3 is standard detection/align loss.

4. Training and Inference Procedure

Dementia Detection (Liu et al., 26 May 2025):

• Preprocess: Force-align speech, insert pause tokens per silence intervals.
• Stage 1: Train PLM + Head1 on HC vs. Non-HC.
• Stage 2: Train PLM + Head2 on Non-HC subset, MCI vs. Dementia.
• Inference: Apply Head1; if Non-HC, proceed to Head2 for fine discrimination.

Ensemble: Multiple backbones/tasks/seeds (e.g., $$h_i(x) = W_i \varphi(x) + b_i, \quad p_i(x) = \sigma(h_i(x))$$4 models) with majority vote on three-way probabilities.

Incremental SGG (Qi et al., 13 Jun 2026):

• For each new task $$h_i(x) = W_i \varphi(x) + b_i, \quad p_i(x) = \sigma(h_i(x))$$5, new heads for each class are instantiated and trained on new classes only, with prior heads frozen.
• At task completion, logits from old heads are cached for distillation.
• Each batch updates backbone and new heads via focal/new-class loss; old predictions are regularized by $$h_i(x) = W_i \varphi(x) + b_i, \quad p_i(x) = \sigma(h_i(x))$$6 distillation on spatially-adapted features.
• Inference concatenates all heads’ outputs for final multi-label predicate prediction.

Step	Dementia Detection	Incremental SGG
Cascade Structure	2-stage sequential	Parallel, dynamically grown
Backbone Update	Separate or shared	Trainable, old heads frozen
Losses	BCE	Focal + $$h_i(x) = W_i \varphi(x) + b_i, \quad p_i(x) = \sigma(h_i(x))$$7 distillation
Inference Mechanism	Stagewise routing	Multi-head parallel

5. Hyperparameterization and Practical Implementation

Dementia Detection (Liu et al., 26 May 2025):

• Backbone: BERT-large-uncased, RoBERTa-large, or ERNIE-2.0 large.
• Dropout: 0.1 per head. Head MLP: 1024→1024 (tanh)→1.
• Optimizer: AdamW, learning rate $$h_i(x) = W_i \varphi(x) + b_i, \quad p_i(x) = \sigma(h_i(x))$$8, weight decay 0.01.
• Batch size: 8; epochs: 20.
• Consistent pause encoding and ASR fine-tuning are required for reproducibility.
• Ensemble across multiple splits/seeds for stability.

Incremental SGG (Qi et al., 13 Jun 2026):

• Backbone: PointNet and stacked transformer GEL++/SIL++ blocks.
• CBPH head: two-layer MLP per predicate, width $$h_i(x) = W_i \varphi(x) + b_i, \quad p_i(x) = \sigma(h_i(x))$$9.
• Loss: Focal ($\sigma(t) = 1/(1+e^{-t})$0, $\sigma(t) = 1/(1+e^{-t})$1), logit distillation weight $\sigma(t) = 1/(1+e^{-t})$2.
• Optimizer: Adam, $\sigma(t) = 1/(1+e^{-t})$3 LR, $\sigma(t) = 1/(1+e^{-t})$4 decay.
• Training: Each new task up to 100 epochs, mini-batches of 8 scenes.
• Logits cached post-task for distillation.

6. Empirical Validation and Quantitative Impact

CBPH architectures yield pronounced advantages in class-imbalanced and incremental learning scenarios.

Dementia Detection (Liu et al., 26 May 2025):

CBPH achieved a Macro-F1 of 58.6% on the PROCESS test set, versus 55.0% for the best flat multi-class head—a $\sigma(t) = 1/(1+e^{-t})$5 absolute gain. Disaggregation of tasks led to more balanced learning: Stage 1 (HC vs. Non-HC) is near-balanced, and Stage 2 (MCI vs. Dementia) addresses rare-class underfitting.

Incremental SGG (Qi et al., 13 Jun 2026):

In 3D scene graph generation, CBPH improved Predicate A@1 by 4.49% absolute over the second-best continual baseline (EWC), also yielding the highest predicate mean A@1 and triplet mA@50/100 across strategies. Freezing old heads, taskwise head expansion, and logit distillation induced strong resistance to catastrophic forgetting while enabling performance scaling with ontology growth.

Method	Predicate A@1	Predicate mA@1	Triplet mA@50
CBPH	81.70	41.04	51.17
EWC	77.21	38.33	26.13

7. Comparative Advantages and Domain-Specific Rationale

CBPH combines modular decision boundaries with explicit architectural mechanisms for mitigating dataset imbalance and catastrophic forgetting:

• Class imbalance in direct multiclass training (e.g., HC : MCI : Dementia ≈ 5 : 4 : 1) is alleviated by decomposing into more balanced binary problems.
• Fine-to-coarse or coarse-to-fine cascades privilege hard distinctions at early stages, enabling later stages to focus on subtle minority-class separation.
• In the continual learning regime, CBPH’s freezing and expansion protocol stabilizes knowledge over sequential tasks; logit distillation aligns new feature spaces with prior heads, preserving predecessor task accuracy.

A plausible implication is that CBPH generalizes well to domains typified by high class imbalance, shifting taxonomies, or the need for robust continual adaptation, without resorting to parameter overwrites or one-vs-rest degeneracy.

---

References:

• "Leveraging Cascaded Binary Classification and Multimodal Fusion for Dementia Detection through Spontaneous Speech" (Liu et al., 26 May 2025)
• "SGFormer++: Semantic Graph Transformer for Incremental 3D Scene Graph Generation" (Qi et al., 13 Jun 2026)