BioST-GCN: Dual-Stream Fall Prediction

• The paper introduces BioST-GCN, a novel dual-stream architecture that fuses 3D pose and biomechanical data to predict fall risk with improved accuracy.
• The model integrates interpretable spatio-temporal attention by combining graph convolutional and recurrent neural networks to extract key kinematic features.
• BioST-GCN achieves up to 5.32% F1-score gains over ST-GCN baselines, with few-shot personalization boosting zero-shot performance significantly.

The Biomechanical Spatio-Temporal Graph Convolutional Network (BioST-GCN) is a dual-stream neural architecture designed for vision-based fall prediction using both pose and biomechanical features. This model fuses raw 3D skeleton sequences and hand-crafted kinematics via cross-attention and outputs a probabilistic fall risk prediction. BioST-GCN is distinguished by its ability to exploit interpretable spatio-temporal body-attention within a Graph Convolutional Network (GCN) framework and to integrate gravity-aligned, anatomically-referenced biomechanical signals processed through recurrent neural networks. While demonstrating superior accuracy over traditional ST-GCN baselines in simulated environments, BioST-GCN exposes significant challenges in simulation-to-reality generalization and motivates strategies for subject-specific personalization and privacy-preserving real-world deployment (Islam et al., 18 Nov 2025).

1. Model Architecture and Fusion Mechanisms

BioST-GCN comprises two principal streams:

• ST-GCN stream (Stream A): Accepts 3D skeleton trajectories derived from 90-frame windows (3 seconds at 30 Hz) of 33 body landmarks. The stream processes data through stacked spatio-temporal graph convolutional blocks endowed with learnable, joint-specific body attention. This is formalized as a dynamic adjacency matrix $$\widetilde{\mathcal A}_k^{(\ell)} = \mathcal A_k^{(\ell)}\odot \mathcal M_k^{(\ell)}$$ where $\mathcal M_k^{(\ell)}$ is a ConvLSTM-derived, elementwise-gated attention mask.
• Biomechanical stream (Stream B): Operates on a set of 45 kinematic features per frame, encompassing Euler angles, center-of-mass (COM) positions, trunk yaw, and planar hip/shoulder displacements, all normalized in a gravity-aligned and anthropometric reference frame. This stream employs a two-layer Bidirectional LSTM whose output is compressed to an 80-dimensional vector via a ReLU-activated MLP.
• AttFusion cross-attention module: Realizes feature fusion by allowing ST-GCN stream outputs to serve as queries and biomechanical stream outputs as keys/values, applying multi-head dot-product attention. The fused representation augments the ST-GCN features with biomechanical insight and is projected via an MLP yielding the final fall probability $\hat y = \Pr(\text{fall})$.

2. Input Representations and Preprocessing

BioST-GCN operates on temporally windowed pose and biomechanical data segments:

• Pose stream: Takes 90 consecutive frames, resulting in a tensor $\mathcal{P} \in \mathbb{R}^{90 \times 33 \times 3}$. Raw landmark data are obtained using MediaPipe Pose extraction, then undergo Savitzky–Golay smoothing (window size 7, polynomial order 3) and subject-height normalization to reduce noise and inter-individual variance.
• Biomechanical features: Per frame, 45 features $\mathbf{B}\in\mathbb{R}^{90\times45}$ are computed as follows:
  • 27 joint-segment Euler angles (across 9 anatomical segments)
  • 9 COM coordinates (upper, lower, and whole-body)
  • 1 trunk-yaw angle
  • 8 image-plane displacements (hip/shoulder)
  • These are referenced anatomically and apply anthropometric scaling under Plagenhoef's method.

3. Spatio-Temporal Attention and Interpretability

The ST-GCN stream is augmented with body attention for improved interpretability and selectivity. Attention weights ($\alpha_{v, t}$) are computed across joints and frames as

$$\alpha_{v, t} = \frac{\exp(f(\mathbf{F}^{(\ell)}_{v, t}))}{\sum_{v'}\exp(f(\mathbf{F}^{(\ell)}_{v', t}))}$$

Empirically, trunk and hip joints receive consistently elevated attention (mean $\approx 0.71$, $\sigma=0.08$), while wrists and ankles are less attended (mean $\approx 0.23$, $\sigma=0.15$). Early network layers exhibit diffuse attention across joints, but deeper layers increasingly focus on upper-body segments, with temporal attention peaking immediately before predicted fall onset—behavior consistent with established pre-fall kinematic patterns.

4. Training Protocols and Performance Evaluation

BioST-GCN is optimized using weighted binary cross-entropy loss:

$$\mathcal{L} = -\frac{1}{N} \sum_i w_i \left[y_i\log\hat y_i + (1-y_i)\log(1-\hat y_i)\right]$$

The Adam optimizer ($\text{lr}\approx 10^{-4}$) is employed with early stopping over 50 epochs. Training data are segmented via 90-frame sliding windows (stride 15 frames), and positive labels are assigned to temporal windows $[0.5, 1.0, 2.0]$ seconds pre-impact. All non-fall frames are labeled negative.

Quantitative results at a 2-second horizon are summarized as follows:

Data Split	Model	Precision	Recall	F1	AUPRC
MCF-UA (80/20)	ST-GCN	80.2%	89.5%	84.6%	86.6%
	BioST-GCN	84.4%	94.3%	89.1%	91.1%
MUVIM (80/20)	ST-GCN	71.2%	80.9%	75.7%	80.3%
	BioST-GCN	72.9%	83.6%	77.9%	82.3%

BioST-GCN thus achieves substantial F1-score improvements over the ST-GCN baseline: 5.32% on MCF-UA (stunt-actor) and 2.91% on MUVIM datasets.

5. Simulation-to-Reality Gap and Identified Biases

Despite robust intra-dataset performance (F1 $\approx$ 89.1% under full supervision on MCF-UA), cross-subject evaluation exposes an acute generalization deficit. On unseen subjects (zero-shot), F1 falls to approximately 34.8% (AUPRC $\approx$ 35.1%), a $\sim$55 point drop. This simulation-to-reality gap is attributed to:

• "Intent-to-fall" cue artifacts: Marked when stunt actors telegraph instability by pre-emptively leaning.
• Kinematic profile mismatch: Young actors recover differently and display greater joint excursions than frail or diabetic older adults.

A plausible implication is that fall-predictive cues in simulated data may not be representative of the target population, demanding domain adaptation or real-world data integration.

6. Personalization and Privacy-Preserving Deployment

To mitigate the simulation-reality gap, BioST-GCN leverages subject-specific adaptation:

• Few-shot fine-tuning: Incorporating $K=1$–$5$ labeled instances from a new subject increases cross-subject F1-score from $\sim$35% to $\sim$60–75%.
• Meta-learning and self-supervision: Approaches such as MAML and smartphone inertial pretraining are advocated to reduce required labeled data further.

The deployment strategy prioritizes privacy:

• On-device personalization with federated averaging (FedAvg) enables model updates without transmitting raw data.
• Homomorphic encryption (e.g., BatchCrypt) or secure multi-party aggregation ensures biomechanical streams remain device-local.
• Differential privacy and federated distillation further obfuscate individual identities in distributed model updates.

This suggests that real-world viability of BioST-GCN in elder populations is dependent on both personalization and robust, privacy-oriented learning pipelines.

7. Impact and Future Research Directions

BioST-GCN represents an advance in interpretable, multimodal fall prediction by integrating body-attentive ST-GCN representations and principled biomechanical features. Its evaluation reveals robust intra-dataset accuracy but identifies a pressing domain transfer challenge for practical deployment in elderly and clinical populations.

Future research priorities include:

• Collection of real-world, at-risk older adult fall datasets to supplant simulation.
• Enhanced domain adaptation between simulated and real kinematics.
• Refinement of few-shot and federated learning pipelines for scalable, privacy-preserving personalization. This trajectory aligns with the urgent need for validated, ethical fall prediction systems for vulnerable populations (Islam et al., 18 Nov 2025).