RG-Gait: Occlusion-Robust Gait Recognition

• RG-Gait is a residual-based deep learning framework that improves gait recognition under occlusion by learning a corrective residual.
• It employs an adaptive fusion mechanism that combines occluded features with a learned residual scaled by an occlusion confidence score.
• RG-Gait achieves state-of-the-art accuracy gains, with 15–20% rank-1 improvements on datasets such as Gait3D, GREW, and BRIAR.

RG-Gait denotes a residual-based deep learning framework specifically designed to improve gait recognition performance under occlusion, while preserving accuracy on holistic inputs. The term also may refer, more broadly, to statistical or deep learning methods for classifying or reconstructing biomechanical or functional deviations in gait using residual signals or robust feature strategies. The principal instance in the modern literature is "RG-Gait: Residual Gap Correction for Occluded Gait Recognition," which defines both the problem formulation and the contemporary model architecture for residual correction of occluded gait signatures (Gupta et al., 15 Jul 2025).

1. Motivation and Context within Gait Recognition

Unconstrained gait recognition systems must operate effectively despite occlusions arising from environmental clutter, other people, or carried objects. Most prior deep learning methods for silhouette-based gait identification experience a large performance drop under occlusions and when models trained with heavy occlusion regularization are deployed on clean data, they tend to lose accuracy on holistic (unoccluded) sequences. RG-Gait addresses this dissociation via a residual learning methodology: it conceptualizes an occluded gait feature as a perturbation of the latent representation corresponding to the unoccluded (holistic) pattern and explicitly learns a correction that is adaptively applied as a function of detected occlusion severity (Gupta et al., 15 Jul 2025).

This approach contrasts with traditional imputation, hallucination, or reconstruction paradigms, such as the ConvLSTM-based silhouette reconstruction of RGait-NET (Das et al., 2019) or autoencoder frameworks, which attempt to recover missing pixel information but do not directly optimize the discriminability of downstream gait signatures.

2. Mathematical Formulation and Residual Correction Principle

Let $S_o$ denote an input sequence of silhouette frames suffering from occlusion. Let $x_o=\mathrm{GSE}(S_o)$ be the $d$-dimensional gait signature extracted from this occluded input by the backbone Gait Signature Extractor (GSE), and $x_h$ be the analogous signature estimated on the hypothetical holistic sequence $S_h$. The occlusion distorts the latent feature as $x_o = x_h + \Delta_r$, with $\Delta_r$ representing the "residual gap". RG-Gait introduces a Feature Restoration Network (FRN) that predicts $R\approx\Delta_r$ from the occluded sequence and associated occlusion descriptor.

A key innovation is the adaptive fusion:

$f = x_o + \alpha\cdot R$

where $\alpha \in [0,1]$ is an occlusion-aware confidence score produced by the Occlusion Evaluation Module (OEM), with $x_o=\mathrm{GSE}(S_o)$0 for holistic sequences and $x_o=\mathrm{GSE}(S_o)$1 under strong occlusion. When occlusion is minimal, $x_o=\mathrm{GSE}(S_o)$2 and the correction is suppressed, preserving holistic identification performance. When occlusion is severe, $x_o=\mathrm{GSE}(S_o)$3 is adjusted toward the holistic manifold by compounding the learned residual $x_o=\mathrm{GSE}(S_o)$4, closing the gap between $x_o=\mathrm{GSE}(S_o)$5 and $x_o=\mathrm{GSE}(S_o)$6.

3. System Architecture

RG-Gait comprises three core modules:

• Gait Signature Extractor (GSE): Accepts silhouette frames $x_o=\mathrm{GSE}(S_o)$7 and extracts a discriminative embedding $x_o=\mathrm{GSE}(S_o)$8 using any state-of-the-art backbone (e.g., GaitBase, DeepGaitV2, SwinGait).
• Occlusion Evaluation Module (OEM): A compact CNN that produces (i) an occlusion feature vector $x_o=\mathrm{GSE}(S_o)$9 (from an intermediate FC layer), and (ii) a scalar $d$0 representing the probability or level of occlusion via classification/regression heads.
• Feature Restoration Network (FRN): Receives both silhouette-encoded features and the occlusion vector $d$1, incorporates spatial occlusion context, and outputs $d$2, a residual feature for correcting $d$3.

These modules are integrated such that $d$4 constitutes the final robust gait descriptor. Training proceeds in three stages: pre-train OEM (synthetic occlusions), train GSE on holistic data, then learn FRN to optimize discrimination under occlusion (backbones frozen).

4. Learning and Optimization

For OEM, a joint loss

$d$5

with $d$6, $d$7, penalizes occlusion type misclassification and misestimation of severity. For GSE and FRN, metric learning dominates:

• Triplet loss with $d$8 on individual embeddings,
• Supplementary cross-entropy ($d$9) for identification,
• Hyperparameters: SGD, weight decay $x_h$0, learning rate 0.01, training in batches of $x_h$1.

Training data is strongly augmented with synthetic occlusions: top, middle, bottom masks, and lateral occluding strips up to 60% of the body, randomized per batch. Silhouettes are extracted using Detectron2, centered and resized to $x_h$2 (Gupta et al., 15 Jul 2025).

5. Quantitative Performance and Comparative Evaluation

RG-Gait achieves state-of-the-art recognition under occlusion on multiple public and proprietary datasets. On Gait3D, GREW, and BRIAR, substantial gains (15–20% rank-1 absolute) are established over OccAware and MimicGait, and large improvements relative to backbones (e.g., baseline-1, baseline-2):

Model/Dataset	Gait3D (R1/R5)	GREW (R1/R5)	BRIAR (R1/R5)
Baseline-1	7.6 / 15.7	14.9 / 25.6	1.3 / 12.0
Baseline-2	17.1 / 31.4	16.4 / 30.4	6.1 / 27.5
OccAware	18.2 / 34.9	22.6 / 38.0	8.7 / 35.4
MimicGait	22.7 / 40.8	28.4 / 45.4	10.9 / 42.3
RG-Gait	40.8 / 57.3	43.0 / 59.3	13.9 / 45.3

On unobstructed (holistic) sequences, RG-Gait incurs less than 2% performance drop relative to maximal backbone performance, attesting to its ability to modulate correction only when occlusions are detected (Gupta et al., 15 Jul 2025). Zero-shot transfer and adaptation to new occlusion types indicate high generalizability.

6. Comparison with Alternative Occlusion-Handling Paradigms

Conventional recovery approaches, such as RGait-NET, combine supervised occlusion detection (e.g., fine-tuned VGG-16) with generative sequence completion (e.g., ConvLSTM) trained on binary cross-entropy plus Dice overlap loss. RGait-NET delivers strong results when a sufficient number of clean frames are present but its ability to reconstruct is limited above 40% occlusion, and computational overhead is higher (approx. 0.4 s/subject). It offers a means to fill missing silhouettes but does not optimize the discriminability of the resulting latent features for identification per se (Das et al., 2019).

In contrast, RG-Gait directly optimizes the downstream feature representation for identification under both holistic and occluded conditions, without requiring explicit paired clean/occluded supervision.

7. Limitations, Practical Considerations, and Outlook

RG-Gait's limitations include a persistent, though small, holistic accuracy deficit versus backbone upper bounds, and the fact that synthetic occlusion augmentations may not perfectly capture the diversity of real-world occlusion phenomena. Other constraints: the approach relies on accurate silhouette extraction and the presence of sufficient data for OEM calibration across occlusion types.

Future work is anticipated in sophisticated multi-residual fusion schemes, improved self-supervised residual learning, real-world occlusion dataset curation, and model refinement for unseen occlusion patterns (Gupta et al., 15 Jul 2025). A plausible implication is that the residual correction paradigm may generalize to other video-based biometric tasks suffering from partial input corruption.

RG-Gait thus constitutes the state-of-the-art in occlusion-robust, holistically retaining gait recognition, achieving a principled tradeoff—not only restoring accuracy in challenging real-world deployments but also preserving maximal performance when the gait data is uncorrupted.