RAST-G@ for Home Stroke Rehabilitation Assessment

• RAST-G@ is a deep learning framework that maps full skeleton motion sequences to continuous movement-quality scores mimicking physiotherapists’ judgments.
• It integrates a spatio-temporal graph convolutional network with transformer-based temporal attention to assess upper-limb ADL and ROM exercises from RGB-D capture.
• The system supports home-based rehabilitation by providing per-trial and longitudinal feedback, enhancing both patient self-assessment and clinical monitoring.

RAST-G@ is a deep learning model for domiciliary stroke rehabilitation assessment that maps a full skeleton motion sequence to a continuous movement-quality score intended to mimic physiotherapists’ judgments. It is the analytical core of a home-based rehabilitation exercise and feedback system comprising RGB-D capture, wearable sensing, a mobile application, and an AI server. The model combines a spatio-temporal graph convolutional network (ST-GCN) with transformer-based temporal attention to assess upper-limb Activities of Daily Living (ADL) and Range-of-Motion (ROM) exercises, and it was evaluated on the KIMORE and NRC datasets using MAD, RMSE, and MAPE (Lim et al., 27 Sep 2025).

1. Clinical scope and problem setting

RAST-G@ addresses a specific gap in post-stroke rehabilitation: the need for frequent, quantitative, home-based assessment of upper-limb exercise quality. The motivating premise is that stroke survivors often require intensive, repetitive upper-limb practice for months or years after discharge, while much of the clinically consequential recovery process occurs at home rather than in supervised settings. In that setting, patients may exercise without supervision, perform incorrect or unsafe movements, and receive only intermittent in-person evaluation (Lim et al., 27 Sep 2025).

The framework is explicitly oriented toward quality assessment rather than mere action recognition. Existing automatic systems are characterized as often focusing on identifying which exercise is being performed rather than how well it is executed; relying on hand-crafted kinematic features or distance metrics such as DTW and Mahalanobis distance; or evaluating short, isolated motion segments rather than the full temporal evolution of a functional action. The paper further contrasts this with coarse clinical instruments such as Fugl-Meyer-style scoring, described as subjective and limited in granularity.

The target movement repertoire spans two clinically distinct categories. ADL exercises include daily functional tasks such as drinking, brushing hair or teeth, using a smartphone, folding paper or a towel, and related upper-limb actions. ROM exercises include shoulder flexion, abduction, and external rotation. This pairing is clinically consequential because it combines gross shoulder and elbow motion, fine hand and bimanual coordination, and task completion in functionally meaningful contexts.

2. Domiciliary rehabilitation system and processing pipeline

RAST-G@ is embedded in a complete home-rehabilitation system rather than presented as an isolated estimator. The hardware stack comprises an Intel RealSense D435i RGB-D camera, Movella Xsens Dot wearable inertial sensors on both wrists, and an Android tablet running the rehabilitation application. For the model reported here, only the RGB-D-derived skeleton sequence is used directly by RAST-G@; IMU streams are collected for possible future multimodal models (Lim et al., 27 Sep 2025).

The operational workflow is sequential. The tablet first provides exercise guidance. During execution, RGB-D video and wrist-sensor signals are recorded, and the application tags the exercise class. Pose estimation is then performed server-side with MMPose using HRNet-DarkPose trained on COCO-WholeBody, producing a 25-joint skeleton per frame. The resulting sequence is temporally standardized to 288 frames using a frame-dropping/grouping strategy. The standardized tensor and the corresponding skeleton graph are passed to RAST-G@, which returns a scalar assessment score. The server then generates user-facing outputs including the score, longitudinal trend graphs, and optional skeleton heatmaps.

This architecture is explicitly described as supporting both patient-centered assessment and monitoring. The patient receives per-trial and longitudinal summaries, while therapists can inspect trends and visual overlays rather than only a single scalar endpoint.

3. Computational formulation and model architecture

RAST-G@ is formulated as a regression model

$f_\theta : X \rightarrow Y,$

where

$$X \in \mathbb{R}^{N \times C \times T \times V \times M}$$

is the input motion tensor and

$Y \in \mathbb{R}$

is the predicted quality score. Here, $N$ is batch size, $C$ the input channels, $T$ the temporal length, $V$ the number of joints, and $M$ the number of persons in the scene; in the reported setup, $T=288$ and $V=25$ (Lim et al., 27 Sep 2025).

The skeletal representation is treated as a graph $$X \in \mathbb{R}^{N \times C \times T \times V \times M}$$0. For hop distance $$X \in \mathbb{R}^{N \times C \times T \times V \times M}$$1, the normalized adjacency used by the ST-GCN is

$$X \in \mathbb{R}^{N \times C \times T \times V \times M}$$2

where $$X \in \mathbb{R}^{N \times C \times T \times V \times M}$$3 is the degree matrix, $$X \in \mathbb{R}^{N \times C \times T \times V \times M}$$4 is a learnable edge-importance mask, and $$X \in \mathbb{R}^{N \times C \times T \times V \times M}$$5 is the identity matrix. Spatial graph convolution in layer $$X \in \mathbb{R}^{N \times C \times T \times V \times M}$$6 is given by

$$X \in \mathbb{R}^{N \times C \times T \times V \times M}$$7

followed by temporal convolution

$$X \in \mathbb{R}^{N \times C \times T \times V \times M}$$8

and residual composition

$$X \in \mathbb{R}^{N \times C \times T \times V \times M}$$9

After the ST-GCN backbone, the feature map

$Y \in \mathbb{R}$0

is reshaped to

$Y \in \mathbb{R}$1

so that each joint trajectory is treated as a temporal token sequence. Temporal attention is then applied per joint: $Y \in \mathbb{R}$2 with scaled dot-product attention

$Y \in \mathbb{R}$3

The output of the attention module is globally pooled and passed to a fully connected regression head,

$Y \in \mathbb{R}$4

yielding a single predicted score per sequence. Training uses a Huber loss with $Y \in \mathbb{R}$5: $Y \in \mathbb{R}$6

Architecturally, the ST-GCN backbone preserves anatomical graph structure and local spatio-temporal dependencies, while the temporal-attention module assigns nonuniform weight to different phases of execution. The paper’s interpretation is that diagnostically salient moments such as initial reach, peak ROM, or return phases need not contribute equally to the final score.

4. Input representation, datasets, and supervision

The raw model input is a skeleton sequence derived from RGB-D capture. Pose estimation is performed with MMPose and HRNet-DarkPose pretrained on COCO-WholeBody, yielding a 25-keypoint full-body skeleton with upper limb and hand coverage. To standardize variable-duration trials, the sequence is divided into groups, one frame is randomly sampled from each group, and the sampled frames are concatenated in temporal order until a fixed length of 288 frames is obtained. This procedure is intended to preserve the global temporal pattern while also acting as augmentation (Lim et al., 27 Sep 2025).

Two datasets are central to the reported evaluation:

Dataset	Scope	Key properties
NRC	Upper-limb ADL and ROM	10 ADL, 5 ROM, 1,142 motion-score pairs
KIMORE	Whole-body rehabilitation	5 exercises, therapist-provided quality scores

The NRC dataset was introduced specifically for domiciliary upper-limb stroke rehabilitation. It contains 15 exercise classes: 10 ADL and 5 ROM. Participants are split into non-disabled (ND) and stroke groups. The ND group contains 325 sequences (293 train, 32 val, no test), while the stroke group contains 817 sequences (633 train, 70 val, 114 test), for a total of 1,142 motion-score pairs with an approximate 8:1:1 split. Stroke inclusion criteria include age $Y \in \mathbb{R}$7, subacute or chronic stage, Fugl-Meyer upper limb score $Y \in \mathbb{R}$8, Brunnstrom hand recovery stage 4–4.5, and MOCA $Y \in \mathbb{R}$9.

Supervision is provided through a 10-item questionnaire, each item scored from 0 to 5, yielding a total score in $N$0. The items are: (1) achievement of the main objective of the action, (2) body stability, (3) smoothness and continuity, (4) correct head movement, (5) right-arm correctness, (6) left-arm correctness, (7) trunk correctness, (8) tool handling, (9) control of force, speed, and direction, and (10) self-regulated trajectory. These annotations were provided by licensed physiotherapists; the questionnaire was developed by 3 therapists and validated by 2 PhDs in PT.

The KIMORE dataset serves as an external benchmark. It contains RGB-D videos and kinematic features for 5 whole-body exercises and uses therapist-provided clinical quality scores on a 50-point scale similar to NRC. In the reported study, KIMORE functions both as a benchmark for comparison against prior rehabilitation-assessment models and as a cross-domain test of whether the architecture generalizes beyond upper-limb domiciliary tasks.

5. Training protocol and empirical performance

Training was implemented in PyTorch on Ubuntu 22.04 using an Intel i9-10920X CPU and an NVIDIA RTX 4090 GPU. The reported hyperparameters are 200 epochs, batch size 64, AdamW, and learning rate 0.003. Performance is measured with three regression metrics: $N$1

$N$2

$N$3

On KIMORE, RAST-G@ is reported to achieve the lowest average RMSE, MAD, and MAPE across the five exercises. The details summarize this as an average RMSE $N$4, compared with values greater than 1.3 for many baselines. Performance is described as consistent across exercises rather than concentrated on a single motion type. One noted exception is Ex4 (pelvis rotation), where Kuang et al. perform particularly well because quaternion-based orientation features better express rotational motion (Lim et al., 27 Sep 2025).

On NRC, the reported numbers are explicit. RAST-G@ achieves RMSE 0.291, MAPE 0.259, and MAD 0.321. The corresponding values for Kuang et al. are RMSE 1.961, MAPE 2.836, and MAD 0.739; for Deb et al., RMSE 1.030, MAPE 3.805, and MAD 0.892. The authors summarize these differences as roughly 3–7× lower RMSE, about 10× lower MAPE, and 2–3× lower MAD relative to the cited baselines.

Ablation studies isolate the contribution of the main design choices. Removing frame dropping slightly changes the error trade-off and is reported to worsen relative error behavior. Removing Temporal Attention increases MAD and RMSE, and also degrades MAPE, which the authors interpret as evidence that temporal attention is important for fine-grained quality assessment. Training only on stroke data yields a slightly better MAD but worse RMSE and MAPE than mixed ND + Stroke training, suggesting that non-disabled examples help the model capture smoother movement flow.

6. Feedback, interpretability, and clinical role

The output of RAST-G@ is integrated into two feedback modes. Period feedback aggregates scores over longer intervals, such as monthly averages, and supports longitudinal monitoring by overall score or by movement categories such as UNI, BIA, and BIS. Discrete feedback summarizes a single exercise trial, displaying a user-facing score rescaled to 0–100, skeleton heatmaps, and optional therapist comments (Lim et al., 27 Sep 2025).

Interpretability is provided through skeleton heatmaps derived from intermediate feature maps and attention-related activations. In the implementation described, higher-contribution nodes and edges are rendered with larger size or stronger color intensity. An example is given for folding paper, where the visualization highlights the hand and trunk joints most responsible for the assessment. Clinicians in beta tests reportedly found these overlays useful for identifying postural misalignment, trunk compensation, and other movement-quality issues, and for explaining corrections to patients.

Clinically, the system is framed as patient-centered rather than template-normalizing. The assessment does not force stroke motions to match healthy reference trajectories; instead, it learns a regression from motion patterns to therapist scores that encode functional success and movement quality under impairment. A plausible implication is that the model’s target space is closer to actual rehabilitation decision-making than a purely biomechanical distance-to-template criterion.

The authors also note several limitations. The current system assesses performance but does not yet generate automated recovery plans or exercise prescriptions. The dataset, though larger than many stroke corpora, remains modest for broad clinical deployment. The present model uses only skeletons derived from RGB-D, leaving multimodal fusion with IMU signals as future work.

7. Position within the literature and nomenclature

Within rehabilitation AI, RAST-G@ is positioned against three main lines of prior work. The first consists of distance-based systems, such as DTW- or Mahalanobis-based comparisons to a healthy reference, which are described as sensitive to noise and poorly aligned with therapist judgment. The second comprises handcrafted kinematic feature pipelines that require manual feature engineering and often target coarse clinical scales. The third includes deep architectures such as CNN+LSTM and GCN+LSTM models, or ST-GCN variants adapted from action recognition, which the authors characterize as less effective at emphasizing clinically critical motion phases (Lim et al., 27 Sep 2025).

The model’s central novelty is the combination of ST-GCN with per-joint temporal attention for rehabilitation-quality regression. The graph component preserves anatomical structure and local spatio-temporal dependencies, while the attention component reweights long-range temporal structure. The paper further emphasizes three additional contributions: a new NRC dataset for upper-limb domiciliary rehabilitation, a 10-item function-oriented score designed for stroke assessment, and deployment within a practical home-based system that combines sensing, scoring, longitudinal monitoring, and visualization.

The designation RAST-G@ is specific to this rehabilitation model and should not be conflated with unrelated arXiv uses of RAST, including radius-adaptive stochastic undersampling in MRI (Bschorr et al., 21 Apr 2026), Reasoning Activation in LLMs via Small-model Transfer (Ouyang et al., 30 May 2025), Retrieval-Augmented Style Transfer for question generation (Gou et al., 2023), and Rast as a language for resource-aware session types (Das et al., 2020). In the rehabilitation context, RAST-G@ refers to the ST-GCN-plus-attention assessment model for skeleton-based scoring of stroke rehabilitation exercises.