ASL360: 360° Sign and VR Systems

• ASL360 is a cross-domain label representing systems that integrate 360° technologies with American Sign Language for VR communication, radar recognition, and video streaming.
• Research highlights include a VR study with 83.3% overall ASL comprehension, a radar system achieving 88.89% accuracy, and a scheduler that improved video quality by 2 dB.
• Each system uses distinct modalities—from body-mounted cameras and multi-antenna beamforming to deep reinforcement learning—to advance immersive sign language applications.

Searching arXiv for the cited ASL360-related papers to ground the article in current metadata. {"query":"all:ASL360 OR id:(Rajmane et al., 26 Feb 2026) OR id:(MacLaughlin et al., 2022) OR id:(Mohammadhosseini et al., 7 Sep 2025)","max_results":10,"sort_by":"submittedDate","sort_order":"descending"} I’ll use the retrieved ASL360-related records together with the provided paper details to write a precise encyclopedia-style synthesis. ASL360 is a label used in recent arXiv literature for multiple technically distinct systems at the intersection of American Sign Language (ASL), 360-degree sensing, immersive media, and wireless delivery. In one line of work, it denotes a 360-degree video approach for ASL communication in virtual reality (VR), centered on body-mounted cameras and comprehensibility analysis (Rajmane et al., 26 Feb 2026). In another, it refers to a radar-based ASL recognition architecture that combines multi-antenna beamforming with deep learning and discusses extension to 360° coverage (MacLaughlin et al., 2022). In a third, it names an adaptive scheduler for layered 360° video streaming over UAV-assisted 5G/mm-Wave networks for mobile VR users (Mohammadhosseini et al., 7 Sep 2025). The shared theme is not a single standardized framework, but the use of 360-degree capture, spatial coverage, or immersive delivery to support sign communication or VR media systems.

1. Taxonomy of the term

The term ASL360 spans at least three research contexts: VR-mediated sign communication, contactless ASL recognition, and adaptive immersive-video delivery. The term therefore functions as a cross-domain label rather than a uniquely defined architecture.

Usage	Core modality	Reported outcome
VR sign communication	Body-mounted 360° video in VR	Overall comprehension rate: 83.3%; shoulder-mounted: 85% (Rajmane et al., 26 Feb 2026)
Simultaneous ASL recognition	77 GHz multi-antenna radar + CNN fusion	Overall accuracy: 88.89% on held-out 20% test set (MacLaughlin et al., 2022)
Immersive video delivery	CMDP + PPO layered 360° streaming	≈2 dB higher average video quality; 80% lower average rebuffering time; 57% lower video quality variation (Mohammadhosseini et al., 7 Sep 2025)

A common source of confusion is that the same label can denote either an ASL-specific communication or recognition system, or a general 360° video streaming scheduler. The literature does not support treating these as instances of one unified protocol. Instead, the label is attached to modality-specific problem formulations: visual ASL intelligibility in VR, mmWave sensing of simultaneous signers, and QoE-constrained adaptive transport of 360° video.

2. ASL360 as 360-degree ASL video in VR

The VR-oriented ASL360 work studies the comprehensibility and user experience of viewing ASL videos captured with body-mounted 360-degree cameras in VR (Rajmane et al., 26 Feb 2026). The experimental setup used $N = 10$ participants $$(6\ \mathrm{F}, 4\ \mathrm{M};\ \text{ages }18\text{–}55)$$, including nine Deaf or Hard-of-Hearing signers and one hearing, ASL-fluent participant. All had prior VR experience, with four having experience in collaborative VR. Participants were recruited via flyers and word-of-mouth and were compensated $20.

Capture employed a Ricoh Theta V dual fish-eye camera with $360^\circ$ horizontal $\times 180^\circ$ vertical coverage. Video was captured at native dual fish-eye resolution and converted to equirectangular in Adobe After Effects. Viewing took place on a Meta Quest running a Unity3D application. Three mounting positions were evaluated. The head-mounted position approximated the signer’s POV and leveraged headset-integrated camera positions; the shoulder-mounted position provided a stable third-person vantage that favored dominant-hand visibility and minimal camera shake; the chest-mounted position offered central placement for symmetric hand view and was motivated in part by the possibility of a necklace-style wearable in future designs.

The task structure comprised minimal pairs, non-minimal single words, and sentences. Per mount, there were five minimal-pair tasks, two non-minimal single-word tasks, and three sentence tasks, for a total of 10 videos recorded per mount and 30 distinct ASL stimuli overall. The signer wore a VR headset during capture to simulate an in-VR signer, and recording was performed in a controlled lab environment using body-mounted straps for camera placement.

Methodologically, the study used a within-subjects design in which each participant completed all three mounting conditions with randomized order. For each trial, participants viewed a 360° ASL video and selected from three on-screen options: correct sign, distractor, or “skip.” The Unity3D application presented videos in an overlay “video box” with no time limit and unlimited replays. Participants could reposition the video box to the four corners and choose dynamic versus fixed attachment to head. Data collection included response type, number of replays, time to decision, a post-condition NASA TLX questionnaire, and an exit survey addressing clarity, distortion, preferred mount, and comparison to text-based chat.

3. Comprehensibility, statistical analysis, and user feedback

Quantitatively, the shoulder-mounted configuration achieved the highest reported accuracy at 85% correct $(\pm \text{approx. }10\%)$, followed by 84% correct for the head-mounted condition and 81% correct for the chest-mounted condition (Rajmane et al., 26 Feb 2026). The overall comprehension rate was 83.3%, corresponding to 25 of 30 tasks on average. Accuracy was defined as

$$\text{Accuracy (\%)} = \left(\frac{n_{\text{correct}}}{n_{\text{total}}}\right)\times 100\%.$$

The study analyzed differences across mounting positions using a repeated measures ANOVA on accuracy. No significant effect of mount position was found $(p > 0.05)$, with the reported statistic $F(2,18) = 1.12, p = 0.34$ and $\eta^2 = 0.11$, described as a small effect. This matters methodologically: the shoulder mount was numerically best, but the data do not support a statistically significant superiority claim within this sample.

The task-level breakdown is informative about which ASL distinctions survived the capture and playback pipeline. Location pairs were recognized at 100%, movement pairs at 96.7%, handshape and expression pairs at 90%, palm/orientation pairs at 80%, single words at 96.7%, and sentences at 70%. Fine finger movements and palm orientation were therefore the hardest to resolve, whereas location and movement contrasts were substantially more robust under the 360° video conditions.

Qualitative feedback identified peripheral distortion as a central limiting factor. Distortion was most pronounced at the FOV edges, especially in the chest mount, and spatial warping affected fine-grained handshape perception. Barrel distortion from fish-eye lenses stretched hands at the FOV periphery, and stitching seams became visible when the signer moved across the stitch line. One participant summarized the perceptual failure mode directly: “Signs on the opposite side got cut off or warped.” The shoulder mount required less head-tracking effort than the head or chest mount, while the head-mounted angle sometimes felt too “peering down” or unnatural.

Preference data were mixed. Six of 10 participants preferred the ASL360 signing method over text-based VR chat for DHH-to-DHH interaction, but there was no consensus on a single best mount: four preferred chest, four preferred shoulder, and two had no preference. A frequent misconception would be that the best objective mount must also be the universally preferred one; the reported results do not support that equivalence.

4. Distortion mitigation and deployment guidance for VR systems

The technical guidance associated with the VR ASL360 study recommends the shoulder mount as the default because it combined the highest overall accuracy with the lowest TLX workload (Rajmane et al., 26 Feb 2026). For ambidextrous signing or bilateral clarity, the recommendations include dual shoulder mounts or a chest + shoulder combination. Angle adjustment is specified more concretely: tilt the camera approximately $10^\circ$ downward from horizontal to center the signer’s hands in the mid-FOV, and use a short arm-extension mount to avoid occlusion by the torso.

The recommended video-processing stack includes equirectangular reprojection with anti-aliasing to soften barrel distortion, GPU-accelerated real-time stitching pipelines using the OpenCV fisheye model, and post-capture mesh unwrapping to flatten the central signing region with minimal warping. These interventions directly target the failure modes identified in the distortion analysis: stretched peripheral geometry, seam artifacts, and degraded perception of fine handshape and palm orientation.

Integration guidance for VR applications emphasizes interface flexibility. Recommended controls include user-controlled repositioning and scaling of the ASL video window, skip and replay functions to reduce guessing and improve confidence, synchronization of audio-visual cues such as haptic confirmation on selection, and multimodal caption overlays for hybrid access in noisy or latency-constrained scenarios. Suggested future improvements include non-fish-eye or multi-camera rigs, a static third-person remote camera for a more natural viewpoint, and real-time streaming with adaptive bitrate and lower-latency stitching.

A plausible implication is that ASL intelligibility in VR is constrained at least as much by optical geometry and viewport design as by linguistic complexity. The reported sentence accuracy of 70%, together with the peripheral-distortion comments, suggests that systems intended for real-time conversational signing must treat capture placement, distortion correction, and UI placement as first-order design variables rather than secondary implementation details.

5. ASL360 as multi-antenna radar recognition

A distinct ASL360 usage appears in radar-based ASL recognition, where the system is not optical but relies on a 77 GHz FMCW radar with a 4-element receive array and a spatio-temporal deep learning pipeline (MacLaughlin et al., 2022). The radar captures raw complex returns $$(6\ \mathrm{F}, 4\ \mathrm{M};\ \text{ages }18\text{–}55)$$0, and preprocessing produces three $$(6\ \mathrm{F}, 4\ \mathrm{M};\ \text{ages }18\text{–}55)$$1 micro-Doppler images: a combined view without beamforming, a beamformed view toward azimuth $$(6\ \mathrm{F}, 4\ \mathrm{M};\ \text{ages }18\text{–}55)$$2, and a beamformed view toward azimuth $$(6\ \mathrm{F}, 4\ \mathrm{M};\ \text{ages }18\text{–}55)$$3. Each image is processed by a small CNN with three convolutional layers, and the three branches are fused in higher fully-connected layers to predict one of nine joint ASL classes.

The array model and beamforming formulation are explicit. At each fast-time sample $$(6\ \mathrm{F}, 4\ \mathrm{M};\ \text{ages }18\text{–}55)$$4, the $$(6\ \mathrm{F}, 4\ \mathrm{M};\ \text{ages }18\text{–}55)$$5-sensor snapshot is modeled as

$$(6\ \mathrm{F}, 4\ \mathrm{M};\ \text{ages }18\text{–}55)$$6

with steering vector

$$(6\ \mathrm{F}, 4\ \mathrm{M};\ \text{ages }18\text{–}55)$$7

For look angle $$(6\ \mathrm{F}, 4\ \mathrm{M};\ \text{ages }18\text{–}55)$$8, the weights are $$(6\ \mathrm{F}, 4\ \mathrm{M};\ \text{ages }18\text{–}55)$$9, yielding a spatially filtered signal $360^\circ$0. A range DFT, range collapse, and STFT then produce the micro-Doppler representation

$360^\circ$1

The network architecture uses three parallel image-specific CNN branches, one per spectrogram input. Each layer applies $360^\circ$2 and $360^\circ$3 kernels in parallel, followed by ReLU activation and max-pooling. The branch outputs are flattened and concatenated, then passed through two shared fully-connected layers with dropout, culminating in a softmax output over nine classes corresponding to all combinations of $360^\circ$4. Training used categorical cross-entropy, Adam with learning rate approximately $360^\circ$5, an 80%/20% train-test split, per-image log scaling and normalization, and no explicit data augmentation.

The experimental dataset consisted of two pairs of individuals, for a total of four signers with varying ASL experience, seated at $360^\circ$6 azimuth and approximately 2 m from the radar. Each class was recorded 20 times per pair, producing $360^\circ$7 examples, each yielding three $360^\circ$8 spectrograms. The TI AWR2243 cascade radar operated at $360^\circ$9 with $\times 180^\circ$0, $\times 180^\circ$1 receive channels, inter-element spacing $\times 180^\circ$2, PRI $\times 180^\circ$3, $\times 180^\circ$4 PRIs over 4 s, and fast-time ADC $\times 180^\circ$5, giving $\times 180^\circ$6 samples per PRI and $\times 180^\circ$7 samples.

Performance was reported as 88.89% overall accuracy on the held-out 20% test set of 72 samples. The confusion matrix showed perfect separation for some classes and minor confusions in C–B versus D–B and C–D versus D–D. The paper attributes robustness to the fusion of combined and beamformed views, which can recover information lost by spatial filtering alone when beamformer sidelobes cause leakage. In this usage, the “360” concept is tied to angular coverage and multi-signer separation rather than spherical video capture: the paper states that multi-antenna spatio-temporal fusion provides wide angular coverage $\times 180^\circ$8 and can be extended to $\times 180^\circ$9 by deploying multiple radars or rotating arrays.

The proposed deployment advantages are real-time feasibility, low-power mmWave sensing, and privacy preservation because no optical imaging is used. Applications listed include smart-home HCI for Deaf users, contactless ASL translation interfaces, in-vehicle gesture controls, and surveillance where visual line-of-sight is blocked. The paper also states clear limitations: the dataset is small, the vocabulary contains only three signs, and only two simultaneous users are modeled.

6. ASL360 as adaptive streaming of layered 360° video

In a third usage, ASL360 denotes an adaptive deep reinforcement learning-based scheduler for on-demand 360° video streaming to mobile VR users over a UAV-assisted 5G wireless network (Mohammadhosseini et al., 7 Sep 2025). The system model considers a single VR user served by two mm-Wave base stations: a macro base station on the ground and a UAV-mounted base station hovering at a fixed altitude. At time slot $(\pm \text{approx. }10\%)$0, the instantaneous downlink rate is

$(\pm \text{approx. }10\%)$1

where $(\pm \text{approx. }10\%)$2 is the assigned RB bandwidth and $(\pm \text{approx. }10\%)$3 is the mm-Wave path loss.

The 360° video is encoded using tiled DASH and scalable coding. Each frame is divided into an $(\pm \text{approx. }10\%)$4 grid of tiles. A Base Layer encodes all tiles in a GOP at low quality with $(\pm \text{approx. }10\%)$5, and one Enhancement Layer re-encodes only the tiles within the predicted FoV at higher quality with $(\pm \text{approx. }10\%)$6. The user maintains two playback buffers, $(\pm \text{approx. }10\%)$7 for the BL and $(\pm \text{approx. }10\%)$8 for the EL, each with capacity 36 s, evolving according to

$(\pm \text{approx. }10\%)$9

with segment length $$\text{Accuracy (\%)} = \left(\frac{n_{\text{correct}}}{n_{\text{total}}}\right)\times 100\%.$$0.

Scheduling is formulated as a Constrained Markov Decision Process. The state is

$$\text{Accuracy (\%)} = \left(\frac{n_{\text{correct}}}{n_{\text{total}}}\right)\times 100\%.$$1

where $$\text{Accuracy (\%)} = \left(\frac{n_{\text{correct}}}{n_{\text{total}}}\right)\times 100\%.$$2, $$\text{Accuracy (\%)} = \left(\frac{n_{\text{correct}}}{n_{\text{total}}}\right)\times 100\%.$$3, $$\text{Accuracy (\%)} = \left(\frac{n_{\text{correct}}}{n_{\text{total}}}\right)\times 100\%.$$4, and $$\text{Accuracy (\%)} = \left(\frac{n_{\text{correct}}}{n_{\text{total}}}\right)\times 100\%.$$5 stores the last $$\text{Accuracy (\%)} = \left(\frac{n_{\text{correct}}}{n_{\text{total}}}\right)\times 100\%.$$6 throughput samples per BS. The action space is binary: $$\text{Accuracy (\%)} = \left(\frac{n_{\text{correct}}}{n_{\text{total}}}\right)\times 100\%.$$7 downloads a BL segment from the MBS, and $$\text{Accuracy (\%)} = \left(\frac{n_{\text{correct}}}{n_{\text{total}}}\right)\times 100\%.$$8 downloads an EL segment from the UAV. The objective maximizes expected discounted cumulative video-quality reward subject to long-term rebuffering and smoothness constraints, with targets $$\text{Accuracy (\%)} = \left(\frac{n_{\text{correct}}}{n_{\text{total}}}\right)\times 100\%.$$9 and $(p > 0.05)$0. The immediate QoE is

$(p > 0.05)$1

where $(p > 0.05)$2, $(p > 0.05)$3 is PSNR, and $(p > 0.05)$4.

The rebuffering cost is incurred when the BL buffer underflows,

$(p > 0.05)$5

and the smoothness cost penalizes EL-buffer empty↔non-empty toggles,

$(p > 0.05)$6

Dynamic cost-weight adjustment updates the Lagrange weights with $(p > 0.05)$7.

Learning uses PPO with stochastic policy $(p > 0.05)$8, value function $(p > 0.05)$9, GAE, learning rate $F(2,18) = 1.12, p = 0.34$0, discount factor $F(2,18) = 1.12, p = 0.34$1, clip parameter $F(2,18) = 1.12, p = 0.34$2, and episode length 36 s. Training ran on an Intel 13th-gen CPU and an NVIDIA RTX 4090 GPU. The implementation used the SJTU 8K 360° dataset, 36 s duration at 30 fps with 1 s per GOP, FoV tiles from real head-motion traces, two 36 s buffers, and real 5G mm-Wave throughput traces from Lumos5G for both MBS and UAV.

Evaluation compared ASL360 against a threshold-based baseline and Pensieve. Averaged across six 360° videos and multiple traces, the threshold-based method achieved approximately 34.5 dB average PSNR, approximately 3.2 s rebuffer time, and approximately 4.1 quality variation; Pensieve achieved approximately 35.0 dB, approximately 2.5 s, and approximately 3.8; ASL360 achieved approximately 37.0 dB, approximately 0.5 s, and approximately 1.6. The paper summarizes these improvements as approximately 2 dB higher average video quality, 78–80% lower rebuffering time than baselines, and approximately 57% lower quality variation than the threshold-based method. Here, ASL360 does not denote sign-language capture or recognition; it is a QoE-optimization mechanism for immersive 360° media transport.

7. Conceptual convergence and distinctions

Across these three usages, ASL360 consistently links ASL or immersive VR scenarios to $F(2,18) = 1.12, p = 0.34$3 sensing, coverage, or delivery, but the operational problem changes substantially from one paper to another. The VR communication study is centered on human comprehensibility under optical distortion and viewpoint constraints. The radar study is centered on simultaneous signer separation via joint spatio-temporal preprocessing and multi-branch CNN fusion. The adaptive streaming study is centered on constrained control of layered 360° media under volatile mm-Wave throughput.

This divergence matters for interpretation. It would be inaccurate to treat performance figures across the three papers as directly comparable, because the target variables differ: human sign identification accuracy and NASA TLX in VR, supervised classification accuracy for multi-user radar sensing, and PSNR/rebuffering/smoothness for video delivery. Likewise, “360” refers to different technical objects: spherical video geometry in the VR and streaming work, and potential angular sensing coverage in the radar work.

At the same time, the literature suggests a coherent broader research direction. The VR paper emphasizes accessibility and collaboration for DHH users in virtual environments, the radar paper emphasizes privacy-preserving contactless recognition and simultaneous signer separation, and the streaming paper emphasizes robust QoE for mobile VR users over challenging wireless links. A plausible implication is that future end-to-end systems could require all three layers simultaneously: intelligible 360° ASL capture, robust recognition or indexing where needed, and low-latency layered transport in immersive networks. The current papers, however, remain separate contributions rather than components of an integrated ASL360 stack.