---
title: 'SANA-Streaming: UAV ABR & Video Editing'
url: https://www.emergentmind.com/topics/sana-streaming
type: topic
---

# SANA-Streaming: UAV ABR & Video Editing

SANA-Streaming encompasses two distinct, widely cited frameworks for real-time streaming applications: (1) sensor-augmented neural adaptive bitrate (ABR) streaming over UAV links via reinforcement learning and (2) high-resolution, latency-bounded video-to-video editing using a hybrid diffusion transformer pipeline. Both variants implement real-time constraints and system/model co-design, but target different modalities and technical challenges: network adaptation (ABR) and temporally consistent generative editing. The following sections delineate the technical composition, methodologies, and empirical results reported in [1909.10914] and [2605.30409].

## 1. Sensor-Augmented Neural Adaptive Bitrate (SA-ABR) Streaming over UAVs

SA-ABR, also referenced as SANA-Streaming in the UAV/ABR literature, is a client-side reinforcement learning solution for air-to-ground adaptive video streaming. It incorporates real-time UAV sensor data into the bitrate selection process to better adapt to volatile wireless channels.

### System Architecture

Key modules include:

- **Sensor Data Collector** integrates GPS, IMU (accelerometer, gyroscope), barometer, and compass, sampling once per video chunk to yield quantized flight-state vectors (distance, velocity, acceleration).
- **Network Monitor** computes application-layer throughput using TCP packet counts, averaging over every chunk (2 s).
- **Buffer Manager** simulates playback state and rebuffering without an actual decoder, updating buffer occupancy $b_t$ and rebuffer time $T_t$ as:
  - If $b_t \geq f_t$:
    $$ b_{t+1} = b_t + t_\text{chunk} - f_t,\qquad T_t = 0 $$
  - Else:
    $$ b_{t+1} = t_\text{chunk},\qquad T_t = \lceil (f_t-b_t)/0.5\rceil\cdot 0.5 $$
- **DRL Agent (Actor–Critic)** ingests quantized sensors and 8-step throughput history, passes them through a 2-layer LSTM (64 hidden units/layer) and two dense layers (30/10 units), outputting a distribution over possible next-bitrate actions.
- **Video Encoder** exposes the fixed bitrate set $A = \{300, 750, 1850, 2850\}$ Kbps (covering 240p–1080p).

### Dataflow

For each chunk, sensor/network features and an 8-step throughput vector are input to the agent, which samples the next bitrate. Upon download completion, the buffer state and reward are updated, and a transition tuple $(s_t, a_t, r_t, s_{t+1})$ is fed back for training.

## 2. MDP and Deep Reinforcement Learning Formulation

The ABR control loop is formulated as a Markov Decision Process $(\mathcal S, \mathcal A, \mathcal P, \mathcal R)$:

- **State** $s_t = (u_t, v_t, x_t)$ with:
  - $u_t = (d_t, v_t, a_t)$, quantized UAV flight state: $d_t \in \{0,1\}$ (distance $\leq 50$ m or $> 50$ m), $v_t \in \{0,1,2\}$, and $a_t \in \{0,1\}$.
  - $v_t = (b_t, l_{t-1})$: current buffer occupancy $b_t \in [0,20]$ s and immediately previous bitrate $l_{t-1}$.
  - $x_t = [x_{t-8},\ldots,x_{t-1}] \in \mathbb R^8$: recent throughput history in Mbps.
- **Action space**: $a_t \in A$.
- **Transition**: determined by real throughput trace and buffer simulation dynamics.
- **Reward**: Per-chunk QoE,
  $$
  \mathrm{QoE}_t = q(l_t) - \mu T_t - |q(l_t) - q(l_{t-1})|
  $$
  where $q(l) = \log\frac{l}{l_\text{min}}$, $l_\text{min}=300$ Kbps, $\mu = 2.26$.

Actor–Critic training uses 10 parallel actors running in varied UAV environments, n-step TD updates (n = 41), and no replay buffer.

## 3. LSTM Temporal Modeling and Sensor Quantization

The throughput LSTM, with sequence length 8, encodes both short- and medium-range temporal dependencies arising from rapidly varying wireless channels attributable to UAV motion. Ablation shows that performance (QoE) significantly degrades with shorter history (e.g., $>$15% QoE drop for length 2 vs 8). Quantization of sensor readings (distance, velocity, acceleration) reduces the variance and improves mean QoE versus unquantized/raw side-information.

## 4. Experimental Evaluation and Results: UAV ABR

The system was implemented on a DJI Matrice 100 UAV platform (25 m altitude, speeds up to 19.5 m/s), streaming to a ground laptop over IEEE 802.11n (2.4 GHz). Over 1,000 real 100 s throughput traces, SANA-Streaming yields:

- **QoE improvement**: 21.4% over Pensieve (next-best ABR baseline)
- **Component gains**: utility +10.8%, smoothness penalty –35.3%, rebuffer penalty unchanged
- **Ablations**: removing sensors leads to QoE reductions of 7–30%; raw sensor input (continuous) increases variance and reduces mean QoE relative to quantized
- **Network temporal modeling**: LSTM outperforms CNN baseline, delivering +17.5% QoE and much lower rebuffer/smoothness penalties

The agent selects bitrate actions that stabilize the buffer and track channel changes efficiently, maintaining playback stability under highly non-stationary UAV mobility [1909.10914].

## 5. SANA-Streaming for Real-Time Video-to-Video Editing

The more recent SANA-Streaming system (2026) addresses the video-to-video editing domain, integrating a hybrid Diffusion Transformer (DiT) model, a novel cycle-reverse regularization scheme, and highly optimized system-level GPU kernels and quantization strategies.

### Hybrid Diffusion Transformer Architecture

- **Backbone**: 20 blocks—15 Gated DeltaNet (GDN, linear attention) and 5 softmax-attention blocks (evenly interleaved).
- **GDN blocks** provide global recurrent memory via compressed states, maintaining constant time/memory.
- **Softmax blocks** inject local, high-fidelity modeling via windowed cross-chunk attention and a persistent "sink" token.
- **Attention complexity & kernel design**: Linear blocks incur $O(FD^2+FND)$ cost and require only $O(D^2)$ cached state; softmax windows reduce cost to $O((wN)^2 D)$.

### Cycle-Reverse Regularization

- **Forward edit**: Applies the desired prompt ($p^+$), generating a chunk $\tilde I_t$.
- **Reverse branch**: Attempts to reconstruct the source $I_{t-1}$ from $\tilde I_t$ using an inverse prompt ($p^-$), penalized via a flow-matching L2 loss.
- **Key property**: Enables enforcement of semantic/temporal consistency over unpaired training data, as only raw videos and synthetic "reverse" prompts are required.

The total per-chunk training loss is
$$
\mathcal L = \mathcal L_{\rm DMD} + \lambda_{\rm flow}\,\mathcal L_{\rm flow}
$$
where $\mathcal L_{\rm DMD}$ is the distillation loss from a short-clip teacher.

## 6. System-Level Optimizations: Fused Kernels and Mixed-Precision Quantization

SANA-Streaming leverages:

- **Fused GDN kernels**: Three-phase, Triton-based GPU kernels optimize memory and computation, yielding 1.5×–2.2× per-block and 1.59× end-to-end speed-up over naïve PyTorch on RTX 5090.
- **Mixed-Precision Quantization**:
  - **Policies**: BF16 (reference), FP8 TensorCore, NVFP4 Grouped 4-bit.
  - **Final policy**: FP4 on 62% parameters (67% FLOPs), FP8 on robustness-critical layers, BF16 on highly sensitive modules.
  - **Performance**: DiT latency for one-minute video reduced from 26.8 s (BF16) to 16.8 s, with LPIPS within 0.17 of reference.

Empirically, this system achieves 24 FPS at 1280×704 resolution (end-to-end), 58 FPS within the DiT module, and matches or exceeds existing SOTA in OpenVE benchmarks [2605.30409].

### Benchmarking Table

| Method           | E2E FPS (1280×704) | Avg. OpenVE Score |
|------------------|-------------------|-------------------|
| Lucy-Edit        | ~6.7              | 2.50              |
| SANA-Streaming   | 24                | 2.62              |
| SANA (distilled) | 762 (DiT core)    | 2.42              |

## 7. Temporal Coherence and Editing Quality

Cycle-Reverse regularization yields a ≈30% reduction in inter-frame flicker over long sequences, measured via non-edited region stability. The mixture of global memory (GDN) and local softmax attention enables both long-range consistency and high-fidelity edits without paired edited video.

SANA-Streaming's design, combining modeling, training, and hardware-aware optimization, establishes new throughput and quality benchmarks for real-time streaming video editing and temporally robust UAV video streaming. Empirical results validate its effectiveness in both highly dynamic network environments and low-latency GPU-accelerated video generation contexts [1909.10914][2605.30409].

Source: https://www.emergentmind.com/topics/sana-streaming