Artic: AI-RTC for MLLM Assistants

• Artic is a specialized AI-oriented real-time communication framework designed to optimize video streaming for cloud-based multimodal large language model assistants.
• It employs innovative modules like Response Capability-aware Adaptive Bitrate and Zero-overhead Context-aware Streaming to dynamically allocate bandwidth and manage latency.
• Experimental evaluations show up to 135 ms latency reduction, 15% improvement in MLLM accuracy, and 45–70% uplink bandwidth savings over conventional RTC systems.

The term Artic refers to a specialized AI-oriented real-time communication (RTC) framework that re-engineers the video streaming pipeline for cloud-based Multimodal LLM (MLLM) video assistants. Distinct from traditional RTC systems that optimize for human perceptual quality, Artic shifts network resource allocation and protocol logic to maximize MLLM response accuracy under stringent end-to-end latency constraints, focusing on ultra-low uplink bandwidth, adaptive content-aware encoding, and automatic accuracy benchmarking. Recent implementations of Artic systems have demonstrated substantial improvements in both model response accuracy and responsiveness over standard RTC stacks, and have established new paradigms for network design in AI-driven interactive video (Wu et al., 13 Feb 2026, Wu et al., 14 Jul 2025).

1. Problem Definition and Quality of Experience Shift

Conventional RTC frameworks—such as WebRTC, adaptive bitrate streaming (ABR), and error correction protocols—are designed around optimizing quality of experience (QoE) for humans, targeting metrics like PSNR, SSIM, stalling rates, and resolution fluency. In AI Video Assistant scenarios, however, the “consumer” is an MLLM (e.g., Qwen2.5-Omni, Gemini Live), shifting the QoE objective to MLLM response accuracy plus low end-to-end latency (typically <300 ms) (Wu et al., 13 Feb 2026, Wu et al., 14 Jul 2025).

This paradigm introduces three fundamental architectural shifts:

• Uplink-Centric Bottleneck: Video must be streamed at high fidelity towards the cloud, while downlink traffic (text/audio) is negligible.
• MLLM Bandwidth Efficiency: Model input requirements (e.g., 2 FPS sampling, <1 MPix per frame) are far below the output rates of mobile cameras, motivating aggressive bitrate reduction.
• Accuracy–Latency Trade-off: Raising video bitrate improves human perceptual quality but—beyond a threshold—yields negligible gains in MLLM performance, instead consuming network headroom and introducing queuing delays.

Prototype experiments confirm that conventional RTC stacks cause unsustainable latency spikes and rapid accuracy degradation under uplink fluctuations, especially during rapid bandwidth collapse events (e.g., elevator rides, cellular handoffs) (Wu et al., 13 Feb 2026).

2. Response Capability-Aware Adaptive Bitrate (ReCapABR)

Artic introduces the Response Capability-aware Adaptive Bitrate module, which explicitly models the relationship between video bitrate ($b$) and MLLM accuracy ($A(b)$). The accuracy curve is monotonically increasing and concave, saturating at $A_\text{sat}$ for moderate $b$. The key strategy is to:

• Proactively cap bitrate at the minimum $b^*$ such that $A(b^*) \geq A_\text{sat} - \epsilon$ for a small tolerance $\epsilon$, preserving bandwidth headroom ($\Delta b = \hat{B} - b^*$, $\hat{B}$: instantaneous bandwidth estimate).
• Dynamically adjust bitrate based on feedback from server-side MLLM response confidence $C_t$, using a nonlinear control law:

$A(b)$0

where $A(b)$1 is the target confidence threshold (Wu et al., 13 Feb 2026).

This allocation suppresses latency spikes by reserving headroom for transient network drops and avoids wasteful bandwidth utilization beyond accuracy saturation. Experimental results show up to 135 ms lower average transmission latency and +15% MLLM accuracy relative to WebRTC+GCC (Wu et al., 13 Feb 2026).

3. Zero-Overhead Context-Aware Streaming (ZeCoStream)

Artic’s Zero-overhead Context-aware Streaming (ZeCoStream) offloads the task of determining which image regions are essential for understanding to the cloud-side MLLM. The MLLM is prompted to return bounding boxes $A(b)$2 that are critical for the current response, effectively generating a region-importance mask with no additional client-side inference (Wu et al., 13 Feb 2026, Wu et al., 14 Jul 2025).

Bitrate allocation is then performed spatially:

• Each frame is divided into $A(b)$3 patches; for patch $A(b)$4 at distance $A(b)$5 to the nearest box boundary, importance is set as $A(b)$6, with empirically-tuned $A(b)$7 (Wu et al., 13 Feb 2026).
• Per-patch quantization parameter $A(b)$8 is set via a quadratic rule:

$A(b)$9

Thus, high-importance regions are preserved at high fidelity, while unimportant background is heavily quantized. Under ultra-low bitrates (e.g., $A_\text{sat}$0290 kbps), context-aware streaming retains >50% higher MLLM QA accuracy than uniform encoding (Wu et al., 13 Feb 2026).

Prediction horizon compensation is accomplished by having the MLLM predict bounding boxes 1.5 s into the future for moving scenes, synchronizing encoding decisions with model input time.

4. Loss-Resilient Adaptive Frame Rate and Redundancy Control

Since most MLLMs operate at modest frame rates ($A_\text{sat}$1 FPS), but networks can support much higher transport rates $A_\text{sat}$2, Artic optimizes frame redundancy to mitigate packet loss without resorting to ARQ retransmissions. For measured packet loss $A_\text{sat}$3 and $A_\text{sat}$4 packets per frame, the probability a frame arrives in time is $A_\text{sat}$5, and the receiver samples $A_\text{sat}$6 candidates per model step (Wu et al., 14 Jul 2025).

Redundancy is increased only to the level required to ensure $A_\text{sat}$7, eliminating ARQ-induced latency and reducing average stalling to $A_\text{sat}$8 ms (Wu et al., 14 Jul 2025).

5. Degraded Video Understanding Benchmark (DeViBench)

DeViBench is the first large-scale benchmark dedicated to evaluating the sensitivity of MLLM responses to task-irrelevant video degradation. The construction pipeline comprises:

• Pre-encoding videos with aggressive quantization;
• Generating QA pairs by prompting MLLMs to focus on details visible only in the high-quality half of paired video frames;
• Filtering for truly “quality-sensitive” QAs by cross-verification and answer comparison between MLLMs (Wu et al., 13 Feb 2026, Wu et al., 14 Jul 2025).

DeViBench comprises 1,968 QA samples, 88,680 s total video, and enables systematic assessment of how encoding and channel strategies impact application-level AI accuracy; roughly 8% of QAs require inter-frame temporal context.

6. Quantitative Evaluation and Performance Impact

Combined, Artic’s optimizations yield:

• 45–70% uplink bandwidth reduction vs. standard RTC stacks, with MLLM QA accuracy gains of 5–15 percentage points across all tested conditions (Wu et al., 13 Feb 2026).
• Latency reduction: Mean transmission latencies decrease by 95–135 ms, and the frequency of frames arriving within 200 ms increases from ~66% to >82% (Wu et al., 13 Feb 2026).
• Energy and cost trade-offs: Minimal server-side inference overhead for context extraction; monetary API costs rise by ~27% due to increased MLLM feedback traffic, offset by reduced uplink data volumes (Wu et al., 13 Feb 2026).

A summary of core Artic components and their functions is provided below.

Module	Key Technique	Impact
ReCapABR	Accuracy-based bitrate cap	Stabilizes latency, prevents wastage
ZeCoStream	MLLM-driven region encoding	Preserves accuracy at low bitrates
Redundancy Control	Adaptive frame duplication	Minimizes retransmission latency
DeViBench	Degradation-sensitive QA	Enables application-level optimization

7. Limitations, Extensions, and Future Directions

Current Artic deployments address only quantization-based degradation, with planned extensions to include channel loss, frame-dropping, and advanced forward error correction (Wu et al., 13 Feb 2026). Scaling to rapidly dynamic scenes or adversarial prompt scenarios may erode context alignment, motivating joint model–codec co-design.

Future work will explore:

• Joint spatial-temporal saliency prediction at the client, semi-autonomous redundancy scheduling,
• Model-internal pruning of vision tokens to further reduce inference latency,
• Cooperative audio–video control where AI assistants rely on multimodal input (e.g., lip-reading),
• Integration with next-generation codecs (VVC/H.266), neural codecs, and model-intensive AR endpoints.

Artic demonstrates that a full-stack rethinking of networked video—for the needs and constraints of AI consumers, rather than humans—enables significant resource and latency efficiencies without MLLM output loss, offering a blueprint for broader MLLM-driven RTC systems (Wu et al., 13 Feb 2026, Wu et al., 14 Jul 2025).