MVStylizer: Mobile Edge Video Stylization

• MVStylizer is a mobile-edge-cloud system that delivers photorealistic video stylization by processing key frames on edge servers and reconstructing intermediate frames on mobile devices.
• It utilizes an optical-flow-based interpolation and a meta-smoothing module to maintain structural realism and upscale stylized outputs effectively.
• The system achieves a 75.5× speedup for high-resolution videos and employs federated updating to continually enhance the edge stylization model.

MVStylizer is an edge-assisted photorealistic video style transfer system for mobile phones that was proposed to make video stylization feasible under mobile resource constraints by stylizing only key frames on a nearby edge server and generating the remaining stylized intermediate frames on the handset through optical-flow-based interpolation. The system combines a pretrained edge-side stylization network, an on-device interpolation stage, a meta-smoothing module that jointly upscales and smooths stylized frames, and an edge-cloud federated updating scheme for continual model improvement (Li et al., 2020).

1. Problem setting and design objective

MVStylizer targets photorealistic video style transfer rather than artistic frame rendering. In this setting, an input video and a user-specified style image are transformed into a stylized video whose frames inherit the style image’s appearance while preserving the semantic content and structural realism of the original video. The paper explicitly treats this as a mobile systems problem: mobile phones have limited computation resources, photorealistic stylization is already expensive on powerful GPUs, and direct frame-by-frame video stylization multiplies this cost across hundreds to tens of thousands of frames (Li et al., 2020).

The motivating argument is numerical as well as architectural. For a 10-minute video at 1920×1080 and 30 fps, there are about 18,000 frames. The reported edge-side DNN stylizer requires 1.51 s/frame at that resolution, so direct frame-by-frame stylization would take about 7.55 hours even on the edge server. MVStylizer therefore reduces the number of frames that undergo expensive neural stylization by selecting key frames, stylizing only those frames remotely, and reconstructing the intermediate frames locally.

A central distinction in the paper is that photorealistic stylization imposes stricter structural requirements than artistic transfer. Distortions that may be tolerable in painterly stylization are treated as unacceptable when the output is expected to remain realistic. This design goal explains why MVStylizer emphasizes structure-preserving interpolation and post-stylization smoothing rather than unconstrained per-frame synthesis.

2. Mobile-edge-cloud architecture

The system is organized as a three-tier architecture spanning the mobile device, a nearby edge server, and a cloud server (Li et al., 2020).

Tier	Primary functions
Mobile phone	key-frame extraction, optical-flow-based interpolation, final video reconstruction
Edge server	key-frame stylization with a pretrained DNN, local retraining, synchronization with cloud
Cloud server	global model maintenance, parameter aggregation, backup for edge models

The end-to-end workflow is procedural. The mobile user provides an input video and a style image. Key frames are then identified using either standard H.264 video codec information or a content-based method from prior work. These key frames, together with the style image, are uploaded to a nearby edge server. The edge server stylizes the key frames with a pretrained neural network and returns the stylized key frames to the phone. The mobile device then synthesizes stylized intermediate frames using an optical-flow-based interpolation algorithm. The stylized key frames and interpolated frames are finally composed into the output stylized video.

The motivation for placing inference on the edge rather than the cloud is latency. The edge server is geographically closer to the mobile device and therefore reduces round-trip delay. The paper’s communication measurements support this deployment choice: uploading a 1920×1080 frame from Boston takes 0.031 s to an edge server in New York, versus 0.318 s to a cloud server in Los Angeles and 0.925 s to a cloud server in Hong Kong.

This architecture makes MVStylizer primarily a systems decomposition of video stylization rather than a monolithic end-to-end video generation model. Heavy stylization is concentrated on sparse key frames, while lightweight geometry-based propagation is performed on the handset.

3. Edge-side stylization network

The edge-side stylizer contains four modules: a pretrained VGG encoder, a colorization module, a decoder with mirrored VGG layers, and the proposed meta-smoothing module (Li et al., 2020). The inputs are a content image $\mathbf{I}_c$ and a style image $\mathbf{I}_s$, and the output is a stylized image $\mathbf{I}_{cs}$, followed by the upscaled and smoothed output $\mathbf{I}_{cs}^r$.

After resizing the content and style images to the same size, the frozen VGG encoder extracts feature maps $\mathbf{F}_c$ and $\mathbf{F}_s$ of size $H \times W \times C$. The paper denotes these as

$$\mathbf{F}_c = \{\mathbf{x}_c^i \mid i=1,\dots,H\times W\}, \qquad \mathbf{F}_s = \{\mathbf{y}_s^j \mid j=1,\dots,H\times W\},$$

with each feature vector having length $C$. The colorized content features are

$$\mathbf{F}_{cs} = \{\mathbf{z}_{cs}^k \mid k=1,\dots,H\times W\},$$

and the paper defines the colorization step as

$\mathbf{I}_s$0

The decoder then reconstructs a stylized image from the fused representation.

Training uses a perceptual content-style objective: $\mathbf{I}_s$1 Here $\mathbf{I}_s$2 is a perceptual distance, $\mathbf{I}_s$3 is a resizing operator to match upscaling factor $\mathbf{I}_s$4, and $\mathbf{I}_s$5 in the experiments. The VGG encoder is frozen, while the colorization module, decoder, and meta-smoothing module are trained jointly with Adam, using $\mathbf{I}_s$6, $\mathbf{I}_s$7, an initial learning rate of $\mathbf{I}_s$8, batch size 2, and filter sizes

$\mathbf{I}_s$9

A common misunderstanding is to treat MVStylizer as introducing a fully specified new photorealism regularizer. The paper does not present a separate photorealism constraint equation such as a matting Laplacian or explicit temporal loss. Its photorealistic behavior is instead attributed to the stylization network design, the perceptual objective, and the meta-smoothing stage.

4. On-device interpolation and meta-smoothing

The mobile-side acceleration mechanism is an optical-flow-based frame interpolation algorithm (Li et al., 2020). Suppose a video has $\mathbf{I}_{cs}$0 total frames, $\mathbf{I}_{cs}$1 key frames $\mathbf{I}_{cs}$2, and $\mathbf{I}_{cs}$3 intermediate frames $\mathbf{I}_{cs}$4, with $\mathbf{I}_{cs}$5. For an intermediate frame $\mathbf{I}_{cs}$6 located between key frames $\mathbf{I}_{cs}$7 and $\mathbf{I}_{cs}$8, the optical flow is defined as

$\mathbf{I}_{cs}$9

where $\mathbf{I}_{cs}^r$0 is an optical flow estimator cited as Lucas-Kanade. After the edge server returns stylized key frames $\mathbf{I}_{cs}^r$1, each stylized intermediate frame is generated by

$\mathbf{I}_{cs}^r$2

where $\mathbf{I}_{cs}^r$3 is a bilinear interpolation kernel.

The paper’s temporal consistency mechanism is therefore implicit rather than loss-based. Intermediate frames inherit motion coherence from the source video through optical flow, and the algorithm is described as a simple one-sided warping scheme from the preceding key frame. The paper does not provide an explicit occlusion model, forward-backward consistency check, or visibility mask.

The meta-smoothing module addresses two tasks simultaneously: arbitrary-resolution upscaling and removal of stylization-related distortions. Given the decoder output $\mathbf{I}_{cs}^r$4, it produces

$\mathbf{I}_{cs}^r$5

where $\mathbf{I}_{cs}^r$6 is a deconvolution operator, $\mathbf{I}_{cs}^r$7 is a resolution-dependent upscaling kernel, $\mathbf{I}_{cs}^r$8 is a smoothing kernel, and $\mathbf{I}_{cs}^r$9 is the user-specified upscaling factor. The conditioning rule for the upscaling kernel is

$\mathbf{F}_c$0

with $\mathbf{F}_c$1 defined as a function that constructs a matrix filled with $\mathbf{F}_c$2.

This design is not presented as a generic super-resolution stage appended after stylization. Instead, it is an integrated module trained end to end to both upscale and smooth stylized outputs. A plausible implication is that the module is intended to absorb artifact patterns specific to the stylization network rather than only perform resolution enhancement.

5. Federated updating and model synchronization

MVStylizer supplements its inference pipeline with an edge-cloud federated learning scheme intended to keep the edge DNN improving over time (Li et al., 2020). The cloud maintains a global model $\mathbf{F}_c$3, edge servers hold local models $\mathbf{F}_c$4, and synchronization occurs when an edge server is idle.

The paper’s synchronization procedure is as follows. A pretrained DNN is first deployed on the cloud and distributed to edge servers for initialization. At round $\mathbf{F}_c$5, each participating edge server receives the latest global parameters $\mathbf{F}_c$6, sets its local model accordingly, retrains on local data $\mathbf{F}_c$7, and uploads updated parameters back to the cloud. The cloud then aggregates the received parameters. The paper writes the aggregation rule as

$\mathbf{F}_c$8

while the accompanying text describes this process as “averaging.” The displayed equation omits the normalization factor, so the intended procedure is described textually as parameter averaging, but the printed formula is an unnormalized sum.

In the evaluation protocol, each participating edge server trains on 4000 randomly sampled images before synchronization, and the experiments consider 1, 2, and 4 participating edge servers. With 4 edge servers, the total loss reaches 0.0748 after 12,000 images trained on each edge server; to reach similar performance, 1 server requires 40,000 images, and 2 servers require 32,000 images on each server.

The paper motivates this module as a way to leverage the diversity of collected data from various mobile clients. It does not, however, provide a formal privacy analysis or secure aggregation mechanism. Raw key frames are still sent to the edge for inference, and no encryption, differential privacy, or poisoning-robust aggregation method is described.

6. Empirical evaluation

The stylizer is trained on MS-COCO, using 328k images and 91 object types, with both content and style images sampled from that dataset (Li et al., 2020). Evaluation uses 100 videos downloaded from Videvo.net, totaling around 41,500 frames, with one style image assigned to each video. Runtime and image-quality experiments use 1000 sampled frames.

The paper compares MVStylizer chiefly against Luan et al. and Li et al. For edge-side stylization runtime, MVStylizer is markedly faster. At 512×256, it requires 0.52 s/frame, compared with 186.52 s for Luan et al. and 2.95 s for Li et al. At 1920×1080, MVStylizer requires 1.51 s/frame, while Li et al. requires 38.72 s, and Luan et al. is reported as >1000.00 s. The mobile interpolation stage is substantially cheaper still: 0.0006 s at 512×256, 0.002 s at 768×384, 0.006 s at 1024×512, and 0.02 s at 1920×1080.

The paper formalizes the resulting acceleration as

$\mathbf{F}_c$9

where $\mathbf{F}_s$0 is the DNN stylization time per frame and $\mathbf{F}_s$1 is the interpolation time per intermediate frame. The headline result is a 75.5× speedup for 1920×1080 videos.

Quality evaluation is reported in several forms. On 1000 sampled 512×256 frames, MVStylizer achieves Inception Score 135.20 and FID 164.23, compared with 129.54 and 168.71 for Li et al. The interpolated stylized frames are also close to directly stylized frames, with MS-SSIM values of 0.9845 at 512×256, 0.9841 at 768×384, 0.9847 at 1024×512, and 0.9849 at 1920×1080. The paper interprets these values as indicating that optical-flow-generated intermediate frames are perceptually very similar to frames stylized directly by the DNN.

7. Limitations, interpretation, and place in the literature

The paper is explicit about several absences in the formulation and leaves other weaknesses implicit (Li et al., 2020). MVStylizer does not provide an explicit temporal consistency loss, an explicit occlusion-handling model, a new key-frame-selection algorithm of its own, or a formal privacy/security treatment for the federated component. It also does not report memory use, bandwidth consumption beyond upload latency, or mobile energy cost. Sensitivity to difficult motion, scene cuts, and severe occlusion is not systematically studied. This suggests that the system’s empirical strengths are clearest when videos can be reconstructed well from sparse key-frame stylization plus optical-flow warping.

A second common misconception is to read MVStylizer as a modern end-to-end video stylization foundation model. It is more accurately classified as a mobile-edge-cloud systems paper whose central novelty lies in computation placement and pipeline integration. Its main contribution is not a radically new stylization objective, but a deployment strategy that makes photorealistic video stylization substantially more practical on mobile hardware.

In later stylization literature, different problem settings became prominent. Stylos is a multi-view 3D stylization method based on single-forward Gaussian Splatting and global style-content coupling (Liu et al., 30 Sep 2025), while V-Stylist is a training-free text-driven video stylization framework built around MLLM/LLM agents, shot decomposition, style retrieval, and self-reflection (Yue et al., 15 Mar 2025). Relative to such later systems, MVStylizer belongs to an earlier line of work centered on efficient mobile inference, sparse remote processing, and photorealistic reconstruction. This contrast highlights MVStylizer’s historical role: it is best understood as an early practical architecture for mobile video stylization rather than as a 3D, multimodal, or agentic stylization framework.