SemFi: Semantic Frame Interpolation & Fuzzing

• SemFi is a label for semantics-centered systems applied to generative video via Semantic Frame Interpolation and network protocol security via semantics-aware fuzzing.
• In video generation, it utilizes endpoint conditioning, text guidance, and a Mixture-of-LoRA module to achieve smooth, multi-rate frame interpolation.
• In network security, SemFi (alias SemFuzz) extracts structured semantic rules from RFCs to generate test cases that target protocol vulnerabilities.

Searching arXiv for the provided identifiers and the term "SemFi" to ground the article in the cited literature. SemFi is a label used in recent arXiv literature for two unrelated semantics-centered systems. In generative video, it denotes a model introduced with the task of Semantic Frame Interpolation (SFI), where intermediate frames are generated between a first frame and a last frame under text guidance and for variable target lengths (Hong et al., 7 Jul 2025). In network security, “SemFi” is used as an alias for SemFuzz, a semantics-aware fuzzing framework that extracts structured semantic rules from RFC documents, generates protocol test cases that intentionally violate those rules, and detects vulnerabilities by comparing observed and expected responses (Sun et al., 6 Mar 2026). This suggests that “SemFi” is not yet a standardized designation but a local shorthand attached to distinct research programs.

1. Terminological scope and disambiguation

The two documented uses of SemFi differ in objective, data modality, and formal apparatus.

Usage of “SemFi”	Domain	Core object
SemFi model	Video generation	Semantic Frame Interpolation
Alias for SemFuzz	Network protocol security	Semantics-aware fuzzing

In the video setting, the relevant paper formally defines SFI as conditional sequence generation from two endpoint frames, a text prompt, and a target number of intermediate frames (Hong et al., 7 Jul 2025). In the protocol-security setting, the relevant paper defines semantics-aware fuzzing through structured semantic rules extracted from RFC specifications and a semantic oracle that flags deviations between observed and expected responses (Sun et al., 6 Mar 2026).

The shared lexical element is “semantic,” but the technical meanings are distinct. In SFI, semantics concerns text-conditioned visual transition and consistency across frame counts. In SemFuzz, semantics concerns compliance with specification semantics and receiver behavior under rule violations.

2. SemFi in Semantic Frame Interpolation: task definition

Semantic Frame Interpolation is defined by the inputs $I_f \in \mathbb{R}^{H \times W \times C}$ for the first frame, $I_l \in \mathbb{R}^{H \times W \times C}$ for the last frame, a text prompt $T$, and a positive integer $N \in \mathbb{Z}^+$ specifying the number of intermediate frames to generate (Hong et al., 7 Jul 2025). The output is a sequence $\{I_1, I_2, \dots, I_N\}$, each in $\mathbb{R}^{H \times W \times C}$, forming a smooth semantic transition from $I_f$ to $I_l$ under guidance $T$ (Hong et al., 7 Jul 2025).

The paper models SFI as conditional sequence generation under a diffusion or other generative model

$p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),$

with inference defined by sampling

$I_l \in \mathbb{R}^{H \times W \times C}$0

This formulation unifies previously separated settings (Hong et al., 7 Jul 2025). When $I_l \in \mathbb{R}^{H \times W \times C}$1 is small and $I_l \in \mathbb{R}^{H \times W \times C}$2, the task recovers classic short-range Video Frame Interpolation. When $I_l \in \mathbb{R}^{H \times W \times C}$3 is large, it recovers long-range frame-to-frame generation as in Foundation Video Models (Hong et al., 7 Jul 2025).

The paper’s motivation is that traditional frame interpolation emphasizes a small number of frames, no text control, and minimal differences between the first and last frames, whereas recent community use of large video models represented by Wan yields frame-to-frame generation with a fixed number of frames and often unsatisfactory behavior for certain frame lengths (Hong et al., 7 Jul 2025). The proposed SFI task is therefore positioned as a practical academic definition that covers both short-range interpolation and long-range generation while supporting inference at multiple frame rates (Hong et al., 7 Jul 2025).

3. SemFi model architecture and multi-rate inference

The SemFi model is built upon Wan2.1, specifically Wan2.1-I2V, and uses a Wan-VAE, a Wan-DiT diffusion transformer, endpoint-image embeddings, text conditioning, and a Mixture-of-LoRA module (Hong et al., 7 Jul 2025). The backbone encodes a concatenated temporal tensor $I_l \in \mathbb{R}^{H \times W \times C}$4, where positions $I_l \in \mathbb{R}^{H \times W \times C}$5 and $I_l \in \mathbb{R}^{H \times W \times C}$6 hold $I_l \in \mathbb{R}^{H \times W \times C}$7 and $I_l \in \mathbb{R}^{H \times W \times C}$8, and positions $I_l \in \mathbb{R}^{H \times W \times C}$9 are zero-filled (Hong et al., 7 Jul 2025). A binary mask

$T$0

with $T$1 and $T$2 is encoded in parallel (Hong et al., 7 Jul 2025).

Conditional information injection proceeds through three channels. First, the guidance tensor is $T$3. Second, the mask is $T$4. Third, a CLIP image encoder produces embeddings $T$5 and $T$6, which are summed and projected as $T$7, with $T$8 a linear projection; $T$9 is inserted via cross-attention in DiT layers alongside text-prompt tokens (Hong et al., 7 Jul 2025). Standard Wan text conditioning remains in place, and the endpoint image embedding and text tokens jointly attend in each block, aligning visual boundary conditions and semantic intent (Hong et al., 7 Jul 2025).

The principal architectural novelty is the Mixture-of-LoRA module. Its motivation is that a single LoRA adapts poorly across widely varying $N \in \mathbb{Z}^+$0 (Hong et al., 7 Jul 2025). The module contains a universal LoRA $N \in \mathbb{Z}^+$1 trained on all $N \in \mathbb{Z}^+$2, together with expert LoRAs $N \in \mathbb{Z}^+$3 for discrete frame counts $N \in \mathbb{Z}^+$4 (Hong et al., 7 Jul 2025). For a target $N \in \mathbb{Z}^+$5, the selected expert index is

$N \in \mathbb{Z}^+$6

and the combined low-rank update is

$N \in \mathbb{Z}^+$7

In each forward pass, the original weight $N \in \mathbb{Z}^+$8 in all linear and attention projections is replaced by

$N \in \mathbb{Z}^+$9

The paper attributes distinct interpolation regimes to different scales: short-range $\{I_1, I_2, \dots, I_N\}$0 emphasizes high-frequency motion and texture, mid-range $\{I_1, I_2, \dots, I_N\}$1 emphasizes smooth semantic transitions, and long-range $\{I_1, I_2, \dots, I_N\}$2 emphasizes large semantic drift and creative generation (Hong et al., 7 Jul 2025).

The multi-rate inference mechanism follows this adapter-selection logic during sampling. A latent $\{I_1, I_2, \dots, I_N\}$3 is initialized; at each denoising step, the DiT predicts updates conditioned on $\{I_1, I_2, \dots, I_N\}$4, $\{I_1, I_2, \dots, I_N\}$5, $\{I_1, I_2, \dots, I_N\}$6, $\{I_1, I_2, \dots, I_N\}$7, and $\{I_1, I_2, \dots, I_N\}$8; and the decoded $\{I_1, I_2, \dots, I_N\}$9 yields $\mathbb{R}^{H \times W \times C}$0 through Wan-VAE (Hong et al., 7 Jul 2025). Because the trained expert set $\mathbb{R}^{H \times W \times C}$1 follows a geometric progression of frame counts and nearest-neighbor selection is used, $\mathbb{R}^{H \times W \times C}$2 need not lie exactly in $\mathbb{R}^{H \times W \times C}$3; the paper states that the universal LoRA helps prevent catastrophic performance drop for unseen $\mathbb{R}^{H \times W \times C}$4 (Hong et al., 7 Jul 2025).

4. SFI-300K, SFIBench, and empirical behavior

The SFI paper introduces SFI-300K as the first general-purpose dataset and benchmark specifically designed for SFI (Hong et al., 7 Jul 2025). The source pool consists of high-quality clips from Open Sora Plan datasets. Clips are filtered by frame rate $\mathbb{R}^{H \times W \times C}$5 FPS and clip length $\mathbb{R}^{H \times W \times C}$6 with $\mathbb{R}^{H \times W \times C}$7 (Hong et al., 7 Jul 2025). For each clip $\mathbb{R}^{H \times W \times C}$8, the preprocessing computes a CLIP similarity

$\mathbb{R}^{H \times W \times C}$9

and a flow score

$I_f$0

via RAFT; thresholds on $I_f$1 and $I_f$2 retain only clips with meaningful visual change (Hong et al., 7 Jul 2025). For each $I_f$3, centered segments are extracted by multi-frame trimming, and captions are produced with Qwen2.5-VL-32B using a specialized prompt. The final dataset is $I_f$4 (Hong et al., 7 Jul 2025).

SFIBench evaluates multiple dimensions. Video fidelity uses per-frame average LPIPS and FID between distributions of generated and ground-truth frames. Frame fidelity evaluates endpoints using PSNR, SSIM, and LPIPS. Semantic fidelity uses ViCLIP-score, defined as the cosine between video and text features. Additional video quality dimensions taken from VBench are Temporal Flickering, Motion Smoothness, Dynamic Degree, Aesthetic Quality, and Imaging Quality (Hong et al., 7 Jul 2025).

On aggregate across all $I_f$5, SemFi is compared against GI, FCVG, and Wan. Reported values are LPIPS $I_f$6 and FID $I_f$7 for Video Fidelity; PSNR $I_f$8, SSIM $I_f$9, and LPIPS $I_l$0 for Frame Fidelity; Semantic Fidelity $I_l$1; Flickering $I_l$2; Smoothness $I_l$3; Dynamic $I_l$4; Aesthetic $I_l$5; and Imaging $I_l$6 (Hong et al., 7 Jul 2025). The paper states that SemFi consistently outperforms or matches the best baseline across five out of eight metrics (Hong et al., 7 Jul 2025).

Performance varies by frame count. At small $I_l$7, SemFi is reported to dramatically reduce LPIPS and FID relative to Wan; at $I_l$8, LPIPS changes from $I_l$9 to $T$0, and FID from $T$1 to $T$2 (Hong et al., 7 Jul 2025). At mid-range $T$3, SemFi trades a slight fidelity drop for higher motion smoothness and aesthetic scores. At long range $T$4, it maintains semantic consistency and superior flicker and smoothness (Hong et al., 7 Jul 2025). The variance analysis reports that $T$5-variance over the six frame counts is lower for SemFi on nearly all metrics, with an example given as FID variance $T$6 versus $T$7, which the paper interprets as greater stability across $T$8 (Hong et al., 7 Jul 2025).

Ablation studies isolate the importance of multi-frame training and the Mixture-of-LoRA module. Without multi-frame training, LPIPS increases from $T$9 to $p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),$0, FID from $p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),$1 to $p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),$2, Flickering changes from $p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),$3 to $p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),$4, and Smoothness from $p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),$5 to $p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),$6 (Hong et al., 7 Jul 2025). Without MoL, LPIPS is $p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),$7, FID is $p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),$8, semantic fidelity remains approximately $p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),$9, Flickering is $I_l \in \mathbb{R}^{H \times W \times C}$00, and Smoothness is $I_l \in \mathbb{R}^{H \times W \times C}$01, while dynamic degree and aesthetic/imaging quality slightly degrade (Hong et al., 7 Jul 2025). The authors conclude that multi-frame data are crucial for scale adaptivity and that MoL refines temporal distribution and prevents artifacts (Hong et al., 7 Jul 2025).

The limitations explicitly noted are a current upper bound of $I_l \in \mathbb{R}^{H \times W \times C}$02, attributed to training-data constraints, and a domain gap for synthetic or heavily stylized footage. Proposed future directions are to extend MoL experts to larger $I_l \in \mathbb{R}^{H \times W \times C}$03, incorporate synthetic data, and investigate continuous-expert weighting instead of nearest-neighbor selection (Hong et al., 7 Jul 2025).

5. SemFi as an alias for SemFuzz: formal semantics-aware fuzzing

In the network-security usage, SemFi denotes SemFuzz, a semantics-aware fuzzing framework for network protocol implementations (Sun et al., 6 Mar 2026). The formal setting introduces a protocol $I_l \in \mathbb{R}^{H \times W \times C}$04 specified by an RFC, a set of message types $I_l \in \mathbb{R}^{H \times W \times C}$05, a set of real-world seed messages $I_l \in \mathbb{R}^{H \times W \times C}$06, and a set of raw specification items $I_l \in \mathbb{R}^{H \times W \times C}$07 extracted from the RFC (Sun et al., 6 Mar 2026). A structured semantic rule is represented as

$I_l \in \mathbb{R}^{H \times W \times C}$08

where $I_l \in \mathbb{R}^{H \times W \times C}$09 is the protocol name, $I_l \in \mathbb{R}^{H \times W \times C}$10 is a message type, $I_l \in \mathbb{R}^{H \times W \times C}$11 is a field or subtree path, $I_l \in \mathbb{R}^{H \times W \times C}$12 is the construction constraint, and $I_l \in \mathbb{R}^{H \times W \times C}$13 is the processing expectation (Sun et al., 6 Mar 2026).

Semantics-aware fuzzing is then the generation of test cases $I_l \in \mathbb{R}^{H \times W \times C}$14 that systematically violate one or more construction constraints while remaining syntactically valid, followed by vulnerability detection through mismatch between observed and expected responses (Sun et al., 6 Mar 2026). For each rule $I_l \in \mathbb{R}^{H \times W \times C}$15 and seed $I_l \in \mathbb{R}^{H \times W \times C}$16, the target is

$I_l \in \mathbb{R}^{H \times W \times C}$17

with a vulnerability flagged whenever

$I_l \in \mathbb{R}^{H \times W \times C}$18

The framework therefore differs from crash-centric fuzzing by making semantic expectations explicit (Sun et al., 6 Mar 2026).

Rule extraction is implemented through two LLM-mediated stages. After cleaning the RFC and splitting it into paragraphs prefixed with section paths, a prompt named “IdentifySR” maps paragraph text and an allowed message-type list to zero or more triples $I_l \in \mathbb{R}^{H \times W \times C}$19 (Sun et al., 6 Mar 2026). A second prompt named “CompleteSR” takes a raw spec and a field-path list extracted from a matching seed and outputs $I_l \in \mathbb{R}^{H \times W \times C}$20, completing the structured rule (Sun et al., 6 Mar 2026). Few-shot examples distinguish correctly extracted rules from irrelevant paragraphs, and seed-message structure is provided so that the model can select the correct field path (Sun et al., 6 Mar 2026).

Mutation proceeds via a Mutation Strategy Generator that produces strategies

$I_l \in \mathbb{R}^{H \times W \times C}$21

where $I_l \in \mathbb{R}^{H \times W \times C}$22 specifies how to violate $I_l \in \mathbb{R}^{H \times W \times C}$23 and $I_l \in \mathbb{R}^{H \times W \times C}$24 is the expected feedback (Sun et al., 6 Mar 2026). Test-case generation has two phases: an LLM prompt “GenerateActions” emits an action sequence $I_l \in \mathbb{R}^{H \times W \times C}$25, with each action belonging to $I_l \in \mathbb{R}^{H \times W \times C}$26, and then a deterministic message mutation engine applies add, remove, and update operations to produce a new test packet (Sun et al., 6 Mar 2026). The deterministic engine recalculates length fields, checksums, and related syntax, so all test cases remain syntactically valid (Sun et al., 6 Mar 2026).

The oracle is explicitly semantics-aware. Each mutation strategy carries an expected response $I_l \in \mathbb{R}^{H \times W \times C}$27; the actual response $I_l \in \mathbb{R}^{H \times W \times C}$28 is observed from live interaction; and the vulnerability predicate is

$I_l \in \mathbb{R}^{H \times W \times C}$29

Rather than using a distance metric, responses are mapped into protocol-specific categories such as HTTP 2xx versus 4xx/5xx, TLS Alert versus HandshakeContinue, and DNS Answer versus Error (Sun et al., 6 Mar 2026).

6. End-to-end workflow, evaluation, and comparative significance

The end-to-end SemFuzz workflow begins from RFC documents $I_l \in \mathbb{R}^{H \times W \times C}$30, a message-type list $I_l \in \mathbb{R}^{H \times W \times C}$31, seed traffic $I_l \in \mathbb{R}^{H \times W \times C}$32, and a target implementation $I_l \in \mathbb{R}^{H \times W \times C}$33 (Sun et al., 6 Mar 2026). It then performs rule extraction, mutation strategy generation, test-case generation, execution and response collection, and analysis that flags all cases where observed response differs from expectation (Sun et al., 6 Mar 2026). Diagrammatically, the system is described as a pipeline from RFC preprocessing through LLM-based rule identification and completion, then LLM-based mutation and action generation, deterministic mutation, execution, and semantic verification (Sun et al., 6 Mar 2026).

The evaluation covers seven implementations: dns.exe for DNS, tcpip.sys for IPv6, schannel.dll, OpenSSL 3.1.3, and LibreSSL 3.4.0 for TLS 1.3, and http.sys and nginx 1.27.2 for HTTP/1.1 (Sun et al., 6 Mar 2026). Reported extraction metrics are average precision $I_l \in \mathbb{R}^{H \times W \times C}$34, recall $I_l \in \mathbb{R}^{H \times W \times C}$35, and $I_l \in \mathbb{R}^{H \times W \times C}$36 for semantic rule extraction, and average precision $I_l \in \mathbb{R}^{H \times W \times C}$37, recall $I_l \in \mathbb{R}^{H \times W \times C}$38, and $I_l \in \mathbb{R}^{H \times W \times C}$39 for rule completion (Sun et al., 6 Mar 2026). The framework generates $I_l \in \mathbb{R}^{H \times W \times C}$40 mutation strategies with accuracy $I_l \in \mathbb{R}^{H \times W \times C}$41, and $I_l \in \mathbb{R}^{H \times W \times C}$42 test cases with accuracy $I_l \in \mathbb{R}^{H \times W \times C}$43 (Sun et al., 6 Mar 2026).

In vulnerability discovery, the paper reports $I_l \in \mathbb{R}^{H \times W \times C}$44 potential vulnerabilities, $I_l \in \mathbb{R}^{H \times W \times C}$45 confirmed real bugs, $I_l \in \mathbb{R}^{H \times W \times C}$46 precision, $I_l \in \mathbb{R}^{H \times W \times C}$47 previously unknown vulnerabilities, and $I_l \in \mathbb{R}^{H \times W \times C}$48 assigned CVEs, including CVE-2021-24074, CVE-2022-34718, CVE-2023-28233, and CVE-2023-28234 (Sun et al., 6 Mar 2026). The confirmed bug classes include cache pollution in DNS, integer overflows and buffer overflows in IPv6 and HTTP.sys, use-after-free in TLS 1.3 and HTTP.sys, and request smuggling in HTTP.sys (Sun et al., 6 Mar 2026). Relative to the baselines BLEEM, ChatAFL, Hdiff, and Fuzztruction-Net, SemFuzz found $I_l \in \mathbb{R}^{H \times W \times C}$49 unique confirmed bugs and $I_l \in \mathbb{R}^{H \times W \times C}$50 of $I_l \in \mathbb{R}^{H \times W \times C}$51 unique bugs discovered by any tool (Sun et al., 6 Mar 2026).

Taken together, the two uses of SemFi exemplify a shared methodological preference for explicit semantic structure, but they should not be conflated. In Semantic Frame Interpolation, SemFi is a multi-rate generative video model centered on endpoint conditioning, text guidance, and Mixture-of-LoRA adaptation (Hong et al., 7 Jul 2025). In SemFuzz, “SemFi” names a fuzzing framework centered on rule extraction from RFC semantics, intent-driven mutation, and an expected-response oracle (Sun et al., 6 Mar 2026). A plausible implication is that the label currently functions as a paper-local abbreviation rather than as a stable term of art across machine learning and systems security.