---
title: 'SemFi: Semantic Frame Interpolation & Fuzzing'
url: https://www.emergentmind.com/topics/semfi
type: topic
---

# SemFi: Semantic Frame Interpolation & Fuzzing

Searching arXiv for the provided identifiers and the term "SemFi" to ground the article in the cited literature.
SemFi is a label used in recent arXiv literature for two unrelated semantics-centered systems. In generative video, it denotes a model introduced with the task of Semantic Frame Interpolation (SFI), where intermediate frames are generated between a first frame and a last frame under text guidance and for variable target lengths [2507.05173]. In network security, “SemFi” is used as an alias for SemFuzz, a semantics-aware fuzzing framework that extracts structured semantic rules from RFC documents, generates protocol test cases that intentionally violate those rules, and detects vulnerabilities by comparing observed and expected responses [2603.05989]. This suggests that “SemFi” is not yet a standardized designation but a local shorthand attached to distinct research programs.

## 1. Terminological scope and disambiguation

The two documented uses of SemFi differ in objective, data modality, and formal apparatus.

| Usage of “SemFi” | Domain | Core object |
|---|---|---|
| SemFi model | Video generation | Semantic Frame Interpolation |
| Alias for SemFuzz | Network protocol security | Semantics-aware fuzzing |

In the video setting, the relevant paper formally defines SFI as conditional sequence generation from two endpoint frames, a text prompt, and a target number of intermediate frames [2507.05173]. In the protocol-security setting, the relevant paper defines semantics-aware fuzzing through structured semantic rules extracted from RFC specifications and a semantic oracle that flags deviations between observed and expected responses [2603.05989].

The shared lexical element is “semantic,” but the technical meanings are distinct. In SFI, semantics concerns text-conditioned visual transition and consistency across frame counts. In SemFuzz, semantics concerns compliance with specification semantics and receiver behavior under rule violations.

## 2. SemFi in Semantic Frame Interpolation: task definition

Semantic Frame Interpolation is defined by the inputs
\(I_f \in \mathbb{R}^{H \times W \times C}\) for the first frame,
\(I_l \in \mathbb{R}^{H \times W \times C}\) for the last frame,
a text prompt \(T\),
and a positive integer \(N \in \mathbb{Z}^+\) specifying the number of intermediate frames to generate [2507.05173]. The output is a sequence \(\{I_1, I_2, \dots, I_N\}\), each in \(\mathbb{R}^{H \times W \times C}\), forming a smooth semantic transition from \(I_f\) to \(I_l\) under guidance \(T\) [2507.05173].

The paper models SFI as conditional sequence generation under a diffusion or other generative model
$$
p_\theta(I_1,\dots,I_N \mid I_f, I_l, T, N),
$$
with inference defined by sampling
$$
\{I_1,\dots,I_N\} \sim p_\theta(I_{1:N} \mid I_f, I_l, T, N).
$$
This formulation unifies previously separated settings [2507.05173]. When \(N\) is small and \(T=\varnothing\), the task recovers classic short-range Video Frame Interpolation. When \(N\) is large, it recovers long-range frame-to-frame generation as in Foundation Video Models [2507.05173].

The paper’s motivation is that traditional frame interpolation emphasizes a small number of frames, no text control, and minimal differences between the first and last frames, whereas recent community use of large video models represented by Wan yields frame-to-frame generation with a fixed number of frames and often unsatisfactory behavior for certain frame lengths [2507.05173]. The proposed SFI task is therefore positioned as a practical academic definition that covers both short-range interpolation and long-range generation while supporting inference at multiple frame rates [2507.05173].

## 3. SemFi model architecture and multi-rate inference

The SemFi model is built upon Wan2.1, specifically Wan2.1-I2V, and uses a Wan-VAE, a Wan-DiT diffusion transformer, endpoint-image embeddings, text conditioning, and a Mixture-of-LoRA module [2507.05173]. The backbone encodes a concatenated temporal tensor
\(I_c \in \mathbb{R}^{N \times H \times W \times C}\),
where positions \(0\) and \(N-1\) hold \(I_f\) and \(I_l\), and positions \(1 \dots N-2\) are zero-filled [2507.05173]. A binary mask
\(M \in \{0,1\}^N\)
with \(M_0=M_{N-1}=1\) and \(M_{1:N-2}=0\) is encoded in parallel [2507.05173].

Conditional information injection proceeds through three channels. First, the guidance tensor is
\(I_c \leftarrow \mathrm{concat}([I_f, 0,\dots,0, I_l])\). Second, the mask is
\(M \leftarrow [1,0,\dots,0,1]\). Third, a CLIP image encoder produces embeddings \(E_f\) and \(E_l\), which are summed and projected as
\(E_0 = \phi(E_f + E_l)\),
with \(\phi\) a linear projection; \(E_0\) is inserted via cross-attention in DiT layers alongside text-prompt tokens [2507.05173]. Standard Wan text conditioning remains in place, and the endpoint image embedding and text tokens jointly attend in each block, aligning visual boundary conditions and semantic intent [2507.05173].

The principal architectural novelty is the Mixture-of-LoRA module. Its motivation is that a single LoRA adapts poorly across widely varying \(N\) [2507.05173]. The module contains a universal LoRA \(\Delta W_U\) trained on all \(N\), together with expert LoRAs \(\Delta W_{E_s}\) for discrete frame counts
\(s \in S = \{5,9,17,33,65,81\}\) [2507.05173]. For a target \(N\), the selected expert index is
$$
s^* = \arg\min_{s \in S} |N-s|,
$$
and the combined low-rank update is
$$
\Delta W = \Delta W_U + \Delta W_{E_{s^*}}.
$$
In each forward pass, the original weight \(W\) in all linear and attention projections is replaced by
$$
W' = W + \Delta W.
$$
The paper attributes distinct interpolation regimes to different scales: short-range \((5,9)\) emphasizes high-frequency motion and texture, mid-range \((17,33)\) emphasizes smooth semantic transitions, and long-range \((65,81)\) emphasizes large semantic drift and creative generation [2507.05173].

The multi-rate inference mechanism follows this adapter-selection logic during sampling. A latent \(z_T \sim \mathcal{N}(0,I)\) is initialized; at each denoising step, the DiT predicts updates conditioned on \(W+\Delta W\), \(I_c\), \(M\), \(T\), and \(E_0\); and the decoded \(z_0\) yields \(\{I_1,\dots,I_N\}\) through Wan-VAE [2507.05173]. Because the trained expert set \(S\) follows a geometric progression of frame counts and nearest-neighbor selection is used, \(N\) need not lie exactly in \(S\); the paper states that the universal LoRA helps prevent catastrophic performance drop for unseen \(N\) [2507.05173].

## 4. SFI-300K, SFIBench, and empirical behavior

The SFI paper introduces SFI-300K as the first general-purpose dataset and benchmark specifically designed for SFI [2507.05173]. The source pool consists of high-quality clips from Open Sora Plan datasets. Clips are filtered by frame rate \(>30\) FPS and clip length \(f \in [f_{\max}, 4f_{\max}]\) with \(f_{\max}=81\) [2507.05173]. For each clip \(v\), the preprocessing computes a CLIP similarity
\(S_c = \mathrm{Cosine}(E(\mathrm{First}), E(\mathrm{Last}))\)
and a flow score
\(S_f = \text{average } \| \mathrm{Flow}(\mathrm{Last}\rightarrow \mathrm{First}) \|_2\)
via RAFT; thresholds on \(S_c\) and \(S_f\) retain only clips with meaningful visual change [2507.05173]. For each \(s \in S\), centered segments are extracted by multi-frame trimming, and captions are produced with Qwen2.5-VL-32B using a specialized prompt. The final dataset is
\(D=\{(V_i,C_i)\}_{i=1}^{300\,000}\) [2507.05173].

SFIBench evaluates multiple dimensions. Video fidelity uses per-frame average LPIPS and FID between distributions of generated and ground-truth frames. Frame fidelity evaluates endpoints using PSNR, SSIM, and LPIPS. Semantic fidelity uses ViCLIP-score, defined as the cosine between video and text features. Additional video quality dimensions taken from VBench are Temporal Flickering, Motion Smoothness, Dynamic Degree, Aesthetic Quality, and Imaging Quality [2507.05173].

On aggregate across all \(N\), SemFi is compared against GI, FCVG, and Wan. Reported values are LPIPS \(=0.2664\) and FID \(=74.52\) for Video Fidelity; PSNR \(=21.96\), SSIM \(=0.6576\), and LPIPS \(=0.2025\) for Frame Fidelity; Semantic Fidelity \(=0.2217\); Flickering \(=0.9816\); Smoothness \(=0.9901\); Dynamic \(=0.2333\); Aesthetic \(=0.5114\); and Imaging \(=0.6129\) [2507.05173]. The paper states that SemFi consistently outperforms or matches the best baseline across five out of eight metrics [2507.05173].

Performance varies by frame count. At small \(N\in\{5,9\}\), SemFi is reported to dramatically reduce LPIPS and FID relative to Wan; at \(N=5\), LPIPS changes from \(0.4772\) to \(0.2472\), and FID from \(275.1\) to \(86.63\) [2507.05173]. At mid-range \(N\in\{17,33\}\), SemFi trades a slight fidelity drop for higher motion smoothness and aesthetic scores. At long range \(N\in\{65,81\}\), it maintains semantic consistency and superior flicker and smoothness [2507.05173]. The variance analysis reports that \(\log_{10}\)-variance over the six frame counts is lower for SemFi on nearly all metrics, with an example given as FID variance \(-1.842\) versus \(-3.741\), which the paper interprets as greater stability across \(N\) [2507.05173].

Ablation studies isolate the importance of multi-frame training and the Mixture-of-LoRA module. Without multi-frame training, LPIPS increases from \(0.2664\) to \(0.2841\), FID from \(74.52\) to \(109.2\), Flickering changes from \(0.9816\) to \(0.9700\), and Smoothness from \(0.9901\) to \(0.9801\) [2507.05173]. Without MoL, LPIPS is \(0.2476\), FID is \(77.04\), semantic fidelity remains approximately \(0.2208\), Flickering is \(0.9815\), and Smoothness is \(0.9900\), while dynamic degree and aesthetic/imaging quality slightly degrade [2507.05173]. The authors conclude that multi-frame data are crucial for scale adaptivity and that MoL refines temporal distribution and prevents artifacts [2507.05173].

The limitations explicitly noted are a current upper bound of \(N=81\), attributed to training-data constraints, and a domain gap for synthetic or heavily stylized footage. Proposed future directions are to extend MoL experts to larger \(N\), incorporate synthetic data, and investigate continuous-expert weighting instead of nearest-neighbor selection [2507.05173].

## 5. SemFi as an alias for SemFuzz: formal semantics-aware fuzzing

In the network-security usage, SemFi denotes SemFuzz, a semantics-aware fuzzing framework for network protocol implementations [2603.05989]. The formal setting introduces a protocol \(P\) specified by an RFC, a set of message types \(\mathcal{M}\), a set of real-world seed messages \(\mathcal{S}=\{S_1,\dots,S_n\}\), and a set of raw specification items \(\mathcal{R}=\{R_1,\dots,R_k\}\) extracted from the RFC [2603.05989]. A structured semantic rule is represented as
$$
SR_i = \bigl(p,m,f,\;C,\;E\bigr),
$$
where \(p \in P\) is the protocol name, \(m \in \mathcal{M}\) is a message type, \(f\) is a field or subtree path, \(C=(r_{\text{role}}, c_{\text{constraint}})\) is the construction constraint, and \(E=(r'_{\text{role}}, e_{\text{expectation}})\) is the processing expectation [2603.05989].

Semantics-aware fuzzing is then the generation of test cases \(\mathcal{T}\) that systematically violate one or more construction constraints while remaining syntactically valid, followed by vulnerability detection through mismatch between observed and expected responses [2603.05989]. For each rule \(SR_i\) and seed \(S_j\), the target is
$$
T_{i,j} = \mathit{Mutate}(S_j,\;{\sf violate}(C)),
$$
with a vulnerability flagged whenever
$$
O(T_{i,j}) \neq e_{\text{expectation}}.
$$
The framework therefore differs from crash-centric fuzzing by making semantic expectations explicit [2603.05989].

Rule extraction is implemented through two LLM-mediated stages. After cleaning the RFC and splitting it into paragraphs prefixed with section paths, a prompt named “IdentifySR” maps paragraph text and an allowed message-type list to zero or more triples \(R_i=(p,m,c)\) [2603.05989]. A second prompt named “CompleteSR” takes a raw spec and a field-path list extracted from a matching seed and outputs \((f,C,E)\), completing the structured rule [2603.05989]. Few-shot examples distinguish correctly extracted rules from irrelevant paragraphs, and seed-message structure is provided so that the model can select the correct field path [2603.05989].

Mutation proceeds via a Mutation Strategy Generator that produces strategies
$$
M_{i,k}=(p,m,f,\;d_{i,k},\;e_{i,k}),
$$
where \(d_{i,k}\) specifies how to violate \(C\) and \(e_{i,k}\) is the expected feedback [2603.05989]. Test-case generation has two phases: an LLM prompt “GenerateActions” emits an action sequence
\(\mathcal{A}=[a_1,\dots,a_\ell]\),
with each action belonging to \(\{\mathtt{add},\mathtt{remove},\mathtt{update}\}\), and then a deterministic message mutation engine applies add, remove, and update operations to produce a new test packet [2603.05989]. The deterministic engine recalculates length fields, checksums, and related syntax, so all test cases remain syntactically valid [2603.05989].

The oracle is explicitly semantics-aware. Each mutation strategy carries an expected response \(e_{i,k}\); the actual response \(o_{i,k}=O(T_{i,k})\) is observed from live interaction; and the vulnerability predicate is
$$
\mathit{vuln}(T_{i,k})=
\begin{cases}
1 & \text{if } o_{i,k}\neq e_{i,k},\\
0 & \text{otherwise.}
\end{cases}
$$
Rather than using a distance metric, responses are mapped into protocol-specific categories such as HTTP 2xx versus 4xx/5xx, TLS Alert versus HandshakeContinue, and DNS Answer versus Error [2603.05989].

## 6. End-to-end workflow, evaluation, and comparative significance

The end-to-end SemFuzz workflow begins from RFC documents \(\mathcal{D}\), a message-type list \(\mathcal{L}\), seed traffic \(\mathcal{S}\), and a target implementation \(Tgt\) [2603.05989]. It then performs rule extraction, mutation strategy generation, test-case generation, execution and response collection, and analysis that flags all cases where observed response differs from expectation [2603.05989]. Diagrammatically, the system is described as a pipeline from RFC preprocessing through LLM-based rule identification and completion, then LLM-based mutation and action generation, deterministic mutation, execution, and semantic verification [2603.05989].

The evaluation covers seven implementations: dns.exe for DNS, tcpip.sys for IPv6, schannel.dll, OpenSSL 3.1.3, and LibreSSL 3.4.0 for TLS 1.3, and http.sys and nginx 1.27.2 for HTTP/1.1 [2603.05989]. Reported extraction metrics are average precision \(0.80\), recall \(0.88\), and \(F1=0.84\) for semantic rule extraction, and average precision \(0.77\), recall \(0.84\), and \(F1=0.80\) for rule completion [2603.05989]. The framework generates \(5\,940\) mutation strategies with accuracy \(0.92\), and \(5\,509\) test cases with accuracy \(0.87\) [2603.05989].

In vulnerability discovery, the paper reports \(16\) potential vulnerabilities, \(10\) confirmed real bugs, \(62.5\%\) precision, \(5\) previously unknown vulnerabilities, and \(4\) assigned CVEs, including CVE-2021-24074, CVE-2022-34718, CVE-2023-28233, and CVE-2023-28234 [2603.05989]. The confirmed bug classes include cache pollution in DNS, integer overflows and buffer overflows in IPv6 and HTTP.sys, use-after-free in TLS 1.3 and HTTP.sys, and request smuggling in HTTP.sys [2603.05989]. Relative to the baselines BLEEM, ChatAFL, Hdiff, and Fuzztruction-Net, SemFuzz found \(10\) unique confirmed bugs and \(11\) of \(11\) unique bugs discovered by any tool [2603.05989].

Taken together, the two uses of SemFi exemplify a shared methodological preference for explicit semantic structure, but they should not be conflated. In Semantic Frame Interpolation, SemFi is a multi-rate generative video model centered on endpoint conditioning, text guidance, and Mixture-of-LoRA adaptation [2507.05173]. In SemFuzz, “SemFi” names a fuzzing framework centered on rule extraction from RFC semantics, intent-driven mutation, and an expected-response oracle [2603.05989]. A plausible implication is that the label currently functions as a paper-local abbreviation rather than as a stable term of art across machine learning and systems security.

Source: https://www.emergentmind.com/topics/semfi