FreeLOC: Training-Free Long Video Generation

• FreeLOC is a training-free inference framework that extends pre-trained short-video diffusion models to generate long videos by addressing two out-of-distribution issues.
• It employs Video-based Relative Position Re-encoding (VRPR) to remap temporal positions and Tiered Sparse Attention (TSA) to structure attention over extended token contexts.
• Layer-adaptive probing selectively applies VRPR and TSA across transformer layers, resulting in improved video consistency, quality, and efficient long-video generation.

Searching arXiv for FreeLOC and closely related long-video generation work. FreeLOC, introduced in "Free-Lunch Long Video Generation via Layer-Adaptive O.O.D Correction" (Tian et al., 26 Mar 2026), is a training-free inference framework for extending pre-trained short-video diffusion transformers to substantially longer video generation without retraining. Its central thesis is that long-video degradation in short-video video DiTs arises from two distinct out-of-distribution effects: frame-level relative position O.O.D., caused by temporal relative positions moving beyond the pretrained RoPE regime, and context-length O.O.D., caused by attention operating over far more tokens than the model was trained to handle. FreeLOC addresses these with Video-based Relative Position Re-encoding (VRPR), Tiered Sparse Attention (TSA), and an offline layer-adaptive probing procedure that determines where each correction should be applied (Tian et al., 26 Mar 2026).

1. Problem formulation and architectural scope

FreeLOC is designed for transformer-based video diffusion models that were trained on short clips, such as Wan2.1-T2V-1.3B and HunyuanVideo, and are then asked to generate much longer videos at inference time (Tian et al., 26 Mar 2026). It is not a new generative model and does not require finetuning; rather, it is a plug-in, inference-time modification of self-attention and positional handling inside existing video DiTs.

The baseline attention form used in the paper is

$$\text{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \text{softmax}\left(\frac{\text{RoPE}(\mathbf{Q})\text{RoPE}(\mathbf{K})^\top}{\sqrt{d}}\right)\mathbf{V},$$

where $\mathbf{Q}$, $\mathbf{K}$, and $\mathbf{V}$ are query, key, and value tokens, and $d$ is key dimensionality (Tian et al., 26 Mar 2026). For frame indices $i$ and $j$, the temporal relative position is

$P_{\text{ori}} = i-j.$

When the video length $L$ is extended beyond the pretrained regime, the temporal relative-position range expands to $[-(L-1), L-1]$, and the attention context grows proportionally (Tian et al., 26 Mar 2026).

The intended input is a pretrained video DiT, a text prompt, a target length larger than the pretrained length, a precomputed layer sensitivity profile, and the hyperparameters for VRPR and TSA. The output is a generated long video at the requested length. The method is explicitly training-free: no gradient updates, no test-time optimization, and no retraining are required (Tian et al., 26 Mar 2026).

2. The two out-of-distribution pathologies

FreeLOC treats long-video failure not as a single extrapolation problem but as the superposition of two O.O.D. effects (Tian et al., 26 Mar 2026).

The first is frame-level relative position O.O.D.. Because video DiTs use 3D RoPE to encode spatiotemporal relations, extending generation length forces the model to extrapolate temporal relative positions beyond the range seen during training. The paper argues that this causes the model to misinterpret temporal dependencies, with visible consequences such as degraded visual quality and temporal inconsistency. It also reports that this effect is layer-dependent: some layers are much more sensitive than others (Tian et al., 26 Mar 2026).

The second is context-length O.O.D.. Even if relative positions are remapped into the pretrained domain, the self-attention softmax still operates over far more tokens than the model was trained to distribute probability mass across. The paper diagnoses this through rising attention entropy, which corresponds to more diffuse attention and weaker preservation of local detail. For a token-wise post-softmax attention distribution $\mathbf{Q}$0, the Shannon entropy is

$\mathbf{Q}$1

As context length increases, the reported entropy increases as well, which the paper associates with blurred details, weaker motion precision, and less stable local visual structure (Tian et al., 26 Mar 2026).

This decomposition is important because it motivates distinct remedies. A positional correction alone does not resolve attention diffusion, while a sparsity pattern alone does not correct RoPE extrapolation. FreeLOC therefore pairs a position re-encoding mechanism with a context-structuring mechanism and applies them selectively.

3. Core mechanisms: VRPR and TSA

VRPR

VRPR, or Video-based Relative Position Re-encoding, remaps temporal relative positions back into the pretrained RoPE regime using a hierarchical multi-granularity scheme (Tian et al., 26 Mar 2026). The method partitions temporal distances into three regions, controlled by thresholds $\mathbf{Q}$2 and $\mathbf{Q}$3:

1. $\mathbf{Q}$4: fine-grained region
2. $\mathbf{Q}$5: medium-grained region
3. $\mathbf{Q}$6: coarse-grained region

In the fine-grained region, relative positions are preserved exactly. In the medium- and coarse-grained regions, positions are quantized with group sizes $\mathbf{Q}$7 and $\mathbf{Q}$8, respectively, so that local temporal precision is retained while longer-range relations are compressed back into the pretrained positional range (Tian et al., 26 Mar 2026). The design premise is that nearby frames require exact temporal precision for motion and detail, whereas distant frames primarily need approximate ordering for global coherence.

The paper states that the re-encoded maximum relative position must remain within the pretrained range. For target length $\mathbf{Q}$9 and pretrained length $\mathbf{K}$0, the hyperparameters are chosen so that

$\mathbf{K}$1

Reported settings include $\mathbf{K}$2 for Wan 4× extension, and $\mathbf{K}$3 for Hunyuan 4× extension (Tian et al., 26 Mar 2026).

TSA

TSA, or Tiered Sparse Attention, addresses context-length O.O.D. by structuring temporal attention density across scales rather than applying either full attention or a fixed sliding window (Tian et al., 26 Mar 2026). It uses three tiers:

• dense local attention for nearby frames,
• striped attention for mid-range dependencies,
• pruning of very long-range direct attention, with a persistent attention sink.

The mask is defined over frames and within-frame token indices as

$\mathbf{K}$4

Here $\mathbf{K}$5 and $\mathbf{K}$6 are temporal thresholds, $\mathbf{K}$7 is the stripe width, and $\mathbf{K}$8 implements the attention sink, which is the initial frame and remains visible to all later frames (Tian et al., 26 Mar 2026). The stripe width is chosen as

$\mathbf{K}$9

where $\mathbf{V}$0 is the number of tokens per frame and $\mathbf{V}$1 is a density reduction factor (Tian et al., 26 Mar 2026). Reported settings include $\mathbf{V}$2 for Wan 4× and $\mathbf{V}$3 for Hunyuan 4× (Tian et al., 26 Mar 2026).

The resulting division of labor is explicit: VRPR corrects positional extrapolation, while TSA keeps local attention dense, allows structured mid-range access, and preserves a global anchor through the first-frame sink.

4. Layer-adaptive probing and selective application

A defining feature of FreeLOC is that VRPR and TSA are not applied uniformly across all transformer layers (Tian et al., 26 Mar 2026). Instead, the paper performs an offline layer-wise probing procedure to estimate which layers are sensitive to which O.O.D. source.

For frame-level relative position O.O.D., the probe perturbs one self-attention layer at a time by shifting only the RoPE of keys while leaving queries unchanged. Sensitivity is then measured with two signals: Vision Reward, used as a proxy for human-rated generation quality, and Attention Logits Difference (ALD),

$\mathbf{V}$4

where $\mathbf{V}$5 is the average attention logits of layer $\mathbf{V}$6 across heads and sampling steps (Tian et al., 26 Mar 2026). Lower Vision Reward and higher ALD indicate greater positional sensitivity.

For context-length O.O.D., the paper first generates long videos with VRPR already enabled, then replaces only one layer at a time with conventional sliding-window attention and measures the change in attention entropy. The Context Length Sensitivity Score is

$\mathbf{V}$7

Higher $\mathbf{V}$8 indicates that the layer is more sensitive to context-length O.O.D. (Tian et al., 26 Mar 2026).

The supplementary reports that the top two-thirds of layers with the largest degradation or score increase are marked sensitive for each O.O.D. type, while the practical allocation used in FreeLOC assigns the top 50% of layers most sensitive to context length to VRPR+TSA, and the remaining layers to VRPR only; the authors describe this 50/50 split as the best sweet spot (Tian et al., 26 Mar 2026). For Wan2.1-T2V-1.3B, published layer profiles are given for a 30-layer model; for HunyuanVideo, profiles are given for a 60-layer model (Tian et al., 26 Mar 2026).

This selectivity is not merely an efficiency heuristic. The ablations show that layer-adaptive assignment outperforms uniform application, random assignment, reverse assignment, and simple contiguous partitions. This suggests that different transformer layers specialize in different forms of temporal reasoning, and that indiscriminate correction can over-regularize insensitive layers (Tian et al., 26 Mar 2026).

5. Empirical performance, efficiency, and ablations

FreeLOC is evaluated on 100 prompts randomly sampled from VBench-long at 480p ($\mathbf{V}$9) on a single NVIDIA A100 80GB GPU, using 50 denoising steps and CFG $d$0. Wan uses UniPC sampling and Hunyuan uses Euler sampling (Tian et al., 26 Mar 2026). The paper reports VBench metrics for video consistency—Subject Consistency (SC), Background Consistency (BC), Motion Smoothness (MS)—and video quality—Imaging Quality (IQ), Aesthetic Quality (AQ), Dynamic Degree (DD) (Tian et al., 26 Mar 2026).

The main results are summarized below.

Setting	Consistency: SC / BC / MS	Quality: IQ / AQ / DD
Wan2.1-T2V-1.3B, 161 frames	98.06 / 97.49 / 98.98	68.31 / 62.33 / 39.41
Wan2.1-T2V-1.3B, 321 frames	98.44 / 97.78 / 98.97	67.44 / 61.21 / 36.27
HunyuanVideo, 253 frames	97.92 / 97.34 / 98.99	68.92 / 62.38 / 40.22
HunyuanVideo, 509 frames	98.47 / 98.12 / 98.95	67.92 / 61.09 / 39.28

Across both model families and both extension ratios, the paper reports that FreeLOC is the strongest all-around method among training-free baselines such as Direct Sampling, Sliding Window, FreeNoise, FreeLong, and RIFLEx (Tian et al., 26 Mar 2026). A notable result is that direct sampling can retain superficially high consistency scores on Wan 4×—for example SC $d$1 and BC $d$2—while collapsing in Dynamic Degree to $d$3, indicating that long-video failure is not adequately captured by consistency metrics alone (Tian et al., 26 Mar 2026).

The ablations attribute the gains to all three components. For positional handling on Wan 4×, VRPR outperforms clipping, grouping, and reported RoPE scaling alternatives such as Linear Interpolation, YaRN, and NTK-aware scaling (Tian et al., 26 Mar 2026). For attention structure, TSA outperforms Sliding Window and Selected Frame Attention on the reported balance of consistency and quality metrics (Tian et al., 26 Mar 2026). For system composition, the best result on Wan 4× is obtained by the full layer-adaptive FreeLOC configuration rather than by Direct+TSA, Direct+VRPR, or uniform VRPR+TSA (Tian et al., 26 Mar 2026).

The user study also favors FreeLOC, with reported scores of $d$4 for Content Consistency, $d$5 for Video Quality, and $d$6 for Video-Text Alignment (Tian et al., 26 Mar 2026). On efficiency, Wan 321-frame generation requires $d$7 s/step and $d$8 GB with FreeLOC, compared with $d$9 s/step and $i$0 GB for direct sampling, and $i$1 s/step and $i$2 GB for FreeLong (Tian et al., 26 Mar 2026). The one-time probing stage is reported at about 4 hours on an RTX 4090 for Wan, and the derived layer profile is highly stable, with Spearman correlation $i$3 when estimated from a single prompt (Tian et al., 26 Mar 2026).

6. Reproduction, limitations, and disambiguation

For reproduction, the paper emphasizes several implementation choices (Tian et al., 26 Mar 2026). VRPR is implemented by modifying query and key position indices before RoPE rather than editing a full relative-position matrix, which preserves compatibility with efficient attention kernels such as FlashAttention2. TSA requires construction of a four-dimensional mask with dense local attention, striped mid-range attention, and an always-visible first-frame sink. The layer profile is architecture-specific and must be precomputed once per backbone. The reported code is available at https://github.com/Westlake-AGI-Lab/FreeLOC (Tian et al., 26 Mar 2026).

The paper also delineates clear limitations. FreeLOC assumes a transformer-based video diffusion model with RoPE-style positional encoding; its layer profiles are model-specific; and its evidence is concentrated on 2× and 4× extension rather than arbitrary unbounded length extrapolation (Tian et al., 26 Mar 2026). The fixed first-frame attention sink is effective in the reported benchmarks, but the authors note that it may be suboptimal for videos with large scene changes, suggesting adaptive sinks as a plausible extension (Tian et al., 26 Mar 2026). A further practical implication is that FreeLOC introduces an offline characterization stage even though runtime sampling remains training-free.

A common source of confusion is nomenclature. Despite its name, FreeLOC is not a geolocation, location-proof, 3D occupancy, or LiDAR localization system. arXiv contains similarly named but unrelated systems including HLOC for IP geolocation (Scheitle et al., 2017), FedLoc for federated localization (Yin et al., 2020), ProLoc for region proofs in hindsight (Viti et al., 2024), VerLoc for decentralized geo-location verification (Kohls et al., 2021), LOcc for open-vocabulary occupancy prediction (Yu et al., 2024), LOC for open-set 3D occupancy prediction (Gao et al., 25 Oct 2025), and TreeLoc++ for forest LiDAR localization (Jung et al., 4 Mar 2026). FreeLOC, by contrast, belongs to the literature on long-video generation with pretrained video diffusion transformers (Tian et al., 26 Mar 2026).

In the design space of training-free long-video generation, FreeLOC is best understood as a three-part synthesis: a diagnosis that long-video extrapolation induces two separate O.O.D. failures, a pair of corrections specialized to those failures, and a layer-adaptive selection rule that determines where those corrections are beneficial. Its significance lies less in inventing a new generative architecture than in showing that long-video quality can be improved materially at inference time by restructuring positional encoding and attention only where the pretrained model is demonstrably sensitive (Tian et al., 26 Mar 2026).