---
title: 'AnyFlow: Continuous Mapping in Vision'
url: https://www.emergentmind.com/topics/anyflow
type: topic
---

# AnyFlow: Continuous Mapping in Vision

Searching arXiv for the two "AnyFlow" papers to ground the article in the cited literature.
“AnyFlow” denotes two distinct research systems that share a name but address different technical problems. In optical flow estimation, AnyFlow is a network that “for the first time, treats the flow field itself as a continuous function of image coordinates,” enabling accurate motion prediction at arbitrary spatial scales from low-resolution inputs [2303.16493]. In video generation, AnyFlow is “the first any-step video diffusion distillation framework based on flow maps,” designed to support arbitrary inference budgets while preserving the test-time scaling behavior of probability-flow ODE sampling [2605.13724]. The shared term therefore refers not to a single unified framework, but to two separate lines of work centered on continuity across scales or time intervals.

## 1. Name and scope

The 2023 AnyFlow paper addresses optical flow under input resizing, especially the degradation of small-object and boundary accuracy when images are downsized for efficiency [2303.16493]. Its central claim is that optical flow can be represented as a continuous coordinate-based function and queried at arbitrary output resolutions. The method is built on a RAFT-style iterative refinement backbone augmented with an implicit neural representation, multi-scale feature warping, and dynamic correlation lookup.

The 2026 AnyFlow paper addresses few-step and any-step video diffusion distillation [2605.13724]. Its starting point is the observation that consistency-distilled video models often degrade when more sampling steps are used at test time. AnyFlow replaces endpoint-only consistency with flow-map transition learning over arbitrary time intervals, together with on-policy flow map distillation.

This suggests that the common semantic thread behind the two systems is not task overlap but a methodological emphasis on continuous mappings: spatial continuity in the optical-flow variant and temporal-transition continuity in the diffusion variant.

## 2. AnyFlow for arbitrary-scale optical flow

In the optical-flow formulation, the flow field is modeled as a continuous function
$$
f^s = \mathbb{I}(I_1, I_2), \qquad f^s \in \mathbb{R}^{sH\times sW\times 2},
$$
where $s$ is any positive scale factor, with $s<1$ for downsize and $s>1$ for upsample [2303.16493]. The method takes a low-resolution image pair as input and can directly produce flow at the original resolution, a down-sampled resolution, or a super-resolved resolution without relying on naive interpolation.

After the encoder and GRU update inherited from RAFT produce a hidden state $M\in\mathbb{R}^{H/8\times W/8\times C}$ and an accumulated low-resolution flow $f_i\in\mathbb{R}^{H/8\times W/8\times 2}$, AnyFlow upsamples $f_i$ to an arbitrary output resolution via a small MLP $f_\theta$ [2303.16493]. For a continuous query coordinate $x_q=(u,v)$ in $[0,H/8]\times[0,W/8]$, the network locates its nine nearest integer neighbors in the coarse flow grid, identifies the nearest feature $z^\*$ at integer coordinate $v^\*$, and feeds the tuple
$$
(z^\*, \Delta x = x_q - v^\*, \psi(\Delta x))
$$
into the MLP, where $\psi(\cdot)$ is the positional encoding from LIIF. The output
$$
\mathcal{O}(x_q) = f_\theta(z^\*, \Delta x, \psi(\Delta x)) \in \mathbb{R}^{(3\times 3)\cdot n^2}
$$
is interpreted as convex weights on the $3\times 3$ local neighbors of $f_i$ together with an arrangement that simultaneously produces an $n\times n$ patch of higher-resolution flow [2303.16493]. By sampling $\{H_o/n\}\times\{W_o/n\}$ query points and assembling patches, the method reconstructs $f_i$ at resolution $H_o\times W_o$ in one shot.

A key consequence is that $H_o$ and $W_o$ may be chosen arbitrarily, including non-integer scalings, without changing the model [2303.16493]. The paper presents this as a departure from fixed-grid optical flow prediction.

## 3. Optical-flow architecture, training, and inference

AnyFlow extends RAFT’s iterative design with multi-scale feature warping [2303.16493]. Whereas RAFT warps only at $1/8$ scale, AnyFlow also extracts features $F^{1/2}$ and $F^{1/4}$, in addition to $F^{1/8}$. After each GRU iteration $i$, the current flow estimate is upsampled via the implicit upsampler to obtain $f_i^{1/2}$ and $f_i^{1/4}$, and second-frame features are warped:
$$
W(F_2^s, f_i^s).
$$
These warped features are concatenated with first-frame features, processed by a $1\times1$ convolution, and then PixelShuffle is used to bring them back to the $1/8$ grid:
$$
F^{1/2'} = PS( Conv_{1\times1}([F_1^{1/2}, W(F_2^{1/2}, f_i^{1/2})]) )
$$
$$
F^{1/4'} = PS( Conv_{1\times1}([F_1^{1/4}, W(F_2^{1/4}, f_i^{1/4})]) )
$$
$$
\Delta f_{i+1} = Conv([ M, Conv_{1\times1}([F^{1/2'},F^{1/4'}]) ]).
$$
According to the paper, this injects high-frequency spatial cues into the update and improves boundary localization and tiny-object capture [2303.16493].

The method also replaces RAFT’s fixed-radius correlation lookup with a dynamic lookup strategy [2303.16493]. At each iteration and for each pixel, a residual radius $\Delta r_i$ is predicted and accumulated as $r_i=\Delta r_i+r_{i-1}$. The local search window therefore grows or shrinks per pixel depending on motion magnitude, while the number of sampled points remains $(2r_i+1)^2$. To address blind spots when $r_i$ becomes large, the model defines nine auxiliary sub-pixel offsets around each sample location, collects their correlation values, and feeds them into a small MLP $g_\phi$ together with $r_i$. The paper terms this “region encoding.”

Training follows the original RAFT loss:
$$
L = \sum_{i=1}^N \gamma^{N-i} \|f_{gt} - f_i^1\|_1.
$$
In all experiments, $N=12$ updates during training and $\gamma=0.8$, and no additional photometric or smoothness terms are used [2303.16493]. To encourage robustness to arbitrary scales, with probability $p$ the input image pair is downsampled by an independently and uniformly chosen scale $s\in(min_s,max_s)$, typically down to 50%–90% of the original, while the model is required to recover the flow at the original resolution.

At inference, the procedure is explicit: optionally downsample the original frames; encode multi-scale features; build $1/8$-scale all-pairs correlations; initialize $f_0=0$ and $r_0=4$; iterate GRU updates and dynamic correlation sampling; upsample each $f_i$ to arbitrary scale $s'$ via $f_\theta$; and return the final upsampled flow [2303.16493]. The implicit upsampler $f_\theta$ and region-encoding module $g_\phi$ add approximately $0.1$ M parameters beyond RAFT’s $5.3$ M, and the runtime overhead is described as small.

## 4. Empirical profile of the optical-flow model

The optical-flow AnyFlow is reported to establish “a new state-of-the-art performance of cross-dataset generalization on the KITTI dataset,” while achieving comparable benchmark accuracy to other SOTA methods [2303.16493]. In cross-dataset generalization experiments trained on FlyingChairs+Things only, RAFT achieves EPE $=1.43$ on Sintel-clean, while AnyFlow (dynamic) reduces this to $1.17$, and region encoding further reduces it to $1.10$ [2303.16493]. On KITTI-train, RAFT has F1-epe $=5.04$ and F1-all $=17.4\%$; AnyFlow (dynamic) achieves $3.95$ EPE and $13.01\%$ outliers, and region encoding yields $3.76/12.44$.

After fine-tuning, the paper reports that on the public Sintel test set AnyFlow (dynamic) scores $1.23$ EPE on “clean” and $2.44$ on “final,” compared with RAFT’s $1.61$ and $2.86$ respectively [2303.16493]. On KITTI-test, AnyFlow (dynamic) records $4.41\%$ F1-all versus $5.10$ for RAFT. The paper characterizes this as second best overall and top among non-Transformer, non-ImageNet-pretrained methods.

A major emphasis is robustness to input downsampling [2303.16493]. When methods are fed inputs downsampled by 50%–90% and evaluated after re-upsampling flow to original size, RAFT’s Sintel-clean EPE rises from $1.47$ at 100% to $1.92$ at 50%, while AnyFlow rises from $1.10$ to $1.41$. On KITTI, RAFT rises from $5.04$ to $7.50$, whereas AnyFlow rises from $3.76$ to $4.88$. The paper interprets this stability as evidence for the advantage of continuous flow representation.

Qualitatively, the reported comparisons indicate that when inputs are halved or lower, RAFT, GMA, and GMFlow lose fine boundaries around wheels, limbs, or thin rods, and small motions vanish entirely, whereas AnyFlow retains crisp object outlines and captures tiny displacements [2303.16493]. A plausible implication is that the model is especially relevant in resource-constrained settings where aggressive input resizing is operationally necessary.

## 5. AnyFlow for any-step video diffusion

In the 2026 work, AnyFlow denotes a video diffusion distillation framework based on flow maps rather than endpoint consistency [2605.13724]. A flow map model parameterized by $\theta$ implements
$$
\Phi_{t\to r}(z_t)\approx z_r,\qquad 0\le r\le t\le T,
$$
with $\Phi_{t\to r}(z_t)=f_\theta(z_t,t,r)$. Endpoint consistency, written as $z_t\mapsto z_0$, is treated as the special case $r=0$.

The underlying teacher model is described by the probability-flow ODE
$$
\frac{dz_t}{dt}
=
f_{\mathrm{ODE}}(z_t,t)
=
-\tfrac12\,\beta(t)\,z_t
-\beta(t)\,s_\phi(z_t,t),
$$
where $s_\phi$ is the teacher score network and $\beta(t)$ is the noise schedule [2605.13724]. The distilled model is trained to approximate the exact ODE solution mapping
$$
\Phi^{\mathrm{ODE}}_{t\to r}(z_t)
=
z_t + \int_t^r f_{\mathrm{ODE}}(z_u,u)\,du,
$$
so that $\Phi^{\mathrm{ODE}}_{T\to 0}(z_T)=z_0$.

The paper’s central claim is that distilling full ODE transitions rather than only endpoints preserves the desirable property that increasing the number of solver steps at test time monotonically improves approximation of the ODE trajectory [2605.13724]. Test-time sampling can therefore use an Euler scheme with $N$ steps:
$$
z_{t_{i-1}} = f_\theta(z_{t_i}, t_i, t_{i-1}),
\qquad
t_i = T\frac{i}{N},\; i=N,\dots,0.
$$
The framework is presented as the first any-step video diffusion distillation method built around this two-time transition formulation.

## 6. Distillation, backward simulation, and empirical results in video generation

After an initial forward flow-map training stage, the diffusion AnyFlow refines $\theta$ via an on-policy Distribution-Matching Distillation loss [2605.13724]. For a sampled interval $[t,r]$ derived from a target budget $N$, the model produces a three-segment rollout:
$$
z_t = \Phi^\theta_{T\to t}(z_T),\qquad
z_r = \Phi^\theta_{t\to r}(z_t),\qquad
\hat z_0 = \Phi^\theta_{r\to 0}(z_r).
$$
It then re-noises $\hat z_0$ back to $z_T$ via the known forward process and minimizes the KL divergence between teacher and student predicted distributions at $\hat z_0$:
$$
\mathcal L_{\mathrm{DMD}}
=
\mathbb E_{N\sim\mathcal U,\;t,r,\;z_T,c}
\Big[
\operatorname{KL}\big(q^T(z_r\mid z_0)\parallel q^\theta(z_r\mid \hat z_0)\big)
\Big].
$$
The expectation is over the target step budget $N\in\{1,\dots,T\}$, the sampled split, initial noise, and prompt context [2605.13724].

The paper further introduces Flow Map Backward Simulation, which decomposes a full Euler rollout into shortcut flow-map transitions [2605.13724]. In practice, shortcut calls $f_\theta(\cdot,t,r)$ replace long multistep Euler rollouts and thereby reduce training cost. The method is motivated by two failure modes: discretization error in few-step sampling and exposure bias in causal generation. By training over all intervals $[t,r]$ and using on-policy rollouts, AnyFlow is reported to correct coarse-step ODE errors and reduce drift in causal models.

The implementation spans both bidirectional and causal architectures [2605.13724]. The bidirectional variant uses a standard UNet backbone, Wan2.1, distilled via the AnyFlow pipeline. The causal variant uses FAR context compression with three “full-token” chunks at patch size 2, the remainder at patch size 4, and a first chunk of size 1 for precise first-frame conditioning. KV caches are reused between flow-map calls to speed up backward simulation. Reported parameter scales are 1.3 B for Wan2.1-1.3B and 14 B for Wan2.1-14B.

The paper also specifies forward training tricks and hyperparameters [2605.13724]. These include interpolated timestep conditioning with $g\cdot \mathrm{emb}(t)+(1-g)\cdot \mathrm{emb}(r)$ and $g=0.25$, guidance-fused training, adaptive loss reweighting anchored at boundary cases $t=r$, and a time sampler uniform over $[0,T]$ with an optional timestep reweight function $w(t)$. Typical hyperparameters are AdamW, learning rate $1e\!-\!5$ in stage 1 and $5e\!-\!6$ in stage 2, weight decay $0.01$, batch size $128$ GPUs times $8$ samples each, stage 1 of approximately $200$ K iterations, stage 2 of approximately $100$ K iterations, and guidance scale matched to the pretrained teacher.

Selected VBench totals are reported for the 14 B models [2605.13724]. For bidirectional text-to-video, rCM at 4 NFE scores $83.73$, AnyFlow at 4 NFE scores $84.04$, and AnyFlow at 32 NFE scores $84.10$. For causal text-to-video, Krea-Realtime at 4 NFE scores $83.25$, AnyFlow-FAR at 4 NFE scores $84.05$, and AnyFlow-FAR at 32 NFE scores $84.41$. For image-to-video, Wan2.1-I2V at 100 NFE scores $87.71$, while AnyFlow-FAR at 4 NFE scores $87.87$. The paper interprets these results as evidence that AnyFlow matches or surpasses consistency-based counterparts in the few-step regime while scaling with sampling step budgets.

## 7. Relationship, distinctions, and limitations

The two AnyFlow systems are unrelated at the task level. One is an optical-flow estimator for dense motion between image pairs; the other is a video diffusion distillation framework for generative sampling [2303.16493] [2605.13724]. The shared name can therefore be misleading if treated as designating a single method family.

Their methodological parallel lies in representing outputs through continuous mappings rather than fixed discrete endpoints. In the optical-flow case, the continuity is spatial and coordinate-based, allowing arbitrary-scale querying of flow fields [2303.16493]. In the diffusion case, the continuity is temporal and interval-based, allowing arbitrary transition learning between times $t$ and $r$ and supporting any-step inference [2605.13724]. This suggests that “AnyFlow” functions as a label for models that seek robustness under variable resolution or variable solver budget by learning over continuous domains.

The limitations discussed in the two papers are likewise domain-specific. For optical flow, the iterative GRU backbone remains the dominant compute cost; the region encoding and dynamic-radius MLPs introduce hyperparameters such as the initial $r_0$ and auxiliary patch size; and the authors identify unsupervised or self-supervised photometric losses and integration with Transformer-style global matching as future directions [2303.16493]. For video diffusion, the paper frames its contribution against the limitations of consistency distillation and positions flow-map learning as a remedy for degraded performance at higher sampling budgets, but its concrete claims are primarily about on-policy distillation, ODE-consistent scaling, and broad applicability across bidirectional and causal architectures rather than about unresolved weaknesses [2605.13724].

Taken together, the term “AnyFlow” currently refers to two separate contributions that use continuity as a design principle to overcome rigidity in standard formulations: fixed-grid prediction in optical flow, and fixed-step or endpoint-only distillation in video diffusion.

Source: https://www.emergentmind.com/topics/anyflow