---
title: 'Blur2Vid: Recovering Video from Blurred Images'
url: https://www.emergentmind.com/topics/blur2vid
type: topic
---

# Blur2Vid: Recovering Video from Blurred Images

Blur2Vid denotes the problem of recovering a temporally ordered sharp video from a single motion-blurred image. In its basic form, the observation is modeled as the temporal average of latent sharp frames during exposure, so the task is to invert an averaging process that destroys both high-frequency detail and temporal order. Early formulations treated the blurred image as a compressed record of a short latent sequence, while later work reframed the task as motion-latent decoding or conditional video generation with large-scale video priors [1804.04065; 2201.12010; 2512.19817].

## 1. Problem definition and image formation

The canonical Blur2Vid model writes a motion-blurred image as a temporal integration or average of latent sharp frames. One formulation is
\[
y = g\left(\frac{1}{\tau}\int_0^\tau \tilde x(t)\,dt\right)
= g\left(\frac{1}{T}\sum_{i=0}^{T-1} x[i]\right),
\]
where \(y\) is the observed blurred image, \(g(\cdot)\) is the camera response function, and \(x[i]\) are latent frames over discretized exposure bins [1804.04065]. A later formulation states the same principle as
\[
x_B \approx \frac{1}{N}\sum_{n=1}^{N} x_n,
\]
with \(x_B\) the blurred image and \(\{x_n\}_{n=1}^N\) the latent sharp frames [2201.12010]. In the 2025 generative formulation, blur is written as
\[
I = g\left(\int_{t\in \mathcal{T}} E(t)\,dt\right),
\]
emphasizing that a video is a physically faithful explanation of the measurement [2512.19817].

Two ambiguities define the field. First, the inversion is a blind deconvolution problem, so many sharp sequences can average to the same blurred observation. Second, averaging destroys temporal ordering: a forward sequence and its reversal can produce the same blur. This makes Blur2Vid distinct from conventional deblurring, which seeks one sharp image, and from standard video restoration, which already has multiple time-indexed observations [1804.04065].

A recurrent theme across the literature is that motion blur is not treated merely as corruption. It is treated as a compressed temporal signal encoding scene motion, camera motion, occlusion patterns, and sometimes depth variation. This suggests that Blur2Vid is fundamentally a spatio-temporal inverse problem rather than a single-frame enhancement problem [2201.12010; 2512.19817].

## 2. Early single-image sequence extraction

The earliest deep formulations reconstructed the latent sequence from one blurred image by exploiting symmetry around a middle frame. In "Learning to Extract a Video Sequence from a Single Motion-Blurred Image" [1804.04065], the main experiments used \(T=7\) latent frames, with one network \(\phi_4\) for the middle frame and separate networks for symmetric pairs \((x[3],x[5])\), \((x[2],x[6])\), and \((x[1],x[7])\). The middle frame was supervised directly using
\[
{\cal L}_\text{middle} = |\phi_4(y)-x[4]|^2 + {\cal L}_\text{perceptual}(\phi_4(y),x[4]),
\]
while non-middle frames used ordering-invariant losses over symmetric pairs.

The critical technical device was the pairwise ordering-invariant loss
\[
{\cal L}_\text{pair}= \sum_{i=1}^3 \Big| |\hat{x}[i] + \hat{x}[8-i]| - |x[i] + x[8-i]| \Big|_1 + \Big| |\hat{x}[i] - \hat{x}[8-i]| - |x[i] - x[8-i]| \Big|_1,
\]
which compares sums and absolute differences of symmetric frame pairs and therefore does not force a unique temporal direction [1804.04065]. The same work reported that a global ordering-invariant loss based only on frame sums was too weak, especially when the blur averages more than 3 frames.

Architecturally, this line of work used a large receptive field, resampling convolution with factor 4, residual blocks, and dilated convolutions to address the ill-posedness of deblurring from a single image. The middle-frame estimator was also evaluated as a practical deblurring network. On a test set matched to its training blur, it reported **32.20 dB** PSNR versus **30.52 dB** for Nah et al.; on its own test set, **29.02 dB** versus **28.19 dB**; and on Nah’s original harder test images, **26.98 dB** versus **28.48 dB** [1804.04065]. These numbers already exposed a persistent property of Blur2Vid systems: performance depends strongly on the blur regime represented during training.

## 3. Motion-latent recurrent formulations

A more structured formulation was introduced in "Unfolding a blurred image" [2201.12010]. It proposed a two-stage, fully convolutional, end-to-end differentiable framework: first learn motion representation from sharp videos using a self-supervised video autoencoder, then transfer that representation to blurred images by training a Blurred Image Encoder (BIE) under guidance from the learned decoder.

The video autoencoder consists of a Recurrent Video Encoder (RVE) and Recurrent Video Decoder (RVD). For input frames \(x_1,\dots,x_N\), the encoder recurrence is
\[
h^{enc}_n = enc(h^{enc}_{n-1}, x_n),
\]
and the final hidden state \(h^{enc}_N\) becomes the learned motion representation [2201.12010]. The encoder uses 4 convolutional blocks with \(3\times 3\) filters, feature map sizes \(16, 32, 64, 128\), and a \(3\times 3\) ConvLSTM kernel. The decoder is initialized by
\[
h^{dec}_0 = h^{enc}_N
\]
and recurrently predicts multi-scale optical flow maps \(f_{n,1}, f_{n,2}, f_{n,3}, f_{n,4}\), which warp a central frame through a differentiable transformer:
\[
\hat{x}_n = T(x_{\lfloor N/2 \rfloor}, f_n).
\]

This formulation does not generate pixels directly. It decodes motion and applies that motion to a sharp reference frame. The surrogate reconstruction loss for the video autoencoder is
\[
\mathcal{L}_{rec} = \sum_{n=1}^{N} \left\| \hat{x}_n - x_n \right\|.
\]
At test time, a separate Deblurring Module (DM) first estimates a sharp center frame,
\[
\hat{x}_{center} = DM(x_B),
\]
then BIE extracts a motion embedding \(z = BIE(x_B)\), and the RVD decodes the video
\[
\hat{x}_{1..N} = RVD(z, \hat{x}_{center}).
\]

This framework separates content restoration, motion extraction, and temporal generation. The DM is an encoder-decoder with Residual Dense Blocks in the encoder, bottleneck blocks in the decoder, skip connections at 3 scales, and a U-Net-like structure [2201.12010]. The BIE–RVD pair is trained with a reconstruction term plus a motion smoothness regularizer,
\[
\min \mathcal{L}_{rec}(RVD(BIE(x_B)), x_{1..N}) + \lambda \mathcal{L}_{smooth}.
\]

A central issue is temporal ambiguity. The paper states that blur is temporally ambiguous and directional ambiguity remains, especially with multiple moving objects. To address this, the BIE–RVD pair is trained with an ordering-invariant frame reconstruction loss and a spatial motion smoothness loss, while staged training is preferred over direct joint training because otherwise the model must learn motion extraction from blur and frame generation from latent motion simultaneously [2201.12010].

Quantitatively, using the ambiguity-invariant loss, the method reports **44.12** versus **49.06** for Jin et al. in the **7-frame blur setting**, and **48.24** in the **9-frame blur setting**, where Jin et al. is not applicable due to rigid 7-frame design [2201.12010]. It also reports a more compact model, **34 MB** versus **70 MB**, deblurring runtime **0.02 s** versus **0.45 s**, and video generation runtime **0.39 s** versus **1.10 s** [2201.12010].

| Representative work | Core mechanism | Distinctive property |
|---|---|---|
| "Learning to Extract a Video Sequence from a Single Motion-Blurred Image" [1804.04065] | Sequential pairwise reconstruction with ordering-invariant losses | Recovers a **7-frame** sequence around a middle frame |
| "Unfolding a blurred image" [2201.12010] | RVE–RVD motion-latent framework with DM and BIE | Recurrent flow-based decoding from a single blurred image |
| "HyperCUT" [2304.01686] | Hyperplane-based self-supervised ordering regularization | Assigns an explicit temporal order label |
| "Generating the Past, Present and Future from a Motion-Blurred Image" [2512.19817] | Fine-tuned latent video diffusion with exposure-interval encoding | Supports present reconstruction and past/future generation |

## 4. Ordering ambiguity and explicit ordering supervision

The forward/backward ambiguity was made the primary target of optimization in "HyperCUT: Video Sequence from a Single Blurry Image using Unsupervised Ordering" [2304.01686]. Instead of relying only on order-invariant losses, HyperCUT learns an explicit order label in a latent space. It defines a mapping
\[
\mathcal{H}: \mathbb{R}^{2 \times H \times W \times C} \rightarrow \mathbb{R}^{d}
\]
such that a frame pair and its reversal lie on opposite sides of a hyperplane \(h\):
\[
\left< \mathcal{H}\left([x^i_k, x^i_{N - k}]\right), h\right>
\left<\mathcal{H}\left([x^i_{N - k}, x^i_k]\right), h\right> < 0.
\]

The associated separation loss is
\[
\mathcal{L}_{h} = \frac{1}{M} \sum_{i=1}^{M} softplus\!\left( \left< \mathcal{H}\left([x^i_k, x^i_{N- k}]\right), h \right> \times \left<\mathcal{H}\left([x^i_{N - k}, x_k^i]\right), h\right> \right),
\]
and the learned ordering network is then frozen and used as a regularizer for a Blur2Vid backbone:
\[
\mathcal{L}(f) = \frac{1}{M}\sum_{i=1}^{M}\mathcal{L}_{D}(f(y^i)) + \alpha\mathcal{R}_{hyp}(f).
\]
This changes the training problem from “accept both orders” to “choose one consistent order.”

HyperCUT is explicitly a plug-in regularizer. The base loss \(\mathcal{L}_D\) may be either a standard \(L_2\) loss or the order-invariant loss from Jin et al. [2304.01686]. The method was evaluated by measuring whether the hyperplane separates forward and reversed frame pairs correctly. Reported ordering results include **REDS**: hit **95.7**, con@2 **96.5**, con@3 **94.4**; **B-Aist++**: hit **97.5**, con@2 **95.6**, con@3 **91.2**; **RB2V-Street**: hit **98.7**, con@2 **98.3**, con@3 **96.8** [2304.01686].

The same work also introduced the **Real Blur2Vid (RB2V)** dataset, collected with a beam splitter camera system using two GoPro Hero-8 cameras, one at **25 fps** and one at **100 fps** [2304.01686]. RB2V contains three subsets: RB2V-Street, RB2V-Face, and RB2V-Hand, with splits of **9000 / 2053**, **8000 / 2157**, and **12000 / 4722** for training/testing respectively. This is significant because earlier Blur2Vid work depended primarily on synthetic blur generation.

A plausible implication is that explicit ordering supervision reduces one of the main optimization pathologies of earlier methods: regression toward symmetric averages or temporally inconsistent pair selection. The reported improvements on border frames and trajectory recovery support that interpretation, but they do not eliminate the underlying physical ambiguity of the inverse problem [2304.01686].

## 5. Diffusion-based Blur2Vid and past/present/future generation

The 2025 paper "Generating the Past, Present and Future from a Motion-Blurred Image" recasts Blur2Vid as conditional video generation with large-scale generative priors [2512.19817]. Its central claim is that a single motion-blurred image is better treated as a partially observed video than as a corrupted still image. The model fine-tunes **CogVideoX-2B**, a **2B-parameter latent video diffusion transformer**, and learns the conditional distribution
\[
P(V \mid I,\; t_1,\ldots,t_F),
\]
where \(I\) is the blurred image and \(V=[I_{t_1},\ldots,I_{t_F}]\) is the generated video.

The main novel conditioning signal is exposure-interval encoding. Each scalar time value is encoded as
\[
\gamma(t)=\left[\cos(2\pi \nu_1 t),\sin(2\pi \nu_1 t),\ldots,\cos(2\pi \nu_N t),\sin(2\pi \nu_N t)\right],
\]
and latent-frame exposure intervals are concatenated and linearly projected:
\[
\mathrm{Linear}\circ \mathrm{Concat}\left(\gamma(t_i[1]),\ldots,\gamma(t_i[8])\right)\in\mathbb{R}^{\tilde D}.
\]
The blurred image’s own exposure interval is normalized to \([-0.5,0.5]\) and encoded in the same way [2512.19817].

Training minimizes the expected \(\ell_2\) error between the predicted denoised latent video \(\hat V\) and the clean latent video \(V\),
\[
\mathbb{E}\,\|\hat V - V\|_2,
\]
while fine-tuning the entire diffusion transformer plus the exposure-interval projection layer. The model supports two modes. In **present-only** mode, output frames lie within the original exposure interval. In **past/present/future** mode, frames can extend before or after that interval, so the method generates plausible temporal continuations beyond the observation [2512.19817].

Training data are upsampled to **1920 FPS** via frame interpolation before blur synthesis, specifically to handle long exposures and dead time between frames. Inference uses **50 diffusion steps** with a **DDPM solver** and classifier-free guidance scale **1.1** [2512.19817]. Evaluation adopts bidirectional patch-based metrics that account for time-reversal ambiguity, including bidirectional patch PSNR, SSIM, LPIPS, FVD, EPE, and motion-blur consistency.

On **GoPro**, the model reports **30.01 PSNR\(_p\)**, **0.9359 SSIM\(_p\)**, **0.010 LPIPS\(_p\)**, **FVD 21.46**, and **EPE 0.39**, compared with MotionETR’s **26.54 / 0.8825 / 0.015 / 94.90 / 1.46** and Jin et al.’s **25.23 / 0.8190 / 0.084 / 235.53 / 3.38** [2512.19817]. On **B-AIST++**, it reports **27.37 PSNR\(_p\)**, **0.9306 SSIM\(_p\)**, **0.027 LPIPS\(_p\)**, **FVD 37.16**, and **EPE 1.78**, outperforming Animation from Blur’s **26.69 / 0.9209 / 0.042 / 138.27 / 2.65**. For motion-blur consistency on GoPro, the average of generated frames reaches **35.47 dB**, versus **33.64** for Jin et al. and **32.17** for MotionETR.

This generative formulation also changes the interpretation of output. The paper is explicit that the generated future or past is not guaranteed to be what actually occurred; it is a plausible temporal explanation constrained by the blur [2512.19817]. That statement addresses a common misconception: stronger video priors improve realism and coherence, but they do not remove the non-identifiability of the original physical inverse problem.

## 6. Relation to video deblurring, applications, and limitations

Blur2Vid is adjacent to, but not identical with, video deblurring. Methods such as VDTR, BSSTNet, MB2D, DAVID, and motion-magnitude-guided RNNs assume access to multiple blurry frames and restore a sharp center frame or a restored video sequence [2204.08023; 2406.07551; 2012.12507; 1912.03445; 2207.13374]. Blur2Vid instead asks whether a single blurred observation can be expanded back into a temporally ordered video. This makes it more ill-posed but also more closely tied to exposure-time image formation.

Several applications follow directly from the generated sequences. The 2025 diffusion model reports downstream use for recovering camera trajectories, object motion, and dynamic 3D scene structure, and shows that generated videos can be fed into MegaSaM or head-tracking pipelines [2512.19817]. HyperCUT reports improvements on face trajectory recovery from **5.75** to **4.87** and hand trajectory recovery from **11.67** to **9.2** when added to a Blur2Vid backbone [2304.01686]. Earlier recurrent frameworks emphasized plausible temporally consistent sharp frames on planar scenes, depth variation, moving foreground objects, and real motion-blurred images [2201.12010].

The field also has stable failure modes. Early sequential models are sensitive to middle-frame quality and degrade on very large blur [1804.04065]. Recurrent motion-latent methods assume that a central sharp frame is recoverable or approximable and remain ambiguous under complex multi-object motion [2201.12010]. HyperCUT focuses on standard motion blur and may not directly handle complex nonrigid motion or long-exposure blur [2304.01686]. The diffusion formulation assumes blur from a single contiguous exposure interval and struggles on composited images, timelapse-like images, extremely long exposures, and cases combining strong camera panning with complex scene motion [2512.19817].

Taken together, these results suggest a clear technical trajectory. Early Blur2Vid systems imposed structure through symmetry and ordering-invariant losses; recurrent formulations learned a motion latent space and decoded flow-conditioned videos; explicit ordering methods replaced implicit ambiguity handling with learned temporal direction; and generative video priors extended the task from reconstructing exposure-time video to generating the past, present, and future. Across these variants, the central principle remains unchanged: a motion-blurred image is treated as a temporally integrated observation whose inversion requires simultaneous reasoning about content, motion, and temporal ordering [1804.04065; 2201.12010; 2304.01686; 2512.19817].

Source: https://www.emergentmind.com/topics/blur2vid