FractalPINN-Flow: Unsupervised Optical Flow

• The paper introduces a fractal-inspired multiscale feature extractor paired with a variational brightness constancy objective to estimate dense, smooth optical flow from two grayscale frames.
• It leverages a combination of L1/L2 data terms and anisotropic TV regularization to robustly enforce brightness constancy while preserving motion boundaries.
• The architecture employs a Fractal Deformation Network with recursive encoder-decoder design and skip fusion to capture multi-scale motion details in limited-annotation scenarios.

FractalPINN-Flow is an unsupervised dense optical flow method that combines a fractal-inspired multiscale feature extractor with a classical variational optical-flow objective. It learns directly from two consecutive grayscale frames without ground-truth motion labels and centers on a Fractal Deformation Network (FDN) coupled to a total-variation-regularized energy functional that enforces brightness constancy while promoting smooth, edge-preserving flow fields. The method is presented as especially relevant for high-resolution data and limited-annotation settings, and its use of the term “fractal” refers to recursive multiscale nesting rather than strict mathematical self-similarity (Behnamian et al., 10 Sep 2025).

1. Problem formulation and conceptual scope

FractalPINN-Flow addresses optical flow as an underdetermined inverse problem. For each pixel, one brightness-constancy equation constrains two unknown flow components, $u$ and $v$, so regularization is required. The method is motivated simultaneously by this ill-posedness and by the practical difficulty of obtaining large annotated optical-flow datasets for supervised CNN-based systems. Its stated aim is to learn from raw consecutive grayscale images, avoid flow labels entirely, preserve sharp motion boundaries, and behave well in limited-data, high-resolution, and low-annotation regimes (Behnamian et al., 10 Sep 2025).

The paper’s basic observation model is the standard brightness-constancy relation

$$I_1(x) = I_2(x + w(x)) \quad \text{for all } x \in \Omega,$$

with $w(x)=(u(x),v(x))$. The target is therefore not a physical velocity field governed by Navier–Stokes equations, but an image-plane motion field inferred from photometric consistency across two frames.

This distinction is important for classification. FractalPINN-Flow is a flow-estimation method in the computer-vision sense of “flow,” not a solver for incompressible fluid transport. This suggests that the method occupies a hybrid position: its training objective is variational and unsupervised, but its inductive structure is supplied by a deep multiscale architecture rather than by the strong-form PDE residuals that define many contemporary PINN frameworks.

2. Variational objective and total variation regularization

The core loss is derived from a first-order linearization of brightness constancy,

$$I_2(x + w(x)) \approx I_2(x) + \nabla I_2(x)\cdot w(x),$$

which yields the residual

$\nabla I_2(x)\cdot w(x) + I_2(x) - I_1(x) = 0.$

This residual is used as the data-fidelity term in an unsupervised objective (Behnamian et al., 10 Sep 2025).

The full energy functional is

$$E_{TV}(w) := \lambda_1 \|\nabla I_2 \cdot w + I_2 - I_1\|_1 +\lambda_2 \|\nabla I_2 \cdot w + I_2 - I_1\|_2^2 +\lambda_{TV} \|w\|_1,$$

with $\lambda_1,\lambda_2,\lambda_{TV}\ge 0$. The $L^1$ term is described as more robust to outliers and non-Gaussian noise, while the $L^2$ term more strongly enforces brightness consistency where the linear model is reliable. The TV term promotes piecewise smoothness while allowing discontinuities at motion boundaries.

The paper explicitly defines the anisotropic TV seminorm as

$v$0

with $v$1 implemented as finite differences. The use of finite differences is emphasized because pointwise gradients can miss discontinuities. The reported qualitative behavior follows the standard TV tradeoff: without TV, the flow may be noisy or unstable; with moderate TV, it becomes smoother and more coherent while retaining edges; with too much TV, fine motion structures can be oversmoothed (Behnamian et al., 10 Sep 2025).

A notable feature of the formulation is the simultaneous use of $v$2 and $v$3 data terms. This gives the objective a mixed robust-and-quadratic character rather than committing exclusively to one photometric penalty.

3. Fractal Deformation Network architecture

FractalPINN-Flow uses a Fractal Deformation Network and a separate flow regression head rather than a single monolithic end-to-end CNN. The processing pipeline is

$v$4

The input consists of two normalized grayscale frames concatenated channel-wise,

$v$5

The FDN is described as a symmetric U-Net-style encoder-decoder with depth $v$6, but its main architectural distinction is recursive multiscale nesting, self-similar block reuse across scales, and addition-based skip fusion rather than concatenation (Behnamian et al., 10 Sep 2025).

Component	Specification	Function
Input	Two grayscale frames, channel-wise concatenated	Frame-pair representation
FDN encoder	$v$7	Multiscale feature extraction
FDN decoder	$v$8	Reconstruction to full resolution
Skip fusion	Bilinear interpolation + element-wise addition	Compact cross-scale fusion
FDN output	$v$9	Full-resolution feature map
Projection	$$I_1(x) = I_2(x + w(x)) \quad \text{for all } x \in \Omega,$$0 convolution, $$I_1(x) = I_2(x + w(x)) \quad \text{for all } x \in \Omega,$$1	Channel remapping
Flow head	$$I_1(x) = I_2(x + w(x)) \quad \text{for all } x \in \Omega,$$2	Dense flow regression

Each downsampling block in the encoder uses two $$I_1(x) = I_2(x + w(x)) \quad \text{for all } x \in \Omega,$$3 convolutions, batch normalization, ReLU, and $$I_1(x) = I_2(x + w(x)) \quad \text{for all } x \in \Omega,$$4 max pooling. The decoder mirrors this with $$I_1(x) = I_2(x + w(x)) \quad \text{for all } x \in \Omega,$$5 transposed convolutions, bilinear interpolation for resolution matching, element-wise skip addition, and two $$I_1(x) = I_2(x + w(x)) \quad \text{for all } x \in \Omega,$$6 convolutions with BN and ReLU in each decoder block. The final output is a dense flow field

$$I_1(x) = I_2(x + w(x)) \quad \text{for all } x \in \Omega,$$7

The paper repeatedly frames the model as “fractal-inspired” because of repeated encoder-decoder processing across multiple scales. It also explicitly states that the term refers to repeated processing across scales rather than strict mathematical self-similarity. This makes the designation architectural rather than geometric or operator-theoretic.

4. Training protocol, datasets, and empirical behavior

Training is entirely unsupervised. No ground-truth flow is used; the model is optimized only through the variational loss $$I_1(x) = I_2(x + w(x)) \quad \text{for all } x \in \Omega,$$8. The fixed loss weights are

$$I_1(x) = I_2(x + w(x)) \quad \text{for all } x \in \Omega,$$9

while $w(x)=(u(x),v(x))$0 is tuned. Optimization uses Adam with learning rate $w(x)=(u(x),v(x))$1, batch size $w(x)=(u(x),v(x))$2, and a fixed number of epochs as the stopping criterion. The best checkpoint is selected by lowest training loss. Implementation is in PyTorch with deterministic settings, CUDA acceleration on NVIDIA GPUs, structured logging, JSON configurations, and explicit CUDA cache clearing and garbage collection after each epoch for memory management (Behnamian et al., 10 Sep 2025).

Two evaluation regimes are reported.

Dataset	Protocol	Reported outcome
Synthetic Shepp-Logan phantom	$w(x)=(u(x),v(x))$3, 10,000 epochs, $w(x)=(u(x),v(x))$4 and $w(x)=(u(x),v(x))$5	For $w(x)=(u(x),v(x))$6: best loss $w(x)=(u(x),v(x))$7, best AEE $w(x)=(u(x),v(x))$8, AAE $w(x)=(u(x),v(x))$9
Middlebury benchmark	20,000 epochs, $$I_2(x + w(x)) \approx I_2(x) + \nabla I_2(x)\cdot w(x),$$0	Moderate TV generally best; $$I_2(x + w(x)) \approx I_2(x) + \nabla I_2(x)\cdot w(x),$$1 or $$I_2(x + w(x)) \approx I_2(x) + \nabla I_2(x)\cdot w(x),$$2 often gives the best balance

The synthetic experiment uses a Shepp-Logan phantom with two added circular regions undergoing opposing vertical motion. The model is reported to recover this localized motion well. With $$I_2(x + w(x)) \approx I_2(x) + \nabla I_2(x)\cdot w(x),$$3, the output shows improved smoothness and preserved boundaries around the moving structures. The benchmark evaluation uses Middlebury scenes including Dimetrodon, Grove2, Grove3, Hydrangea, RubberWhale, Urban2, Urban3, and Venus.

Representative Middlebury results include best AEE values of $$I_2(x + w(x)) \approx I_2(x) + \nabla I_2(x)\cdot w(x),$$4 for Dimetrodon at $$I_2(x + w(x)) \approx I_2(x) + \nabla I_2(x)\cdot w(x),$$5, $$I_2(x + w(x)) \approx I_2(x) + \nabla I_2(x)\cdot w(x),$$6 for Grove2 at $$I_2(x + w(x)) \approx I_2(x) + \nabla I_2(x)\cdot w(x),$$7, $$I_2(x + w(x)) \approx I_2(x) + \nabla I_2(x)\cdot w(x),$$8 for RubberWhale at $$I_2(x + w(x)) \approx I_2(x) + \nabla I_2(x)\cdot w(x),$$9, $\nabla I_2(x)\cdot w(x) + I_2(x) - I_1(x) = 0.$0 for Venus at $\nabla I_2(x)\cdot w(x) + I_2(x) - I_1(x) = 0.$1, $\nabla I_2(x)\cdot w(x) + I_2(x) - I_1(x) = 0.$2 for Urban2 at $\nabla I_2(x)\cdot w(x) + I_2(x) - I_1(x) = 0.$3, and $\nabla I_2(x)\cdot w(x) + I_2(x) - I_1(x) = 0.$4 for Grove3 at $\nabla I_2(x)\cdot w(x) + I_2(x) - I_1(x) = 0.$5. The reported failure mode is that strong regularization can be harmful in complex, high-displacement scenes; Urban2 is explicitly given as a case where $\nabla I_2(x)\cdot w(x) + I_2(x) - I_1(x) = 0.$6 is much worse than $\nabla I_2(x)\cdot w(x) + I_2(x) - I_1(x) = 0.$7 (Behnamian et al., 10 Sep 2025).

The qualitative interpretation is consistent across experiments: high TV yields smoother and more coherent fields but can blur fine structures; low or zero TV preserves discontinuities more aggressively but can introduce noise or instability.

5. Position within PINN and multiscale flow-learning research

FractalPINN-Flow is related to, but distinct from, several contemporary PINN lines of work. In FlexPINN, the target problem is incompressible laminar flow and passive species mixing in a 3D T-shaped micromixer with internal fins. That framework uses a first-order nondimensional reformulation of the steady incompressible Navier–Stokes and convection–diffusion equations, a flexible multi-branch network, adaptive loss weighting, penalty constraints for mass-flow conservation, and transfer learning across geometries. It predicts pressure drop coefficient and mixing index with maximum errors of $\nabla I_2(x)\cdot w(x) + I_2(x) - I_1(x) = 0.$8 and $\nabla I_2(x)\cdot w(x) + I_2(x) - I_1(x) = 0.$9, respectively, relative to CFD, and identifies the rectangular fin with configuration C in the double-unit setup at $$E_{TV}(w) := \lambda_1 \|\nabla I_2 \cdot w + I_2 - I_1\|_1 +\lambda_2 \|\nabla I_2 \cdot w + I_2 - I_1\|_2^2 +\lambda_{TV} \|w\|_1,$$0 as the best case, with mixing efficiency $$E_{TV}(w) := \lambda_1 \|\nabla I_2 \cdot w + I_2 - I_1\|_1 +\lambda_2 \|\nabla I_2 \cdot w + I_2 - I_1\|_2^2 +\lambda_{TV} \|w\|_1,$$1 (Hassanzadeh et al., 24 Apr 2025).

FV-PINN addresses steady-state incompressible flow by replacing strong-form residuals at collocation points with finite-volume integral residuals evaluated at Gaussian quadrature points on control-volume boundaries. The method reduces derivative order, uses a stream-function output to satisfy continuity by construction, and reports about a $$E_{TV}(w) := \lambda_1 \|\nabla I_2 \cdot w + I_2 - I_1\|_1 +\lambda_2 \|\nabla I_2 \cdot w + I_2 - I_1\|_2^2 +\lambda_{TV} \|w\|_1,$$2 reduction in training time in the Pipe Bend example, from $$E_{TV}(w) := \lambda_1 \|\nabla I_2 \cdot w + I_2 - I_1\|_1 +\lambda_2 \|\nabla I_2 \cdot w + I_2 - I_1\|_2^2 +\lambda_{TV} \|w\|_1,$$3 s for a traditional PINN to $$E_{TV}(w) := \lambda_1 \|\nabla I_2 \cdot w + I_2 - I_1\|_1 +\lambda_2 \|\nabla I_2 \cdot w + I_2 - I_1\|_2^2 +\lambda_{TV} \|w\|_1,$$4 s for FV-PINN (Su et al., 2024).

MSPINN focuses on inverse reconstruction of transient natural convection from instantaneous temperature data and introduces multiple-scale input reformulation to mitigate vanishing gradients. It reports improvements in maximum and mean errors by $$E_{TV}(w) := \lambda_1 \|\nabla I_2 \cdot w + I_2 - I_1\|_1 +\lambda_2 \|\nabla I_2 \cdot w + I_2 - I_1\|_2^2 +\lambda_{TV} \|w\|_1,$$5 and $$E_{TV}(w) := \lambda_1 \|\nabla I_2 \cdot w + I_2 - I_1\|_1 +\lambda_2 \|\nabla I_2 \cdot w + I_2 - I_1\|_2^2 +\lambda_{TV} \|w\|_1,$$6, respectively, and explicitly argues that encoding physics-based scales helps recover smaller-scale structures in transient flow fields (Ohashi et al., 2024).

Pseudo-differential-enhanced PINNs extend gradient enhancement into Fourier space by adding a pseudo-differential residual term. They are reported to improve neural tangent kernel spectral eigenvalue decay, mitigate frequency bias, pair well with few collocation points, and support fractional derivatives, including examples involving Navier–Stokes (Gracyk, 16 Feb 2026).

Taken together, these works indicate that FractalPINN-Flow belongs to a wider methodological landscape organized around three recurring themes: multiscale representation, regularization of ill-posed inverse structure, and stabilization of learning under sparse supervision. This suggests a useful taxonomy. FractalPINN-Flow treats “flow” as dense image motion and encodes multiscale structure architecturally; FlexPINN, FV-PINN, MSPINN, and pseudo-differential-enhanced PINNs treat “flow” as a PDE-governed field and encode physics directly through residuals, integral balances, scale-augmented inputs, or Fourier-domain operators.

6. Misconceptions, limitations, and interpretive boundaries

Several recurrent misconceptions can be clarified directly from the reported formulation. First, the “fractal” designation does not denote strict mathematical fractal recursion. The paper explicitly states that the term refers to repeated processing across scales rather than strict mathematical self-similarity (Behnamian et al., 10 Sep 2025). This differs sharply from genuine fractal-fluid models, such as the non-integer-dimensional continuum approach for Poiseuille flow in pipes, where the governing operators themselves are generalized to non-integer dimensional space and the flow-rate law scales as $$E_{TV}(w) := \lambda_1 \|\nabla I_2 \cdot w + I_2 - I_1\|_1 +\lambda_2 \|\nabla I_2 \cdot w + I_2 - I_1\|_2^2 +\lambda_{TV} \|w\|_1,$$7 rather than $$E_{TV}(w) := \lambda_1 \|\nabla I_2 \cdot w + I_2 - I_1\|_1 +\lambda_2 \|\nabla I_2 \cdot w + I_2 - I_1\|_2^2 +\lambda_{TV} \|w\|_1,$$8 (Tarasov, 2015).

Second, the method should not be conflated with a classical PDE-residual PINN. Its stated objective is the brightness-constancy residual plus TV regularization, not a Navier–Stokes or conservation-law residual with boundary and initial condition losses. This suggests that the “PINN” portion of the name functions more as a methodological label than as a strict descriptor of the standard PINN formalism.

The limitations reported in the paper are correspondingly specific. Performance is sensitive to the TV weight. The method uses a simple brightness-constancy model, which may fail under illumination changes, occlusion, or non-Lambertian effects. No explicit occlusion handling or sophisticated warping pyramid is described. The results are based on single-pair training per configuration, which may limit broader generalization claims. The paper also states that the method is not a modern large-scale benchmark system and is closer to a principled prototype combining variational regularization with a compact CNN (Behnamian et al., 10 Sep 2025).

Within those boundaries, FractalPINN-Flow is best understood as a hybrid unsupervised optical-flow framework that reintroduces classical variational structure into a compact multiscale neural architecture. Its specific contribution lies less in redefining PINNs as a formal class than in showing how brightness constancy, mixed $$E_{TV}(w) := \lambda_1 \|\nabla I_2 \cdot w + I_2 - I_1\|_1 +\lambda_2 \|\nabla I_2 \cdot w + I_2 - I_1\|_2^2 +\lambda_{TV} \|w\|_1,$$9 data fidelity, and anisotropic TV can be combined with recursive encoder-decoder design to produce dense, smooth, and boundary-aware optical flow from only two grayscale images.