Fractal Deformation Network for Dense Optical Flow

• FDN is a recursive, fractal-inspired encoder-decoder that extracts multiscale motion features from consecutive grayscale frames.
• It employs lightweight skip connections with bilinear interpolation to fuse high-frequency details and capture long-range motion patterns.
• Integrated within FractalPINN-Flow, FDN works with variational and CNN regression layers to achieve robust, unsupervised dense optical flow estimation.

The Fractal Deformation Network (FDN) is the feature-extraction backbone of FractalPINN-Flow, an unsupervised deep learning framework for dense optical flow estimation from consecutive grayscale frames. In that setting, FDN is defined as a recursive, fractal-inspired encoder-decoder that builds motion features at multiple scales while preserving fine detail through skip connections. Its design is motivated by fractals and self-similarity: the network is not a mathematically exact fractal, but it follows the broad idea of repeated structure at different scales in order to capture both fine-grained details and long-range motion patterns (Behnamian et al., 10 Sep 2025).

1. Definition and conceptual role

In FractalPINN-Flow, FDN is the component that converts a pair of grayscale frames into a learned representation of deformation structure. The paper describes it as a recursive or fractal-inspired encoder-decoder whose purpose is to provide multiscale motion features for subsequent flow regression. The immediate motivation is optical flow itself: dense motion estimation must handle small, sharp local motions at object boundaries, larger displacements over extended regions, and the need to preserve spatial structure rather than oversmooth it (Behnamian et al., 10 Sep 2025).

The architecture is presented as fractal-inspired because fractal geometry is characterized by repeated structure across scales, and the network borrows that idea by reusing encoder-decoder-like processing across hierarchical resolutions. The paper explicitly states that the term “fractal” is used loosely. This point is central to the interpretation of FDN: the claim is architectural and multiscale, not a claim of strict mathematical self-similarity.

Within that conceptual framing, FDN is intended to capture fine-grained texture and local deformation cues at higher resolutions while simultaneously encoding long-range motion patterns through deeper, coarser representations. This dual objective distinguishes its role from that of a generic hierarchical extractor. A plausible implication is that the design is meant to reduce the usual tension between boundary preservation and broad contextual reasoning in optical flow.

2. Architectural construction

The implementation described for FDN is a symmetric U-Net-style encoder-decoder with fixed depth $d=4$. The input consists of two grayscale frames concatenated along the channel dimension, so the encoder starts from 2 channels and expands the representation through successive downsampling stages. Each encoder block contains two $3 \times 3$ convolutions, batch normalization, ReLU activations, and $2 \times 2$ max pooling. For $d=4$, the encoder channel progression is

$$2 \rightarrow 32 \rightarrow 64 \rightarrow 128 \rightarrow 256.$$

The decoder mirrors this structure, using $2 \times 2$ transposed convolutions for upsampling, followed after each upsampling step by two $3 \times 3$ convolutions, batch normalization, and ReLU. Its channel progression is

$$256 \rightarrow 128 \rightarrow 64 \rightarrow 32 \rightarrow 32.$$

The output is a 32-channel feature map at the original spatial resolution,

$F \in \mathbb{R}^{B \times 32 \times H \times W}.$

This feature map is explicitly not yet the optical flow; it is a learned representation of the image pair’s deformation structure (Behnamian et al., 10 Sep 2025).

Component	Operations	Channels / output
Input	Concatenated grayscale frames	$2$ channels
Encoder	Two $3 \times 3$0 convolutions, batch normalization, ReLU, $3 \times 3$1 max pooling	$3 \times 3$2
Decoder	$3 \times 3$3 transposed convolutions, then two $3 \times 3$4 convolutions, batch normalization, ReLU	$3 \times 3$5
Skip fusion	Bilinear interpolation plus element-wise addition	Resolution matched to decoder
FDN output	Feature map $3 \times 3$6	$3 \times 3$7

A notable design choice is the skip-connection mechanism. Instead of classical U-Net-style concatenation, encoder features are bilinearly interpolated to the decoder resolution and fused by element-wise addition. The paper states that addition is lighter-weight, reduces parameters, and still allows multiscale fusion. In the stated interpretation, these skips reintroduce high-frequency spatial details that would otherwise be lost in pooling.

The contrast with a standard CNN encoder-decoder is therefore specific rather than generic. The paper emphasizes sequential downsampling and concatenative skips as typical of standard designs, whereas FDN is presented as fractal-inspired or recursively nested in spirit, explicitly multiscale, strongly oriented toward preserving both local texture and global deformation context, and ending with a relatively rich 32-channel map rather than collapsing too early.

3. Placement in the full FractalPINN-Flow system

The full FractalPINN-Flow pipeline is

$3 \times 3$8

Two normalized grayscale frames are concatenated into

$3 \times 3$9

FDN produces

$2 \times 2$0

after which a $2 \times 2$1 convolution maps $2 \times 2$2. This projection layer is described as a shallow channel expansion to adapt the features for regression. The optical flow prediction stage is then a 5-layer CNN with channel pattern

$2 \times 2$3

using $2 \times 2$4 convolutions and ReLU activations, with no pooling so that spatial resolution is preserved throughout. The final output is the dense flow field

$2 \times 2$5

In this decomposition, FDN is the multiscale representation learner, while the later CNN serves as the regression head (Behnamian et al., 10 Sep 2025).

The paper’s implementation details further specify the practical regime in which FDN is used. Zero padding is used to preserve dimensions. The optimizer is Adam with learning rate $2 \times 2$6. The framework is PyTorch with CUDA. The batch size is 1, since only one image pair is used in training in that setup, and training is run for a fixed number of epochs. The report also notes CUDA cache clearing and garbage collection after each epoch. The depth hyperparameter was not tuned; $2 \times 2$7 was used for all experiments.

These details matter because they delimit the reported instantiation of FDN. The architecture is described as compact enough to be practical, with the flow head separated from the feature extractor. That statement pertains to the reported model rather than to an abstract family of possible FDN variants.

4. Variational objective and relation to classical optical flow

FractalPINN-Flow is trained without ground-truth flow labels. The governing assumption is brightness constancy under motion,

$2 \times 2$8

The paper recalls the classical linearized model

$2 \times 2$9

which yields the linear constraint

$d=4$0

On that basis, the training functional is defined as

$d=4$1

with $d=4$2. The anisotropic total variation term is

$d=4$3

using finite differences in horizontal and vertical directions (Behnamian et al., 10 Sep 2025).

The three terms play distinct roles in the formulation. The $d=4$4 term is an $d=4$5 brightness-constancy or data-fidelity term that is described as robust to outliers. The $d=4$6 term encourages overall photometric agreement. The TV term promotes piecewise smooth flow while preserving discontinuities. The paper explicitly contrasts this with classical Horn–Schunck-style optical flow, where quadratic data and smoothness terms tend to oversmooth motion boundaries.

In the optical-flow interpretation advanced by the paper, FDN and the variational objective are complementary. FDN supplies multiscale deformation features that combine local precision and global context, while the variational objective constrains the predicted field to satisfy brightness constancy and spatial coherence. This suggests that the architecture is not intended as a purely data-driven regressor; it is embedded within a classical energy-minimization perspective.

5. Reported empirical behavior

The reported experiments include a synthetic Shepp–Logan case and evaluation on Middlebury scenes. In the synthetic phantom, two moving circular regions were used: one circle moved up by 3 pixels and the other moved down by 3 pixels. Training was run for 10,000 epochs with $d=4$7 and $d=4$8. The paper reports that the model recovered the opposing motions correctly and that TV regularization made the predicted flow smoother and more coherent. For $d=4$9, the loss fell rapidly and reached a best value of

$$2 \rightarrow 32 \rightarrow 64 \rightarrow 128 \rightarrow 256.$$0

with best AEE $$2 \rightarrow 32 \rightarrow 64 \rightarrow 128 \rightarrow 256.$$1 and best AAE $$2 \rightarrow 32 \rightarrow 64 \rightarrow 128 \rightarrow 256.$$2 (Behnamian et al., 10 Sep 2025).

On Middlebury, the tested regularization weights were

$$2 \rightarrow 32 \rightarrow 64 \rightarrow 128 \rightarrow 256.$$3

with training for 20,000 epochs. The main reported trend is that moderate TV weights often performed best, especially $$2 \rightarrow 32 \rightarrow 64 \rightarrow 128 \rightarrow 256.$$4 or $$2 \rightarrow 32 \rightarrow 64 \rightarrow 128 \rightarrow 256.$$5. The same section also states the usual TV tradeoff: moderate TV improves smoothness and stability, too much TV oversmooths and can remove fine motion, and too little TV leaves noisy or unstable flow.

Scene	Best reported setting	Reported AEE
Dimetrodon	$$2 \rightarrow 32 \rightarrow 64 \rightarrow 128 \rightarrow 256.$$6	$$2 \rightarrow 32 \rightarrow 64 \rightarrow 128 \rightarrow 256.$$7
RubberWhale	$$2 \rightarrow 32 \rightarrow 64 \rightarrow 128 \rightarrow 256.$$8	$$2 \rightarrow 32 \rightarrow 64 \rightarrow 128 \rightarrow 256.$$9
Venus	$2 \times 2$0	$2 \times 2$1
Hydrangea	$2 \times 2$2	$2 \times 2$3
Grove3	$2 \times 2$4	$2 \times 2$5
Urban2	$2 \times 2$6 better than $2 \times 2$7	$2 \times 2$8 vs. $2 \times 2$9

These observations are consistent with the architectural claims made for FDN. The paper states that the model is especially effective for high-resolution data and scenarios with limited annotations, that the fractal-inspired multiscale design helps capture both small deformation details and larger motion structures, and that TV regularization gives edge-preserving smoothness. At the same time, the empirical discussion places clear emphasis on sensitivity to the TV weight; the best results usually come from moderate regularization rather than extreme smoothing.

6. Terminological limits and related fractal deformation formulations

A common misconception is to read “fractal” in FDN as a statement of strict mathematical fractality. The paper directly rejects that interpretation: the term is used loosely to denote repeated multiscale processing rather than strict mathematical self-similarity. Another potential misunderstanding is to treat FDN as the complete optical-flow model. In the reported system it is only the feature-extraction backbone; the final flow prediction is produced after the projection layer and the 5-layer CNN head (Behnamian et al., 10 Sep 2025).

A related but distinct line of work appears in “Neural Network Operator-Based Fractal Approximation: Smoothness Preservation and Convergence Analysis” (Bhat et al., 22 Mar 2025). That paper does not define the same optical-flow backbone, but it presents what it calls a clean FDN-style mechanism in which a standard approximation operator is recursively deformed by fractal scaling to produce a new function $3 \times 3$0 or $3 \times 3$1. The construction replaces the classical base function by a neural network operator,

$3 \times 3$2

and yields the self-referential equation

$3 \times 3$3

The same paper proves that $3 \times 3$4 converges uniformly to $3 \times 3$5 as $3 \times 3$6, and, in the smooth construction based on the four-layer neural network operator $3 \times 3$7, states that choosing

$3 \times 3$8

ensures a fractal interpolation function in $3 \times 3$9.

This suggests a broader conceptual family behind the phrase “fractal deformation”: repeated self-referential structure can be used either as a multiscale feature extractor for dense motion estimation, as in FDN within FractalPINN-Flow, or as an operator-theoretic deformation mechanism for interpolation and approximation, as in the $$256 \rightarrow 128 \rightarrow 64 \rightarrow 32 \rightarrow 32.$$0-fractal framework of (Bhat et al., 22 Mar 2025). The two usages are not identical, but both organize deformation through recursive multiscale structure.