---
title: Frequency-Driven Inverse Kernel Prediction Network
url: https://www.emergentmind.com/topics/frequency-driven-inverse-kernel-prediction-network-fdikp
type: topic
---

# Frequency-Driven Inverse Kernel Prediction Network

Searching arXiv for the cited FDIKP-related papers and closely related frequency/kernel papers.
Frequency-Driven Inverse Kernel Prediction Network (FDIKP) is a frequency-centric architecture for single image defocus deblurring that predicts inverse kernels from Fourier-domain representations rather than relying exclusively on spatial-domain features. In its canonical formulation, introduced for single image defocus deblurring, FDIKP addresses the problem of recovering an all-in-focus image from a defocus counterpart whose blur is spatially varying and difficult to estimate in severely blurred regions. The model combines frequency-domain inverse kernel prediction, position-adaptive deconvolution, and dual-domain scale-recurrent refinement in a compact three-stage network [2508.12736].

## 1. Defocus deblurring formulation and frequency-domain motivation

The paper formulates single image defocus deblurring as recovery of an all-in-focus RGB image \(y\) from a defocused observation \(x\) under a spatially varying convolution model,
\[
x = y \otimes k,
\]
where \(k\) is a spatially varying defocus kernel and \(\otimes\) denotes convolution. Locally, the degradation is written as
\[
x(p) = \sum_{q} y(q)\, k_p(p-q),
\]
with \(k_p\) varying over image position because of depth variation. The paper notes that defocus kernels are often approximately disk-shaped, while their scale varies over the image [2508.12736].

The main difficulty is that kernel estimation becomes ill-posed in severely blurred regions. Existing kernel-modeling methods such as GKMNet, NRKNet, INIKNet, and P\(^2\)IKT are described as operating mostly in the spatial domain, where local gradients and textures are suppressed by strong blur. The stated consequence is that different kernels can produce visually similar local patches, making spatial kernel estimation unstable [2508.12736].

FDIKP is motivated by the observation that blurring is inherently a frequency-domain filtering process:
\[
X(\omega) = Y(\omega) \cdot K(\omega).
\]
Within this representation, the amplitude spectrum is used to encode global blur patterns, while the phase spectrum is used to retain local structural cues and alignment. The blur kernel is written as
\[
K = \mathcal{F}(k) = |K| e^{j\phi_K},
\]
and the spatial kernel as
\[
k = \mathcal{F}^{-1}\big(|K| e^{j\phi_K}\big).
\]
For deblurring, the model explicitly targets an inverse kernel \(k^\ast\),
\[
\hat{y} = x \otimes k^{\ast}, \quad
k^{\ast} = \mathcal{F}^{-1}\left(\frac{1}{|K|} \cdot e^{-j\phi_K}\right),
\]
so that, in frequency space, \(\hat{Y}(\omega) \approx X(\omega)/K(\omega)\) [2508.12736].

This formulation places FDIKP in a broader line of work that treats kernel information as complementary to learned image priors. A related deblurring study argues that kernel priors are independent of image context and therefore can improve out-of-domain generalization when fused in the frequency domain [2504.14664]. In FDIKP, that general perspective is specialized to inverse-kernel estimation for defocus blur.

## 2. Network organization and multi-scale data flow

FDIKP is implemented as a three-stage scale-recurrent network operating from coarse to fine. The three input resolutions are
\[
x_{1/4} \in \mathbb{R}^{H/4 \times W/4 \times 3}, \quad
x_{1/2} \in \mathbb{R}^{H/2 \times W/2 \times 3}, \quad
x \in \mathbb{R}^{H \times W \times 3}.
\]
At each scale, the architecture uses two core components: Frequency Inverse Kernel Prediction (FIKP) and the Dual-Domain Scale Recurrent Module (DSRM) [2508.12736].

FIKP takes the defocus image at the current scale, together with the upsampled output from the previous scale, and predicts a small set of inverse kernels in the frequency domain. The default configuration uses \(5\) inverse kernels, each of size \(5 \times 5\). These kernels are then used by Position Adaptive Convolution (PAC) to generate deconvolution feature maps \(F_s\) [2508.12736].

DSRM processes the current-scale image and the recurrent hidden state from the previous scale. It uses an encoder-decoder structure with ResBlocks, a bottleneck Dual-Domain Module (DDM), and an APU-based scale-recurrent unit. Its outputs are coefficient maps \(C_s\), which modulate the deconvolution features, and an updated hidden state \(h_s\) for the next scale. The scale output is produced by
\[
\tilde{y}_s = \text{Conv}\big(F_s \odot C_s\big),
\]
where \(\odot\) denotes feature modulation [2508.12736].

The full model has **5.52 MB** of weights and uses **5 inverse kernels of size 5×5** by default. In the reported implementation, inference time is **0.10 s** per DPDD test image on an NVIDIA RTX 3090 [2508.12736].

| Component | Role | Key specification |
|---|---|---|
| FIKP | Predict inverse kernels and produce deconvolution features | 5 kernels, \(5 \times 5\) |
| PAC | Apply predicted inverse kernels adaptively across the image | Blur-aware dilation map |
| DSRM | Fuse features and refine outputs across scales | Three-stage scale recurrence |

Architecturally, the design separates inverse-kernel estimation from multi-scale reconstruction. This separation is central to the model’s claim that kernel modeling should be frequency-driven, while feature fusion and restoration should remain jointly spatial and spectral.

## 3. Dual-Branch Inverse Kernel Prediction and position-adaptive deconvolution

The core inverse-kernel mechanism is Dual-Branch Inverse Kernel Prediction (DIKP), which operates on a 2D FFT of the input:
\[
X = \mathcal{F}(x) \Rightarrow A = |X|, \quad P = \phi_X.
\]
The amplitude branch uses \(A\) to predict blur-related structural information, while the phase branch uses \(P\) to refine alignment and preserve localization [2508.12736].

In the amplitude branch,
\[
K_A = \text{Predictor}(A),
\]
where the Predictor consists of two units of Conv + ReLU + Adaptive Avg Pool followed by a Conv and Softmax. The phase branch is amplitude-guided:
\[
K_P = \text{Predictor}(P) * \text{Attention}(A).
\]
The attention path uses amplitude residuals to suppress phase information that is irrelevant to kernel structure. DIKP then reconstructs a frequency-domain inverse-kernel representation and returns to the spatial domain via
\[
k^{\ast} = \mathcal{F}^{-1}\big(|K_A| \cdot e^{jK_P'}\big),
\]
where \(K_P'\) is a processed phase-like term [2508.12736].

The paper does not define explicit supervision against ground-truth kernels, because such kernels are not available. Instead, inverse-kernel prediction is trained implicitly through image-level reconstruction, perceptual, and frequency-domain losses. The stated interpretation is that physically plausible inverse kernels emerge indirectly from constraints on the restored image [2508.12736].

PAC addresses the fact that only a limited number of inverse kernels are predicted, while the true blur varies spatially. A blur-aware dilation map \(D(p)\) is predicted by a small Conv → Sigmoid network, with \(D(p) \in (0,1]\). Deconvolution is then performed as
\[
F(p) = \sum_{i=1}^{K \times K} k^{\ast}_i \, x\big(p + \Delta p_i \cdot D(p)\big),
\]
where \(\Delta p_i\) indexes canonical sampling offsets. A subsequent refinement convolution produces corrected feature maps [2508.12736].

This construction differs from per-pixel dynamic kernels and from deformable convolutions. PAC is presented as lightweight because it predicts only a scalar dilation factor per pixel, and as physically motivated because defocus kernels are treated as varying primarily in scale rather than shape. The paper’s ablation attributes a PSNR increase from **26.28 dB** to **26.42 dB** to the addition of PAC on top of DSRM + DIKP [2508.12736].

## 4. Dual-domain recurrent refinement and optimization objective

DSRM performs coarse-to-fine refinement and fuses deconvolution features with recurrently propagated contextual information. At each scale, the encoder extracts features with ResBlocks, the bottleneck DDM performs sequential spatial-frequency processing, and the decoder reconstructs feature maps for recurrent propagation and output prediction [2508.12736].

The DDM is explicitly sequential rather than parallel. Its first stage is spatial self-attention, described as Stripformer-like, for local structures and high-frequency details. Its second stage applies frequency-domain refinement:
\[
F_{\text{freq}} = \mathcal{F}(F_{\text{spatial}}), \quad
F_{\text{freq}}' = \text{Conv}(\text{ReLU}(\text{Conv}(F_{\text{freq}}))), \quad
F_{\text{ddm}} = \mathcal{F}^{-1}(F_{\text{freq}}').
\]
A spatial-domain ResBlock is then added to preserve local high-frequency components [2508.12736].

Recurrence across scales is mediated by an APU unit. Although exact closed-form recurrence equations are not provided, the paper gives the conceptual form
\[
h_s = \text{APU}(F_{\text{dec}, s}, h_{s-1}), \quad
C_s = \text{Head}(h_s),
\]
with the deblurred output at scale \(s\) again written as
\[
\hat{y}_s = \text{Conv}\big(F_s \odot C_s\big).
\]
The intended effect is progressive refinement: quarter scale recovers global structure, half scale restores mid-level detail, and full scale restores fine textures and edges [2508.12736].

Training uses a multi-scale objective,
\[
\mathcal{L} =
\lambda_1 \mathcal{L}(y,\hat{y}) +
\lambda_2 \mathcal{L}(y_{1/2},\hat{y}_{1/2}) +
\lambda_3 \mathcal{L}(y_{1/4},\hat{y}_{1/4}),
\]
with \(\lambda_1=1\), \(\lambda_2=0.2\), and \(\lambda_3=0.1\). The single-scale loss is
\[
\mathcal{L}(y,\hat{y}) =
\alpha \mathcal{L}_2(y,\hat{y}) +
\beta \mathcal{L}_{\text{lpips}}(y,\hat{y}) +
\gamma \mathcal{L}_{\text{freq}}(y,\hat{y}),
\]
with \(\alpha = 1\), \(\beta = 0.2\), and \(\gamma = 0.2\). The components are MSE, LPIPS, and an \(L_1\) discrepancy between Fourier transforms,
\[
\mathcal{L}_{\text{freq}}(y,\hat{y}) = \left\|\mathcal{F}(y) - \mathcal{F}(\hat{y}) \right\|_1.
\]
Optimization uses PyTorch, Adam with \(\beta_1=0.9\) and \(\beta_2=0.999\), an initial learning rate of \(1 \times 10^{-4}\), and MultiStepLR. Training is two-stage, with stochastic weight averaging in the second stage [2508.12736].

The reported loss ablation is consistent with the architecture’s emphasis on spectral consistency. Using only \(\mathcal{L}_2\) yields PSNR **25.694** and LPIPS **0.255**, while adding \(\mathcal{L}_{\text{freq}}\) raises PSNR to **26.258**; combining \(\mathcal{L}_2\), LPIPS, and frequency loss with the multi-scale loss gives **PSNR 26.420**, **SSIM 0.813**, and **LPIPS 0.185** [2508.12736].

## 5. Datasets, quantitative performance, and ablation evidence

The primary training and evaluation setting uses DPDD, a real dual-pixel defocus dataset with **500 scenes (350 train / 74 val / 76 test)** at resolution **1680×1120**. Additional evaluation uses RealDOF, RTF, CUHK, and LFDOF. Reported metrics are PSNR, SSIM, LPIPS, model size, and inference time [2508.12736].

When trained on DPDD, the model reports the following quantitative results. On **DPDD**, FDIKP achieves **PSNR 26.42**, **SSIM 0.813**, and **LPIPS 0.185**. On **RealDOF**, it reports **PSNR 25.58**, **SSIM 0.771**, and **LPIPS 0.283**. On **RTF**, it reports **PSNR 26.52**, **SSIM 0.865**, and **LPIPS 0.163**. The paper states that these results compare favorably to methods including NRKNet, SFHformer, P\(^2\)IKT, PPTformer, and EAMamba, with especially strong performance on RTF and RealDOF [2508.12736].

In the LFDOF-trained setting, the model reports **PSNR 31.12**, **SSIM 0.899**, and **LPIPS 0.114** on LFDOF; **PSNR 24.21**, **SSIM 0.736**, and **LPIPS 0.348** on RealDOF; and **PSNR 28.16**, **SSIM 0.909**, and **LPIPS 0.080** on RTF [2508.12736].

Ablation studies isolate the contribution of the major modules. The baseline SRAM obtains **25.49 dB**. Adding FIKP raises performance to **25.89 dB**. Replacing SRAM with DSRM yields **26.11 dB**. DSRM plus DIKP yields **26.28 dB**. The full model, DSRM + DIKP + PAC, yields **26.42 dB**. A DDM ablation reports **26.24 dB** for spatial-only, **26.17 dB** for frequency-only, **26.25 dB** for a parallel dual-branch design, and **26.42 dB** for the sequential DDM [2508.12736].

The qualitative description is aligned with the quantitative findings. Reported improvements include reduced defocus halos around bright objects, sharper restoration of text contours and narrow gaps, and more faithful reconstruction in heavily blurred backgrounds. On CUHK, which has no ground truth, the paper describes restoration as robust across diverse real-world scenes without over-sharpening or obvious artifacts [2508.12736].

## 6. Broader technical context, conceptual extensions, and limitations

FDIKP belongs to a broader family of methods that couple kernel priors with frequency-domain processing. In blind image deblurring, a related study introduces a Frequency Integration Module for fusing kernel priors with a frequency-based Transformer backbone, arguing that kernel priors remain independent of image context and therefore support stronger generalization under domain shift [2504.14664]. FDIKP differs in targeting inverse-kernel prediction for defocus blur, but the shared premise is that blur structure is more identifiable in the Fourier domain than in local spatial patches.

The accompanying materials also use “FDIKP” as a conceptual lens for inverse problems beyond image deblurring. For inverse scattering with multi-frequency data, a multi-level network is constructed along the frequency axis, with each level updating a reconstruction using a single frequency and progressively recovering higher-order Fourier components as depth increases. This suggests a generalized frequency-driven inverse-kernel viewpoint in which depth is aligned with physical frequency and each stage performs a localized inverse update [2606.29368].

A second conceptual extension arises from kernel inverse optimization in reproducing kernel Hilbert spaces. There, objective functions are lifted into an RKHS, a representer-type theorem reduces the learning problem to a finite-dimensional convex program, and scalability is addressed by Sequential Selection Optimization. This suggests that inverse learning, kernels, and frequency-structured architectures can be related at the level of hypothesis-class enrichment and structured optimization, although that relation is presented as a translation or conceptual bridge rather than as the direct content of the defocus-deblurring model [2410.23952].

Within its original defocus-deblurring setting, the reported limitations are specific. Failure cases include **radial blur** or blur not well described by disk-like defocus kernels, **extremely severe blur** where information is almost completely lost, and **mixed blur types** such as motion plus defocus, which are not explicitly addressed. The paper also notes a potential **domain gap** between synthetic and real data, even though the reported cross-dataset results are strong [2508.12736].

The future directions identified in the materials are correspondingly concrete: extending frequency-driven inverse-kernel prediction to other blur models such as motion blur and camera shake, introducing more flexible kernel priors beyond disk-like defocus, combining the approach with diffusion-based models such as RDDM, and transferring PAC or DSRM to other restoration tasks including super-resolution, denoising, and dehazing [2508.12736]. Collectively, these directions indicate that FDIKP is best understood not only as a specific deblurring network, but also as a design pattern for inverse problems in which spectral structure, kernel modeling, and progressive refinement are tightly coupled.

Source: https://www.emergentmind.com/topics/frequency-driven-inverse-kernel-prediction-network-fdikp