Convolution Direct Image Techniques

• Convolution Direct Image is a family of methods that preserve the inherent blur operator by directly modeling convolution in imaging and computational systems.
• These techniques enhance image restoration and reconstruction by strictly adhering to the physical and mathematical properties of image formation.
• The scope spans from physics-based optical and medical imaging to adaptive convolution operations and categorical constructions in arithmetic geometry.

In the recent literature, “Convolution Direct Image” appears in several technically distinct but structurally related senses. In image formation and restoration, it denotes models that respect the native blur law $y=x*k$ and its Fourier dual $Y=XK$, rather than substituting blur with additive-noise surrogates (Parvathireddy et al., 24 May 2026). In medical imaging, it denotes learned sinogram-to-image mappings implemented by convolutional generators instead of iterative reconstruction loops (Kandarpa et al., 2020). In systems work, it denotes computing convolution directly on image tensors, without materializing im2col matrices or relying on transform-domain detours (Ji, 2019). A mathematically separate homonym occurs in arithmetic geometry, where “convolution direct image” is a direct-image construction for perverse sheaves on commutative group schemes (Rojas-León, 2018).

1. Image formation, convolution, and direct operator structure

For optical blur, the direct image-formation model is convolution. In the continuous setting, a blurred observation $y$ is formed from a latent image $x$ and a point spread function $k$ by

$y(r)=\int x(\tau)k(r-\tau)\,d\tau=(x*k)(r),$

and in the Fourier domain this becomes $Y(\omega)=X(\omega)K(\omega)$. The discrete counterpart is

$y[i,j]=\sum_u\sum_v x[i-u,j-v]\,k[u,v]=(x*k)[i,j],$

with discrete Fourier relation $Y[p,q]=X[p,q]K[p,q]$. For isotropic Gaussian blur, $$k(r)=\frac{1}{2\pi\sigma^2}\exp(-\|r\|^2/(2\sigma^2))$$ and $Y=XK$0; this Gaussian-to-Gaussian duality is what makes closed-form intermediate blur states physically valid in later deblurring constructions (Parvathireddy et al., 24 May 2026).

The same operator structure underlies direct Fourier-domain optimization. With periodic boundary conditions, spatial convolution is represented by a block-circulant with circulant blocks operator, diagonalized by the 2D discrete Fourier transform, so that $Y=XK$1. This yields direct convolution and deconvolution formulas in the frequency domain without explicitly building dense linear systems. The resulting closed forms include inverse filtering, Wiener filtering, and Tikhonov-regularized solves such as

$Y=XK$2

and gradient-regularized variants with denominator $Y=XK$3 (Helou et al., 2018).

A central implication is that “directness” is not restricted to spatial sliding-window implementations. In the image-processing literature it also means respecting the actual forward operator—convolution, projection, or other linear physics—and exploiting the algebraic form of that operator rather than replacing it with a generic surrogate.

2. Physically grounded blur restoration

The most explicit recent formulation of convolution as direct image formation in restoration is ConvDiff. Its forward trajectory is not additive Gaussian corruption but fractional application of the blur transfer function:

$Y=XK$4

For Gaussian blur this gives $Y=XK$5 with $Y=XK$6, so every intermediate state is itself a valid PSF-convolved image. The reverse model learns $Y=XK$7 with an $Y=XK$8 objective over $Y=XK$9, and blind inference re-estimates a temporary blur kernel by Wiener inversion,

$y$0

using $y$1 in the paper. On DIV2K with synthetic Gaussian blur (kernel size $y$2, $y$3), ConvDiff uses 5 steps and reports PSNR/SSIM/LPIPS of 29.59/0.7809/0.1701, compared with 25.35/0.6747/0.3167 for SR3 with 2000 additive-noise diffusion steps and 26.09/0.7038/0.2073 for INDI with 10 interpolation steps (Parvathireddy et al., 24 May 2026).

A different direct strategy is one-shot convolutional deblurring. Instead of iterative inversion, the deblurring filter is synthesized as a linear combination of FIR even-derivative filters approximating the inverse PSF over a target band,

$y$4

and then tempered by a Gaussian low-pass term to decouple denoising from edge restoration. In 2D, the restoration is applied separably, and blind Gaussian or Laplacian PSF statistics are estimated from scale-space radial spectrum ratios. Across 2054 naturally blurred images from six imaging applications, the reported average no-reference focus-quality scores are 0.9283 and 0.9265 for Gaussian PSF modeling with downsample scales $y$5 and $y$6, and 0.8792 and 0.8561 for Laplacian PSF modeling, with strong runtime advantages over iterative deconvolution baselines (Hosseini et al., 2018).

These works share a common premise: blur restoration is better aligned when the algorithmic trajectory is derived from the blur operator itself. The contrast with additive-noise diffusion is not merely implementational; it is a statement about physical validity of intermediate states.

3. Direct reconstruction from measurement space to image space

In tomographic reconstruction, “direct image reconstruction” denotes replacing iterative inversion with a learned convolutional map from projection data to images. DUG-RECON instantiates this idea with a three-stage fully convolutional pipeline: a denoising U-Net operating on sinograms, a Double U-Net Generator consisting of $y$7 and $y$8, and an 8-block residual super-resolution module trained with perceptual loss from VGG-16 features. The reconstruction target is the 2-D mapping from projection space to image space, approximating the inverse Radon transform

$y$9

without embedding explicit analytical inversion or iterative updates (Kandarpa et al., 2020).

The training objective combines denoising MSE on sinograms, $x$0 losses for $x$1 and $x$2, and a consistency term $x$3. The framework was trained on ACRIN 6688 FLT Breast PET/CT data comprising 76,000 PET slices and 21,104 CT slices, with sinograms generated by scikit-learn’s Radon transform and Poisson noise injection. Representative PET results on held-out slices report DUG at SSIM $x$4–$x$5 and RMSE $x$6–$x$7, DUG+SR at SSIM $x$8–$x$9 and RMSE $k$0–$k$1, and DeepPET at SSIM $k$2–$k$3 and RMSE $k$4–$k$5. Inference is described as instantaneous at test time, and parameter counts are reported as $k$6 for AUTOMAP, $k$7 for a patch-based Radon inversion layer, $k$8 for DeepPET, and $k$9 for DUG-RECON. The PET results are stronger than the CT results in this proof-of-concept study, with CT performance limited in part by the smaller training set and incomplete recovery of tissue and bone structures (Kandarpa et al., 2020).

In this usage, “direct image” does not mean direct spatial convolution of already formed images. It means direct inversion from acquisition coordinates into image coordinates by a convolutional generator, bypassing iterative forward/backprojection.

4. Direct convolution as a systems primitive

In systems and compiler literature, direct convolution refers to computing the sliding-window dot product on the native tensor layout, rather than expanding patches into im2col matrices or moving to FFT or Winograd transforms. The motivation is primarily architectural: for single-image inference or bandwidth-limited hardware, memory traffic and synchronization overhead often dominate arithmetic.

Three representative implementations illustrate the design space.

Method	Core mechanism	Reported outcome
ILP-M (Ji, 2019)	Threads mapped to output channels; input tiles cached in shared memory; filters loaded per thread without shared-memory filter caching	$y(r)=\int x(\tau)k(r-\tau)\,d\tau=(x*k)(r),$0 over im2col and $y(r)=\int x(\tau)k(r-\tau)\,d\tau=(x*k)(r),$1 over the fastest existing direct convolution; up to 46.8% lower execution time than Winograd on tested integrated-GPU 3×3 layers
SConv (Ferrari et al., 2023)	Convolution Slicing Analysis and Optimization with cache-aware macro-kernels and Vector-Based Packing	Packing-time reduction of 2.0×–3.9× on Intel x86 and 3.6×–7.2× on IBM POWER10; end-to-end inference speedups of 9%–25% and 10%–42%
Indirect Convolution (Dukhan, 2019)	GEMM-like microkernel fed by an indirection buffer of row pointers instead of im2col materialization	Up to 62% over GEMM-based convolution when im2col is required; minor performance reduction on 1×1 stride-1 convolutions

ILP-M targets single-image CNN inference on mobile GPUs, where thread-level parallelism is limited and latency hiding must rely on instruction-level parallelism. Its central choices are filter layout reorganization to $y(r)=\int x(\tau)k(r-\tau)\,d\tau=(x*k)(r),$2, one barrier per input tile, and mapping threads to output channels so that coalesced per-channel filter loads can be interleaved with many independent FMAs (Ji, 2019). SConv targets CPU inference through a compiler-guided direct-convolution macro-kernel. Its Convolution Slicing Analysis chooses channel, window, and filter tiles to fit L1/L2/L3 constraints, and its Vector-Based Packing uses shift-align instructions for unit-stride packing on AVX-512 and POWER10 MMA paths (Ferrari et al., 2023). Indirect Convolution retains GEMM-style microkernels but replaces im2col with an indirection buffer of pointers to NHWC channel rows, eliminating the large intermediate buffer while preserving contiguous channel loads (Dukhan, 2019).

Across these systems, the unifying principle is that direct convolution is a memory-hierarchy strategy. The principal gains come from avoiding duplicated input materialization, minimizing barrier-induced stalls, and reusing tiles at the granularity that the target architecture can sustain.

5. Adaptive and learned convolution operators

A second major meaning of “direct image” in convolution research concerns modifying the convolution operator itself while still acting directly on image coordinates. Active Convolution Units replace the fixed integer-grid stencil of standard convolution with a learned set of synapse positions $y(r)=\int x(\tau)k(r-\tau)\,d\tau=(x*k)(r),$3, shared across the layer, and evaluate

$y(r)=\int x(\tau)k(r-\tau)\,d\tau=(x*k)(r),$4

where $y(r)=\int x(\tau)k(r-\tau)\,d\tau=(x*k)(r),$5 is obtained by bilinear interpolation at fractional coordinates. Offset learning uses normalized gradients, a small position learning rate of about 0.001, and a warm-up period with frozen offsets. On CIFAR, replacing standard convolutions by ACUs reduces plain-network test error from 8.01% to 7.33% on CIFAR-10 and from 27.85% to 27.11% on CIFAR-100; on Place365, AlexNet improves from 51.68% to 52.28% top-1 accuracy and from 81.29% to 82.08% top-5 accuracy (Jeon et al., 2017).

Weighted convolution keeps the receptive field fixed but modulates each tap by a learnable density function. In the reported implementation, the density is a separable rank-1 matrix $y(r)=\int x(\tau)k(r-\tau)\,d\tau=(x*k)(r),$6 with symmetry, non-negativity, and center anchor $y(r)=\int x(\tau)k(r-\tau)\,d\tau=(x*k)(r),$7. Kernel weights $y(r)=\int x(\tau)k(r-\tau)\,d\tau=(x*k)(r),$8 are optimized by SGD, while the density parameters $y(r)=\int x(\tau)k(r-\tau)\,d\tau=(x*k)(r),$9 are optimized by DIRECT-L in a nested scheme. For image-to-image denoising, learned densities such as $Y(\omega)=X(\omega)K(\omega)$0 for $Y(\omega)=X(\omega)K(\omega)$1 produce up to 53% loss reduction relative to uniform convolution, with an average 11% forward-pass execution-time increase (Cammarasana et al., 30 May 2025).

The differentiable sparse kernel complex addresses large, non-convex, and spatially varying kernels by factorizing a dense target kernel into a sequence of sparse convolution layers,

$Y(\omega)=X(\omega)K(\omega)$2

and training offsets and weights through an impulse-response loss $Y(\omega)=X(\omega)K(\omega)$3 with Charbonnier distance to the target kernel. For spatial variation, a small basis of sparse filters is preoptimized offline and interpolated per pixel in filter space. The method reports higher fidelity than simulated annealing, lower cost than low-rank decompositions, and up to a 20× speedup over direct dense filtering on Gaussian and non-convex kernels. In this paper, “complex kernel” refers to complex shapes or PSFs rather than complex-valued coefficients (Wu et al., 4 Dec 2025).

These approaches preserve the direct action of convolution on image tensors while changing either where samples are taken, how taps are weighted, or how a dense kernel is synthesized from sparse components.

6. Alternative realizations, asynchronous processing, and terminological breadth

Direct convolution is not confined to frame-based digital image pipelines. For event cameras, spatial convolution can be maintained asynchronously as a continuous-time state. If $Y(\omega)=X(\omega)K(\omega)$4 denotes the high-pass filtered convolved image, the per-pixel state evolves by

$Y(\omega)=X(\omega)K(\omega)$5

and each incoming event updates only the support of the kernel after exact exponential decay from the last timestamp. This yields direct convolution of the event stream without synthesizing pseudo-frames and enables asynchronous Harris corner-response states for feature detection (Scheerlinck et al., 2018).

A physically different realization appears in synthetic photonic lattices. There, wave evolution in a translation-symmetric lattice implements convolution intrinsically:

$Y(\omega)=X(\omega)K(\omega)$6

Using a single electro-optic modulator and an incoherent frequency comb, the reported system performs image processing at 13.5 TOPS with about 40 ps latency per convolution step, supports separable 2D filtering of 30×30 images, and exploits complex-valued kernels for reversible unitary evolution or optical encryption (Su et al., 27 Nov 2025).

Astronomical difference image analysis exposes another domain-specific issue: not whether to convolve, but which image to convolve. The reported decision rule depends jointly on seeing and exposure depth,

$Y(\omega)=X(\omega)K(\omega)$7

rather than on FWHM alone. For DECam the combined-fields slope is reported as $Y(\omega)=X(\omega)K(\omega)$8, and for Swope as $Y(\omega)=X(\omega)K(\omega)$9 (Angulo et al., 13 Aug 2025). This directly contradicts the common heuristic that the sharper image should always be the one convolved.

Finally, the phrase has a formally unrelated meaning in arithmetic geometry. Given a finite Galois étale morphism $y[i,j]=\sum_u\sum_v x[i-u,j-v]\,k[u,v]=(x*k)[i,j],$0 of commutative group schemes and a perverse sheaf $y[i,j]=\sum_u\sum_v x[i-u,j-v]\,k[u,v]=(x*k)[i,j],$1 with property $y[i,j]=\sum_u\sum_v x[i-u,j-v]\,k[u,v]=(x*k)[i,j],$2, the convolution direct image $y[i,j]=\sum_u\sum_v x[i-u,j-v]\,k[u,v]=(x*k)[i,j],$3 is the unique perverse sheaf on $y[i,j]=\sum_u\sum_v x[i-u,j-v]\,k[u,v]=(x*k)[i,j],$4 whose pullback is the Galois-orbit convolution $y[i,j]=\sum_u\sum_v x[i-u,j-v]\,k[u,v]=(x*k)[i,j],$5. In the base-change situation $y[i,j]=\sum_u\sum_v x[i-u,j-v]\,k[u,v]=(x*k)[i,j],$6, its Frobenius traces satisfy the norm-convolution identity

$y[i,j]=\sum_u\sum_v x[i-u,j-v]\,k[u,v]=(x*k)[i,j],$7

and allied work proves semisimplicity and Frobenius semisimplicity for generalized convolution morphisms on partial affine flag varieties (Rojas-León, 2018, Cataldo et al., 2016).

Taken together, these usages show that “Convolution Direct Image” is best understood as a family of constructions organized around one idea: preserving the native convolutional or convolution-induced structure of the problem. In image restoration that means following the blur operator; in reconstruction it means learning the measurement-to-image map directly; in systems it means eliminating reshaping overhead; in adaptive operators it means modifying the kernel while remaining in image coordinates; and in arithmetic geometry it means a categorical direct-image construction built from convolution itself.