Direction-Aware Convolution

• Direction-aware convolution refers to methods that embed explicit directional priors into convolution operations to capture anisotropic and orientation-dependent features.
• These techniques utilize anisotropic kernels, directional sampling, and self-attention to improve performance across tasks like image restoration, depth estimation, and segmentation.
• The approach spans diverse domains—from Euclidean vision and geometric deep learning to astronomical analysis—highlighting its adaptability over traditional isotropic filtering.

Searching arXiv for the cited papers and closely related direction-aware convolution work. Direction-aware convolution denotes a family of convolutional constructions in which directional asymmetry is made explicit rather than treated as an incidental by-product of isotropic filtering. In the cited arXiv literature, the phrase spans several distinct but related ideas: feature decomposition that is sensitive to rain orientation in single-image deraining, direction-adjusted sampling and directional aggregation in self-supervised monocular depth estimation, orthogonal directional kernels for segmentation, orientation-modulated filters for recognition, and directional signal propagation on surfaces and point clouds through equivariant or anisotropic operators (Deng et al., 2021, Han et al., 2023, Xu et al., 2024, Qin et al., 2021, Poulenard et al., 2018, Wiersma et al., 2021). In a broader mathematical and domain-specific sense, “direction” also appears in convolution theory for harmonic mappings convex in one direction and in astronomical difference image analysis, where the principal question is which image should be convolved (Kumar et al., 2014, Angulo et al., 13 Aug 2025).

1. Scope and meanings of the term

The literature does not use “direction-aware convolution” to denote a single canonical operator. Instead, the same phrase refers to several mechanisms that encode directional priors at different levels of abstraction.

Context	Directional mechanism	Representative paper
Euclidean vision	Anisotropic kernels, oriented filter banks, affine resampling, directional routing	(Deng et al., 2021, Han et al., 2023, Xu et al., 2024, Qin et al., 2021)
Geometric deep learning	Directional functions, equivariant angular channels, scalar/vector streams	(Poulenard et al., 2018, Wiersma et al., 2021)
Other uses of “convolution direction”	Convexity in one direction; choosing which image to convolve	(Kumar et al., 2014, Angulo et al., 13 Aug 2025)

A common misconception is that direction-aware convolution is merely a larger or elongated kernel. The surveyed works collectively indicate a broader design space. Some methods alter the sampling grid before convolution, some maintain explicit directional state across layers, some use orthogonal basis decompositions, and some formulate anisotropy through geometric differential operators rather than image-space kernels (Han et al., 2023, Poulenard et al., 2018, Wiersma et al., 2021). Another recurring distinction is between direction-aware processing and generic attention: several papers explicitly build direction into the convolutional operator itself rather than treating it as an auxiliary weighting mechanism (Xu et al., 2024, Qin et al., 2021).

2. Feature-level directional decomposition in image restoration

In single-image deraining, direction-aware convolution appears as the direction-aware Cross-Median Filter (dCMF) inside a dual-branch frequency decomposition network with a detail learning branch and a structure learning branch (Deng et al., 2021). The paper’s central claim is that rain is not merely high frequency; it is directional high-frequency structure. Ordinary convolution uses isotropic 2D kernels that respond similarly in all directions, whereas rain streaks are elongated and directionally coherent. The proposed network therefore performs frequency decomposition at feature level rather than image level and inserts dCMF into the cross-branch frequency communication paths HILO (“High In Low Out”) and LIHO (“Low In High Out”) (Deng et al., 2021).

The dCMF modifies naive low-pass filtering in three ways. First, it uses median pooling instead of average pooling or averaging-based filtering, with the stated purpose of reducing the influence of outliers caused by rain streaks. Second, it replaces a conventional 2D isotropic kernel with 1D directional lines, motivated by the claim that a line kernel orthogonal to the rain direction is least affected by streak pixels. The paper extends a single CMF operator to a set

$\{CMF_1, CMF_2, CMF_3\},$

each corresponding to a different direction. Third, because rain direction is scene-dependent, it applies self-attention over the directional branches: the filtering results are aggregated by addition and global average pooling to compute attentions for each group, and a weighted sum forms the final dCMF output (Deng et al., 2021).

The “cross” in Cross-Median Filter is operationally important. Each 1D kernel is followed by a crisscross counterpart, yielding a two-step cross-shaped median filtering scheme. The paper states that each CMF has the same receptive field as the corresponding 2D kernel, but is more robust to rain because the second kernel operates after the first has already reduced much of the rain pattern. Within the network, dCMF is not a universal replacement for all convolutions; it is the explicit frequency separator on the information exchange path between the two branches. In HILO, the low-frequency output of dCMF is adapted by channel-wise attention using Squeeze-and-Excitation before being sent to the structure branch. In LIHO, the residual of the dCMF output is sent to the detail branch as the high-frequency component (Deng et al., 2021).

This separation is supervised explicitly. The branch losses and reconstruction loss are

$$L_d = \| f(\mathbf{I}) - \tilde{\mathbf{D}} \|_1,\qquad L_s = \| g(\mathbf{I}) - \tilde{\mathbf{S}} \|_2,$$

$$\mathbf{I}_p = f(\mathbf{I}) + g(\mathbf{I}), \qquad L_c = \lambda_1 L_d + \lambda_2 L_s + \lambda_3 L_r,$$

with $\lambda_1=\lambda_2=\lambda_3=1$ in experiments, and

$$L_r = \|\mathbf{I}_p-\mathbf{I}_{gt}\|_1 + 1 - SSIM(\mathbf{I}_p,\mathbf{I}_{gt}).$$

The ablation reported on Rain200H directly compares dCMF with generic low-pass alternatives: a $3\times 3$ median filter yields PSNR 27.86 and SSIM 0.9290, a Gaussian filter yields PSNR 22.58 and SSIM 0.8064, and dCMF yields PSNR 28.09 and SSIM 0.9316 (Deng et al., 2021). Within the paper’s design logic, this indicates that directional median-based decomposition is better aligned with rain anisotropy than generic filtering.

3. Direction-adjusted sampling and cumulative aggregation for monocular depth

In self-supervised monocular depth estimation, direction-aware convolution is motivated by two empirical claims: feature representation is direction-sensitive and environment-dependent, and the most important contextual cues for depth often lie in the connection region between the camera and an object, especially in the vertical direction (Han et al., 2023). The Direction-aware Cumulative Convolution Network (DaCCN) addresses these observations with two components: a direction-aware module (DaM) for feature extraction and a cumulative convolution (CC) for feature aggregation (Han et al., 2023).

DaM is inserted into each encoder branch and consists of an affinity transformation $\mathbf{A}$, a convolutional feature extractor $\mathcal{F}$, and a back-projection transformation $\mathbf{A}^{-1}$:

$$\mathbf{x} = \mathbf{A}^{-1}\,\mathcal{F}(\mathbf{A}\mathbf{I}).$$

The affine transform is

$$L_d = \| f(\mathbf{I}) - \tilde{\mathbf{D}} \|_1,\qquad L_s = \| g(\mathbf{I}) - \tilde{\mathbf{S}} \|_2,$$0

where $$L_d = \| f(\mathbf{I}) - \tilde{\mathbf{D}} \|_1,\qquad L_s = \| g(\mathbf{I}) - \tilde{\mathbf{S}} \|_2,$$1 and $$L_d = \| f(\mathbf{I}) - \tilde{\mathbf{D}} \|_1,\qquad L_s = \| g(\mathbf{I}) - \tilde{\mathbf{S}} \|_2,$$2 are learnable scaling factors in the horizontal and vertical directions. The paper’s interpretation is that the module reallocates sampling density across directions: directions needing more detailed information should receive denser sampling and a smaller effective receptive field, whereas directions needing more global context should receive sparser sampling and a larger effective receptive field. The back-projection restores consistency with the original image coordinate system after convolution in the transformed space (Han et al., 2023). The paper explicitly contrasts this mechanism with deformable convolution, arguing that deformable convolution learns spatial offsets but does not directly control direction-wise sampling density.

The second component, cumulative convolution, aggregates environmental information asymmetrically rather than through a square neighborhood centered on each pixel. It is defined by

$$L_d = \| f(\mathbf{I}) - \tilde{\mathbf{D}} \|_1,\qquad L_s = \| g(\mathbf{I}) - \tilde{\mathbf{S}} \|_2,$$3

The accumulator performs a cumulative sum from the bottom row upward:

$$L_d = \| f(\mathbf{I}) - \tilde{\mathbf{D}} \|_1,\qquad L_s = \| g(\mathbf{I}) - \tilde{\mathbf{S}} \|_2,$$4

and normalization divides by row extent:

$$L_d = \| f(\mathbf{I}) - \tilde{\mathbf{D}} \|_1,\qquad L_s = \| g(\mathbf{I}) - \tilde{\mathbf{S}} \|_2,$$5

The geometric rationale given in the paper is that points in the connection region tend to project to larger $$L_d = \| f(\mathbf{I}) - \tilde{\mathbf{D}} \|_1,\qquad L_s = \| g(\mathbf{I}) - \tilde{\mathbf{S}} \|_2,$$6-coordinates, so bottom-up accumulation captures relevant context efficiently (Han et al., 2023).

These modules are trained end-to-end under the standard self-supervised photometric reprojection loss and smoothness regularization. In the KITTI ablation, the baseline reports Abs Rel 0.102, RMSE 4.483, and $$L_d = \| f(\mathbf{I}) - \tilde{\mathbf{D}} \|_1,\qquad L_s = \| g(\mathbf{I}) - \tilde{\mathbf{S}} \|_2,$$7 0.896; adding DaM gives Abs Rel 0.100, RMSE 4.372, and $$L_d = \| f(\mathbf{I}) - \tilde{\mathbf{D}} \|_1,\qquad L_s = \| g(\mathbf{I}) - \tilde{\mathbf{S}} \|_2,$$8 0.899; adding CC gives Abs Rel 0.101, RMSE 4.380, and $$L_d = \| f(\mathbf{I}) - \tilde{\mathbf{D}} \|_1,\qquad L_s = \| g(\mathbf{I}) - \tilde{\mathbf{S}} \|_2,$$9 0.897; and adding both yields Abs Rel 0.099, RMSE 4.316, and $$\mathbf{I}_p = f(\mathbf{I}) + g(\mathbf{I}), \qquad L_c = \lambda_1 L_d + \lambda_2 L_s + \lambda_3 L_r,$$0 0.897 (Han et al., 2023). The paper interprets these gains as evidence that depth estimation benefits from directional feature extraction and from asymmetric aggregation of environmental context.

4. Orthogonal directional bases and orientation-modulated filters

For polyp segmentation, ODC-SA Net introduces an Orthogonal Direction Convolutional (ODC) block that extracts horizontal and vertical directional responses using transposed rectangular kernels and treats them as an orthogonal feature vector basis (Xu et al., 2024). The deepest encoder feature map, after a preceding RFB stage, is represented as

$$\mathbf{I}_p = f(\mathbf{I}) + g(\mathbf{I}), \qquad L_c = \lambda_1 L_d + \lambda_2 L_s + \lambda_3 L_r,$$1

and is passed into two directional branches. The horizontal branch applies a 3-layer $$\mathbf{I}_p = f(\mathbf{I}) + g(\mathbf{I}), \qquad L_c = \lambda_1 L_d + \lambda_2 L_s + \lambda_3 L_r,$$2 convolution stack,

$$\mathbf{I}_p = f(\mathbf{I}) + g(\mathbf{I}), \qquad L_c = \lambda_1 L_d + \lambda_2 L_s + \lambda_3 L_r,$$3

and the vertical branch applies a 3-layer $$\mathbf{I}_p = f(\mathbf{I}) + g(\mathbf{I}), \qquad L_c = \lambda_1 L_d + \lambda_2 L_s + \lambda_3 L_r,$$4 stack,

$$\mathbf{I}_p = f(\mathbf{I}) + g(\mathbf{I}), \qquad L_c = \lambda_1 L_d + \lambda_2 L_s + \lambda_3 L_r,$$5

The paper frames the resulting features as orthogonal basis components:

$$\mathbf{I}_p = f(\mathbf{I}) + g(\mathbf{I}), \qquad L_c = \lambda_1 L_d + \lambda_2 L_s + \lambda_3 L_r,$$6

Operationally, ODC expands the directional features to a richer representation and then recombines them with the original deep feature map through a residual-style fusion (Xu et al., 2024).

The paper’s stated distinction from standard convolution is threefold: rectangular anisotropic kernels, orthogonal decomposition, and efficiency through structured directional branches rather than deeper networks or larger square kernels. In ablation, removing ODC reduces mDice from 0.803 to 0.788 on ETIS-LaribPolypDB and from 0.943 to 0.937 on CVC-ClinicDB (Xu et al., 2024). The full model reports ClinicDB mDice 0.943 and mIoU 0.899, Kvasir mDice 0.921 and mIoU 0.875, ETIS mDice 0.803 and mIoU 0.728, CVC-300 mDice 0.902 and mIoU 0.836, and ColonDB mDice 0.810 and mIoU 0.730 (Xu et al., 2024).

For image recognition, Orientation Convolutional Networks (OCNs) introduce orientation-aware convolution by modulating learned filters with Landmark Gabor Filters (LGFs) (Qin et al., 2021). The LGF construction begins from a dense oriented Gabor filter bank

$$\mathbf{I}_p = f(\mathbf{I}) + g(\mathbf{I}), \qquad L_c = \lambda_1 L_d + \lambda_2 L_s + \lambda_3 L_r,$$7

and learns a smaller set of representative filters through a structured factorization

$$\mathbf{I}_p = f(\mathbf{I}) + g(\mathbf{I}), \qquad L_c = \lambda_1 L_d + \lambda_2 L_s + \lambda_3 L_r,$$8

with low-rank and sparse regularization. The relaxed learning objective is

$$\mathbf{I}_p = f(\mathbf{I}) + g(\mathbf{I}), \qquad L_c = \lambda_1 L_d + \lambda_2 L_s + \lambda_3 L_r,$$9

The effective convolution weights are then modulated as

$\lambda_1=\lambda_2=\lambda_3=1$0

and the output feature map is

$\lambda_1=\lambda_2=\lambda_3=1$1

The paper presents OCNs as a drop-in replacement for standard convolution layers, with max-pooling, ReLU, and residual blocks left unchanged, and argues that the resulting models are less sensitive to orientation changes and have few parameters to learn (Qin et al., 2021).

Taken together, ODC and OCN exemplify two different Euclidean-grid interpretations of direction-aware convolution: one uses explicitly orthogonal spatial branches as a compact directional basis, and the other uses orientation-modulated filter banks derived from structured factorization (Xu et al., 2024, Qin et al., 2021).

5. Directional convolution on surfaces and point clouds

On curved surfaces, the absence of a globally consistent tangent-frame orientation makes direction-aware convolution fundamentally different from image-space anisotropy. “Multi-directional Geodesic Neural Networks via Equivariant Convolution” addresses this by replacing scalar surface signals with directional functions

$\lambda_1=\lambda_2=\lambda_3=1$2

and defining directional convolution through the completed exponential map (Poulenard et al., 2018). For a directional function $\lambda_1=\lambda_2=\lambda_3=1$3 and template $\lambda_1=\lambda_2=\lambda_3=1$4, the operator is

$\lambda_1=\lambda_2=\lambda_3=1$5

A central proposition shows that earlier geodesic convolution is recovered by max pooling over directions:

$\lambda_1=\lambda_2=\lambda_3=1$6

This establishes the paper’s main conceptual claim: preserving the full angular response across layers is strictly more informative than collapsing orientation immediately by a max over alignments (Poulenard et al., 2018). The method is further shown to be equivariant to local reference-frame changes, with frame rotations producing circular shifts in angular channels rather than destroying consistency.

The discrete implementation on triangle meshes uses geodesic polar coordinates, barycentric interpolation, and a discretized parallel transport tensor. Empirically, the paper reports substantial gains over GCNN on CIFAR-10 mapped to meshes, with GCNN $\lambda_1=\lambda_2=\lambda_3=1$7 versus MDGCNN $\lambda_1=\lambda_2=\lambda_3=1$8 on sphere embeddings and GCNN $\lambda_1=\lambda_2=\lambda_3=1$9 versus MDGCNN $$L_r = \|\mathbf{I}_p-\mathbf{I}_{gt}\|_1 + 1 - SSIM(\mathbf{I}_p,\mathbf{I}_{gt}).$$0 on grid embeddings. On human body segmentation, examples include 3D coordinates: GCNN $$L_r = \|\mathbf{I}_p-\mathbf{I}_{gt}\|_1 + 1 - SSIM(\mathbf{I}_p,\mathbf{I}_{gt}).$$1, MDGCNN $$L_r = \|\mathbf{I}_p-\mathbf{I}_{gt}\|_1 + 1 - SSIM(\mathbf{I}_p,\mathbf{I}_{gt}).$$2; SHOT$$L_r = \|\mathbf{I}_p-\mathbf{I}_{gt}\|_1 + 1 - SSIM(\mathbf{I}_p,\mathbf{I}_{gt}).$$3: GCNN $$L_r = \|\mathbf{I}_p-\mathbf{I}_{gt}\|_1 + 1 - SSIM(\mathbf{I}_p,\mathbf{I}_{gt}).$$4, MDGCNN $$L_r = \|\mathbf{I}_p-\mathbf{I}_{gt}\|_1 + 1 - SSIM(\mathbf{I}_p,\mathbf{I}_{gt}).$$5; and SHOT$$L_r = \|\mathbf{I}_p-\mathbf{I}_{gt}\|_1 + 1 - SSIM(\mathbf{I}_p,\mathbf{I}_{gt}).$$6: GCNN $$L_r = \|\mathbf{I}_p-\mathbf{I}_{gt}\|_1 + 1 - SSIM(\mathbf{I}_p,\mathbf{I}_{gt}).$$7, MDGCNN $$L_r = \|\mathbf{I}_p-\mathbf{I}_{gt}\|_1 + 1 - SSIM(\mathbf{I}_p,\mathbf{I}_{gt}).$$8 (Poulenard et al., 2018).

For point clouds, DeltaConv constructs anisotropic convolution layers from vector-calculus operators acting on scalar fields and tangential vector fields (Wiersma et al., 2021). The method maintains a scalar stream and a vector stream connected by gradient, co-gradient, divergence, curl, norm, identity, and Hodge Laplacian. The Hodge Laplacian is written as

$$L_r = \|\mathbf{I}_p-\mathbf{I}_{gt}\|_1 + 1 - SSIM(\mathbf{I}_p,\mathbf{I}_{gt}).$$9

with discrete form

$3\times 3$0

A cleaned-up form of the layer update given in the paper is

$3\times 3$1

$3\times 3$2

The paper explicitly invokes Perona–Malik anisotropic diffusion as a conceptual template and emphasizes that no global tangent coordinate system is required (Wiersma et al., 2021).

DeltaConv reports ModelNet40 overall accuracy 93.8%, ScanObjectNN up to 84.7%, ShapeNet part segmentation 86.9 mIoU with the U-ResNet variant, and SHREC11 99.6% (Wiersma et al., 2021). It also reports that the vector stream reduces error by up to 25% relative on ModelNet40 and up to 21% relative on ShapeNet in ablations, and gives inference about 1.5–2× faster and backward pass about 2.5–30× faster compared with edge-based features, depending on the variant (Wiersma et al., 2021).

6. Mathematical and domain-specific alternative meanings

Outside modern neural architectures, convolution and direction meet in substantially different ways. In the theory of univalent harmonic mappings convex in one direction, the relevant object is the convolution of analytic and harmonic maps in the open unit disk $3\times 3$3 (Kumar et al., 2014). A harmonic map is written as $3\times 3$4, with dilatation

$3\times 3$5

and is locally univalent and sense-preserving if $3\times 3$6 and $3\times 3$7 in $3\times 3$8. The paper builds on earlier results of Pokhrel and uses the Hengartner–Schober criterion and the Clunie–Sheil-Small lemma. Its main theorem states that if

$3\times 3$9

and if there exist $\mathbf{A}$0 with $\mathbf{A}$1 such that

$\mathbf{A}$2

then $\mathbf{A}$3 for every analytic $\mathbf{A}$4 (Kumar et al., 2014). Here “direction” refers to convexity in the direction $\mathbf{A}$5, not to anisotropic feature extraction.

An even more domain-specific usage appears in astronomical difference image analysis. “Optimizing Convolution Direction and Template Selection for Difference Image Analysis” studies which image should be convolved before subtraction, rather than how a learned kernel should encode orientation (Angulo et al., 13 Aug 2025). For image pair subtraction

$\mathbf{A}$6

the paper introduces a diagnostic based on reduced chi-square and

$\mathbf{A}$7

and argues that the optimal convolution direction depends on both FWHM and depth $\mathbf{A}$8, not on seeing alone. The empirical boundary is a sloped line in the $\mathbf{A}$9–$\mathcal{F}$0 plane rather than a vertical line, and for DECam the combined fit gives $\mathcal{F}$1, while Swope fields yield roughly $\mathcal{F}$2–$\mathcal{F}$3 (Angulo et al., 13 Aug 2025). The same paper defines a Figure-of-Merit for template ranking,

$\mathcal{F}$4

with lower or more negative values preferred (Angulo et al., 13 Aug 2025).

These cases illustrate that “direction-aware convolution” is not semantically uniform across disciplines. In complex analysis it concerns geometric properties preserved by convolution; in astronomical subtraction it concerns the direction of kernel matching between images; in deep learning it usually concerns anisotropic representation learning.

7. Recurrent principles, misconceptions, and limitations

Across the neural-network literature surveyed here, three recurrent design principles appear. First, directional asymmetry is often encoded by changing support geometry or sampling geometry: 1D directional median lines in dCMF, affine scaling by $\mathcal{F}$5 and $\mathcal{F}$6 in DaM, and $\mathcal{F}$7 versus $\mathcal{F}$8 kernels in ODC (Deng et al., 2021, Han et al., 2023, Xu et al., 2024). Second, several methods preserve directional state explicitly instead of collapsing it immediately: MDGCNN keeps angular channels across layers, and DeltaConv maintains a vector stream of tangential features (Poulenard et al., 2018, Wiersma et al., 2021). Third, direction selection is frequently learned rather than fixed, as in self-attention over dCMF directions and LGF-based orientation modulation in OCNs (Deng et al., 2021, Qin et al., 2021).

A common misunderstanding is that all such methods aim at the same invariance objective. The cited papers suggest otherwise. Some methods seek robustness to orientation changes in recognition, some seek better disentanglement of directional nuisance structure such as rain, some seek exploitation of directional depth cues, and some seek intrinsic anisotropy on manifolds or point clouds (Qin et al., 2021, Deng et al., 2021, Han et al., 2023, Poulenard et al., 2018, Wiersma et al., 2021). Another misconception is that direction-aware processing must replace an entire backbone. The deraining work explicitly places dCMF in cross-branch communication rather than “as a generic backbone convolution block,” and the depth work inserts DaM at the start of each encoder branch while keeping a standard encoder-decoder structure (Deng et al., 2021, Han et al., 2023).

A plausible implication is that the term is best understood as a design principle rather than a single layer type. The principle is to embed directional priors—orientation coherence, anisotropic context, or tangent-space directionality—into convolutional processing in a way that ordinary isotropic kernels do not. The cited works differ sharply in mathematical formalism, but they converge on the same underlying claim: when the signal, nuisance process, or geometry is direction-dependent, explicitly modeling direction can improve representation quality, stability, or task performance (Deng et al., 2021, Han et al., 2023, Xu et al., 2024, Poulenard et al., 2018, Wiersma et al., 2021).