Affine Self Convolution (ASC) Overview

• ASC is a term encompassing three constructions: attention-weighted affine filtering in vision, infinite convolution in self-affine measures, and affine-group convolution in time-frequency analysis.
• In computer vision, ASC combines local self-attention with a learned affine map to yield translation-equivariant filters that reduce parameter count while enhancing performance.
• In fractal and time-frequency analysis, ASC frameworks utilize affine transformations to construct spectral measures and smooth Wigner distributions, offering new insights into signal processing.

Affine Self Convolution (ASC) is a term that appears in at least three distinct technical settings in the arXiv literature: a vision operator that combines self-attention with convolution to produce data-dependent local filters while preserving translation equivariance (Diaconu et al., 2019); an infinite convolution construction for self-affine probability measures associated with expanding integer matrices and digit sets, central to the spectral-measure theory of Hadamard triples (Dutkay et al., 2016); and the affine-group self-convolution of affine Wigner distributions on the upper half-plane, used in time-scale analysis and scalogram identities (Berge et al., 2019). The shared phrase therefore names different objects in computer vision, fractal harmonic analysis, and affine time-frequency analysis, with the common vocabulary of “affine” and “self-convolution” instantiated through different algebraic and analytic structures.

1. Terminological scope and disambiguation

In vision models, ASC is an operator on feature maps $X \in \mathbb{R}^{H\times W\times C}$ defined over spatial neighborhoods $\Omega \subset \mathbb{Z}^2$. It augments local self-attention with a learned affine map consisting of multiplicative parameters $\psi$ and additive parameters $\beta$, and the resulting operator is explicitly translation equivariant (Diaconu et al., 2019).

In harmonic analysis on self-affine sets, ASC refers to a self-affine measure

$$\mu(R,B)=\delta_{R^{-1}B} * \delta_{R^{-2}B} * \delta_{R^{-3}B} * \cdots,$$

where $R$ is an expanding integer matrix and $B$ is a finite digit set. The central question is whether $L^2(\mu)$ admits an orthonormal basis of exponentials, and the main theorem states that Hadamard triples generate spectral measures (Dutkay et al., 2016).

In affine time-scale analysis, ASC is the affine convolution of an affine Wigner distribution with itself: $$\mathrm{ASC}[f](x,a):=\big(W_{\mathrm{Aff}}^{\phi} *_{\mathrm{Aff}} W_{\mathrm{Aff}}^{\phi}\big)(x,a),$$ where convolution is taken on the affine group $\mathrm{Aff}=\mathbb{R}\times\mathbb{R}_+$ with its non-unimodular Haar structure (Berge et al., 2019).

A recurrent misconception is to treat these usages as variants of a single formalism. The cited works do not support that reading. Instead, they use the same phrase for three domain-specific constructions: local attention-weighted affine filtering, infinite convolution of rescaled atomic measures, and group convolution of affine Wigner distributions.

2. ASC as local attention-weighted affine filtering in vision

The vision formulation starts from the contrast between conventional convolution and self-attention. Convolution performs template matching with data-independent filters $\Omega \subset \mathbb{Z}^2$0, while self-attention computes data-dependent weights $\Omega \subset \mathbb{Z}^2$1 but does not explicitly perform filtering and lacks relative spatial structure unless positional information is added. ASC unifies these two mechanisms by applying a learned affine map to local neighborhoods and then using attention to modulate the resulting local responses (Diaconu et al., 2019).

The simplified affine map centered at position $\Omega \subset \mathbb{Z}^2$2 is

$\Omega \subset \mathbb{Z}^2$3

and the output is

$\Omega \subset \mathbb{Z}^2$4

with attention weights

$\Omega \subset \mathbb{Z}^2$5

Here, $\Omega \subset \mathbb{Z}^2$6 provides multiplicative template matching, $\Omega \subset \mathbb{Z}^2$7 acts as positional bias, and $\Omega \subset \mathbb{Z}^2$8 supplies the data dependence.

The practical Q/K/V version uses $\Omega \subset \mathbb{Z}^2$9 convolutions

$\psi$0

followed by separate affine maps $\psi$1, $\psi$2, and $\psi$3. Attention is then computed by comparing the center query with neighbor keys,

$\psi$4

and the output is

$\psi$5

This construction differs from standard self-attention because the aggregation is not a free linear combination of values; it is an attention-weighted affine filtering of the local neighborhood. It also differs from generic dynamic convolution because the affine parameters are tied across positions through convolutional translation and the conditioning enters through $\psi$6, which the paper identifies as a source of parameter efficiency. The paper also connects $\psi$7 to relative positional embeddings: the additive embeddings in local self-attention correspond to $\psi$8, and the product $\psi$9 appears implicitly in the score computation.

3. Equivariance, group extension, implementation, and empirical behavior

A central structural claim is that ASC is translation equivariant. With translation operator $\beta$0, the operator satisfies

$\beta$1

The proof sketch in the paper relies on three conditions: $\beta$2 and $\beta$3 are shared across spatial positions through relative offsets, the attention is normalized over local neighbors $\beta$4, and the same local rule is applied at each position (Diaconu et al., 2019).

The same paper extends ASC and Squeeze-and-Excitation to the roto-translation group $\beta$5, with practical experiments on the discrete subgroup $\beta$6, where orientations are taken in $\beta$7. The affine map becomes

$\beta$8

and the simplified group ASC is

$\beta$9

The paper proves equivariance under the action of $$\mu(R,B)=\delta_{R^{-1}B} * \delta_{R^{-2}B} * \delta_{R^{-3}B} * \cdots,$$0, and the associated group Squeeze-and-Excitation module is group-invariant because averaging over $$\mu(R,B)=\delta_{R^{-1}B} * \delta_{R^{-2}B} * \delta_{R^{-3}B} * \cdots,$$1 commutes with the group action.

Architecturally, ASC replaces the $$\mu(R,B)=\delta_{R^{-1}B} * \delta_{R^{-2}B} * \delta_{R^{-3}B} * \cdots,$$2 spatial convolution inside each bottleneck block of a CIFAR-style ResNet29, while the stem remains convolutional. When downsampling is required, the attention layer is followed by $$\mu(R,B)=\delta_{R^{-1}B} * \delta_{R^{-2}B} * \delta_{R^{-3}B} * \cdots,$$3 average pooling with stride $$\mu(R,B)=\delta_{R^{-1}B} * \delta_{R^{-2}B} * \delta_{R^{-3}B} * \cdots,$$4. The implementation uses kernel size $$\mu(R,B)=\delta_{R^{-1}B} * \delta_{R^{-2}B} * \delta_{R^{-3}B} * \cdots,$$5, $$\mu(R,B)=\delta_{R^{-1}B} * \delta_{R^{-2}B} * \delta_{R^{-3}B} * \cdots,$$6 heads, $$\mu(R,B)=\delta_{R^{-1}B} * \delta_{R^{-2}B} * \delta_{R^{-3}B} * \cdots,$$7 Q/K/V projections, batch norms, He initialization for convolutions, $$\mu(R,B)=\delta_{R^{-1}B} * \delta_{R^{-2}B} * \delta_{R^{-3}B} * \cdots,$$8, Nesterov momentum $$\mu(R,B)=\delta_{R^{-1}B} * \delta_{R^{-2}B} * \delta_{R^{-3}B} * \cdots,$$9, weight decay $R$0, and learning-rate warmup for $R$1 epochs to $R$2.

On CIFAR10, the reported mean $R$3 std over $R$4 runs are: ResNet29 $R$5 with $R$6k parameters, ResNet29+Simple_ASC $R$7 with $R$8k, ResNet29+ASC $R$9 with $B$0k, and $B$1ResNet29+ASC $B$2 with $B$3k. On CIFAR100, ResNet29 obtains $B$4 with $B$5k parameters, ResNet29+ASC $B$6 with $B$7k, $B$8ResNet29 $B$9 with $L^2(\mu)$0k, and $L^2(\mu)$1ResNet29+ASC $L^2(\mu)$2 with $L^2(\mu)$3k. The paper’s stated interpretation is that ASC variants consistently reduce parameter counts compared to convolutional baselines while achieving comparable or better accuracy, whereas SASA underperforms the baseline, indicating that the affine map is important.

4. ASC as self-affine measure construction and spectrality theory

In the self-affine-measure literature, the basic objects are an expanding integer matrix $L^2(\mu)$4 and finite digit sets $L^2(\mu)$5 with $L^2(\mu)$6. The affine maps are

$L^2(\mu)$7

and the corresponding ASC measure is

$L^2(\mu)$8

Equivalently, $L^2(\mu)$9 is the unique Borel probability measure satisfying

$$\mathrm{ASC}[f](x,a):=\big(W_{\mathrm{Aff}}^{\phi} *_{\mathrm{Aff}} W_{\mathrm{Aff}}^{\phi}\big)(x,a),$$0

and its support is the self-affine attractor

$$\mathrm{ASC}[f](x,a):=\big(W_{\mathrm{Aff}}^{\phi} *_{\mathrm{Aff}} W_{\mathrm{Aff}}^{\phi}\big)(x,a),$$1

The measure is spectral if there exists a countable set $$\mathrm{ASC}[f](x,a):=\big(W_{\mathrm{Aff}}^{\phi} *_{\mathrm{Aff}} W_{\mathrm{Aff}}^{\phi}\big)(x,a),$$2 such that $$\mathrm{ASC}[f](x,a):=\big(W_{\mathrm{Aff}}^{\phi} *_{\mathrm{Aff}} W_{\mathrm{Aff}}^{\phi}\big)(x,a),$$3 is an orthonormal basis of $$\mathrm{ASC}[f](x,a):=\big(W_{\mathrm{Aff}}^{\phi} *_{\mathrm{Aff}} W_{\mathrm{Aff}}^{\phi}\big)(x,a),$$4 (Dutkay et al., 2016).

The relevant algebraic condition is the Hadamard triple property. The triple $$\mathrm{ASC}[f](x,a):=\big(W_{\mathrm{Aff}}^{\phi} *_{\mathrm{Aff}} W_{\mathrm{Aff}}^{\phi}\big)(x,a),$$5 is Hadamard if

$$\mathrm{ASC}[f](x,a):=\big(W_{\mathrm{Aff}}^{\phi} *_{\mathrm{Aff}} W_{\mathrm{Aff}}^{\phi}\big)(x,a),$$6

is unitary. The paper proves the main theorem that if $$\mathrm{ASC}[f](x,a):=\big(W_{\mathrm{Aff}}^{\phi} *_{\mathrm{Aff}} W_{\mathrm{Aff}}^{\phi}\big)(x,a),$$7 is a Hadamard triple, then the ASC self-affine measure $$\mathrm{ASC}[f](x,a):=\big(W_{\mathrm{Aff}}^{\phi} *_{\mathrm{Aff}} W_{\mathrm{Aff}}^{\phi}\big)(x,a),$$8 is spectral, thereby settling the Jorgensen–Pedersen conjecture in full generality for $$\mathrm{ASC}[f](x,a):=\big(W_{\mathrm{Aff}}^{\phi} *_{\mathrm{Aff}} W_{\mathrm{Aff}}^{\phi}\big)(x,a),$$9.

Fourier analysis enters through the mask function

$\mathrm{Aff}=\mathbb{R}\times\mathbb{R}_+$0

for which

$\mathrm{Aff}=\mathbb{R}\times\mathbb{R}_+$1

Hadamard unitarity yields the partition-of-unity identity

$\mathrm{Aff}=\mathbb{R}\times\mathbb{R}_+$2

which is crucial for orthogonality and completeness arguments.

When the periodic zero set

$\mathrm{Aff}=\mathbb{R}\times\mathbb{R}_+$3

is empty, one may take the explicit tower spectrum

$\mathrm{Aff}=\mathbb{R}\times\mathbb{R}_+$4

When $\mathrm{Aff}=\mathbb{R}\times\mathbb{R}_+$5, the paper derives a quasi-product structure after a unimodular change of coordinates and proves spectrality by product-spectrum methods. It also introduces an almost-Parseval-frame condition which, under $\mathrm{Aff}=\mathbb{R}\times\mathbb{R}_+$6, yields Fourier frames rather than orthonormal bases.

The paper’s examples include the one-fourth Cantor measure with $\mathrm{Aff}=\mathbb{R}\times\mathbb{R}_+$7, $\mathrm{Aff}=\mathbb{R}\times\mathbb{R}_+$8, $\mathrm{Aff}=\mathbb{R}\times\mathbb{R}_+$9, which is spectral with

$\Omega \subset \mathbb{Z}^2$00

and the middle-third Cantor measure with $\Omega \subset \mathbb{Z}^2$01, $\Omega \subset \mathbb{Z}^2$02, presented as non-spectral but potentially frame-admitting under almost-Parseval towers.

5. ASC on the affine group and affine Wigner analysis

In affine Wigner analysis, the ambient group is

$\Omega \subset \mathbb{Z}^2$03

with inverse $\Omega \subset \mathbb{Z}^2$04, left Haar measure $\Omega \subset \mathbb{Z}^2$05, right Haar measure $\Omega \subset \mathbb{Z}^2$06, and modular function $\Omega \subset \mathbb{Z}^2$07 (Berge et al., 2019). The affine convolution is

$\Omega \subset \mathbb{Z}^2$08

The affine Wigner distribution for $\Omega \subset \mathbb{Z}^2$09 is

$\Omega \subset \mathbb{Z}^2$10

where

$\Omega \subset \mathbb{Z}^2$11

Its diagonal version $\Omega \subset \mathbb{Z}^2$12 satisfies the orthogonality and marginal identities

$\Omega \subset \mathbb{Z}^2$13

$\Omega \subset \mathbb{Z}^2$14

The paper’s convolution theorem states that for $\Omega \subset \mathbb{Z}^2$15, with $\Omega \subset \mathbb{Z}^2$16 and $\Omega \subset \mathbb{Z}^2$17 supported in $\Omega \subset \mathbb{Z}^2$18,

$\Omega \subset \mathbb{Z}^2$19

where $\Omega \subset \mathbb{Z}^2$20 is the right involution

$\Omega \subset \mathbb{Z}^2$21

Within this framework, ASC is defined by

$\Omega \subset \mathbb{Z}^2$22

A Mellin-domain description is obtained via the affine ambiguity function

$\Omega \subset \mathbb{Z}^2$23

and the multiplier

$\Omega \subset \mathbb{Z}^2$24

The paper proves

$\Omega \subset \mathbb{Z}^2$25

This gives ASC a natural FFT/FFTLog implementation route on a uniform $\Omega \subset \mathbb{Z}^2$26-grid and a logarithmic $\Omega \subset \mathbb{Z}^2$27-grid.

Two structural properties are emphasized. First, no affine Wigner distribution is analytic, because $\Omega \subset \mathbb{Z}^2$28 contains no analytic or anti-analytic functions; ASC therefore also lives outside holomorphic frameworks. Second, positivity is exceptional: the Morse ground state

$\Omega \subset \mathbb{Z}^2$29

has nonnegative affine Wigner distribution, and the paper formulates an affine positivity conjecture asserting that generalized Klauder wavelets are the only $\Omega \subset \mathbb{Z}^2$30 functions with this property.

6. Comparative perspective, limitations, and open problems

Across the three literatures, “affine” refers to different transformations. In vision ASC, it denotes a learned local affine map $\Omega \subset \mathbb{Z}^2$31 translated across the spatial lattice (Diaconu et al., 2019). In self-affine measure theory, it refers to an iterated function system generated by $\Omega \subset \mathbb{Z}^2$32 and the resulting infinite convolution of atomic measures (Dutkay et al., 2016). In affine Wigner analysis, it refers to the noncommutative $\Omega \subset \mathbb{Z}^2$33 group and its Haar-compatible convolution law (Berge et al., 2019). This suggests that the common label is terminological rather than structural.

The limitations are equally domain-specific. For vision ASC, the paper states that attention-based implementations are slower than standard convolution, that the presented operator is local and therefore does not directly model very long-range dependencies, that the practical rotation-equivariant experiments use $\Omega \subset \mathbb{Z}^2$34 rather than continuous $\Omega \subset \mathbb{Z}^2$35, and that richer affine maps than the channel-wise $\Omega \subset \mathbb{Z}^2$36 used in experiments would increase cost. For self-affine measures, the stated assumptions include expanding integer $\Omega \subset \mathbb{Z}^2$37, finite integer digit sets, equal weights, and, frequently, no-overlap conditions; the open problems include whether every spectral self-affine measure arises from a Hadamard triple, whether spectral measures necessarily have no overlap, and how to construct explicit almost-Parseval towers for non-spectral fractals. For affine Wigner analysis, the paper highlights non-commutativity, the need to track involution and modular factors in convolution, the non-analyticity of all affine Wigner distributions, and the positivity conjecture as an unresolved issue.

The three uses of ASC therefore occupy different research programs: equivariant local operators for vision, Fourier bases and frames for self-affine fractal measures, and affine-group smoothing of quadratic time-scale representations. Their intersection lies mainly in vocabulary and in the broad idea that convolution can be modified by an affine structure; their definitions, objectives, and proof techniques are otherwise distinct.