Core AOT Block Overview

• Core AOT Block is a computational module that unifies multi-scale aggregation, adaptive filtering, and causal optimization across diverse domains.
• It leverages modular constructs such as multi-branch convolutions, adaptive pivoting, and mixnet designs to address challenges in image inpainting, graph algorithms, and secure communications.
• Its implementation enhances efficiency and scalability in applications ranging from high-resolution image inpainting to transformer modeling and adapted optimal transport.

A Core AOT Block refers to the atomic algorithmic module or architectural motif underlying a diverse set of contemporary frameworks, each denoted by the acronym “AOT” in their respective domains: image inpainting (Aggregated Contextual Transformation), triangle listing (Adaptive Orientation Technique), anonymous communication (Anonymization by Oblivious Transfer), transformer models (Attention-Only Transformers), and adapted optimal transport (Adapted Optimal Transport). While differing substantially in specifics and context, these “core blocks” manifest as the minimal algorithmic or architectural unit encapsulating the defining computational, combinatorial, or statistical principle of the respective AOT approach.

1. Aggregated Contextual Transformation in Image Inpainting

The AOT block in high-resolution image inpainting as described in “Aggregated Contextual Transformations for High-Resolution Image Inpainting” implements a multi-branch, multi-scale convolutional operator designed for superior context aggregation and local pattern synthesis (Zeng et al., 2021). The block addresses two challenges: extracting long-range structure for global consistency, and generating fine, non-repetitive texture for local realism.

Structure and Forward Pass

Given an input feature tensor $x \in \mathbb{R}^{C \times H \times W}$:

• Split: Partition along channel dimension into $M$ branches, each convolved with a $3 \times 3$ kernel and distinct dilation rate $d_i$.
• Transform: Each branch applies a dilated convolution and ReLU activation:

$$t_i = \mathrm{ReLU}\bigl( \mathrm{Conv2D}(x; W_i, d_i) \bigr)$$

• Merge: Concatenate all branches, then fuse via another convolution and ReLU:

$$t_{\mathrm{cat}} = [t_1, ..., t_M], \quad r = \mathrm{ReLU}(\mathrm{Conv2D}(t_{\mathrm{cat}}; W_{\text{fuse}}, 1))$$

• Spatially-Variant Gated Residual: Compute a 1-channel mask gate

$g = \sigma(\mathrm{Conv2D}(x; W_g, 1))$

and combine:

$x_{\mathrm{out}} = x \cdot g + r \cdot (1 - g)$

where $\sigma$ is the logistic sigmoid, and $\odot$ denotes broadcasted multiplication.

This structure enables per-pixel, per-channel adaptive blending between the input and the aggregated multi-scale residual, crucial for edge coherence and preventing texture bleeding.

Integration and Stacking

A generator network sequences multiple such AOT blocks—empirically, $K=8$ is effective—enabling the receptive field to grow multiplicatively and permitting both fine and global structure to be preserved throughout the inpainting pipeline. All convolutions are followed by ReLU (sigmoid for the gate); normalization is omitted, and each block operates independently in parameterization.

2. Adaptive Orientation Technique in Triangle Listing

Within fast triangle listing algorithms on graphs, the core AOT block (Adaptive Orientation Technique) combines directed acyclic orientation with adaptive per-edge pivoting, providing the tightest in-memory triangle listing complexity to date (Yu et al., 2020).

Principle and Algorithm

• Orientation: Assign a total order $\eta$ to the vertices, orient edges $u \rightarrow v$ iff $\eta(u) < \eta(v)$.
• Adaptive Pivoting: For each oriented edge $\langle u, v \rangle$, pivot at the endpoint with smaller out-degree—without altering the global acyclic orientation.
• Computation: For each vertex $u$, build a hash table of its out-neighbors $N^+(u)$. For each out-neighbor $v$ with $\deg^+(v) < \deg^+(u)$, search for triangles via intersection; symmetrically, iterate over in-neighbors $x$ with $\deg^+(x) < \deg^+(u)$.

The principal complexity improvement stems from each edge $\langle u, v \rangle$ being processed in

$\Theta(\min\{\deg^+(u), \deg^+(v)\})$

work, yielding a global time bound: $$T(n, m) = \Theta\left( \sum_{\langle u, v \rangle \in \vec{E}} \min\{\deg^+(u), \deg^+(v)\} \right)$$ matching the lower bound for edge-iterator in-memory methods.

3. Anonymization by Oblivious Transfer

The core block of the AOT system for anonymous messaging is a three-level mixnet architecture utilizing tags, oblivious transfer, batch mixing, and dummy queries to guarantee receiver anonymity and resilience to blending attacks (Javani et al., 2021).

Protocol Mechanics

• Handshake: Sender and receiver establish a shared secret $\sigma_{A,B}$ using an in-band, indistinguishable handshake.
• Tag Computation: For each message, compute a one-time tag $k_{A,B} = f(\sigma_{A,B}, c)$, using a KDF and per-message counter.
• Three-Level Mix Cascade:
  • Level 1: Entry nodes strip sender address, bucket inputs.
  • Level 2: Decrypt, permute, append dummies.
  • Level 3: Publish tags on a bulletin board; deliver payloads via $1$-out-$\gamma$ OT, ensuring the selector is only known to the recipient.
• Dummy Requests: All honest users periodically issue dummy OT requests, providing indistinguishability between real and cover traffic.

Security Properties

Receiver anonymity is provable under conventional OT security, and the batch/dummy design ensures robust resistance against blending (n−1) and traffic analysis attacks. Each layer horizontally scales and is functionally agnostic to sender/receiver activity.

4. Attention-Only Block in Transformers via Subspace Denoising

Wang et al. identify the core attention-only transformer (AOT) block as the fundamental operator derived from iterated denoising of token embeddings toward low-dimensional subspaces, obviating the need for MLPs in transformer layers (Wang et al., 4 Jun 2025).

Theoretical and Practical Formulation

Given $Z^{(l)} \in \mathbb{R}^{d \times N}$:

• Update Rule:

$Z^{(l+1)} = Z^{(l)} + \eta\, \mathrm{SA}(Z^{(l)})$

where $\mathrm{SA}$ is multi-head (shared QKV) self-attention. In the idealized denoising regime, all projection matrices per head are tied to subspace bases, and outputs are weighted by similarity (softmax or hard-thresholded).

• Signal-to-Noise Ratio Guarantee:

$$\mathrm{SNR}_k^{(l+1)} = (1+\eta \tau)\, \mathrm{SNR}_k^{(l)}$$

indicating linear improvement in denoising depth.

Empirical Performance

Models using only this block, omitting MLPs and complex normalization, remain competitive with standard architectures on both vision (ImageNet) and NLP (OpenWebText, LAMBADA), supporting the block’s sufficiency in deep representation learning.

5. Adapted Optimal Transport Block

In computational optimal transport for stochastic processes, the core adapted optimal transport (AOT) block encodes the causal (filtration-preserving) constraint during pathwise coupling and optimization (Eckstein et al., 2022).

Formulation

• Primal Problem:

$\min_{\pi \geq 0} \sum_{x, y} C(x, y)\pi_{x, y}$

subject to fixed marginals and causality:

$$\pi[X_{t+1} \in A \mid X_{1:t} = x_{1:t}, Y_{1:t} = y_{1:t}] = \mu[X_{t+1} \in A \mid X_{1:t} = x_{1:t}]$$

• Entropic Regularization:

$$E_c^\epsilon(\mu, \nu, C) = \min_{\pi \in \Pi_c(\mu, \nu)} \sum_{x, y} C(x, y)\pi_{x, y} + \epsilon\, D_{KL}(\pi \| \mu \otimes \nu)$$

• Blockwise Sinkhorn Iteration: Alternate scaled updates (blockwise generalized Sinkhorn) on prefix-wise arrays to enforce both marginal constraints and adaptedness, exploiting block-Gibbs structure:

$$u_{1:t}^{(k+1)}(x_{1:t}) = \mu_{1:t}(x_{1:t}) / \sum_{y_{1:t}} K_{1:t,1:t}(x_{1:t},y_{1:t})\, v_{1:t}^{(k)}(y_{1:t})$$

and analogously for $v_{1:t}$.

Convergence and Stability

As $\epsilon \rightarrow 0$, the entropic minimizer converges to a true adapted coupling. Stability results hold under metric perturbations (adapted Wasserstein distance), and complexity is dominated by the prefixwise supports, making the block tractable for moderate horizons.

6. Comparative Summary Table

Domain	Algorithmic Block Principle	Mathematical Core/Formula
Image Inpainting	Multi-dilated aggregated convolution + gating	$x_\mathrm{out} = x \cdot g + r \cdot(1-g)$
Triangle Listing	Adaptive orientation with per-edge pivot selection	$$T(n, m) = \Theta(\sum_{\langle u,v \rangle} \min\{\deg^+(u),\deg^+(v)\})$$
Anonymous Communication	Three-level mixnet, tag-based rendezvous, OT	$k_{A,B} = f(\sigma_{A,B}, c)$; security via OT
Transformer (AOT)	Residual shared-QKV self-attention (no MLPs)	$Z^{(l+1)} = Z^{(l)} + \eta\, \mathrm{SA}(Z^{(l)})$
Adapted Optimal Transport (AOT)	Blockwise causal scaling in Sinkhorn iterations	$u_{1:t}(x_{1:t})$ and $v_{1:t}(y_{1:t})$ blockwise updates

7. Significance and Interpretations

The core AOT block, despite disciplinary differences, typically encapsulates (1) efficient aggregation of dispersed or multi-scale information, (2) filtration or adaptivity in information flow, and (3) architectural or algorithmic parsimony: minimal but sufficient operations to capture the heart of the task. In learning settings, this yields superior balance between bias and variance (inpainting, transformers); in combinatorial and cryptographic applications, it aligns computational cost exactly to the structure of the problem instance (graph degree, OT causality, protocol anonymity).

A plausible implication is that the AOT paradigm—modular, minimal, context-aggregating blocks—may generalize as a design pattern wherever the joint inference or computation over structured, multi-scale, or causally filtered data is required.