---
title: Linear Positional Channel Overview
url: https://www.emergentmind.com/topics/linear-positional-channel
type: topic
---

# Linear Positional Channel Overview

Searching arXiv for recent papers on "linear positional channel" and closely related positional encoding mechanisms.
Linear positional channel denotes a family of positional-injection mechanisms designed to preserve the factorized or streaming structure of efficient sequence models while still encoding order, lag, or geometry. Across recent work, the term is used in several closely related senses: explicit positional feature columns concatenated to queries and keys in linear attention, position-indexed transformations of queries and keys that preserve kernel separability, learnable causal positional kernels acting on value streams, continuous-time linear maps from timestamps to embedding space, and projected positional subspaces that reduce interference between channel content and position [2302.01925][2307.09270][2602.23671][2409.20092][2606.04752]. In wireless modeling, the same idea extends to linear positional contexts such as linear antenna arrays, 1D time lines, and 1D subcarrier grids, where the positional mechanism is required to preserve relative offsets and axis structure rather than absolute coordinates [2605.00968].

## 1. Conceptual scope of the term

In the linear-attention literature, a linear positional channel is a positional pathway that modulates attention without destroying the linear time/space factorization. The formulation in Linearized Relative Positional Encoding defines it as a “position-dependent transformation path” acting on queries and keys so that relative position enters attention through a decomposable, kernelizable form [2307.09270]. In FourierLearner-Transformers, the same idea is realized by appending learned positional feature blocks to the query and key matrices before kernel linearization, so that relative positional encoding is present while complexity remains linear in sequence length [2302.01925].

In long-sequence recommendation, the term is used even more explicitly. FuXi-Linear introduces a Linear Positional Channel as a learnable, strictly linear-time mechanism that injects expressive relative positional signals into linear attention, while remaining compatible with streaming and incremental inference [2602.23671]. In irregular time-series forecasting, Continuous-Time Linear Positional Embedding does not use the phrase as a system name, but it instantiates a closely related notion: a continuous positional map $p(t)=a t+b$ added to value embeddings, with the time gap encoded by $p(t_2)-p(t_1)=a(t_2-t_1)$ [2409.20092].

A broader interpretation appears in signal and wireless domains. In a multi-channel signal Transformer audit, projected positional encoding is analyzed as a learned linear projection of the positional basis that rotates positional information away from the channel-content subspace [2606.04752]. In wireless CSI modeling, Adaptive 3D-RoPE describes a “linear positional channel” scenario as one in which positions are indexed along a spatial line, a time line, or a subcarrier grid, and relative phase laws rather than absolute coordinates are the essential inductive bias [2605.00968].

Taken together, these uses indicate that the term does not identify one single architecture. It names a design principle: positional information should enter through a path that remains compatible with linearized attention, recurrent state updates, or other efficient factorization, while retaining relative structure.

## 2. Canonical mathematical constructions

Several algebraic constructions recur in the literature.

| Family | Positional object | Injection site |
|---|---|---|
| FLTs | Low-rank positional blocks $N_1(R), N_2(R)$ | Concatenate with $Q,K$ before kernel map |
| LRPE | Position-indexed transforms $W_s$ | Transform $Q,K$ before linear attention |
| FuXi-Linear | Learnable kernel $k(n)$ and recurrent state $S_p$ | Dedicated value-side positional stream |
| CTLPE | Continuous map $p(t)=a t+b$ | Add to token embeddings |
| Rotary/Jordan variants | Relative operator from rotations or Jordan blocks | Bilinear query-key logit |

In FourierLearner-Transformers, the starting point is an RPE mask $N$ with entries $N_{ij}=f(r_i-r_j)$. Rather than learning $f$ directly in the token domain, FLTs learn its Fourier transform $g(\xi)$, construct random-feature positional blocks $N_1$ and $N_2$, and then form
$$
\widehat{Q}=\big[N_1,\; Q d^{-1/4}\big], \qquad \widehat{K}=\big[N_2,\; K d^{-1/4}\big].
$$
This realizes the positional channel as explicit positional columns appended to the content channels, while keeping overall complexity at $O(L(m+r)d)$ time and $O(L(m+r)+(m+r)d+Ld)$ space [2302.01925].

LRPE gives a more abstract criterion for linearizability. Relative position is represented through a family of operators satisfying
$$
W_{t-s}=W_s^{\mathsf H}W_t,
$$
so that
$$
f_{\mathrm{rel}}(q_s,k_t)=q_s^{\mathsf H}W_{t-s}k_t=(W_s q_s)^{\mathsf H}(W_t k_t).
$$
Because the positional dependence is separated into a query-side transform and a key-side transform, any kernel feature map can then be applied to $\tilde q_s=W_s q_s$ and $\tilde k_t=W_t k_t$ without breaking linear aggregation [2307.09270].

FuXi-Linear adopts a distinct design. It relaxes strict translation invariance and approximates a relative kernel by a learnable separable kernel $g(x,y)=k(x)^\top k(y)$. With $V_p=XW_p$, its positional channel is
$$
(S_p)_n=(S_p)_{n-1}+k(n)^\top (V_p)_n,\qquad
(y_p)_n=\alpha\, k(n)(S_p)_n+\beta\,(V_p)_n.
$$
This makes the positional channel a causal recurrent stream on top of values, with $O(d_p d)$ per-token cost in streaming mode [2602.23671].

CTLPE is the simplest of the family in form, but not in implication. It learns
$$
p(t)=a t+b,
$$
and uses additive fusion,
$$
h_i=x_i+p(t_i).
$$
The positional channel is therefore continuous in time, defined for arbitrary irregular timestamps, and its differences encode elapsed time exactly linearly [2409.20092].

In wireless rotary formulations, the positional channel is neither appended columns nor additive timestamps, but relative phase geometry. Adaptive 3D-RoPE uses
$$
\theta_{m,i}^{(h)}=t_m \tilde{\Omega}_{T,i}^{(h)}+k_m \tilde{\Omega}_{K,i}^{(h)}+u_m \tilde{\Omega}_{U,i}^{(h)},
$$
which reduces in a strictly linear spatial context to
$$
\theta_{m,i}^{(h)}=u_m \tilde{\Omega}_{U,i}^{(h)}.
$$
Jordan-RoPE further enlarges the primitive basis by replacing a semisimple rotary block with a defective complex Jordan block, thereby generating not only $e^{i\omega d}$ but also the distance-modulated phase $d e^{i\omega d}$ [2605.00968][2605.04217].

## 3. Relative position, invariance, and geometry

A unifying property of these mechanisms is that they privilege relative offsets over absolute coordinates. FLTs are built directly from a mask of the form $f(r_i-r_j)$, with positional interactions specified as functions of displacement in $\mathbb{R}^\ell$ rather than of absolute positions [2302.01925]. LRPE formalizes the same principle at the operator level: once $W_{t-s}=W_s^{\mathsf H}W_t$ holds, the positional effect depends only on lag $t-s$ [2307.09270].

The irregular-time literature makes the invariance question explicit. CTLPE states monotonicity, translation invariance, symmetry, inductive extension, data-driven learnability, and irregularity-adaptability as desiderata for ideal positional embeddings, and gives a theorem that under monotonicity and translation invariance with Euclidean distance, the positional map must be linear, $p(t)=k t+b$ [2409.20092]. Within those assumptions, linearity is not merely a computational convenience; it is the class singled out by the geometry of the embedding distance.

Wireless formulations sharpen the distinction between relative and absolute position. For CSI $H(t,k,u)$, Adaptive 3D-RoPE uses axis-wise autocorrelation functions and coherence extents to argue that the positional prior must encode pure relative decay and preserve the 3D spatio-temporal-frequency structure. The paper explicitly contrasts this with absolute positional encodings, which mix coordinates with content, and with standard 1D-RoPE, which enforces a relative-only interaction but flattens CSI into a 1D order, thereby discarding axis anisotropy [2605.00968].

Jordan-RoPE exposes a further subtlety. Relative positional encoding is not limited to pure phase or pure distance bias. A non-semisimple block supplies oscillatory-polynomial lag functions such as $e^{-\gamma d}\cos(\omega d)$ and $d e^{-\gamma d}\cos(\omega d)$ inside the primitive attention logit. This is presented not as a separate additive distance channel, but as a coupled distance-modulated phase basis arising from the same defective block [2605.04217].

A common misconception is therefore that “linear positional channel” always means a linear function of an absolute index. The literature suggests a more precise reading: linearity may refer to linear-time computation, linearizable factorization, linear continuous-time maps, or linear operators on feature space, while the encoded positional dependence is often explicitly relative.

## 4. Domain-specific realizations

In language modeling, image classification, and molecular modeling, linear positional channels arise primarily as kernel-preserving relative-position mechanisms. FLTs learn a spectral representation of the desired relative mask and use low-rank positional factors that generalize naturally from 1D sequences to 2D images and 3D molecular coordinates [2302.01925]. LRPE instead organizes existing relative schemes under a unitary or orthogonal transformation family, including complex diagonal, real block-rotation, and permutation solutions [2307.09270].

In time-aware recommendation, FuXi-Linear decouples semantics, time, and position into separate streams. Its Linear Positional Channel operates only on projected values and positional codes, while semantic retention and temporal retention are computed in parallel and fused later by normalization, concatenation, and gating. This architectural separation is intended to avoid crosstalk between temporal and semantic signals while restoring fine-grained order sensitivity [2602.23671].

In irregularly sampled forecasting, CTLPE interprets positional encoding as a continuous-time signal rather than an index lookup. Because $p(t)$ is defined for all $t\in\mathbb{R}$, irregular observation patterns and arbitrary prediction timestamps are handled without a fixed positional table. The paper also introduces NCDE-PE as a more expressive continuous-time alternative, but reports that the learned trajectory collapses to a near-linear function of time, empirically reinforcing the linear positional channel design [2409.20092].

Wireless work introduces an explicitly physical notion of linear positional context. Adaptive 3D-RoPE treats linear antenna arrays, 1D time lines, and 1D subcarrier grids as settings where the positional law should follow axis-specific phase increments and preserve relative-only interactions. In the strictly linear spatial case, the phase angle depends only on antenna index $u_m$ and the adaptive spatial bank $\tilde{\Omega}_U$, allowing extrapolation to longer arrays by preserving the relative spatial phase law [2605.00968].

A different wireless usage appears in channel charting. “Efficient channel charting via phase-insensitive distance computation” states that the overall map from CSI to chart coordinates is nonlinear because both the phase-insensitive distance and the Isomap geodesic construction are nonlinear in the raw channels [2104.13184]. “Channel Charting for Position and Orientation” then describes how an optional affine map $ \hat r = A z + b $ can place a learned latent chart into real-world coordinates, which provides a clean example of a linear positional channel layered on top of a nonlinear chart [2606.18151]. This suggests that in wireless localization literature the phrase can also refer to a final linear or affine positional head, not only to the internal attention mechanism.

In multi-channel signal Transformers, the positional channel may be neither relative-mask factorization nor recurrent kernel, but a learned linear projection of a fixed sinusoidal basis. The projected positional encoding variant computes $p_t=\mathrm{PE}_t W_{\text{pos}}+b_{\text{pos}}$ before addition to $x_t^\top W+b$, and the reported mechanism is positional-channel orthogonalisation: the learned projection rotates the positional subspace away from the channel subspace [2606.04752].

## 5. Empirical evidence and comparative behavior

The empirical record shows that linear positional channels are usually valuable, but that the best realization depends strongly on domain and inductive bias.

| Setting | Reported result | Citation |
|---|---|---|
| FLTs, WikiText-103 | PPL 30.1 for FLT (local RPE) and 30.3 for FLT (Gaussian mixture) | [2302.01925] |
| LRPE, WikiText-103 | Type 1/2 validation/test PPL 31.9/31.6 versus 32.86/32.53 for PermuteFormer | [2307.09270] |
| FuXi-Linear, Kuairand-27K | LPC: NG@10 0.0609, HR@10 0.1124, MRR 0.0540; up to 10× prefill and 21× decode speedup | [2602.23671] |
| Adaptive 3D-RoPE, wireless CSI | Up to 10.7 dB NMSE reduction under 8× antenna scale extrapolation | [2605.00968] |
| Encoder audit, synthetic $C=4$ | linear-ppe 2.114 ± 0.029 versus linear 2.155 ± 0.019 | [2606.04752] |
| Jordan-RoPE, synthetic LM | Stabilized Jordan-RoPE 0.906 ± 0.054 at length 8192 | [2605.04217] |

FLTs report linear scaling close to Performer in memory and speed while improving over several other linear Transformers on language, image, and molecular tasks. Their learned spectral channel is described as parameter efficient, with less than $0.03$M extra parameters [2302.01925]. LRPE reports state-of-the-art performance in language modeling, text classification, and image classification, while emphasizing a general paradigm rather than a single encoding instance [2307.09270].

FuXi-Linear offers one of the clearest ablations of a dedicated Linear Positional Channel. Replacing LPC with RoPE, T5Bias, ALiBi, or no positional channel degrades recommendation metrics, and removing the positional channel entirely drops NG@10 from $0.0609$ to $0.0568$, HR@10 from $0.1124$ to $0.1067$, and MRR from $0.0540$ to $0.0501$ [2602.23671]. The same paper reports robust power-law scaling at thousand-length scale.

In wireless CSI modeling, Adaptive 3D-RoPE reports gains in both extrapolation and zero-shot transfer, including $1.07$ dB zero-shot NMSE improvement across unseen mobility scenarios and $0.90$ dB in low-frequency-to-millimeter-wave transfer [2605.00968]. The paper attributes the gains to preserving 3D axis integrity and adapting the rotary bank per sample.

The input-encoder audit supplies a useful counterpoint. On synthetic multi-channel signals, a wide “top tier” emerges in which a standard per-channel linear projection, block-partitioned concatenation, linear-ortho, and projected positional encoding are practically close, with projected positional encoding showing a small advantage at small channel count. The same study finds that the shared-scalar baseline and the channel-independent baseline lose decisively, and that the practical default should remain $\mathrm{nn.Linear}(C,d_{\text{model}})$ unless there is a specific reason for something more elaborate [2606.04752].

Jordan-RoPE is deliberately more cautious. Its paper states that the evidence is structural rather than a broad performance claim. On a small WikiText-103 byte language model, the scaled-exact Jordan variant improves over RoPE and direct-sum baselines within the Jordan family, but RoPE+ALiBi remains strongest overall [2605.04217]. This is one of the clearest instances where a richer linear positional channel expands the primitive lag basis without yet dominating standard baselines in general-purpose language modeling.

## 6. Limitations, ambiguities, and open directions

The main limitation of the term is semantic rather than mathematical: it is not standardized across fields. In some papers, “linear” refers to linear-time complexity and kernel factorization; in CTLPE it refers to the functional form $p(t)=a t+b$; in channel charting it can refer to an affine chart-to-coordinate head [2307.09270][2409.20092][2606.18151]. This suggests that any use of the phrase should be read relative to the surrounding modeling framework.

Each realization also has specific constraints. FLTs rely on a good match between the sampling distribution $p$ and the learned spectrum $g$; when $g$ and $p$ are badly mismatched, the variance constant grows and more positional random features are needed [2302.01925]. LRPE only covers relative encodings that admit the decomposable form $W_{t-s}=W_s^{\mathsf H}W_t$, and the paper identifies specialized 2D positional design as an open problem [2307.09270].

FuXi-Linear gains expressivity by abandoning strict Toeplitz structure, but its default positional codes are learned per index, $k(n)=E_p[n]$, so extrapolation to unseen lengths requires extending or replacing the table [2602.23671]. CTLPE, by contrast, extrapolates trivially because the positional map is defined on all real-valued timestamps, but its theorem is tied to monotonicity and translation invariance under Euclidean distance; the result should therefore be read within those assumptions rather than as a universal theorem for all positional objectives [2409.20092].

Wireless formulations reveal another boundary. Adaptive 3D-RoPE does not introduce explicit amplitude-decay modeling, such as path loss, in the positional mechanism, and its controller uses compact global CSI descriptors rather than explicit physical parameter estimation for Doppler or delay spread. The paper also notes that stage-wise sharing may limit layer-specific specialization [2605.00968]. Jordan-RoPE shows a different trade-off: stabilized bounded-shear variants improve numerical behavior but break the exact one-parameter group law, and even in the exact or scaled-exact forms the learned shear tends to remain small on natural-language data [2605.04217].

Finally, the wireless charting literature highlights that not every positional map in channel modeling is linear even when a linear head is present. Phase-insensitive distance computation followed by Isomap is explicitly nonlinear overall [2104.13184]. A plausible implication is that “linear positional channel” is best reserved for the positional injection mechanism itself, not for the entire end-to-end map, unless the model’s final coordinate placement is the object under discussion.

Across these lines of work, the most stable conclusion is not that one linear positional channel dominates, but that efficient models require an explicit positional pathway matched to their algebraic structure: separable when attention is kernelized, recurrent when inference is streaming, continuous when timestamps are irregular, and axis-aware when the underlying geometry is physical rather than textual.

Source: https://www.emergentmind.com/topics/linear-positional-channel