Linear Positional Channel Overview

Updated 4 July 2026

Linear Positional Channel is a design principle that injects relative positional signals into sequence models while preserving linear factorization and streaming efficiency.
Multiple implementations use learned spectral features, continuous-time mappings, and recurrent or additive mechanisms to encode positional information effectively.
Empirical studies demonstrate that leveraging linear positional channels improves performance in language, recommendation, and wireless modeling, though trade-offs remain.

Searching arXiv for papers on "linear positional channel" and closely related positional encoding mechanisms. Linear positional channel denotes a family of positional-injection mechanisms designed to preserve the factorized or streaming structure of efficient sequence models while still encoding order, lag, or geometry. Across recent work, the term is used in several closely related senses: explicit positional feature columns concatenated to queries and keys in linear attention, position-indexed transformations of queries and keys that preserve kernel separability, learnable causal positional kernels acting on value streams, continuous-time linear maps from timestamps to embedding space, and projected positional subspaces that reduce interference between channel content and position (Choromanski et al., 2023, Qin et al., 2023, Ye et al., 27 Feb 2026, Kim et al., 2024, Lehtinen, 3 Jun 2026). In wireless modeling, the same idea extends to linear positional contexts such as linear antenna arrays, 1D time lines, and 1D subcarrier grids, where the positional mechanism is required to preserve relative offsets and axis structure rather than absolute coordinates (Zhang et al., 1 May 2026).

1. Conceptual scope of the term

In the linear-attention literature, a linear positional channel is a positional pathway that modulates attention without destroying the linear time/space factorization. The formulation in Linearized Relative Positional Encoding defines it as a “position-dependent transformation path” acting on queries and keys so that relative position enters attention through a decomposable, kernelizable form (Qin et al., 2023). In FourierLearner-Transformers, the same idea is realized by appending learned positional feature blocks to the query and key matrices before kernel linearization, so that relative positional encoding is present while complexity remains linear in sequence length (Choromanski et al., 2023).

In long-sequence recommendation, the term is used even more explicitly. FuXi-Linear introduces a Linear Positional Channel as a learnable, strictly linear-time mechanism that injects expressive relative positional signals into linear attention, while remaining compatible with streaming and incremental inference (Ye et al., 27 Feb 2026). In irregular time-series forecasting, Continuous-Time Linear Positional Embedding does not use the phrase as a system name, but it instantiates a closely related notion: a continuous positional map $p(t)=a t+b$ added to value embeddings, with the time gap encoded by $p(t_2)-p(t_1)=a(t_2-t_1)$ (Kim et al., 2024).

A broader interpretation appears in signal and wireless domains. In a multi-channel signal Transformer audit, projected positional encoding is analyzed as a learned linear projection of the positional basis that rotates positional information away from the channel-content subspace (Lehtinen, 3 Jun 2026). In wireless CSI modeling, Adaptive 3D-RoPE describes a “linear positional channel” scenario as one in which positions are indexed along a spatial line, a time line, or a subcarrier grid, and relative phase laws rather than absolute coordinates are the essential inductive bias (Zhang et al., 1 May 2026).

Taken together, these uses indicate that the term does not identify one single architecture. It names a design principle: positional information should enter through a path that remains compatible with linearized attention, recurrent state updates, or other efficient factorization, while retaining relative structure.

2. Canonical mathematical constructions

Several algebraic constructions recur in the literature.

Family	Positional object	Injection site
FLTs	Low-rank positional blocks $N_1(R), N_2(R)$	Concatenate with $Q,K$ before kernel map
LRPE	Position-indexed transforms $W_s$	Transform $Q,K$ before linear attention
FuXi-Linear	Learnable kernel $k(n)$ and recurrent state $S_p$	Dedicated value-side positional stream
CTLPE	Continuous map $p(t)=a t+b$	Add to token embeddings
Rotary/Jordan variants	Relative operator from rotations or Jordan blocks	Bilinear query-key logit

In FourierLearner-Transformers, the starting point is an RPE mask $N$ with entries $p(t_2)-p(t_1)=a(t_2-t_1)$ 0. Rather than learning $p(t_2)-p(t_1)=a(t_2-t_1)$ 1 directly in the token domain, FLTs learn its Fourier transform $p(t_2)-p(t_1)=a(t_2-t_1)$ 2, construct random-feature positional blocks $p(t_2)-p(t_1)=a(t_2-t_1)$ 3 and $p(t_2)-p(t_1)=a(t_2-t_1)$ 4, and then form

$p(t_2)-p(t_1)=a(t_2-t_1)$ 5

This realizes the positional channel as explicit positional columns appended to the content channels, while keeping overall complexity at $p(t_2)-p(t_1)=a(t_2-t_1)$ 6 time and $p(t_2)-p(t_1)=a(t_2-t_1)$ 7 space (Choromanski et al., 2023).

LRPE gives a more abstract criterion for linearizability. Relative position is represented through a family of operators satisfying

$p(t_2)-p(t_1)=a(t_2-t_1)$ 8

so that

$p(t_2)-p(t_1)=a(t_2-t_1)$ 9

Because the positional dependence is separated into a query-side transform and a key-side transform, any kernel feature map can then be applied to $N_1(R), N_2(R)$ 0 and $N_1(R), N_2(R)$ 1 without breaking linear aggregation (Qin et al., 2023).

FuXi-Linear adopts a distinct design. It relaxes strict translation invariance and approximates a relative kernel by a learnable separable kernel $N_1(R), N_2(R)$ 2. With $N_1(R), N_2(R)$ 3, its positional channel is

$N_1(R), N_2(R)$ 4

This makes the positional channel a causal recurrent stream on top of values, with $N_1(R), N_2(R)$ 5 per-token cost in streaming mode (Ye et al., 27 Feb 2026).

CTLPE is the simplest of the family in form, but not in implication. It learns

$N_1(R), N_2(R)$ 6

and uses additive fusion,

$N_1(R), N_2(R)$ 7

The positional channel is therefore continuous in time, defined for arbitrary irregular timestamps, and its differences encode elapsed time exactly linearly (Kim et al., 2024).

In wireless rotary formulations, the positional channel is neither appended columns nor additive timestamps, but relative phase geometry. Adaptive 3D-RoPE uses

$N_1(R), N_2(R)$ 8

which reduces in a strictly linear spatial context to

$N_1(R), N_2(R)$ 9

Jordan-RoPE further enlarges the primitive basis by replacing a semisimple rotary block with a defective complex Jordan block, thereby generating not only $Q,K$ 0 but also the distance-modulated phase $Q,K$ 1 (Zhang et al., 1 May 2026, Zhang, 5 May 2026).

3. Relative position, invariance, and geometry

A unifying property of these mechanisms is that they privilege relative offsets over absolute coordinates. FLTs are built directly from a mask of the form $Q,K$ 2, with positional interactions specified as functions of displacement in $Q,K$ 3 rather than of absolute positions (Choromanski et al., 2023). LRPE formalizes the same principle at the operator level: once $Q,K$ 4 holds, the positional effect depends only on lag $Q,K$ 5 (Qin et al., 2023).

The irregular-time literature makes the invariance question explicit. CTLPE states monotonicity, translation invariance, symmetry, inductive extension, data-driven learnability, and irregularity-adaptability as desiderata for ideal positional embeddings, and gives a theorem that under monotonicity and translation invariance with Euclidean distance, the positional map must be linear, $Q,K$ 6 (Kim et al., 2024). Within those assumptions, linearity is not merely a computational convenience; it is the class singled out by the geometry of the embedding distance.

Wireless formulations sharpen the distinction between relative and absolute position. For CSI $Q,K$ 7, Adaptive 3D-RoPE uses axis-wise autocorrelation functions and coherence extents to argue that the positional prior must encode pure relative decay and preserve the 3D spatio-temporal-frequency structure. The paper explicitly contrasts this with absolute positional encodings, which mix coordinates with content, and with standard 1D-RoPE, which enforces a relative-only interaction but flattens CSI into a 1D order, thereby discarding axis anisotropy (Zhang et al., 1 May 2026).

Jordan-RoPE exposes a further subtlety. Relative positional encoding is not limited to pure phase or pure distance bias. A non-semisimple block supplies oscillatory-polynomial lag functions such as $Q,K$ 8 and $Q,K$ 9 inside the primitive attention logit. This is presented not as a separate additive distance channel, but as a coupled distance-modulated phase basis arising from the same defective block (Zhang, 5 May 2026).

A common misconception is therefore that “linear positional channel” always means a linear function of an absolute index. The literature suggests a more precise reading: linearity may refer to linear-time computation, linearizable factorization, linear continuous-time maps, or linear operators on feature space, while the encoded positional dependence is often explicitly relative.

4. Domain-specific realizations

In language modeling, image classification, and molecular modeling, linear positional channels arise primarily as kernel-preserving relative-position mechanisms. FLTs learn a spectral representation of the desired relative mask and use low-rank positional factors that generalize naturally from 1D sequences to 2D images and 3D molecular coordinates (Choromanski et al., 2023). LRPE instead organizes existing relative schemes under a unitary or orthogonal transformation family, including complex diagonal, real block-rotation, and permutation solutions (Qin et al., 2023).

In time-aware recommendation, FuXi-Linear decouples semantics, time, and position into separate streams. Its Linear Positional Channel operates only on projected values and positional codes, while semantic retention and temporal retention are computed in parallel and fused later by normalization, concatenation, and gating. This architectural separation is intended to avoid crosstalk between temporal and semantic signals while restoring fine-grained order sensitivity (Ye et al., 27 Feb 2026).

In irregularly sampled forecasting, CTLPE interprets positional encoding as a continuous-time signal rather than an index lookup. Because $W_s$ 0 is defined for all $W_s$ 1, irregular observation patterns and arbitrary prediction timestamps are handled without a fixed positional table. The paper also introduces NCDE-PE as a more expressive continuous-time alternative, but reports that the learned trajectory collapses to a near-linear function of time, empirically reinforcing the linear positional channel design (Kim et al., 2024).

Wireless work introduces an explicitly physical notion of linear positional context. Adaptive 3D-RoPE treats linear antenna arrays, 1D time lines, and 1D subcarrier grids as settings where the positional law should follow axis-specific phase increments and preserve relative-only interactions. In the strictly linear spatial case, the phase angle depends only on antenna index $W_s$ 2 and the adaptive spatial bank $W_s$ 3, allowing extrapolation to longer arrays by preserving the relative spatial phase law (Zhang et al., 1 May 2026).

A different wireless usage appears in channel charting. “Efficient channel charting via phase-insensitive distance computation” states that the overall map from CSI to chart coordinates is nonlinear because both the phase-insensitive distance and the Isomap geodesic construction are nonlinear in the raw channels (Magoarou, 2021). “Channel Charting for Position and Orientation” then describes how an optional affine map $W_s$ 4 can place a learned latent chart into real-world coordinates, which provides a clean example of a linear positional channel layered on top of a nonlinear chart (Richner et al., 16 Jun 2026). This suggests that in wireless localization literature the phrase can also refer to a final linear or affine positional head, not only to the internal attention mechanism.

In multi-channel signal Transformers, the positional channel may be neither relative-mask factorization nor recurrent kernel, but a learned linear projection of a fixed sinusoidal basis. The projected positional encoding variant computes $W_s$ 5 before addition to $W_s$ 6, and the reported mechanism is positional-channel orthogonalisation: the learned projection rotates the positional subspace away from the channel subspace (Lehtinen, 3 Jun 2026).

5. Empirical evidence and comparative behavior

The empirical record shows that linear positional channels are usually valuable, but that the best realization depends strongly on domain and inductive bias.

Setting	Reported result	Citation
FLTs, WikiText-103	PPL 30.1 for FLT (local RPE) and 30.3 for FLT (Gaussian mixture)	(Choromanski et al., 2023)
LRPE, WikiText-103	Type 1/2 validation/test PPL 31.9/31.6 versus 32.86/32.53 for PermuteFormer	(Qin et al., 2023)
FuXi-Linear, Kuairand-27K	LPC: NG@10 0.0609, HR@10 0.1124, MRR 0.0540; up to 10× prefill and 21× decode speedup	(Ye et al., 27 Feb 2026)
Adaptive 3D-RoPE, wireless CSI	Up to 10.7 dB NMSE reduction under 8× antenna scale extrapolation	(Zhang et al., 1 May 2026)
Encoder audit, synthetic $W_s$ 7	linear-ppe 2.114 ± 0.029 versus linear 2.155 ± 0.019	(Lehtinen, 3 Jun 2026)
Jordan-RoPE, synthetic LM	Stabilized Jordan-RoPE 0.906 ± 0.054 at length 8192	(Zhang, 5 May 2026)

FLTs report linear scaling close to Performer in memory and speed while improving over several other linear Transformers on language, image, and molecular tasks. Their learned spectral channel is described as parameter efficient, with less than $W_s$ 8M extra parameters (Choromanski et al., 2023). LRPE reports state-of-the-art performance in language modeling, text classification, and image classification, while emphasizing a general paradigm rather than a single encoding instance (Qin et al., 2023).

FuXi-Linear offers one of the clearest ablations of a dedicated Linear Positional Channel. Replacing LPC with RoPE, T5Bias, ALiBi, or no positional channel degrades recommendation metrics, and removing the positional channel entirely drops NG@10 from $W_s$ 9 to $Q,K$ 0, HR@10 from $Q,K$ 1 to $Q,K$ 2, and MRR from $Q,K$ 3 to $Q,K$ 4 (Ye et al., 27 Feb 2026). The same paper reports robust power-law scaling at thousand-length scale.

In wireless CSI modeling, Adaptive 3D-RoPE reports gains in both extrapolation and zero-shot transfer, including $Q,K$ 5 dB zero-shot NMSE improvement across unseen mobility scenarios and $Q,K$ 6 dB in low-frequency-to-millimeter-wave transfer (Zhang et al., 1 May 2026). The paper attributes the gains to preserving 3D axis integrity and adapting the rotary bank per sample.

The input-encoder audit supplies a useful counterpoint. On synthetic multi-channel signals, a wide “top tier” emerges in which a standard per-channel linear projection, block-partitioned concatenation, linear-ortho, and projected positional encoding are practically close, with projected positional encoding showing a small advantage at small channel count. The same study finds that the shared-scalar baseline and the channel-independent baseline lose decisively, and that the practical default should remain $Q,K$ 7 unless there is a specific reason for something more elaborate (Lehtinen, 3 Jun 2026).

Jordan-RoPE is deliberately more cautious. Its paper states that the evidence is structural rather than a broad performance claim. On a small WikiText-103 byte LLM, the scaled-exact Jordan variant improves over RoPE and direct-sum baselines within the Jordan family, but RoPE+ALiBi remains strongest overall (Zhang, 5 May 2026). This is one of the clearest instances where a richer linear positional channel expands the primitive lag basis without yet dominating standard baselines in general-purpose language modeling.

6. Limitations, ambiguities, and open directions

The main limitation of the term is semantic rather than mathematical: it is not standardized across fields. In some papers, “linear” refers to linear-time complexity and kernel factorization; in CTLPE it refers to the functional form $Q,K$ 8; in channel charting it can refer to an affine chart-to-coordinate head (Qin et al., 2023, Kim et al., 2024, Richner et al., 16 Jun 2026). This suggests that any use of the phrase should be read relative to the surrounding modeling framework.

Each realization also has specific constraints. FLTs rely on a good match between the sampling distribution $Q,K$ 9 and the learned spectrum $k(n)$ 0; when $k(n)$ 1 and $k(n)$ 2 are badly mismatched, the variance constant grows and more positional random features are needed (Choromanski et al., 2023). LRPE only covers relative encodings that admit the decomposable form $k(n)$ 3, and the paper identifies specialized 2D positional design as an open problem (Qin et al., 2023).

FuXi-Linear gains expressivity by abandoning strict Toeplitz structure, but its default positional codes are learned per index, $k(n)$ 4, so extrapolation to unseen lengths requires extending or replacing the table (Ye et al., 27 Feb 2026). CTLPE, by contrast, extrapolates trivially because the positional map is defined on all real-valued timestamps, but its theorem is tied to monotonicity and translation invariance under Euclidean distance; the result should therefore be read within those assumptions rather than as a universal theorem for all positional objectives (Kim et al., 2024).

Wireless formulations reveal another boundary. Adaptive 3D-RoPE does not introduce explicit amplitude-decay modeling, such as path loss, in the positional mechanism, and its controller uses compact global CSI descriptors rather than explicit physical parameter estimation for Doppler or delay spread. The paper also notes that stage-wise sharing may limit layer-specific specialization (Zhang et al., 1 May 2026). Jordan-RoPE shows a different trade-off: stabilized bounded-shear variants improve numerical behavior but break the exact one-parameter group law, and even in the exact or scaled-exact forms the learned shear tends to remain small on natural-language data (Zhang, 5 May 2026).

Finally, the wireless charting literature highlights that not every positional map in channel modeling is linear even when a linear head is present. Phase-insensitive distance computation followed by Isomap is explicitly nonlinear overall (Magoarou, 2021). A plausible implication is that “linear positional channel” is best reserved for the positional injection mechanism itself, not for the entire end-to-end map, unless the model’s final coordinate placement is the object under discussion.

Across these lines of work, the most stable conclusion is not that one linear positional channel dominates, but that efficient models require an explicit positional pathway matched to their algebraic structure: separable when attention is kernelized, recurrent when inference is streaming, continuous when timestamps are irregular, and axis-aware when the underlying geometry is physical rather than textual.