Target-Decoupled Positional Encoding

• Target-Decoupled Positional Encoding is a strategy that separates positional data from semantic content, assigning distinct computational roles to each in transformer models.
• It implements multiple channels (absolute, relative, and semantic) to prevent positional information from being overwritten during training and attention scoring.
• Empirical results show that decoupling improves linguistic probing and task performance on benchmarks like GLUE, SQuAD, and Flash-Holmes without sacrificing efficiency.

Searching arXiv for the cited papers to ground the article. arxiv_search: (Lequeu et al., 28 May 2026) Give it Space! Explicit Disentangling of Positional and Semantic Representations in Encoders Searching arXiv for related positional encoding papers: (Chen et al., 2021, Gopalakrishnan et al., 5 Sep 2025, Kenney et al., 30 Jun 2026, Ali et al., 11 May 2026). Target-decoupled positional encoding is best understood as a family of positional-design strategies that prevent positional information from being forced to share the same representation, supervision pathway, or geometric prior as semantic content. In the clearest encoder formulation, semantic, absolute positional (AP), and relative positional (RP) information are assigned distinct computational roles, and the masked-language-modeling (MLM) objective is applied only to the semantic stream, so the positional channel is preserved rather than being overwritten by the prediction target (Lequeu et al., 28 May 2026). Related work broadens the same design impulse to attention-level decoupling of content and position (Chen et al., 2021), removal of content-dependent phase shifts from rotary scoring (Gopalakrishnan et al., 5 Sep 2025), factorization of target identity and within-target progress in multi-target decoding (Coquenet et al., 2023), inter-modal positional independence in large vision-LLMs (Wang et al., 22 May 2025), and explicit separation of rotation and translation in camera-aware attention (Kenney et al., 30 Jun 2026).

1. Conceptual scope and meanings of decoupling

The literature does not use a single uniform definition of decoupling. In encoder pretraining, decoupling can mean prediction-target decoupling: positional channels remain in the hidden state, but the prediction head reads only the semantic subspace, so the objective no longer directly pressures positional dimensions (Lequeu et al., 28 May 2026). In attention design, it can mean content-position decoupling: position and segment information are moved out of the summed input embedding $X+P$ and inserted directly into the attention score as separate terms, avoiding both a rank bottleneck and identical gradients on token and positional embeddings at the point of summation (Chen et al., 2021). In rotary reformulations, it can mean phase-content decoupling: PoPE replaces RoPE’s token-dependent phase shift with a score of the form

$$a^{\text{PoPE}}_{ts}=\sum_{c=1}^{d}|q_{tc}|\,|k_{sc}|\cos\!\big((s-t)\theta_c+\delta_c\big),$$

thereby removing the $\phi_{k_{sc}}-\phi_{q_{tc}}$ term that makes RoPE’s preferred relative offset content-dependent (Gopalakrishnan et al., 5 Sep 2025).

The same term also appears in structurally different settings. In Faster DAN, the decoder position is factorized into line index $j$ and token index within line $i$, so target identity and within-target progress are encoded separately as

$$P^{\text{doc}}_{j,i}=\text{concatenate}(P^{\text{1D}'}_j,P^{\text{1D}'}_i),\qquad q_i^j=E_{\hat y_i^j}+P^{\text{doc}}_{j,i},$$

which is a concrete form of target-decoupling for multi-target output streams (Coquenet et al., 2023). In LVLMs, Circle-RoPE defines decoupling as equalizing the positional distance from each text token to all image tokens, so inter-modal positional dependence is removed while intra-text and intra-image structure are retained (Wang et al., 22 May 2025). In multi-view vision, DPPE uses the term more narrowly for pose-decoupling, separating rotation and translation because storing them in the same value dimensions creates non-identifiability (Kenney et al., 30 Jun 2026).

A central misconception is that decoupling is equivalent to using relative position. The papers collectively reject that identification. DIET is explicitly decoupled while offering both absolute and relative variants (Chen et al., 2021). The disentangled encoder uses both AP and RP, but assigns them different roles and excludes AP from MLM supervision (Lequeu et al., 28 May 2026). Conversely, Jordan-RoPE is explicitly a coupled relative mechanism, designed to bind phase and distance in the same defective Jordan block (Zhang, 5 May 2026).

2. Architectural patterns

A first pattern is attention-level decoupling. DIET replaces input-level positional addition with a score decomposition

$$A_{i,j}^{\mathrm{ABS}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +(P_QP_K^\top)_{i,j}+E_S(S(i),S(j)),$$

or, in the relative variant,

$$A_{i,j}^{\mathrm{REL}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +R_{i-j}+E_S(S(i),S(j)).$$

Its formal claim is that additive input embeddings induce a bottleneck: $\mathrm{rank}(A_a)\le d_h$, whereas there exist choices such that $\mathrm{rank}(A_r)=d_p+d_h>d_h$. The same paper also proves that if the loss is $$a^{\text{PoPE}}_{ts}=\sum_{c=1}^{d}|q_{tc}|\,|k_{sc}|\cos\!\big((s-t)\theta_c+\delta_c\big),$$0, then $$a^{\text{PoPE}}_{ts}=\sum_{c=1}^{d}|q_{tc}|\,|k_{sc}|\cos\!\big((s-t)\theta_c+\delta_c\big),$$1, showing that token and positional embeddings receive identical gradients at the point where they are summed (Chen et al., 2021).

A second pattern is stream-level disentanglement with target isolation. The encoder in "Give it Space! Explicit Disentangling of Positional and Semantic Representations in Encoders" (Lequeu et al., 28 May 2026) uses a semantic stream, an AP stream, and an RP attention-logit bias. The hidden width is partitioned as

$$a^{\text{PoPE}}_{ts}=\sum_{c=1}^{d}|q_{tc}|\,|k_{sc}|\cos\!\big((s-t)\theta_c+\delta_c\big),$$2

with a main configuration $$a^{\text{PoPE}}_{ts}=\sum_{c=1}^{d}|q_{tc}|\,|k_{sc}|\cos\!\big((s-t)\theta_c+\delta_c\big),$$3, $$a^{\text{PoPE}}_{ts}=\sum_{c=1}^{d}|q_{tc}|\,|k_{sc}|\cos\!\big((s-t)\theta_c+\delta_c\big),$$4, and $$a^{\text{PoPE}}_{ts}=\sum_{c=1}^{d}|q_{tc}|\,|k_{sc}|\cos\!\big((s-t)\theta_c+\delta_c\big),$$5. Pre-attention and post-attention RMSNorm are applied separately to AP and semantic embeddings. Queries and keys are projected separately in each stream, but both are mapped to the same $$a^{\text{PoPE}}_{ts}=\sum_{c=1}^{d}|q_{tc}|\,|k_{sc}|\cos\!\big((s-t)\theta_c+\delta_c\big),$$6, so the larger semantic stream does not automatically dominate attention-logit scale. Attention logits are then formed by summing semantic, AP, and RP contributions,

$$a^{\text{PoPE}}_{ts}=\sum_{c=1}^{d}|q_{tc}|\,|k_{sc}|\cos\!\big((s-t)\theta_c+\delta_c\big),$$7

while value transport remains separated by stream-specific value and output projections. The feed-forward block is the only explicit AP–semantic mixing mechanism, and MLM is applied only to the semantic part of the last hidden state.

A third pattern is asymmetric query/key control over positional effect. GAPE does not introduce a new positional basis; it augments RoPE with a content-aware additive mask

$$a^{\text{PoPE}}_{ts}=\sum_{c=1}^{d}|q_{tc}|\,|k_{sc}|\cos\!\big((s-t)\theta_c+\delta_c\big),$$8

with

$$a^{\text{PoPE}}_{ts}=\sum_{c=1}^{d}|q_{tc}|\,|k_{sc}|\cos\!\big((s-t)\theta_c+\delta_c\big),$$9

Here $\phi_{k_{sc}}-\phi_{q_{tc}}$0 is a query-dependent gate controlling contraction, and $\phi_{k_{sc}}-\phi_{q_{tc}}$1 is a key-dependent landmark gate controlling protection. This is not target-decoupling in the strict stream-separation sense, but it is an explicit decoupling of query-conditioned distance suppression from key-conditioned token preservation (Ali et al., 11 May 2026).

3. Mechanistic evidence from disentangled encoders

The most detailed mechanistic account comes from the disentangled encoder study (Lequeu et al., 28 May 2026). Its central empirical result is that the AP stream collapses into a low-frequency two-dimensional manifold. PCA on the learned AP embedding matrix shows that the first two principal components explain $\phi_{k_{sc}}-\phi_{q_{tc}}$2 of total variance, whereas the entangled AP baseline allocates only $\phi_{k_{sc}}-\phi_{q_{tc}}$3 of variance to its first two components. A Type-II discrete cosine transform of the top PCs shows that $\phi_{k_{sc}}-\phi_{q_{tc}}$4 and $\phi_{k_{sc}}-\phi_{q_{tc}}$5 of spectral power for the first two PCs lies in the first four frequency bins, while the third component, which explains only $\phi_{k_{sc}}-\phi_{q_{tc}}$6 of variance, is high-frequency. In the hidden states across layers, PCA still reveals about $\phi_{k_{sc}}-\phi_{q_{tc}}$7 variance in the first two PCs, and tokens from the same sentence form clusters that become more separated in deeper layers.

This positional manifold is not merely a geometric curiosity. The paper distinguishes AP from RP by persistence and scale. AP acts as persistent structural memory, retaining normalized token position in the document, segment index, sentence and paragraph grouping, and overall progress through document structure. RP is a local support signal injected at attention time; it does not persist in hidden states and is not found to independently encode macroscopic structure. In probing with ridge regression and $\phi_{k_{sc}}-\phi_{q_{tc}}$8, token-level AP and segment-level AP are near-perfect in earlier layers for the disentangled model, while RoPE and RP are weak. The entangled AP baseline is also strong early, but collapses in the final layer: for token-level AP, AP-NeoBERT falls to $\phi_{k_{sc}}-\phi_{q_{tc}}$9, and for segment-level AP it falls to $j$0. Intra-segment position behaves differently: all models do reasonably well, and in the disentangled model the signal is recoverable almost entirely from the semantic stream, with AP contributing almost nothing.

Attention-head specialization is also quantitatively separated. For each head, the paper removes one component $j$1 before softmax, computes $j$2, and measures

$j$3

Averaging over 500 WikiText documents yields normalized influence vectors that cluster near semantic-dominant or AP-dominant corners, with essentially no purely RP-dominant heads. RP therefore appears as an auxiliary modulator, especially for semantic heads, rather than as an independent positional pathway.

The same paper argues that target-decoupling improves linguistic representation without catastrophic loss on standard encoder benchmarks. On GLUE, MTEB, and SQuAD, the disentangled model is roughly comparable to baselines: averaged GLUE is $j$4 versus $j$5 for AP and RoPE; MTEB average is $j$6, matching RoPE; SQuAD F1 is $j$7, matching RoPE and slightly above AP. On Flash-Holmes, however, the disentangled model performs best in every macro field: morphology $j$8 vs $j$9, reasoning $i$0 vs $i$1-$i$2, semantics $i$3 vs $i$4-$i$5, discourse $i$6 vs $i$7-$i$8, and syntax $i$9 vs $$P^{\text{doc}}_{j,i}=\text{concatenate}(P^{\text{1D}'}_j,P^{\text{1D}'}_i),\qquad q_i^j=E_{\hat y_i^j}+P^{\text{doc}}_{j,i},$$0. It performs best on 49 of 65 phenomena.

4. Representative instantiations across domains

The same design principle recurs in markedly different modalities. In some systems, the decoupled object is the prediction target; in others it is the attention pathway, the output-target identity, the modality interface, or the pose factorization.

System	Decoupled object	Representative evidence
DIET (Chen et al., 2021)	Position and segment from token/content at attention score level	GLUE $$P^{\text{doc}}_{j,i}=\text{concatenate}(P^{\text{1D}'}_j,P^{\text{1D}'}_i),\qquad q_i^j=E_{\hat y_i^j}+P^{\text{doc}}_{j,i},$$1; XTREME $$P^{\text{doc}}_{j,i}=\text{concatenate}(P^{\text{1D}'}_j,P^{\text{1D}'}_i),\qquad q_i^j=E_{\hat y_i^j}+P^{\text{doc}}_{j,i},$$2; Diet-Abs and Diet-Rel remain near baseline overhead
Disentangled encoder (Lequeu et al., 28 May 2026)	Semantic stream, AP stream, RP bias, and MLM target placement	First two AP PCs explain $$P^{\text{doc}}_{j,i}=\text{concatenate}(P^{\text{1D}'}_j,P^{\text{1D}'}_i),\qquad q_i^j=E_{\hat y_i^j}+P^{\text{doc}}_{j,i},$$3; Flash-Holmes best on 49/65 phenomena
Faster DAN (Coquenet et al., 2023)	Line identity $$P^{\text{doc}}_{j,i}=\text{concatenate}(P^{\text{1D}'}_j,P^{\text{1D}'}_i),\qquad q_i^j=E_{\hat y_i^j}+P^{\text{doc}}_{j,i},$$4 from within-line index $$P^{\text{doc}}_{j,i}=\text{concatenate}(P^{\text{1D}'}_j,P^{\text{1D}'}_i),\qquad q_i^j=E_{\hat y_i^j}+P^{\text{doc}}_{j,i},$$5 in decoder queries	No line encoding yields CER $$P^{\text{doc}}_{j,i}=\text{concatenate}(P^{\text{1D}'}_j,P^{\text{1D}'}_i),\qquad q_i^j=E_{\hat y_i^j}+P^{\text{doc}}_{j,i},$$6, $$P^{\text{doc}}_{j,i}=\text{concatenate}(P^{\text{1D}'}_j,P^{\text{1D}'}_i),\qquad q_i^j=E_{\hat y_i^j}+P^{\text{doc}}_{j,i},$$7, $$P^{\text{doc}}_{j,i}=\text{concatenate}(P^{\text{1D}'}_j,P^{\text{1D}'}_i),\qquad q_i^j=E_{\hat y_i^j}+P^{\text{doc}}_{j,i},$$8; prediction-time speedups from $$P^{\text{doc}}_{j,i}=\text{concatenate}(P^{\text{1D}'}_j,P^{\text{1D}'}_i),\qquad q_i^j=E_{\hat y_i^j}+P^{\text{doc}}_{j,i},$$9 to $$A_{i,j}^{\mathrm{ABS}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +(P_QP_K^\top)_{i,j}+E_S(S(i),S(j)),$$0
PoPE (Gopalakrishnan et al., 5 Sep 2025)	Content magnitudes from positional phase in rotary scoring	Indirect Indexing $$A_{i,j}^{\mathrm{ABS}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +(P_QP_K^\top)_{i,j}+E_S(S(i),S(j)),$$1 vs RoPE $$A_{i,j}^{\mathrm{ABS}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +(P_QP_K^\top)_{i,j}+E_S(S(i),S(j)),$$2; OpenWebText gains from 124M to 774M
Circle-RoPE (Wang et al., 22 May 2025)	Text-image positional dependence across modalities	PTD $$A_{i,j}^{\mathrm{ABS}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +(P_QP_K^\top)_{i,j}+E_S(S(i),S(j)),$$3 vs spatial embedding $$A_{i,j}^{\mathrm{ABS}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +(P_QP_K^\top)_{i,j}+E_S(S(i),S(j)),$$4; average score $$A_{i,j}^{\mathrm{ABS}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +(P_QP_K^\top)_{i,j}+E_S(S(i),S(j)),$$5 on Qwen2.5-VL-3B
DPPE (Kenney et al., 30 Jun 2026)	Rotation from translation in camera-based value/output encoding	MVImgNet2: PRoPE $$A_{i,j}^{\mathrm{ABS}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +(P_QP_K^\top)_{i,j}+E_S(S(i),S(j)),$$6, DPPE$$A_{i,j}^{\mathrm{ABS}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +(P_QP_K^\top)_{i,j}+E_S(S(i),S(j)),$$7 $$A_{i,j}^{\mathrm{ABS}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +(P_QP_K^\top)_{i,j}+E_S(S(i),S(j)),$$8

Faster DAN is the clearest example of target identity as a positional axis. Its two-pass decoder predicts layout tokens and first characters of lines in a first pass, then completes all lines in parallel in a second pass. The positional encoding $$A_{i,j}^{\mathrm{ABS}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +(P_QP_K^\top)_{i,j}+E_S(S(i),S(j)),$$9 lets the same decoder process multiple line-prefix targets simultaneously. The paper reports that removing line encoding and reverting to standard 1D positional encoding makes the model fail: CER rises to $$A_{i,j}^{\mathrm{REL}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +R_{i-j}+E_S(S(i),S(j)).$$0 on RIMES 2009, $$A_{i,j}^{\mathrm{REL}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +R_{i-j}+E_S(S(i),S(j)).$$1 on READ 2016 single-page, and $$A_{i,j}^{\mathrm{REL}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +R_{i-j}+E_S(S(i),S(j)).$$2 on READ 2016 double-page. By contrast, the full scheme is at least 4 times faster on whole single-page and double-page images and reaches speedups of $$A_{i,j}^{\mathrm{REL}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +R_{i-j}+E_S(S(i),S(j)).$$3, $$A_{i,j}^{\mathrm{REL}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +R_{i-j}+E_S(S(i),S(j)).$$4, $$A_{i,j}^{\mathrm{REL}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +R_{i-j}+E_S(S(i),S(j)).$$5, $$A_{i,j}^{\mathrm{REL}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +R_{i-j}+E_S(S(i),S(j)).$$6, and $$A_{i,j}^{\mathrm{REL}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +R_{i-j}+E_S(S(i),S(j)).$$7 on the reported datasets (Coquenet et al., 2023).

PoPE and Circle-RoPE both decouple within RoPE-style systems, but they do so differently. PoPE removes the token-dependent phase shift from RoPE, so content enters only through nonnegative magnitudes and position enters only through phase; it improves validation perplexity on OpenWebText at 124M, 253M, and 774M, and exhibits strong zero-shot length extrapolation relative to RoPE and YaRN (Gopalakrishnan et al., 5 Sep 2025). Circle-RoPE instead targets multimodal cross-bias. It maps image token positions onto a circle in a plane orthogonal to the text direction, yielding PTD $$A_{i,j}^{\mathrm{REL}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +R_{i-j}+E_S(S(i),S(j)).$$8 in the ideal construction and improving tasks such as MMMU, MathVista, AI2D, and TextVQA under fine-tuning on Qwen2.5-VL-3B (Wang et al., 22 May 2025).

DPPE extends the same logic to camera-aware multi-view attention. Its diagnosis is that PRoPE’s value/output transform stores rotation and translation in the same output coordinates, so infinitesimal changes in rotation can be compensated by translation perturbations without changing the transformed value. DPPE$$A_{i,j}^{\mathrm{REL}}=(X_{i:}W_Q)(X_{j:}W_K)^\top/\sqrt d +R_{i-j}+E_S(S(i),S(j)).$$9 fixes this by splitting channels into rotation and translation blocks, using multiplicative rotation encoding, additive translation encoding, and separate attention weights for the two branches. The paper reports that PRoPE stagnates and degrades late in training, whereas DPPE remains stable, including in a 24-layer, 1M-iteration setting (Kenney et al., 30 Jun 2026).

5. Coupled counterpositions and theoretical debate

Target-decoupling is not presented in the literature as an unconditional optimum. The clearest counterposition is "Jordan-RoPE: Non-Semisimple Relative Positional Encoding via Complex Jordan Blocks" (Zhang, 5 May 2026), which argues that some positional interactions are intrinsically coupled at the primitive attention-logit level. Jordan-RoPE places a complex rotary eigenvalue and a nilpotent response in the same defective Jordan block, making basis functions such as

$\mathrm{rank}(A_a)\le d_h$0

available directly in the bilinear score. On the mixed target $\mathrm{rank}(A_a)\le d_h$1, exact/raw Jordan reports MSE $\mathrm{rank}(A_a)\le d_h$2, scaled-exact $\mathrm{rank}(A_a)\le d_h$3 reports $\mathrm{rank}(A_a)\le d_h$4, while RoPE reports $\mathrm{rank}(A_a)\le d_h$5, RoPE+ALiBi $\mathrm{rank}(A_a)\le d_h$6, and direct-sum $\mathrm{rank}(A_a)\le d_h$7. On a Jordan-friendly synthetic language-model task, stabilized Jordan-RoPE reaches $\mathrm{rank}(A_a)\le d_h$8 at evaluation length 8192, compared with RoPE $\mathrm{rank}(A_a)\le d_h$9, RoPE+ALiBi $\mathrm{rank}(A_r)=d_p+d_h>d_h$0, and direct-sum $\mathrm{rank}(A_r)=d_p+d_h>d_h$1. Yet on the small WikiText-103 byte LM, RoPE+ALiBi remains strongest overall, and the paper explicitly describes the evidence as structural rather than a broad performance claim.

A second counterposition comes from "Unpacking Positional Encoding in Transformers: A Spectral Analysis of Content-Position Coupling" (Gu et al., 19 May 2025). That paper treats relative positional effects as Toeplitz structure and argues that multiplicative content-position coupling, exemplified by RoPE, induces spectral contraction. On its content-position dependent task, RoPE performs best, converges fastest, and generalizes best. The same study also identifies a mechanistic cost of hard coupling: “single-head deposit,” in which a shallow-layer head becomes disproportionately responsible for positional processing. Its MLA formulation partially separates positional and non-positional channels and removes the deposit while retaining high Task 1 accuracy $\mathrm{rank}(A_r)=d_p+d_h>d_h$2 and greatly improving Task 2 accuracy $\mathrm{rank}(A_r)=d_p+d_h>d_h$3 relative to RoPE’s $\mathrm{rank}(A_r)=d_p+d_h>d_h$4. A plausible implication is that the most effective designs may be hybrid rather than purely decoupled or purely coupled.

Broader operator frameworks complicate the picture further. Algebraic Positional Encodings factorize relative relations through separate query-side and key-side orthogonal operators, but are not explicitly target-decoupled; absolute positions can be applied to queries or keys, but not both, and the framework is primarily about preserving algebraic structure across sequences, trees, and grids (Kogkalidis et al., 2023). GridPE is likewise not explicitly target-decoupled: each token receives an absolute Fourier/grid-cell code, but query–key inner products become functions of relative displacement, making the positional effect implicitly relative rather than explicitly target-separated (Li et al., 2024).

6. Empirical profile, limitations, and open questions

Across the surveyed work, decoupling is valuable when positional information would otherwise be overwritten, aliased, or forced through an unsuitable bottleneck. The encoder study shows that preserving a dedicated positional channel improves linguistic probing without crippling GLUE, MTEB, or SQuAD (Lequeu et al., 28 May 2026). DIET shows that moving position and segment into attention can be both simpler and faster than heavier relative methods, with Diet-Rel using only 1.6% of the time and 17.6% of the space complexity of positional embedding variants for that positional component in the reported BERT setting (Chen et al., 2021). GAPE shows that protected tokens remain accessible while attention mass on unprotected distant tokens decays as a function of the query gate, and reports sharper attention and improved long-context robustness over rotary baselines (Ali et al., 11 May 2026). PoPE improves evaluation loss and downstream task performance in music, genomics, and natural language, with gains persisting from 124M to 774M parameters (Gopalakrishnan et al., 5 Sep 2025).

The limitations are equally consistent. The disentangled encoder study is modest in scale—6 layers, 6 heads, around 22B training tokens, encoder-only, and maximum sequence length 512—and does not validate its long-context motivation directly on very long contexts (Lequeu et al., 28 May 2026). DIET’s central results are on encoder-style or encoder–decoder settings rather than decoder-only language modeling (Chen et al., 2021). Circle-RoPE’s best practical setting uses fusion weight $\mathrm{rank}(A_r)=d_p+d_h>d_h$5, which means the deployed model intentionally relaxes exact inter-modal decoupling to preserve image spatial fidelity (Wang et al., 22 May 2025). GAPE is evaluated up to 124M parameters in the reported long-context tables, so its large-scale LLM behavior remains open (Ali et al., 11 May 2026). Jordan-RoPE explicitly warns that coupled primitive bases are beneficial only when the target interaction is itself phase-distance coupled, and its natural-language evidence does not establish broad superiority (Zhang, 5 May 2026).

Several open questions recur. The disentangled encoder paper explicitly leaves open whether the same three-stream separation works in decoder-only LMs, how much AP dimensionality is actually needed, and whether the low-dimensional AP manifold can be exploited for long-context extrapolation, retrieval, or caching semantic states independently of position (Lequeu et al., 28 May 2026). DPPE suggests that the value/output path may require stronger decoupling than the query–key path, since DPPE$\mathrm{rank}(A_r)=d_p+d_h>d_h$6 works best when applied only to VO, with PRoPE retained for QK (Kenney et al., 30 Jun 2026). The spectral and Jordan papers jointly suggest that some tasks may require coupled primitive score functions, whereas others benefit from keeping positional structure separate until later fusion (Gu et al., 19 May 2025, Zhang, 5 May 2026). This suggests that target-decoupled positional encoding is less a single mechanism than a design space: one defined by where position enters, which variables are allowed to mix, and which parts of the model or objective are allowed to overwrite positional state.