---
title: 'Lasformer: Ambiguous Transformer Interpretations'
url: https://www.emergentmind.com/topics/lasformer
type: topic
---

# Lasformer: Ambiguous Transformer Interpretations

Searching arXiv for the relevant papers and naming variants to ground the article.
arxiv_search query: Lasformer transformer LATFormer Lformer GLassoformer LAformer 
Within the cited arXiv literature, **“Lasformer” does not designate a single uniformly named architecture**. Instead, the label is associated with several nearby names and usages: it appears as a likely typo for **LATFormer**, the **Locality-Aware Point-View Fusion Transformer** for 3D shape recognition [2109.01291]; it is closely confusable with **Lformer**, a semi-autoregressive text-to-image generator based on L-shape block parallel decoding [2303.03800]; it overlaps conceptually with **GLassoformer**, where a “Lasformer” reading corresponds to a lasso-regularized transformer with group-sparse queries for post-fault power grid voltage prediction [2201.09145]; and it resembles **LAformer**, a lane-aware transformer for autonomous-driving trajectory prediction [2302.13933]. The term therefore has to be interpreted contextually, by domain and by the mechanism attached to the specific paper.

## 1. Nomenclature and scope

The principal source of ambiguity is that several arXiv papers use near-identical names while addressing unrelated problems. In the 3D recognition setting, the manuscript explicitly states that **“Lasformer” appears to be a typo for LATFormer**, and that the manuscript and results consistently refer to **LATFormer** [2109.01291]. In the power-systems setting, “Lasformer” can be read as **Lasso + Transformer**, with **GLassoformer** representing a group-Lasso instantiation of that idea [2201.09145]. In text-to-image generation and trajectory prediction, the relevant published names are **Lformer** and **LAformer**, not “Lasformer” [2303.03800], [2302.13933].

| Name in paper | Expansion | Primary task |
|---|---|---|
| LATFormer | Locality-Aware Point-View Fusion Transformer | 3D shape retrieval and classification |
| Lformer | Text-to-Image Generation with L-shape Block Parallel Decoding | Text-to-image generation |
| GLassoformer | Group-Lasso query-sparse transformer | Post-fault power grid voltage prediction |
| LAformer | Lane-Aware scene-constraint transformer | Autonomous-driving trajectory prediction |

A common misconception is therefore to treat “Lasformer” as the title of a single recognized architecture. The cited literature supports the opposite conclusion: the name is **domain-dependent**, and in some contexts it is simply incorrect.

## 2. LATFormer: locality-aware point-view fusion for 3D recognition

**LATFormer** is a multimodal architecture for **3D shape retrieval and classification** that fuses **point clouds** and **multi-view images** by modeling **where to fuse**, rather than only aggregating pooled global descriptors [2109.01291]. The problem setting is motivated by the complementary properties of the two modalities: point clouds capture precise 3D geometry but can be sparse and sensitive to noise and occlusions, whereas multi-view images capture rich appearance cues and view-based silhouette or contour information but are sensitive to camera and view configurations. The method targets the gap in prior approaches that typically fuse global features and thereby ignore spatial correspondence between local parts across modalities.

The architecture takes a point cloud \(P\) and multi-view images \(V=\{I_1,\dots,I_M\}\). A point encoder produces multi-scale local features \(\{F_{p,1},F_{p,2},\dots,F_{p,S}\}\), and a view encoder produces multi-scale features \(\{F_{v,1},F_{v,2},\dots,F_{v,S}\}\). The core module is **Locality-Aware Fusion (LAF)**, which organizes fusion at the level of local regions or tokens, computes co-occurrence between local point features and local view features, filters low-saliency pairs, and fuses information **bidirectionally** and **hierarchically**. In the provided technical summary, a faithful template consistent with the manuscript describes the co-occurrence mechanism as
\[
s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad
\alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad
m_{ij}=[s_{ij}\ge \tau],
\]
followed by point-to-view and view-to-point aggregation,
\[
\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad
\beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad
\tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.
\]
The manuscript excerpt does not provide a more specific similarity function, and the summary notes that these formulas should be viewed as a faithful template rather than verbatim manuscript equations.

Hierarchical fusion is performed across scales, for example at \(s=1,2,3\), and the fused representation is pooled for classification or retrieval. The paper states that LATFormer “obtains higher performance but with much fewer parameters” than previous multimodal methods and is robust to arbitrary view settings. A plausible implication is that the thresholded co-occurrence mask reduces redundant cross-modal aggregation by retaining only salient correspondences.

The empirical results reported in the provided summary are strongest on **ScanObjectNN** and **ModelNet40**. On **ScanObjectNN**, LATFormer with points plus depth images achieves **OA 88.7** and **mAcc 87.1**. This exceeds the best depth-view-only baseline, **SimpleView++**, by **+3.9% OA**, and slightly surpasses strong point-only baselines such as **PointNeXt+HyCoRe** and **PointVector-S**. On **ModelNet40**, single-scale fusion \(\{(V1,P1)\}\) already reaches **OA 93.8** and **mAcc 91.2**, while three corresponding scales \(\{(V1,P1),(V2,P2),(V3,P3)\}\) reach **OA 94.4** and **mAcc 92.2**. The reported ablation indicates that **matching scales** \(V_s\) with \(P_s\) tends to perform slightly better than cross-scale pairings, suggesting that locality alignment across modalities matters.

## 3. Lformer: L-shape block parallel decoding for text-to-image generation

**Lformer** addresses a different problem: **semi-autoregressive text-to-image generation** using **L-shape block parallel decoding** [2303.03800]. Standard autoregressive text-to-image transformers generate discrete image tokens one by one in raster order, which is slow for long sequences. Lformer accelerates sampling while retaining unidirectional decoding benefits by dividing an \(h\times h\) token grid into \(h\) mirrored **L-shape blocks** and decoding all tokens in each block in parallel.

The first stage uses a **VQ-VAE/VQGAN-style** tokenizer with encoder \(E\), decoder \(D\), and codebook \(e=\{e_k\}_{k=1}^K\subset\mathbb{R}^{n_e}\). Quantization is described as
\[
\mathbf{q}(\hat e_{ij})=e_k,\qquad
k=\arg\min_k\|\hat e_{ij}-e_k\|,
\]
with reconstruction
\[
\hat x=D(\tilde e)=D(\mathbf q(E(x))).
\]
Text conditioning is provided by a **frozen CLIP text encoder**, and Lformer further augments conditioning with a **CVAE latent** \(z\) mapped by an MLP into a StyleGAN-like \(W\) space, yielding
\[
c=[\hat c;f(z)].
\]

The distinctive factorization replaces standard autoregressive token ordering with blockwise L-order decoding:
\[
p(s|c)=p(s_1|c)\cdot\prod_{t=2}^{h}
p\big(s_{[(t-1)^2+1:t^2]}\,\big|\,s_{[1:(t-1)^2]},c\big).
\]
At step \(t\), the model predicts the entire \(t\)-th L-block adjacent to the previously generated square context. The training objective is token-level cross-entropy with teacher forcing,
\[
L=\frac{1}{h^2}\sum_{t=1}^{h}\mathrm{CE}(b_t',b_t),
\]
and the paper also introduces a **CVAE objective** to mitigate conditional independence within a block:
\[
\mathcal{L}_{\mathrm{CVAE}}=
-\mathrm{KL}\big(q_\phi(z|x,\hat c)\,\|\,p_\theta(z|\hat c)\big)
+\mathbb{E}_{q_\phi(z|x,\hat c)}\big(\log p_\theta(x|z,\hat c)\big).
\]

The implementation uses a decoder-style transformer with cross-attention. Reported backbone configurations include **L=19, H=16, D=1024** for speed tests, **L=24, H=16, D=1024** on MMCelebA-HQ, and **L=24, H=16, D=1536** for large-scale MS-COCO/LAION experiments. The paper also reports **PB-relax**, **Sandwich LayerNorm**, mixed precision, activation checkpointing, **ZeRO optimization**, and block-sparse attention.

The quantitative profile emphasizes both speed and quality. On a **V100 GPU** with batch size 32 and **32×32 tokens**, **Lformer-cache** achieves **3.57 s/batch**, **0.112 s/img**, and **8.96 img/s**, compared with **33.94 s/batch**, **1.06 s/img**, and **0.942 img/s** for **AR-cache**. On **MMCelebA-HQ** at **256×256**, the **Lformer-L** configuration reports **FID 19.91** at **0.046 s/img**, while the **1B-parameter Lformer-E** reaches **FID 18.60** at **0.076 s/img**. On **MS-COCO 2014**, **Lformer-E** pretrained on **LAION90M** reports **FID 15.01**, **IS 26.22**, and **CLIPSIM 0.3289**, with a finetuned **FID 9.57**. The same model also supports **image editing without finetuning**, including rollback-based regeneration, repainting, and bounding-box-guided inpainting.

## 4. GLassoformer and the lasso-regularized interpretation of “Lasformer”

In the power-systems literature, the most direct “Lasformer” interpretation is **Lasso + Transformer**, instantiated concretely by **GLassoformer**, an encoder–decoder transformer for **post-fault power grid voltage prediction** [2201.09145]. The model introduces **structured sparsity on the query vectors in self-attention via group Lasso**, with the explicit goal of reducing active queries, cutting compute and memory, and improving stability.

The key mechanism applies group Lasso to the **query projection matrix** \(W_Q\). For input sequence \(\tilde X\), the attention projections are
\[
Q=W_Q^\top\tilde X^\top,\qquad
K=W_K^\top\tilde X^\top,\qquad
V=W_V^\top\tilde X^\top,
\]
and standard attention is
\[
\hat X=\operatorname{softmax}\Big(\frac{QK^\top}{\sqrt D}\Big)V.
\]
Columns of \(W_Q\) are partitioned into groups \(\{W_1,\dots,W_N\}\), one per query index, and the penalized objective is
\[
L(\theta,W_Q)=f(\theta,W_Q)+\lambda\|W_Q\|_{GL},\qquad
\|W_Q\|_{GL}=\sum_{g=1}^N\|W_g\|_2.
\]
Training uses the **Relaxed Group-wise Splitting Method (RGSM)** with proximal group soft-thresholding,
\[
y_g^*=\mathbf{Prox}_{GL,\lambda}(W_g)
\triangleq
W_g\cdot \max(\|W_g\|_2-\lambda,0)/\|W_g\|_2.
\]
Because \(Q=W_Q^\top X^\top\), zeroing a column of \(W_Q\) yields a zero row in \(Q\), so entire query vectors are eliminated rather than merely sparsified entrywise.

The architecture uses embeddings formed by **padding and 1D convolutions** concatenated with a **global time-stamp channel**, then an encoder with **two identical blocks** of multi-head group-sparse self-attention plus **1D convolution with ELU**, and a decoder with group-sparse self-attention, canonical multi-head cross-attention, and a fully connected output projection. The reported convergence guarantee assumes a Lipschitz gradient, with the paper noting that **ELU ensures this**.

The application dataset is a simulated **New York/New England 16-generator, 68-bus power system**, containing **2248 fault events**, each spanning **10 seconds**. The train/validation/test split is **1000/350/750**, and the reported optimizer is **Adam** with learning rate \(\eta=1\times 10^{-4}\), decayed by **0.8 every 10 epochs**; group-Lasso hyperparameters are \(\beta=0.9\) and \(\lambda=0.01\), batch size is **30**, and maximum epochs is **80** with early stopping patience **30**.

The quantitative results favor GLassoformer over the cited baselines. In **Case I** with nearby voltage and line-current features, the model achieves **MSE \(3.189\times 10^{-5}\)** and **MAE \(2.374\times 10^{-3}\)**, compared with **Informer** at **\(3.662\times 10^{-5}\)** and **\(2.684\times 10^{-3}\)** and **Lasso** at **\(3.532\times 10^{-5}\)** and **\(2.543\times 10^{-3}\)**. In **Case II** using single-bus information only, GLassoformer reports **MSE \(6.115\times 10^{-5}\)** and **MAE \(3.520\times 10^{-3}\)**. The paper also reports a **pruning rate of 19.09%**, compared with **4.220%** for Informer, **2.674%** for unstructured Lasso, and **2.3%** for a vanilla transformer. Inference time on **GTX-1080Ti** is **18.98 ms** for GLassoformer, faster than **Informer** at **29.76 ms**, though slower than **Lasso** at **14.92 ms** and close to the **vanilla transformer** at **15.32 ms**.

The paper explicitly contrasts **GLassoformer** with a baseline **“LassoFormer”** that uses regular, unstructured Lasso. The distinction is central: unstructured penalties yield scattered sparsity, whereas group Lasso zeros entire query rows, enabling both computational savings and query-level interpretability.

## 5. LAformer: lane-aware trajectory prediction under scene constraints

**LAformer** is another separate model family, aimed at **multimodal trajectory prediction for autonomous driving** under **lane-aware scene constraints** [2302.13933]. It is a **two-stage, map-aware motion forecasting model**. The first stage performs temporally dense lane-aware estimation and anchor-trajectory generation, while the second stage refines the selected prediction to improve temporal consistency across the full forecast horizon.

The inputs are vectorized trajectories and vectorized lane centerlines, normalized around the target agent’s last observed position \(X_0^i\) for translation invariance. Motion features \(h_i\) and lane features \(c_j\) are obtained through **MLPs followed by GRUs**, and the **Global Interaction Graph (GIG)** applies symmetric cross-attention:
\[
h_i\leftarrow h_i+\operatorname{CrossAtt}(Q=h_i,K=c_j,V=c_j),\qquad
c_j\leftarrow c_j+\operatorname{CrossAtt}(Q=c_j,K=h_i,V=h_i),
\]
followed by concatenation and self-attention among agents. The attention operator is standard scaled dot-product attention,
\[
\operatorname{Attention}(Q,K,V)=
\operatorname{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.
\]

A distinctive feature is the **temporally dense lane-aware estimation module**. At each future step \(t\), the model scores every lane segment:
\[
\hat s_{j,t}=
\frac{\exp(\phi(h_i,c_j,A_{i,j},t))}
{\sum_{n=1}^{N_{\text{lane}}}\exp(\phi(h_i,c_n,A_{i,n},t))}.
\]
For each \(t\), it retains only the **top-\(K\)** segments, with **\(K=2\) by default**, and concatenates them across time to form a spatio-temporal lane token sequence
\[
C^\star=\operatorname{Concat}\{(c_{t,1:K},\hat s_{t,1:K})\}_{t=1}^{t_f}.
\]
The lane-selection objective is a binary cross-entropy over all lanes and time steps,
\[
\mathcal L_{\mathrm{lane}}=
\sum_{t=1}^{t_f}\sum_{j=1}^{N_{\text{lane}}}
\mathrm{CE}(s_{j,t},\hat s_{j,t}),
\]
where \(s_{j,t}=1\) for the segment closest to the ground-truth position \(Y_t\).

Stage 1 predicts a **Laplacian Mixture Density Network** over future trajectories,
\[
p(\mathbf y_{1:t_f})\approx
\sum_{m=1}^M \hat\pi_m
\prod_{t=1}^{t_f}\mathrm{Laplace}(\mathbf y_t\mid \mu_{m,t},b_{m,t}),
\]
with a Winner-Takes-All regression loss and a mode-classification loss. Stage 2 refines the selected anchor trajectory via an offset regressor and an **angle-consistency loss**,
\[
\mathcal L_{\mathrm{angle}}=
\frac{1}{t_f}\sum_{t=1}^{t_f}
\big[-\cos(\hat\theta_t-\theta_t)\big].
\]
The total losses are reported as
\[
\mathcal L_{\mathrm{S1}}=\lambda_1\mathcal L_{\mathrm{lane}}+\mathcal L_{\mathrm{reg}}+\mathcal L_{\mathrm{cls}},
\qquad
\mathcal L_{\mathrm{S2}}=\mathcal L_{\mathrm{S1}}+\lambda_2\mathcal L_{\mathrm{off}}+\lambda_3\mathcal L_{\mathrm{angle}},
\]
with \(\lambda_1=10\), \(\lambda_2=5\), and \(\lambda_3=2\).

The reported evaluations cover **Argoverse 1** and **nuScenes**. On **Argoverse 1**, LAformer reports **validation ADE 0.64, FDE 0.92**, and **test ADE 0.77, FDE 1.16**, which is on par or slightly better than the cited **HiVT** configuration. On **nuScenes**, LAformer reports **ADE 1.19, MR 0.48** for **\(K=5\)** and **ADE 0.93, MR 0.33** for **\(K=10\)**. The ablation study attributes much of the gain to temporally dense lane scoring and the second-stage refinement: a baseline without lanes and without stage 2 gives **ADE 0.72, FDE 1.12**, whereas the full model gives **ADE 0.64, FDE 0.92**. The reported parameter count is **2.654M**, with **~115 ms per scene** inference for an average of **12 agents**.

## 6. Comparative interpretation and boundary cases

Across these papers, the nearest coherent reading of “Lasformer” is not a shared architecture but a **family of naming collisions around transformer selectivity mechanisms**. **LATFormer** selects salient **local point-view correspondences** by co-occurrence filtering [2109.01291]. **Lformer** selects an **adjacent L-shape block** of image tokens at each generation step rather than decoding the full raster sequence [2303.03800]. **GLassoformer** selects only the **nonzero query rows** induced by group Lasso on \(W_Q\) [2201.09145]. **LAformer** selects **top-\(K\) lane segments per future time step** before decoding [2302.13933]. This suggests a recurrent design preference for reducing irrelevant interactions before or during attention, although the application domains, supervision schemes, and computational goals differ substantially.

Another boundary case is phonetic rather than architectural. **“Metal Blossom: Laser Forming Complex and Freeform Metal Structures Imitating Flower Blooming”** concerns **laser forming origami (lasergami)** rather than transformers [2111.15083]. Its subject is metal-sheet folding through computational origami, nearly-blooming net generation, and as-folded-as-possible motion planning, not sequence modeling or attention. The similarity between “laser forming” and “Lasformer” can therefore create an additional naming confusion, but the work belongs to computational fabrication rather than transformer design.

In technical usage, precision in capitalization and expansion is therefore decisive. In 3D recognition, the correct name is **LATFormer**; in text-to-image generation, **Lformer**; in power-grid forecasting, **GLassoformer** or **LassoFormer** in the lasso-regularized sense; and in trajectory prediction, **LAformer**. Any encyclopedia treatment of “Lasformer” is most accurate when it treats the term as an **ambiguous label** whose meaning must be resolved by context, task, and the exact arXiv title.

Source: https://www.emergentmind.com/topics/lasformer