Lasformer: Ambiguous Transformer Interpretations

• Lasformer is an ambiguous transformer label encompassing various domain-specific models such as LATFormer, Lformer, GLassoformer, and LAformer.
• It involves distinct mechanisms from locality-aware fusion in 3D recognition to L-shape block parallel decoding in text-to-image generation and group-Lasso regulated sparsity in power systems.
• The context-specific usage of the term underscores the importance of precise naming to ensure proper application and performance interpretation across diverse domains.

Searching arXiv for the relevant papers and naming variants to ground the article. arxiv_search query: Lasformer transformer LATFormer Lformer GLassoformer LAformer Within the cited arXiv literature, “Lasformer” does not designate a single uniformly named architecture. Instead, the label is associated with several nearby names and usages: it appears as a likely typo for LATFormer, the Locality-Aware Point-View Fusion Transformer for 3D shape recognition (He et al., 2021); it is closely confusable with Lformer, a semi-autoregressive text-to-image generator based on L-shape block parallel decoding (Li et al., 2023); it overlaps conceptually with GLassoformer, where a “Lasformer” reading corresponds to a lasso-regularized transformer with group-sparse queries for post-fault power grid voltage prediction (Zheng et al., 2022); and it resembles LAformer, a lane-aware transformer for autonomous-driving trajectory prediction (Liu et al., 2023). The term therefore has to be interpreted contextually, by domain and by the mechanism attached to the specific paper.

1. Nomenclature and scope

The principal source of ambiguity is that several arXiv papers use near-identical names while addressing unrelated problems. In the 3D recognition setting, the manuscript explicitly states that “Lasformer” appears to be a typo for LATFormer, and that the manuscript and results consistently refer to LATFormer (He et al., 2021). In the power-systems setting, “Lasformer” can be read as Lasso + Transformer, with GLassoformer representing a group-Lasso instantiation of that idea (Zheng et al., 2022). In text-to-image generation and trajectory prediction, the relevant published names are Lformer and LAformer, not “Lasformer” (Li et al., 2023, Liu et al., 2023).

Name in paper	Expansion	Primary task
LATFormer	Locality-Aware Point-View Fusion Transformer	3D shape retrieval and classification
Lformer	Text-to-Image Generation with L-shape Block Parallel Decoding	Text-to-image generation
GLassoformer	Group-Lasso query-sparse transformer	Post-fault power grid voltage prediction
LAformer	Lane-Aware scene-constraint transformer	Autonomous-driving trajectory prediction

A common misconception is therefore to treat “Lasformer” as the title of a single recognized architecture. The cited literature supports the opposite conclusion: the name is domain-dependent, and in some contexts it is simply incorrect.

2. LATFormer: locality-aware point-view fusion for 3D recognition

LATFormer is a multimodal architecture for 3D shape retrieval and classification that fuses point clouds and multi-view images by modeling where to fuse, rather than only aggregating pooled global descriptors (He et al., 2021). The problem setting is motivated by the complementary properties of the two modalities: point clouds capture precise 3D geometry but can be sparse and sensitive to noise and occlusions, whereas multi-view images capture rich appearance cues and view-based silhouette or contour information but are sensitive to camera and view configurations. The method targets the gap in prior approaches that typically fuse global features and thereby ignore spatial correspondence between local parts across modalities.

The architecture takes a point cloud $P$ and multi-view images $V=\{I_1,\dots,I_M\}$. A point encoder produces multi-scale local features $\{F_{p,1},F_{p,2},\dots,F_{p,S}\}$, and a view encoder produces multi-scale features $\{F_{v,1},F_{v,2},\dots,F_{v,S}\}$. The core module is Locality-Aware Fusion (LAF), which organizes fusion at the level of local regions or tokens, computes co-occurrence between local point features and local view features, filters low-saliency pairs, and fuses information bidirectionally and hierarchically. In the provided technical summary, a faithful template consistent with the manuscript describes the co-occurrence mechanism as

$$s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad \alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad m_{ij}=[s_{ij}\ge \tau],$$

followed by point-to-view and view-to-point aggregation,

$$\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad \beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad \tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.$$

The manuscript excerpt does not provide a more specific similarity function, and the summary notes that these formulas should be viewed as a faithful template rather than verbatim manuscript equations.

Hierarchical fusion is performed across scales, for example at $s=1,2,3$, and the fused representation is pooled for classification or retrieval. The paper states that LATFormer “obtains higher performance but with much fewer parameters” than previous multimodal methods and is robust to arbitrary view settings. A plausible implication is that the thresholded co-occurrence mask reduces redundant cross-modal aggregation by retaining only salient correspondences.

The empirical results reported in the provided summary are strongest on ScanObjectNN and ModelNet40. On ScanObjectNN, LATFormer with points plus depth images achieves OA 88.7 and mAcc 87.1. This exceeds the best depth-view-only baseline, SimpleView++, by +3.9% OA, and slightly surpasses strong point-only baselines such as PointNeXt+HyCoRe and PointVector-S. On ModelNet40, single-scale fusion $\{(V1,P1)\}$ already reaches OA 93.8 and mAcc 91.2, while three corresponding scales $\{(V1,P1),(V2,P2),(V3,P3)\}$ reach OA 94.4 and mAcc 92.2. The reported ablation indicates that matching scales $V_s$ with $V=\{I_1,\dots,I_M\}$0 tends to perform slightly better than cross-scale pairings, suggesting that locality alignment across modalities matters.

3. Lformer: L-shape block parallel decoding for text-to-image generation

Lformer addresses a different problem: semi-autoregressive text-to-image generation using L-shape block parallel decoding (Li et al., 2023). Standard autoregressive text-to-image transformers generate discrete image tokens one by one in raster order, which is slow for long sequences. Lformer accelerates sampling while retaining unidirectional decoding benefits by dividing an $V=\{I_1,\dots,I_M\}$1 token grid into $V=\{I_1,\dots,I_M\}$2 mirrored L-shape blocks and decoding all tokens in each block in parallel.

The first stage uses a VQ-VAE/VQGAN-style tokenizer with encoder $V=\{I_1,\dots,I_M\}$3, decoder $V=\{I_1,\dots,I_M\}$4, and codebook $V=\{I_1,\dots,I_M\}$5. Quantization is described as

$V=\{I_1,\dots,I_M\}$6

with reconstruction

$V=\{I_1,\dots,I_M\}$7

Text conditioning is provided by a frozen CLIP text encoder, and Lformer further augments conditioning with a CVAE latent $V=\{I_1,\dots,I_M\}$8 mapped by an MLP into a StyleGAN-like $V=\{I_1,\dots,I_M\}$9 space, yielding

$\{F_{p,1},F_{p,2},\dots,F_{p,S}\}$0

The distinctive factorization replaces standard autoregressive token ordering with blockwise L-order decoding: $\{F_{p,1},F_{p,2},\dots,F_{p,S}\}$1 At step $\{F_{p,1},F_{p,2},\dots,F_{p,S}\}$2, the model predicts the entire $\{F_{p,1},F_{p,2},\dots,F_{p,S}\}$3-th L-block adjacent to the previously generated square context. The training objective is token-level cross-entropy with teacher forcing,

$\{F_{p,1},F_{p,2},\dots,F_{p,S}\}$4

and the paper also introduces a CVAE objective to mitigate conditional independence within a block: $\{F_{p,1},F_{p,2},\dots,F_{p,S}\}$5

The implementation uses a decoder-style transformer with cross-attention. Reported backbone configurations include L=19, H=16, D=1024 for speed tests, L=24, H=16, D=1024 on MMCelebA-HQ, and L=24, H=16, D=1536 for large-scale MS-COCO/LAION experiments. The paper also reports PB-relax, Sandwich LayerNorm, mixed precision, activation checkpointing, ZeRO optimization, and block-sparse attention.

The quantitative profile emphasizes both speed and quality. On a V100 GPU with batch size 32 and 32×32 tokens, Lformer-cache achieves 3.57 s/batch, 0.112 s/img, and 8.96 img/s, compared with 33.94 s/batch, 1.06 s/img, and 0.942 img/s for AR-cache. On MMCelebA-HQ at 256×256, the Lformer-L configuration reports FID 19.91 at 0.046 s/img, while the 1B-parameter Lformer-E reaches FID 18.60 at 0.076 s/img. On MS-COCO 2014, Lformer-E pretrained on LAION90M reports FID 15.01, IS 26.22, and CLIPSIM 0.3289, with a finetuned FID 9.57. The same model also supports image editing without finetuning, including rollback-based regeneration, repainting, and bounding-box-guided inpainting.

4. GLassoformer and the lasso-regularized interpretation of “Lasformer”

In the power-systems literature, the most direct “Lasformer” interpretation is Lasso + Transformer, instantiated concretely by GLassoformer, an encoder–decoder transformer for post-fault power grid voltage prediction (Zheng et al., 2022). The model introduces structured sparsity on the query vectors in self-attention via group Lasso, with the explicit goal of reducing active queries, cutting compute and memory, and improving stability.

The key mechanism applies group Lasso to the query projection matrix $\{F_{p,1},F_{p,2},\dots,F_{p,S}\}$6. For input sequence $\{F_{p,1},F_{p,2},\dots,F_{p,S}\}$7, the attention projections are

$\{F_{p,1},F_{p,2},\dots,F_{p,S}\}$8

and standard attention is

$\{F_{p,1},F_{p,2},\dots,F_{p,S}\}$9

Columns of $\{F_{v,1},F_{v,2},\dots,F_{v,S}\}$0 are partitioned into groups $\{F_{v,1},F_{v,2},\dots,F_{v,S}\}$1, one per query index, and the penalized objective is

$\{F_{v,1},F_{v,2},\dots,F_{v,S}\}$2

Training uses the Relaxed Group-wise Splitting Method (RGSM) with proximal group soft-thresholding,

$\{F_{v,1},F_{v,2},\dots,F_{v,S}\}$3

Because $\{F_{v,1},F_{v,2},\dots,F_{v,S}\}$4, zeroing a column of $\{F_{v,1},F_{v,2},\dots,F_{v,S}\}$5 yields a zero row in $\{F_{v,1},F_{v,2},\dots,F_{v,S}\}$6, so entire query vectors are eliminated rather than merely sparsified entrywise.

The architecture uses embeddings formed by padding and 1D convolutions concatenated with a global time-stamp channel, then an encoder with two identical blocks of multi-head group-sparse self-attention plus 1D convolution with ELU, and a decoder with group-sparse self-attention, canonical multi-head cross-attention, and a fully connected output projection. The reported convergence guarantee assumes a Lipschitz gradient, with the paper noting that ELU ensures this.

The application dataset is a simulated New York/New England 16-generator, 68-bus power system, containing 2248 fault events, each spanning 10 seconds. The train/validation/test split is 1000/350/750, and the reported optimizer is Adam with learning rate $\{F_{v,1},F_{v,2},\dots,F_{v,S}\}$7, decayed by 0.8 every 10 epochs; group-Lasso hyperparameters are $\{F_{v,1},F_{v,2},\dots,F_{v,S}\}$8 and $\{F_{v,1},F_{v,2},\dots,F_{v,S}\}$9, batch size is 30, and maximum epochs is 80 with early stopping patience 30.

The quantitative results favor GLassoformer over the cited baselines. In Case I with nearby voltage and line-current features, the model achieves MSE $$s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad \alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad m_{ij}=[s_{ij}\ge \tau],$$0 and MAE $$s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad \alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad m_{ij}=[s_{ij}\ge \tau],$$1, compared with Informer at $$s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad \alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad m_{ij}=[s_{ij}\ge \tau],$$2 and $$s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad \alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad m_{ij}=[s_{ij}\ge \tau],$$3 and Lasso at $$s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad \alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad m_{ij}=[s_{ij}\ge \tau],$$4 and $$s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad \alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad m_{ij}=[s_{ij}\ge \tau],$$5. In Case II using single-bus information only, GLassoformer reports MSE $$s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad \alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad m_{ij}=[s_{ij}\ge \tau],$$6 and MAE $$s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad \alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad m_{ij}=[s_{ij}\ge \tau],$$7. The paper also reports a pruning rate of 19.09%, compared with 4.220% for Informer, 2.674% for unstructured Lasso, and 2.3% for a vanilla transformer. Inference time on GTX-1080Ti is 18.98 ms for GLassoformer, faster than Informer at 29.76 ms, though slower than Lasso at 14.92 ms and close to the vanilla transformer at 15.32 ms.

The paper explicitly contrasts GLassoformer with a baseline “LassoFormer” that uses regular, unstructured Lasso. The distinction is central: unstructured penalties yield scattered sparsity, whereas group Lasso zeros entire query rows, enabling both computational savings and query-level interpretability.

5. LAformer: lane-aware trajectory prediction under scene constraints

LAformer is another separate model family, aimed at multimodal trajectory prediction for autonomous driving under lane-aware scene constraints (Liu et al., 2023). It is a two-stage, map-aware motion forecasting model. The first stage performs temporally dense lane-aware estimation and anchor-trajectory generation, while the second stage refines the selected prediction to improve temporal consistency across the full forecast horizon.

The inputs are vectorized trajectories and vectorized lane centerlines, normalized around the target agent’s last observed position $$s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad \alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad m_{ij}=[s_{ij}\ge \tau],$$8 for translation invariance. Motion features $$s_{ij}=\phi(p_i)^\top\psi(v_j),\qquad \alpha_{ij}=\operatorname{softmax}_j(s_{ij}),\qquad m_{ij}=[s_{ij}\ge \tau],$$9 and lane features $$\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad \beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad \tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.$$0 are obtained through MLPs followed by GRUs, and the Global Interaction Graph (GIG) applies symmetric cross-attention: $$\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad \beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad \tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.$$1 followed by concatenation and self-attention among agents. The attention operator is standard scaled dot-product attention,

$$\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad \beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad \tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.$$2

A distinctive feature is the temporally dense lane-aware estimation module. At each future step $$\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad \beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad \tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.$$3, the model scores every lane segment: $$\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad \beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad \tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.$$4 For each $$\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad \beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad \tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.$$5, it retains only the top-$$\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad \beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad \tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.$$6 segments, with $$\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad \beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad \tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.$$7 by default, and concatenates them across time to form a spatio-temporal lane token sequence

$$\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad \beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad \tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.$$8

The lane-selection objective is a binary cross-entropy over all lanes and time steps,

$$\tilde p_i=\sum_j m_{ij}\alpha_{ij}W_vv_j,\qquad \beta_{ij}=\operatorname{softmax}_i(s_{ij}),\qquad \tilde v_j=\sum_i m_{ij}\beta_{ij}W_pp_i.$$9

where $s=1,2,3$0 for the segment closest to the ground-truth position $s=1,2,3$1.

Stage 1 predicts a Laplacian Mixture Density Network over future trajectories,

$s=1,2,3$2

with a Winner-Takes-All regression loss and a mode-classification loss. Stage 2 refines the selected anchor trajectory via an offset regressor and an angle-consistency loss,

$s=1,2,3$3

The total losses are reported as

$s=1,2,3$4

with $s=1,2,3$5, $s=1,2,3$6, and $s=1,2,3$7.

The reported evaluations cover Argoverse 1 and nuScenes. On Argoverse 1, LAformer reports validation ADE 0.64, FDE 0.92, and test ADE 0.77, FDE 1.16, which is on par or slightly better than the cited HiVT configuration. On nuScenes, LAformer reports ADE 1.19, MR 0.48 for $s=1,2,3$8 and ADE 0.93, MR 0.33 for $s=1,2,3$9. The ablation study attributes much of the gain to temporally dense lane scoring and the second-stage refinement: a baseline without lanes and without stage 2 gives ADE 0.72, FDE 1.12, whereas the full model gives ADE 0.64, FDE 0.92. The reported parameter count is 2.654M, with ~115 ms per scene inference for an average of 12 agents.

6. Comparative interpretation and boundary cases

Across these papers, the nearest coherent reading of “Lasformer” is not a shared architecture but a family of naming collisions around transformer selectivity mechanisms. LATFormer selects salient local point-view correspondences by co-occurrence filtering (He et al., 2021). Lformer selects an adjacent L-shape block of image tokens at each generation step rather than decoding the full raster sequence (Li et al., 2023). GLassoformer selects only the nonzero query rows induced by group Lasso on $\{(V1,P1)\}$0 (Zheng et al., 2022). LAformer selects top-$\{(V1,P1)\}$1 lane segments per future time step before decoding (Liu et al., 2023). This suggests a recurrent design preference for reducing irrelevant interactions before or during attention, although the application domains, supervision schemes, and computational goals differ substantially.

Another boundary case is phonetic rather than architectural. “Metal Blossom: Laser Forming Complex and Freeform Metal Structures Imitating Flower Blooming” concerns laser forming origami (lasergami) rather than transformers (Hao et al., 2021). Its subject is metal-sheet folding through computational origami, nearly-blooming net generation, and as-folded-as-possible motion planning, not sequence modeling or attention. The similarity between “laser forming” and “Lasformer” can therefore create an additional naming confusion, but the work belongs to computational fabrication rather than transformer design.

In technical usage, precision in capitalization and expansion is therefore decisive. In 3D recognition, the correct name is LATFormer; in text-to-image generation, Lformer; in power-grid forecasting, GLassoformer or LassoFormer in the lasso-regularized sense; and in trajectory prediction, LAformer. Any encyclopedia treatment of “Lasformer” is most accurate when it treats the term as an ambiguous label whose meaning must be resolved by context, task, and the exact arXiv title.