Point-Excited Wavelet Attention (PEWA)

• Point-Excited Wavelet Attention (PEWA) is a mechanism that refines backbone features using point-guided discrete wavelet transforms to hallucinate object boundaries in point-supervised multi-object tracking.
• It decouples low-frequency semantic content from high-frequency edge details by applying Gaussian-based modulation on wavelet coefficients, thus mitigating location-boundary mismatch.
• Empirical results on benchmarks like DanceTrack demonstrate that PEWA improves metrics such as HOTA and AssA with minimal training overhead while being bypassed during inference.

Searching arXiv for the cited papers to ground the article in the source literature. Point-Excited Wavelet Attention (PEWA) is a model-level mechanism for point-supervised multi-object tracking introduced within PS-Track, a hierarchical “points-to-instances” pipeline for PS-MOT. In this setting, each object is annotated by a single point, ideally near its topological center, rather than a full bounding box. PEWA addresses the resulting location–boundary mismatch by using annotated points as frequency exciters in a wavelet decomposition of backbone features, selectively activating high-frequency components near those points and reconstructing boundary-aware, instance-focused features before transformer encoding. In PS-Track, PEWA operates alongside Temporal-Feedback Prompting (TFP) at the data level and Uncertainty-Guided Gaussian Learning (UGL) at the loss level, and the overall system is built on MOTIP with Deformable DETR and a ResNet-50 backbone (Luo et al., 29 Jun 2026).

1. Position within point-supervised multi-object tracking

PS-MOT is defined as a cost-effective alternative to bounding-box supervision in which dense boxes are replaced by single-point annotations. This supervision regime reduces annotation burden but leaves object extent, scale, and boundary structure fundamentally under-constrained. The PS-Track framework organizes the response to this problem into three levels: TFP evolves points into temporally consistent pseudo-labels using SAM, motion priors, and negative cues; PEWA refines the feature representation itself; and UGL models pseudo-boxes as probabilistic observations and calibrates regression via uncertainty (Luo et al., 29 Jun 2026).

Within that hierarchy, PEWA is the model-level mechanism. It is inserted after the backbone and before the transformer encoder, where it refines multi-scale features using point-guided heatmaps during training; during inference, this branch is bypassed and the tracker follows the original MOTIP inference graph without point inputs (Luo et al., 29 Jun 2026). The role of PEWA is therefore not to supply explicit geometric annotations, but to alter the representation so that sparse point supervision induces boundary-sensitive and instance-aware features.

The motivating problem is stated in terms of spatial ambiguity and identity drift. A point indicates where an object is located but not where its boundaries lie, and under such supervision local convolutions tend to leak signal into background regions or adjacent instances. The paper attributes qualitative improvements in tighter boxes, reduced jitter, and fewer identity switches under abnormal poses and occlusions to PEWA’s “ability to hallucinate structural boundaries in the frequency domain” (Luo et al., 29 Jun 2026).

2. Conceptual basis: points, frequency decomposition, and boundary hallucination

PEWA is motivated by two linked observations. First, the paper draws inspiration from top-down human vision, in which global semantic gist guides local detail. Second, it adopts the frequency-domain view that low-frequency components encode coarse semantics and context, whereas high-frequency components encode edges, contours, and fine structural detail (Luo et al., 29 Jun 2026).

The module operationalizes that view by applying a Discrete Wavelet Transform (DWT) to a feature map, generating one low-frequency subband and three high-frequency subbands. Annotated points are converted into Gaussian activation masks that excite high-frequency coefficients near the point and suppress them elsewhere. The inverse wavelet transform then reconstructs a feature map in which regions around points are encouraged to contain sharp boundaries and object-centric structure (Luo et al., 29 Jun 2026).

The term “Point-Excited” refers specifically to this use of points as the source of spatial excitation in the wavelet domain. The “attention” is not token-wise self-attention with queries, keys, and values; instead, it is realized as multiplicative masking of wavelet-domain coefficients conditioned on point-derived heatmaps (Luo et al., 29 Jun 2026). The paper explicitly contrasts this with self-attention and deformable attention, characterizing PEWA as a frequency-selective, point-conditioned attention layer.

A useful comparative background is provided by wavelet-based attention in image classification. “WaveNets: Wavelet Channel Attention Networks” generalizes channel attention by replacing Global Average Pooling with wavelet-based compression and shows that GAP is equivalent to recursive approximate Haar DWT. That work keeps the SE-style excitation structure but changes the compression stage, emphasizing that wavelet decompositions preserve richer spectral information than a single scalar channel summary (Salman et al., 2022). PEWA differs in purpose and design: it is not a channel-attention replacement for GAP, but a task-specific mechanism that modulates only high-frequency spatial components using point supervision (Luo et al., 29 Jun 2026).

3. Architectural formulation

For a backbone feature map at a given scale,

$X \in \mathbb{R}^{C \times H \times W},$

PEWA applies a single-level Haar DWT:

$X_{LF}, \{X_{HF}^{k}\}_{k=1}^3 = \text{DWT}(X),$

where

$$X_{LF} \in \mathbb{R}^{C \times \frac{H}{2} \times \frac{W}{2}}$$

is the low-frequency approximation subband and the three high-frequency subbands correspond to LH, HL, and HH, each in the same reduced spatial resolution (Luo et al., 29 Jun 2026).

For a target point $p_i$, the module generates a Gaussian heatmap

$G \in \mathbb{R}^{1 \times H \times W},$

centered at $p_i$, with

$$G(x, y) = \exp\left(-\frac{\|(x, y) - p_i\|_2^2}{2\sigma_g^2}\right).$$

This heatmap is downsampled to the wavelet resolution,

$$G_{\downarrow} \in \mathbb{R}^{1 \times \tfrac{H}{2} \times \tfrac{W}{2}},$$

and passed through a lightweight modulator $\phi$, followed by a sigmoid:

$$M_{exc} = \sigma(\phi(G_{\downarrow})), \qquad M_{exc} \in \mathbb{R}^{C \times \tfrac{H}{2} \times \tfrac{W}{2}}.$$

The paper describes $X_{LF}, \{X_{HF}^{k}\}_{k=1}^3 = \text{DWT}(X),$0 as lightweight but does not fully specify its architecture (Luo et al., 29 Jun 2026).

The excitation mask is then applied to each high-frequency subband:

$X_{LF}, \{X_{HF}^{k}\}_{k=1}^3 = \text{DWT}(X),$1

This suppresses high-frequency coefficients far from the point and emphasizes edge and texture information near the point. The low-frequency component is left unmodulated, or minimally processed, in order to preserve global identity consistency; the minimal implementation simply forwards $X_{LF}, \{X_{HF}^{k}\}_{k=1}^3 = \text{DWT}(X),$2 unchanged (Luo et al., 29 Jun 2026).

Finally, PEWA reconstructs the spatial feature map by inverse DWT:

$X_{LF}, \{X_{HF}^{k}\}_{k=1}^3 = \text{DWT}(X),$3

The residual connection stabilizes training and allows the network to revert toward backbone features when the PEWA refinement is not beneficial (Luo et al., 29 Jun 2026). The resulting $X_{LF}, \{X_{HF}^{k}\}_{k=1}^3 = \text{DWT}(X),$4 replaces $X_{LF}, \{X_{HF}^{k}\}_{k=1}^3 = \text{DWT}(X),$5 as the input to the transformer encoder, and the same PEWA block is applied to all four feature levels derived from the backbone.

4. Interaction with supervision and training dynamics

PEWA uses original point annotations during training, not the TFP-generated pseudo-boxes, to drive wavelet excitation. The paper states this explicitly: “Parallel to this data evolution, the Point-Excited Wavelet Attention (PEWA) module at the model level directly treats the original annotated points as frequency exciters” (Luo et al., 29 Jun 2026). Pseudo-box regression and UGL then encourage the model to align the induced boundary-sensitive representation with pseudo-box extents.

In the benchmark setup, point seeds are generated from ground-truth box centers, while retaining imperfections such as occlusion and off-center placement to simulate realistic clicking. For multiple instances in a frame, each point $X_{LF}, \{X_{HF}^{k}\}_{k=1}^3 = \text{DWT}(X),$6 yields a 2D Gaussian $X_{LF}, \{X_{HF}^{k}\}_{k=1}^3 = \text{DWT}(X),$7. The paper does not specify the exact aggregation mechanism, but notes that one natural extension is to aggregate via $X_{LF}, \{X_{HF}^{k}\}_{k=1}^3 = \text{DWT}(X),$8 or $X_{LF}, \{X_{HF}^{k}\}_{k=1}^3 = \text{DWT}(X),$9. This suggests that the essential design principle is multi-instance Gaussian excitation rather than a single mandated aggregation rule (Luo et al., 29 Jun 2026).

A central property of PEWA is its train-time-only usage. During training, it is inserted after the backbone to refine multi-scale features using point-guided heatmaps. During inference, because no points are available in standard MOT evaluation, the PEWA branch is structurally bypassed and the tracker runs on RGB only, following the original MOTIP inference graph (Luo et al., 29 Jun 2026). The supplementary analysis directly studies the training–inference discrepancy by stochastically activating PEWA during training with probability $$X_{LF} \in \mathbb{R}^{C \times \frac{H}{2} \times \frac{W}{2}}$$0. On DanceTrack validation in a 2-epoch setting, deterministic activation at $$X_{LF} \in \mathbb{R}^{C \times \frac{H}{2} \times \frac{W}{2}}$$1 achieves HOTA 44.2 and AssA 30.4, whereas removing PEWA entirely at $$X_{LF} \in \mathbb{R}^{C \times \frac{H}{2} \times \frac{W}{2}}$$2 yields HOTA 43.7 and AssA 29.6; intermediate probabilities underperform the deterministic setting, and the paper concludes that randomizing PEWA causes feature jitter that harms identity association (Luo et al., 29 Jun 2026).

No additional loss is tied specifically to PEWA. It is trained end-to-end through the standard detection and identity losses together with UGL-regulated regression (Luo et al., 29 Jun 2026). This implies that its contribution is representational rather than supervisory: it reshapes intermediate features so that downstream losses induce object-centric structure from point seeds.

5. Empirical evidence and operational cost

The main ablation on DanceTrack validation for 10 epochs isolates PEWA within the PS-Track pipeline. A baseline without TFP, PEWA, or UGL reaches HOTA 30.3, DetA 45.4, IDF1 32.5, MOTA 36.3, and AssA 20.4. Adding TFP alone raises HOTA to 49.0 and AssA to 37.1. Adding UGL yields HOTA 49.4 and AssA 38.0. The full PS-Track configuration, which adds PEWA on top of TFP and UGL, reaches HOTA 50.3, DetA 65.3, IDF1 52.9, MOTA 66.2, and AssA 39.1 (Luo et al., 29 Jun 2026).

The comparison between the last two rows attributes to PEWA a gain of $$X_{LF} \in \mathbb{R}^{C \times \frac{H}{2} \times \frac{W}{2}}$$3 HOTA and $$X_{LF} \in \mathbb{R}^{C \times \frac{H}{2} \times \frac{W}{2}}$$4 AssA, together with a small increase in DetA and a $$X_{LF} \in \mathbb{R}^{C \times \frac{H}{2} \times \frac{W}{2}}$$5 MOTA improvement (Luo et al., 29 Jun 2026). The paper interprets this pattern as evidence of improved instance-aware representation and boundary quality. Because TFP already improves pseudo-label quality and UGL already mitigates supervision noise, the incremental gain associated with PEWA is presented as specifically representational.

The training overhead is limited to the training phase. Relative to the MOTIP baseline, parameters increase from 59.1M to 61.9M with the PEWA branch; training memory rises from 23.7GB to 24.2GB; and training time increases from 7.4 h/epoch to 8.0 h/epoch on DanceTrack using an RTX 5090. During inference, FLOPs and speed are described as almost identical to the MOTIP baseline because the PEWA branch is bypassed (Luo et al., 29 Jun 2026).

The broader PS-Track system is evaluated on DanceTrack, EmboTrack, SportsMOT, and JRDB, where it is reported to establish a new state-of-the-art for point-supervised tracking (Luo et al., 29 Jun 2026). PEWA’s contribution to that result is situated as one component of the three-level pipeline rather than as an isolated standalone tracker module.

6. Relation to adjacent attention mechanisms and frequency-domain modeling

PEWA is explicitly distinguished from three nearby classes of methods. First, it differs from standard convolutional backbones, which are described as entangling semantic and structural information and tending to blur boundaries under point supervision. By decomposing features into low- and high-frequency subbands, PEWA attempts to decouple coarse semantics from edge structure and condition only the structural portion on points (Luo et al., 29 Jun 2026).

Second, it differs from token-based self-attention and deformable attention. The paper states that PEWA is not based on Q/K/V interactions or non-local token mixing; its “attention” is multiplicative masking in the wavelet domain, guided by point-derived heatmaps. This requires fewer parameters and FLOPs than full self-attention and can be inserted between a CNN backbone and a transformer encoder without altering the inference graph (Luo et al., 29 Jun 2026).

Third, it differs from general-purpose frequency-based modules such as WaveFormer and from wavelet-based channel attention exemplified by WaveNets. WaveNets shows that DWT can replace GAP in channel attention, implements DWT as fixed Conv2D filters with stride 2, and argues that wavelet compression preserves richer information for channel-interdependency modeling (Salman et al., 2022). PEWA, by contrast, uses a single-level Haar DWT over spatial feature maps and explicitly modulates only the high-frequency subbands with point-conditioned masks while leaving low-frequency content largely untouched (Luo et al., 29 Jun 2026). This task-specific choice is tailored to point-supervised boundary hallucination rather than general channel recalibration.

A plausible implication is that PEWA occupies a distinct position between structural priors and attention modules. It uses frequency decomposition as an inductive bias for recovering missing object extent from sparse annotations, rather than as a generic means of boosting representational capacity. The paper’s emphasis on training-time-only usage reinforces that interpretation (Luo et al., 29 Jun 2026).

7. Limitations and proposed extensions

The paper identifies several limitations directly relevant to PEWA. Under severe motion blur, high-frequency components physically degrade, making PEWA’s emphasis on edge information less effective for inferring object extent. Under extreme occlusions or entangled limbs, a single point may lie near an ambiguous intersection of multiple bodies; in such cases, SAM pseudo-labels can be fragmented and PEWA may excite high-frequency content along incorrect boundaries, limiting instance separation (Luo et al., 29 Jun 2026).

A further limitation concerns point quality. The benchmark protocol uses ground-truth box centers as points, which may be cleaner than real human clicks. The paper notes that although PEWA together with UGL shows robustness to moderate noise, extreme point misplacement could degrade performance (Luo et al., 29 Jun 2026). This places an upper bound on the expected reliability of point-conditioned boundary hallucination under substantially noisier annotation regimes.

The authors indicate several possible extensions. These include exploring different wavelet bases or multi-level DWT for richer scale decomposition, using richer point encodings such as combinations of points with text prompts or object-type cues, integrating temporally propagated or temporally sparse point cues, and extending the point-driven wavelet mechanism to 3D MOT or BEV representations (Luo et al., 29 Jun 2026). These are framed as future directions rather than validated design choices.

From an implementation perspective, the paper treats certain elements as flexible. Standard Haar filters suffice, and libraries such as pytorch_wavelets, kymatio, or custom convolution kernels can realize DWT and IDWT. The modulator $$X_{LF} \in \mathbb{R}^{C \times \frac{H}{2} \times \frac{W}{2}}$$6 is described only as lightweight, with example forms such as small $$X_{LF} \in \mathbb{R}^{C \times \frac{H}{2} \times \frac{W}{2}}$$7 or $$X_{LF} \in \mathbb{R}^{C \times \frac{H}{2} \times \frac{W}{2}}$$8 convolutional stacks suggested in the details, but not fixed as part of the canonical definition (Luo et al., 29 Jun 2026). By contrast, two aspects are presented as critical: the decoupling into low- and high-frequency components with modulation of only the high-frequency bands, and the use of PEWA only during training with bypass at inference (Luo et al., 29 Jun 2026).