Precise Event Spotting in Time-Series

• Precise Event Spotting is a technique that detects atomic events at frame-level resolution using strict temporal tolerances.
• It employs diverse architectures—CNN-RNN, Transformer, graph-based—to capture both local and global temporal context for high accuracy.
• Challenges include extreme class imbalance, annotation noise, and the need for efficient real-time processing in varied applications.

Precise Event Spotting (PES) is the task of localizing and classifying events in time—most commonly within long, untrimmed sequences such as videos, audio, or time-series—with a strict requirement for frame- or sub-second-level temporal accuracy. Research in PES has evolved rapidly, driven by sports analytics, fine-grained action understanding, expression recognition, astronomical event detection, and sound event localization, demanding methodology that unifies efficient computation, robust label alignment, temporal context, and multimodal input.

1. Formal Definition and Problem Scope

Precise Event Spotting is characterized by its focus on atomic event detection at precise time instants, as opposed to interval-based localization. Given an input sequence of $T$ frames (e.g., video or audio), and a sparse set of ground-truth events $\mathcal{E} = \{(t_j, e_j)\}_{j=1}^N$, the objective is to predict a set of frame positions and their associated classes: $$\hat{\mathcal{E}} = \{(\hat{t}_i, \hat{c}_i)\}_{i=1}^M, \quad \hat{t}_i \in [1, T],~ \hat{c}_i \in \mathcal{C}$$ where each predicted spot is considered correct if $|\hat{t}_i - t_j| \leq \delta$ for some unmatched ground-truth $t_j$ of the same class—$\delta$ is the temporal tolerance, often set to 1–2 frames for PES (Xu et al., 6 May 2025, Hong et al., 2022). Unlike action localization, which requires start/end boundary regression, PES targets singular points.

In sports analytics, this formalism is codified in datasets such as SoccerNet-v2 (events: goal, substitution, card, etc.) and fine-grained domains such as tennis serves, gymnastics moves, or table tennis strokes, with challenging inter-event intervals and strong class imbalance (Giancola et al., 2018, Xu et al., 10 Jul 2025). In micro-expression and sound event spotting, the goal is often to find milliseconds-scale onsets amid substantial background (Yu et al., 24 Oct 2024, Wolters et al., 2021).

2. Methodological Foundations and Model Architectures

PES models are built on the premise that both local and global temporal context are necessary for frame-level precision. Major architecture categories include:

• Feature-based pipelines: Extract fixed frame- or snippet-level features (e.g., via ResNet, I3D), followed by temporal pooling or encoding—mean, max, NetVLAD, or temporal convolutions—and a decision head (Giancola et al., 2018, Vanderplaetse et al., 2020).
• End-to-end CNN-RNN networks: E2E-Spot feeds raw RGB frames to a 2D CNN with Gate Shift Modules, producing per-frame features, which are then input to a bidirectional GRU for temporal context aggregation and frame-wise event classification (Hong et al., 2022).
• Regression-offset models: RMS-Net jointly trains a classification head and a regression head that predicts the normalized offset of the true event within each input clip; the spot is set by $\hat{t} = s + T \cdot \hat{o}$, with joint cross-entropy and regression loss (Tomei et al., 2021).
• Dense anchor-based detectors: Models place temporal detection anchors at each frame (and each class), predicting both a confidence score and a fine-grained offset; temporal displacements are learned via regression (e.g., U-Net or Transformer trunk) (Soares et al., 2022).
• Encoder-decoder and multi-scale architectures: T-DEED employs an encoder–decoder that downsamples through temporal max pooling and upsamples back to frame resolution, enabling aggregation of multiscale context while preserving output temporal fidelity. SGP-Mixer modules drive token discriminability (Xarles et al., 8 Apr 2024).
• Transformer-based and attention architectures: PESFormer uses a multi-head transformer with direct timestamp encoding, classifying each timestamp independently rather than via interval or anchor regression (Yu et al., 24 Oct 2024).
• Graph and keypoint-based networks: UMEG-Net introduces a multi-entity graph (e.g., human pose, object, court keypoints) with spatio-temporal GCN and temporal shifts for few-shot PES, leveraging multimodal distillation for robust training with limited labels (Liu et al., 18 Nov 2025).
• Specialized temporal shift and attention modules: Multi-Scale Attention Gate Shift Module (MSAGSM) extends Gate Shift/Fuse with multi-dilation temporal shifts and multi-head spatial attention, yielding better capture of fine-grained dependencies with low overhead (Xu et al., 10 Jul 2025).

3. Loss Functions, Training, and Label Alignment

The class imbalance and sparse-positive nature of PES motivates tailored losses and sampling:

• Classification and regression multitask loss: Models like RMS-Net and T-DEED optimize both per-frame (or per-clip) cross-entropy for discrete event classes and regression loss (often squared error or Smooth-$L_1$) for offset prediction (Tomei et al., 2021, Xarles et al., 8 Apr 2024).
• Contrastive and soft instance losses: To improve class separation and recall for rare events, some models employ SoftIC loss—a memory bank–based contrastive objective with soft (mixup-augmented) labels—to promote compact intra-class and separated inter-class feature distributions (Santra et al., 28 Feb 2025).
• Focal and Dice losses for severe imbalance: Direct timestamp encoding models such as PESFormer apply focal loss and dice loss to manage extreme foreground-background skew (Yu et al., 24 Oct 2024).
• Dynamic label assignment and temporal misalignment: Recent work introduces dynamic cost-based assignment of predictions to mislabeled or temporally ambiguous ground-truth using a Hungarian algorithm on combined class-similarity and temporal-offset cost, directly mitigating annotation noise and label drift (Tamura, 31 Mar 2025).
• Masking and uniform offset sampling: RMS-Net masks ambiguous frames (with probability $p$ up to offset $q$) within foreground windows, compelling the model to focus on post-event discriminative cues. Offset sampling ensures uniformity for regression (Tomei et al., 2021).

4. Evaluation Metrics, Protocols, and Benchmarks

Strict evaluation protocols enforce the unique demands of PES:

• Frame-tolerance precision and mAP: The canonical metric is mean Average Precision (mAP) under a tight temporal tolerance (e.g., $\delta=1$ frame), computed by one-to-one bipartite matching between predictions and ground-truth events within $\delta$ (Xu et al., 6 May 2025, Hong et al., 2022). Average-mAP aggregates mAP over multiple tolerances for a global view (Giancola et al., 2018).
• Datasets: Key PES benchmarks include SoccerNet(-v2) (soccer, 1s anchors, multi-class), Tennis/FineDiving/FineGym/Figure Skating (frame-accurate), Table Tennis Australia (first table tennis PES dataset), and micro/macro expression corpora such as CAS(ME)$^2$, CAS(ME)$^3$, and SAMM-LV (Xu et al., 10 Jul 2025, Yu et al., 24 Oct 2024).
• Few-shot protocols: UMEG-Net and others formalize the $k$-clip PES paradigm, where models are trained with only a handful of labeled event clips and supplemented by large unlabeled pools (Liu et al., 18 Nov 2025).
• Sound event metrics: In few-shot sound PES, evaluation uses event-level average precision, proposal accuracy, and F1 within IoU matching (Wolters et al., 2021).

5. Recent Advances: Temporal Modules, Attention, and Multimodality

Enhancing temporal context and cross-modality boosts PES:

• Temporal context modules: Gate Shift Module (GSM), Gate Shift Fuse (GSF), and MSAGSM modules are widely used to inject short-term and long-term temporal dependencies into 2D CNNs with minimal computational overhead (Xu et al., 10 Jul 2025). Multi-scale dilations are critical for frame-level accuracy; excessive dilation, however, deteriorates precision.
• Multi-head attention: MSAGSM and other attention mechanisms (e.g., duration-constrained self-attention in PESFormer) focus features on salient spatial regions and efficiently pool temporal evidence (Yu et al., 24 Oct 2024).
• Multimodal fusion: Incorporating audio streams with video—especially in sports PES—yields strong improvements in mAP (e.g., soccer goals benefit from commentator/fan acoustics), with late and mid-level fusion outperforming early fusion (Vanderplaetse et al., 2020). Vision-LLMs or per-frame object/context fusion (e.g., UGL, GLIP) further drive event discriminability (Xu et al., 6 May 2025).
• Wavelet and semi-parametric methods: For time-series PES, robust semi-parametric models with multi-scale wavelet expansions and inferential EM algorithms allow event detection under irregular sampling, structured trends, and non-Gaussian noise (Blocker et al., 2013).

6. Empirical Results and State-of-the-Art

State-of-the-art PES systems achieve high mAP under strict frame-level tolerances (commonly $\gtrsim90\%$ on single-sport, moderate on complex multi-sport or low-resource settings):

Model	Setting	mAP@1	Dataset	Notable Features
E2E-Spot	Tennis	96.1	Tennis	2D CNN + Bi-GRU, end-to-end
T-DEED	FigureSkate	85.15	FS-Comp	Encoder–decoder, SGP-Mixer
RMS-Net	Soccer	77.5	SoccerNet (val)	Offset regression + masking
MSAGSM+E2E-Spot	Table Tennis	69.5	TTA	Multi-scale shift, multi-head attn
PESFormer	Face expr.	83.8 F1	CAS(ME)$^2$	Direct timestamp encoding, ViT
UMEG-Net	Few-shot	64.0*	k-clip average	Multi-entity graph, distill learning

* F1 per-class with $\delta = 1$ frame (Liu et al., 18 Nov 2025, Xu et al., 10 Jul 2025, Yu et al., 24 Oct 2024, Santra et al., 28 Feb 2025, Xarles et al., 8 Apr 2024, Tomei et al., 2021, Hong et al., 2022)

Ablation studies consistently demonstrate: (a) frame-level precision is lost when per-frame outputs are made at pooled resolutions, (b) per-frame discriminability is superior when skip connections and discriminability-enhancing heads (SGP, SoftIC) are used, (c) multimodal and multi-entity features provide robustness to occlusion and ambiguity, and (d) strong handling of extreme class-imbalance and temporal misalignment is essential for rare-event recall (Santra et al., 28 Feb 2025, Tamura, 31 Mar 2025, Xarles et al., 8 Apr 2024, Xu et al., 6 May 2025).

7. Limitations, Open Challenges, and Future Directions

Current limitations and future directions in PES research, as distilled from recent surveys and experimental analyses, include:

• Domain Generalization: Most PES approaches are tuned for broadcast-quality sports video; generalization to diverse camera conditions, sports, or real-world modalities remains limited (Xu et al., 6 May 2025).
• Annotation Noise and Label Ambiguity: Human labelers introduce frame-level inaccuracies, degrading model precision. Dynamic label assignment and robust temporal matching are active research areas (Tamura, 31 Mar 2025).
• Class Imbalance and Annotation Cost: Many event classes are heavily underrepresented. Advanced contrastive losses (SoftIC), selective sampling, and automated annotation/weak supervision are ongoing needs (Santra et al., 28 Feb 2025, Xu et al., 6 May 2025).
• Real-Time and Resource Efficiency: Leading models rely on heavy CNN or transformer backbones; progress toward lightweight, real-time deployable PES remains crucial, particularly with emerging models like MSAGSM (Xu et al., 10 Jul 2025).
• Few-Shot and Transfer Learning: Graph-based, keypoint-driven, and distillation-based few-shot pipelines enable scalable PES with minimal supervision but are sensitive to detector and graph quality (Liu et al., 18 Nov 2025).
• Multimodal, Scene, and Context Integration: Beyond simple audio-video fusion, future systems may leverage contextual data, vision-language alignment, commentary, or cross-sport transfer with unified architectures (Xu et al., 6 May 2025).

Overall, Precise Event Spotting has become a central, rapidly advancing subfield of temporal event analysis, interweaving rigorous annotation, efficient and context-sensitive modeling, and robust evaluation toward frame-accurate understanding of time in multimodal streams.