SOR-Track: Spatial Orthogonal Refinement Tracker

• SOR-Track is a robust RGB-event visual tracking framework that leverages directional event cues to accurately rectify degraded RGB textures.
• It utilizes a two-stage ‘Refine-then-Track’ architecture where the Spatial Orthogonal Refinement module dynamically steers Gabor-like filters to align motion-consistent features.
• The approach achieves superior target localization under challenging conditions, yielding a 3.83% absolute gain in strict overlap metrics compared to leading methods.

Searching arXiv for the specified paper and closely related entries to ground the article in current sources. SOR-Track, short for Spatial Orthogonal Refinement Tracker, is a robust RGB-Event visual object tracking framework introduced for high-speed motion scenarios in which conventional RGB sensing suffers from severe motion blur and degradation. It is formulated as a two-stage, “Refine-then-Track” pipeline that explicitly rectifies degraded RGB textures by leveraging the high-frequency, directional structural cues provided by an event camera. Its central mechanism, the Spatial Orthogonal Refinement (SOR) module, uses dynamically steered orthogonal Gabor-like filters to extract motion-consistent responses from event data and to asymmetrically modulate RGB features, thereby bridging the structural gap induced by motion blur or extreme illumination (Huang et al., 29 Mar 2026).

1. Problem setting and design rationale

Robust visual object tracking remains difficult under high-velocity motion, severe motion blur, and low-light conditions. The motivating observation behind SOR-Track is that RGB imagery and event streams exhibit complementary failure modes: conventional RGB sensors are prone to blur and texture aliasing during rapid motion, whereas event cameras provide microsecond temporal resolution, high dynamic range, and structurally informative intensity-change signals that remain useful when RGB quality deteriorates (Huang et al., 29 Mar 2026).

The framework is positioned against RGB-Event fusion approaches that treat event data as dense intensity representations and rely on black-box fusion strategies. SOR-Track instead assumes that event streams encode directional geometric priors that can be used explicitly to rectify degraded RGB features. In this formulation, event responses are not merely an auxiliary modality for feature concatenation; they serve as geometric anchors for texture refinement. The paper therefore presents the method as a principled and physics-grounded approach to multi-modal feature alignment and texture rectification rather than as a generic fusion backbone (Huang et al., 29 Mar 2026).

A common misconception is to regard SOR-Track as an ordinary dual-stream tracker. The formulation suggests a narrower and more explicit role for cross-modal interaction: structural refinement precedes tracking, and the event modality is used primarily to sharpen, gate, and align motion-consistent structures in the RGB branch.

2. “Refine-then-Track” architecture

Given a template $z$ from the first frame and a search region $x$ from a subsequent frame, SOR-Track processes both an RGB image $$I_{\mathrm{rgb}} \in \mathbb{R}^{3\times H\times W}$$ and an event frame $$I_{\mathrm{event}} \in \mathbb{R}^{C_{\mathrm{event}}\times H\times W}$$. The architecture is organized into three stages: a granular input stem, the Spatial Orthogonal Refinement module, and a lightweight tracking head (Huang et al., 29 Mar 2026).

The granular input stem replaces strided convolutions with a space-to-depth, or pixel-shuffle, preprocessing layer. This produces fine-grained feature maps $$F_{\mathrm{rgb}}, F_{\mathrm{event}} \in \mathbb{R}^{D\times (H/s)\times (W/s)}$$. The stated purpose of this “StemNet” is to preserve sparse, high-frequency event spikes and thin RGB edges, preventing early destruction of exactly the signals on which later structural refinement depends.

The SOR module contains the Orthogonal Directional Module (ODM). ODM constructs $K$ steerable Gabor filters with orientations defined as orthogonal offsets around a principal motion orientation $\phi$. These filters are applied to both modalities, producing directional response tensors $$R_{\mathrm{event}}, R_{\mathrm{rgb}} \in \mathbb{R}^{(K\cdot C)\times (H/s)\times (W/s)}$$. The high-pass, anisotropic event response then serves as a geometric gate that rectifies the lower-fidelity RGB response through element-wise modulation, after which the refined representation is returned to the base channel dimension.

The tracking head is intentionally lightweight. It consumes the refined feature $F_{\mathrm{out}}$ and predicts a center-heatmap for target localization together with bounding-box offsets for regression. The architectural claim is that by decoupling structural refinement from relational matching, the tracker remains lightweight while improving robustness under severe blur or low-light conditions (Huang et al., 29 Mar 2026).

3. Spatial Orthogonal Refinement: mathematical formulation

The SOR module is defined around a principal local motion orientation $\phi$. From this orientation, the method samples $x$0 directions uniformly across the half-circle, described as the orthogonal space:

$x$1

For each $x$2, the ODM defines a 2D Gabor filter

$x$3

with rotated coordinates

$x$4

where $x$5 are learnable parameters (Huang et al., 29 Mar 2026).

Using grouped convolutions, denoted by $x$6, the filter bank $x$7 is applied to both modalities:

$x$8

where $x$9 is Group Normalization applied independently over each directional channel. This yields modality-specific directional responses aligned to the estimated motion orientation.

The cross-modal interaction is asymmetric. Because $$I_{\mathrm{rgb}} \in \mathbb{R}^{3\times H\times W}$$0 is assumed to contain sharp, motion-consistent edges that are relatively immune to RGB blur, SOR-Track constructs a gating mask

$$I_{\mathrm{rgb}} \in \mathbb{R}^{3\times H\times W}$$1

This mask modulates the RGB directional responses:

$$I_{\mathrm{rgb}} \in \mathbb{R}^{3\times H\times W}$$2

and a $$I_{\mathrm{rgb}} \in \mathbb{R}^{3\times H\times W}$$3 projection $$I_{\mathrm{rgb}} \in \mathbb{R}^{3\times H\times W}$$4 restores the original channel dimension with a residual connection:

$$I_{\mathrm{rgb}} \in \mathbb{R}^{3\times H\times W}$$5

The Hadamard product $$I_{\mathrm{rgb}} \in \mathbb{R}^{3\times H\times W}$$6 implements the explicit rectification step. The residual term preserves coarse semantic information. In effect, the module uses event-derived directional structure to amplify or suppress RGB responses according to motion-consistent geometry rather than relying on symmetric feature mixing (Huang et al., 29 Mar 2026).

4. Dynamic orientation guidance and optimization

The steering variable $$I_{\mathrm{rgb}} \in \mathbb{R}^{3\times H\times W}$$7 is derived from the event stream itself. In practice, the method computes per-pixel gradients of $$I_{\mathrm{rgb}} \in \mathbb{R}^{3\times H\times W}$$8, for example with $$I_{\mathrm{rgb}} \in \mathbb{R}^{3\times H\times W}$$9 sobel filters, to obtain $$I_{\mathrm{event}} \in \mathbb{R}^{C_{\mathrm{event}}\times H\times W}$$0 and $$I_{\mathrm{event}} \in \mathbb{R}^{C_{\mathrm{event}}\times H\times W}$$1. It then estimates the local motion orientation at each spatial location as

$$I_{\mathrm{event}} \in \mathbb{R}^{C_{\mathrm{event}}\times H\times W}$$2

These per-pixel orientation estimates are aggregated within the target region, for example by averaging or by a small MLP, to obtain a single $$I_{\mathrm{event}} \in \mathbb{R}^{C_{\mathrm{event}}\times H\times W}$$3 used to sample $$I_{\mathrm{event}} \in \mathbb{R}^{C_{\mathrm{event}}\times H\times W}$$4. Because $$I_{\mathrm{event}} \in \mathbb{R}^{C_{\mathrm{event}}\times H\times W}$$5 is recomputed at every frame or patch, the filter bank dynamically adapts to the instantaneous motion direction, which the paper argues is necessary for alignment with true edges in the event data (Huang et al., 29 Mar 2026).

The implementation details are specific. The backbone is Vision Transformer (ViT-Base/16) with patch size 16, initialized from MAE pre-trained weights. The number of orientations is $$I_{\mathrm{event}} \in \mathbb{R}^{C_{\mathrm{event}}\times H\times W}$$6. Optimization uses AdamW with weight decay $$I_{\mathrm{event}} \in \mathbb{R}^{C_{\mathrm{event}}\times H\times W}$$7, learning rates of $$I_{\mathrm{event}} \in \mathbb{R}^{C_{\mathrm{event}}\times H\times W}$$8 for the SOR-Track head and $$I_{\mathrm{event}} \in \mathbb{R}^{C_{\mathrm{event}}\times H\times W}$$9 for the backbone, and a step decay scheduler at epoch 40 with factor $$F_{\mathrm{rgb}}, F_{\mathrm{event}} \in \mathbb{R}^{D\times (H/s)\times (W/s)}$$0. Training uses batch size 32 on two NVIDIA RTX 4090 GPUs for 50 epochs, approximately $$F_{\mathrm{rgb}}, F_{\mathrm{event}} \in \mathbb{R}^{D\times (H/s)\times (W/s)}$$1 samples per epoch, with no data augmentation.

The loss is

$$F_{\mathrm{rgb}}, F_{\mathrm{event}} \in \mathbb{R}^{D\times (H/s)\times (W/s)}$$2

with $$F_{\mathrm{rgb}}, F_{\mathrm{event}} \in \mathbb{R}^{D\times (H/s)\times (W/s)}$$3, $$F_{\mathrm{rgb}}, F_{\mathrm{event}} \in \mathbb{R}^{D\times (H/s)\times (W/s)}$$4, and $$F_{\mathrm{rgb}}, F_{\mathrm{event}} \in \mathbb{R}^{D\times (H/s)\times (W/s)}$$5. During inference, the first-frame template is cached for both RGB and event features, subsequent search patches are $$F_{\mathrm{rgb}}, F_{\mathrm{event}} \in \mathbb{R}^{D\times (H/s)\times (W/s)}$$6, and the SOR module executes in real time, reported as less than $$F_{\mathrm{rgb}}, F_{\mathrm{event}} \in \mathbb{R}^{D\times (H/s)\times (W/s)}$$7 ms per frame (Huang et al., 29 Mar 2026).

5. Empirical performance on FE108

The principal evaluation reported for SOR-Track is on the large-scale FE108 benchmark, which features high-velocity motion, severe motion blur, and low-light conditions. On this benchmark, the tracker achieves AUC (Success Rate) of $$F_{\mathrm{rgb}}, F_{\mathrm{event}} \in \mathbb{R}^{D\times (H/s)\times (W/s)}$$8, Precision (PR) of $$F_{\mathrm{rgb}}, F_{\mathrm{event}} \in \mathbb{R}^{D\times (H/s)\times (W/s)}$$9, OP50 of $K$0, and OP75 of $K$1 (Huang et al., 29 Mar 2026).

The comparison emphasized in the paper is against CEUTrack, described as the leading Transformer-based fusion baseline. CEUTrack is reported at AUC $K$2 and OP75 $K$3, while SOR-Track yields a $K$4 absolute gain in OP75. The interpretation given is that SOR-Track provides more precise localization under extreme conditions, particularly those dominated by motion blur and low illumination.

The experimental result is significant within the paper’s framing because OP75 is a stricter localization criterion than coarse success or precision measures. This suggests that the benefit of the refinement mechanism is not merely maintaining rough target presence, but recovering boundaries and structures accurately enough to improve high-overlap localization. The source attributes this improvement to explicit structural discrepancy reduction between the two modalities rather than to a larger or more complex tracking head (Huang et al., 29 Mar 2026).

6. Stated sources of effectiveness and terminological ambiguity

The paper provides four reasons for SOR-Track’s effectiveness. First, it invokes physics-grounded priors: event cameras capture instantaneous intensity changes, characterized as high-pass, directional edges, and the ODM Gabor filters are said to mimic V1 cortical cells tuned to specific orientations. Second, it emphasizes structural alignment: $K$5 is used as a gating mask to rectify isotropic, blurred RGB textures where conventional fusion would blur or average away critical boundaries. Third, it highlights dynamic adaptation: steering the filter bank by local motion orientation $K$6 allows the module to remain aligned with variable and unpredictable trajectories. Fourth, it credits granular encoding: the StemNet space-to-depth transformation preserves the high-frequency event information on which the SOR mechanism depends (Huang et al., 29 Mar 2026).

These points also delimit what SOR-Track is not. It is not presented as a modality-agnostic fusion transformer, nor as a dense event-image encoder. Its distinguishing claim is the explicit use of directional event structure to refine RGB textures before prediction.

The name “SOR-Track” is, however, not unique across the supplied literature. In an unrelated numerical linear algebra context, the label is used in a summary of fully parallel SOR/ILU preconditioners on structured grids, where “tracks” denote subdomains in multi-frontal sweeping for structured $K$7-diagonal matrices (Tavakoli, 2010). In another unrelated context, a technical report uses “SOR-Track” for a system extending TrackOR to long-term personalized multi-person tracking in surgical operating rooms using 3D human-pose detection, 8-view depth-map ReID embeddings, Hungarian assignment, and offline recovery (Wang et al., 11 Aug 2025). These usages concern structured-grid sweeping algorithms and operating-room tracking, respectively, rather than RGB-Event visual object tracking.

Within the context of visual tracking, SOR-Track therefore refers specifically to the Spatial Orthogonal Refinement framework for robust RGB-Event VOT, centered on dynamically steered orthogonal directional filtering and asymmetric structural modulation (Huang et al., 29 Mar 2026).