Recurrent Tracking Functions (RTFs)

• RTFs are recurrent mappings that update a tracking-relevant state from sequential observations, enabling continuous tracking without traditional detection pipelines.
• They utilize formulations like hₜ = f(hₜ₋₁, xₜ) and output maps such as yₜ = g(hₜ) to integrate past and current data for accurate predictions.
• Applications span robotics, visual tracking, and control systems, where RTFs support robust state estimation and real-time adaptation under varying conditions.

Recurrent Tracking Functions (RTFs) are recurrent mappings that update a tracking-relevant state from sequential observations and decode task-relevant outputs from that state. In learned tracking systems, a common RTF-style decomposition is $h_t=f_\theta(h_{t-1},x_t)$ together with an output map such as $y_t=g_\theta(h_t)$ or its probabilistic form $P(y_t\mid x_{1:t})=P(y_t\mid h_t)$; in this sense, the tracker is identified with the recurrent state update itself rather than with a separate detection-association-estimation pipeline (Ondruska et al., 2016). In safety-critical control, the term is used explicitly for a continuous function over tracking-error variables that replaces monotonic Lyapunov decrease with a weaker finite-time recurrence condition while still implying exponential tracking and enabling barrier-based safety guarantees (Liu et al., 1 Oct 2025). The resulting literature spans robotics, visual tracking, multi-object tracking, sound source tracking, particle tracking, and layered control, but the term remains domain-dependent rather than fully standardized.

1. Conceptual scope and terminological usage

Across recurrent tracking architectures, the central idea is that past information is compressed into a state that is updated online and reused for prediction. In the robotics formulation of occupancy tracking, this appears as

$$P(y_t\mid x_{1:t}) = P(y_t\mid h_t),\qquad h_t=f(h_{t-1},x_t),$$

with later extensions that decode semantics $c_t$ from the same hidden state (Ondruska et al., 2016). Closely related formulations appear in single-object tracking, where recurrent hidden state, previous localization, and current visual evidence are combined to predict the next target location, and in multi-object tracking, where the recurrent state may represent per-track latent memories, explicit memory windows, fixed track slots, or implicit state-density maps (Gan et al., 2015).

This broader usage suggests that “RTF” functions as an analytic abstraction even in papers that do not adopt the term explicitly. In that abstraction, what varies across domains is not the existence of recurrence, but the object being propagated: a latent scene map, a set of trajectory embeddings, a density field over hypothetical target space, a target-specific filter, or a time-varying parameterization of the tracker itself.

A persistent source of ambiguity is that “RTF” also denotes Relative Transfer Function in acoustic beamforming. That usage concerns channel estimation and wideband spectral-spatial covariance structure, not recurrent state tracking, and it is therefore a distinct concept despite the identical acronym (Bologni et al., 2024).

2. Control-theoretic formalization

The explicit formal definition of a Recurrent Tracking Function is given in layered safety-critical control for a closed-loop full-order model projected onto a reduced-order model with tracking error $e(\cdot):=z(\cdot)-z_s(\cdot)$ and $\dot e(\cdot):=\dot z(\cdot)-\dot z_s(\cdot)$ (Liu et al., 1 Oct 2025). For a compact set $S\subseteq \mathbb{R}^n\times\mathbb{R}^n$ with $0\in \mathrm{int}(S)$, a continuous $$V:\mathbb{R}^n\times\mathbb{R}^n\to\mathbb{R}_{\ge 0}$$ is an RTF over $y_t=g_\theta(h_t)$0 if it satisfies positive definiteness with linear error bounds,

$y_t=g_\theta(h_t)$1

and a $y_t=g_\theta(h_t)$2-exponential $y_t=g_\theta(h_t)$3-recurrence condition,

$y_t=g_\theta(h_t)$4

The formal novelty lies in replacing the usual monotone decay requirement of a tracking Lyapunov function with recurrence within bounded time. Transient growth is therefore admissible: $y_t=g_\theta(h_t)$5 may increase temporarily, provided it recurrently returns below an exponentially shrinking envelope. The paper proves that this weaker condition still implies exponential convergence of $y_t=g_\theta(h_t)$6 with rate $y_t=g_\theta(h_t)$7 on $y_t=g_\theta(h_t)$8, thereby establishing that recurrence is not merely heuristic relaxation but a sufficient certificate for stable tracking (Liu et al., 1 Oct 2025).

RTFs are then combined with reduced-order Control Barrier Functions (CBFs). If $y_t=g_\theta(h_t)$9 is a CBF for the reduced-order model with linear class-$P(y_t\mid x_{1:t})=P(y_t\mid h_t)$0 function $P(y_t\mid x_{1:t})=P(y_t\mid h_t)$1, and if $P(y_t\mid x_{1:t})=P(y_t\mid h_t)$2 is a valid RTF with $P(y_t\mid x_{1:t})=P(y_t\mid h_t)$3, the paper constructs

$P(y_t\mid x_{1:t})=P(y_t\mid h_t)$4

and shows that the zero-superlevel set $P(y_t\mid x_{1:t})=P(y_t\mid h_t)$5 is control $P(y_t\mid x_{1:t})=P(y_t\mid h_t)$6-recurrent. The same framework yields a safety theorem: if the initial full-order state is safe and its projected state and tracking-error derivative lie in $P(y_t\mid x_{1:t})=P(y_t\mid h_t)$7, then the full-order trajectory remains in the safe set for all $P(y_t\mid x_{1:t})=P(y_t\mid h_t)$8. A disturbance-robust extension replaces exact recurrence by a practical recurrence inequality shifted by a disturbance-dependent margin and yields an input-to-state safety statement (Liu et al., 1 Oct 2025).

3. Latent-state RTFs in learned perception and robotics

In robotics mapping and scene understanding, RTFs appear as learned nonlinear filters over partially observed sensor streams. A canonical example uses laser-derived occupancy observations $P(y_t\mid x_{1:t})=P(y_t\mid h_t)$9, latent state $$P(y_t\mid x_{1:t}) = P(y_t\mid h_t),\qquad h_t=f(h_{t-1},x_t),$$0, occlusion-free occupancy $$P(y_t\mid x_{1:t}) = P(y_t\mid h_t),\qquad h_t=f(h_{t-1},x_t),$$1, and semantic labels $$P(y_t\mid x_{1:t}) = P(y_t\mid h_t),\qquad h_t=f(h_{t-1},x_t),$$2, with the target posterior written as $$P(y_t\mid x_{1:t}) = P(y_t\mid h_t),\qquad h_t=f(h_{t-1},x_t),$$3. The recurrent state is implemented by a convolutional GRU-like unit with dilated convolutions, so the hidden representation is a distributed spatial memory that jointly propagates model dynamics and integrates new sensor measurements. Training is unsupervised for occupancy through future prediction on later-observed cells, and semantic prediction is learned by inductive transfer from the tracking state; on a busy road-junction dataset, the method compares favourably to a state-of-the-art model-free tracking solution and outperforms a conventional one-shot semantic training scheme (Ondruska et al., 2016).

Single-object visual tracking produced early end-to-end RTF-like formulations in which current observations are filtered through previous predictions before entering recurrence. In anonymous object tracking, the model computes $$P(y_t\mid x_{1:t}) = P(y_t\mid h_t),\qquad h_t=f(h_{t-1},x_t),$$4, updates $$P(y_t\mid x_{1:t}) = P(y_t\mid h_t),\qquad h_t=f(h_{t-1},x_t),$$5, and predicts $$P(y_t\mid x_{1:t}) = P(y_t\mid h_t),\qquad h_t=f(h_{t-1},x_t),$$6; the intended role of $$P(y_t\mid x_{1:t}) = P(y_t\mid h_t),\qquad h_t=f(h_{t-1},x_t),$$7 is to summarize $$P(y_t\mid x_{1:t}) = P(y_t\mid h_t),\qquad h_t=f(h_{t-1},x_t),$$8 and $$P(y_t\mid x_{1:t}) = P(y_t\mid h_t),\qquad h_t=f(h_{t-1},x_t),$$9 so that the current target estimate depends on the entire past trajectory and observation history rather than on a short Markov window (Gan et al., 2015). This suggests an RTF interpretation in which recurrence substitutes for explicit observation, transition, and filtering models.

Attention-based trackers made the same abstraction more explicit. RATM uses a recurrent attention module whose hidden state predicts the next spatial sampling window, while a differentiable Gaussian read mechanism extracts the current glimpse and feeds the resulting features back into the recurrent update (Kahou et al., 2015). HART extends this into a hierarchical attentive recurrent tracker with spatial attention, target-conditioned appearance attention, and an LSTM state that jointly predicts the current box, the next attention region, and the next appearance-conditioning vector; its factorization of “where” and “what” pathways suggests that effective RTFs need not ingest raw frames directly, but may instead operate on target-filtered observations (Kosiorek et al., 2017).

4. Structured state, association, and multi-target variants

A major line of development concerns the structure of the recurrent state itself. In sound source tracking, PI-RNN replaces conventional vector recurrence with set-valued input and set-valued state,

$c_t$0

and requires permutation invariance of the input set together with permutation equivariance of the state set. Association is performed by multi-head attention over $c_t$1, and each trajectory embedding is updated by a gated recurrent rule derived from a simplified minimal gated recurrent unit. The attention matrix is interpreted as an assignment matrix, so learned association is embedded directly בתוך the recurrence rather than imposed by a separate matching stage (Diaz-Guerra et al., 2023).

Online multi-object tracking introduces further variants. RAN models each active trajectory with an external memory $c_t$2 and an internal GRU state $c_t$3, from which it predicts a Gaussian conditional density for the next appearance or motion feature; the predictive mean $c_t$4 makes the hidden state a controller over memory rather than a full content store, and detection-to-track association is performed by likelihood evaluation under these recurrent autoregressive models (Fang et al., 2017). A different approach represents a fixed number of track slots with recurrent state estimation, recurrent target-existence management, and an LSTM-based sequential approximation to data association, using an existence probability vector $c_t$5 to handle variable effective cardinality (Milan et al., 2016).

Other multi-target RTFs abandon explicit object lists altogether. The Convolutional Recurrent Predictor for multi-target filtering and tracking learns the motion model over probability density difference maps

$c_t$6

where $c_t$7 is a GM-PHD intensity over continuous hypothetical target space. A ConvLSTM predicts the next PDD map, from which a predicted PHD map is reconstructed and then decoded into explicit Gaussian components, after which birth handling, update, pruning, and Hungarian association are performed classically (Emambakhsh et al., 2018). GotFlow3D adopts a related recurrent-refinement view for double-frame 3D particle tracking: a global optimal transport plan is computed once from graph features, then adaptive point-voxel retrieval and a GRU recurrently refine per-particle flow increments $c_t$8 and the accumulated motion field $c_t$9 (Liang et al., 2022). These examples suggest that the recurrently tracked quantity can be a set, a slot bank, a trajectory-conditioned likelihood, or an implicit field rather than a single latent vector.

5. Recurrently updated tracking functions

A distinct interpretation of RTFs locates recurrence not in the forward state estimator, but in the tracker that is being updated over time. ROAM is the clearest example: the forward tracker consists of response generation and bounding-box regression with parameters $e(\cdot):=z(\cdot)-z_s(\cdot)$0, while a coordinate-wise LSTM optimizer recurrently predicts an element-wise learning-rate vector $e(\cdot):=z(\cdot)-z_s(\cdot)$1 from previous learning rate, current gradient, current parameters, and current loss, and then updates the tracking model by

$e(\cdot):=z(\cdot)-z_s(\cdot)$2

The effective tracking function is therefore time-varying, and the recurrence lies in parameter adaptation rather than in the feed-forward predictor itself (Yang et al., 2019).

Recurrent Filter Learning for visual tracking pushes the same idea into target-specific filter generation. A convolutional LSTM maintains hidden and cell states $e(\cdot):=z(\cdot)-z_s(\cdot)$3 from exemplar features $e(\cdot):=z(\cdot)-z_s(\cdot)$4, produces a target filter $e(\cdot):=z(\cdot)-z_s(\cdot)$5, and applies that filter to next-frame search features $e(\cdot):=z(\cdot)-z_s(\cdot)$6 through

$e(\cdot):=z(\cdot)-z_s(\cdot)$7

Here the recurrently updated object model is the appearance memory encoded in $e(\cdot):=z(\cdot)-z_s(\cdot)$8, operationalized as the filter used for matching; the tracker adapts every frame without online SGD, and ablations show that convolutional recurrence and learned initialization materially improve performance (Yang et al., 2017).

Re$e(\cdot):=z(\cdot)-z_s(\cdot)$9 adopts an intermediate position. It uses a two-layer factored LSTM with peephole connections whose cell and hidden states act as the tracker’s latent appearance-and-motion memory, while the second LSTM output is mapped directly to the current box corners in crop coordinates. The paper states that it does not update the network itself at test time; instead, the recurrent parameters represent the tracker state and are updated with a single forward pass, which makes the hidden state functionally equivalent to an online target model (Gordon et al., 2017). Taken together, these systems broaden the RTF concept from recurrent state estimation to recurrent model evolution.

6. Supervision, robustness, and theoretical limits

Many practical RTFs are trained by objectives that force the recurrent state to survive missing data, self-generated inputs, or delayed supervision. In occupancy tracking, future-prediction training compares predictions only on cells that later become observed, making occlusion at deployment analogous to deliberately omitted future inputs during training; the same logic is reused for semantic labels so that classification of occluded objects depends on memory rather than only on current visibility (Ondruska et al., 2016). In visual tracking, Re$\dot e(\cdot):=\dot z(\cdot)-\dot z_s(\cdot)$0 uses curriculum learning up to training length $\dot e(\cdot):=\dot z(\cdot)-\dot z_s(\cdot)$1 and scheduled self-training in which crop generation gradually shifts from ground truth to the tracker’s own predictions, explicitly teaching the recurrent update to operate under closed-loop drift (Gordon et al., 2017).

The strongest theoretical challenge to long-horizon RTF design comes from recent work on error control dynamics. For symbolic state-tracking tasks, affine recurrent models—including model classes encompassing State-Space Models and Linear Attention—can realize transition rules while failing to correct perturbations along the symbolic subspace

$\dot e(\cdot):=\dot z(\cdot)-\dot z_s(\cdot)$2

If a return sequence preserves all symbolic codes $\dot e(\cdot):=\dot z(\cdot)-\dot z_s(\cdot)$3, then the induced affine map satisfies $\dot e(\cdot):=\dot z(\cdot)-\dot z_s(\cdot)$4, so state-relevant perturbations are transported rather than contracted. In the approximate setting this yields

$\dot e(\cdot):=\dot z(\cdot)-\dot z_s(\cdot)$5

and failure is predicted by the distinguishability ratio $\dot e(\cdot):=\dot z(\cdot)-\dot z_s(\cdot)$6, which compares decoder-visible within-class spread to between-class separation. Across 113 affine $\dot e(\cdot):=\dot z(\cdot)-\dot z_s(\cdot)$7 models with $\dot e(\cdot):=\dot z(\cdot)-\dot z_s(\cdot)$8, the first threshold-crossing time $\dot e(\cdot):=\dot z(\cdot)-\dot z_s(\cdot)$9 correlates strongly with downstream max-passing length, with $S\subseteq \mathbb{R}^n\times\mathbb{R}^n$0 and $S\subseteq \mathbb{R}^n\times\mathbb{R}^n$1 (Chung et al., 8 May 2026). This result suggests that recurrent tracking is governed not only by representational expressivity but also by whether the dynamics can repeatedly restore decoder-relevant state geometry.

The literature therefore exhibits a recurring tradeoff. Implicit trackers can avoid hand-designed detection-association-estimation pipelines, but they may remain non-object-centric, place-specific, slot-limited, or dependent on classical post-processing. Examples include occupancy trackers whose static memory is tied to a specific place, set-structured trackers implemented with a fixed maximum number of slots, and implicit density-map predictors that still rely on classical measurement update and Hungarian association outside the recurrent predictor (Ondruska et al., 2016). In that sense, the RTF concept names a family of recurrent tracking mechanisms rather than a single settled architecture, and its mature forms continue to balance memory structure, association strategy, supervision design, and error-control properties.