GPS-MTM: Masked Trajectory Transformer

• The paper introduces GPS-MTM, a framework that reconstructs high-resolution GPS trajectories from degraded inputs using transformer architectures and graph convolutional networks.
• It employs dual methodologies, with one variant restoring geometric details and another inferring masked mobility events to capture patterns of normalcy.
• Experimental results on datasets like GeoLife and Numosim-LA demonstrate notable improvements in trajectory fidelity and infilling accuracy over traditional methods.

Searching arXiv for the cited GPS-MTM and related trajectory-modeling papers. arxiv_search(query="(Yonekura et al., 2024)", max_results=5, sort_by="submittedDate") GPSMasked Trajectory Transformer (GPS-MTM) is a label used in recent trajectory-learning literature for transformer-based GPS mobility models trained with reconstruction-oriented self-supervision, but it does not denote a single canonical architecture. In current arXiv usage, the term refers both to a super-resolution system for restoring high-resolution trajectories from privacy-preserving low-resolution releases and to a later foundation-model framework that learns “patterns of normalcy” in mobility through masked reconstruction over semantically decomposed trajectory events (Yonekura et al., 2024, Garg et al., 28 Sep 2025). A related, but distinct, line is Joint GPS and Route Modeling (JGRM), which combines GPS and route modalities through a shared transformer-based fusion module rather than adopting a single-stream masked trajectory transformer (Ma et al., 2024).

1. Terminological scope and conceptual definition

The shared conceptual core of GPS-MTM is masked or reconstruction-style learning over trajectory data with transformer components. In both uses of the acronym, the model is designed to recover information that is absent from the observed input: in one case, missing spatial precision in degraded GPS traces; in the other, masked semantic components of mobility sequences. This suggests that the term is best understood as a family resemblance across problem formulations rather than as one fixed model definition.

Usage	Representation	Primary setting
Super-resolution GPS-MTM	Low-resolution GPS trajectory plus road graph	Restore high-resolution GPS traces
Foundation-model GPS-MTM	States and actions over stop-point sequences	Learn patterns of normalcy for infilling and prediction
JGRM (related, not identical)	Joint GPS trajectory and route trajectory	Multimodal trajectory representation learning

The super-resolution usage appears in “Restoring Super-High Resolution GPS Mobility Data,” where GPS-MTM denotes a hybrid transformer-plus-GCN reconstruction system operating on rounded, truncated, masked, or synthetic low-resolution GPS traces (Yonekura et al., 2024). The foundation-model usage appears in “GPS-MTM: Capturing Pattern of Normalcy in GPS-Trajectories with self-supervised learning,” where GPS-MTM denotes a bi-directional Transformer trained to reconstruct missing state/action tokens in semantically structured mobility sequences (Garg et al., 28 Sep 2025).

2. Super-resolution GPS-MTM for privacy-degraded trajectories

In the 2024 super-resolution formulation, GPS-MTM addresses a publication scenario in which raw GPS points are not released at full precision. The paper explicitly discusses rounding/truncation of latitude/longitude, noise injection, spatial cloaking / obfuscation, and publication of synthetic trajectories generated from a privacy-preserving model. The motivating problem is the inverse mapping

$$\text{low-resolution trajectory} \rightarrow \text{high-resolution trajectory},$$

with the goal of recovering fine-grained details such as turns, intersections, lane changes, and short deviations that are lost when data are coarsened for privacy or storage reasons (Yonekura et al., 2024).

The architecture combines a Transformer encoder-decoder with a Graph Convolutional Network (GCN). The road network is represented as

$G=(V,E),$

where nodes are road intersections and edges are roads connecting them. Each node has geographic features

$x_v=[\text{lat}_v,\text{lon}_v],$

and each edge is weighted using the inverse haversine distance between neighboring intersections. The paper uses a localized subgraph around the trajectory to reduce memory and computation. A standard GCN update is written as

$$H^{(l+1)}=\sigma\!\left(\tilde{D}^{-1/2}\tilde{A}\tilde{D}^{-1/2}H^{(l)}W^{(l)}\right).$$

The trajectory itself is treated as a multivariate time series. Each time step contains standardized GPS and time features, roughly

$z_t=[\hat{\text{lat}_t,\hat{\text{lon}_t,\hat{\tau}_t}],$

with z-score normalization applied to latitude, longitude, and timestamps. The Transformer encoder captures sequential dependencies through scaled dot-product attention,

$$\text{Attention}(Q,K,V)=\text{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V,$$

and multi-head attention,

$$\text{MHA}(Q,K,V)=\text{Concat}(\text{head}_1,\dots,\text{head}_h)W^O.$$

The decoder then fuses the trajectory embedding from the encoder with the road-network embedding from the GCN to generate a high-resolution output sequence. Conceptually, it predicts

$\hat{Y}=f_{\theta}(X_{\text{low-res}},G),$

and outputs normalized coordinates that are inverse-transformed back to latitude/longitude values.

The training objective uses Soft Dynamic Time Warping (SoftDTW), written as

$$\mathcal{L}_{\text{SoftDTW}(X,Y)}=\operatorname{SoftDTW}_{\gamma}(X,Y),$$

and the optimization problem is

$$\min_{\theta}\sum_{i=1}^{N}\mathcal{L}_{\text{SoftDTW}(\hat{Y}_i,Y_i)}, \qquad \hat{Y}_i=f_{\theta}(X_i,G_i).$$

Because SoftDTW is differentiable and tolerant to time misalignment, it is suited to trajectory reconstruction in which the recovered path may be temporally shifted but geometrically close.

3. Foundation-model GPS-MTM for patterns of normalcy

In the 2025 formulation, GPS-MTM is presented as a self-supervised foundation-model-style framework for mobility data. The model’s central claim is that trajectories should not be flattened into coordinate streams or grid-cell tokens alone; instead, mobility is decomposed into two aligned modalities, states and actions. States are semantic dwell locations expressed as Point-of-Interest (POI) categories such as home, work, gym, and grocery store, while actions are the transition details associated with a stop, especially the temporal/spatial attributes of staying and moving (Garg et al., 28 Sep 2025).

Formally, a trajectory $G=(V,E),$0 is represented as a sequence of stop points with

$G=(V,E),$1

where $G=(V,E),$2 is the POI category, and

$G=(V,E),$3

This representation turns mobility modeling into a multi-modal prediction problem in which one modality can be masked and inferred from the other.

Architecturally, the model uses a bi-directional Transformer encoder in the style of BERT. It has a multi-modal input embedding layer, a multi-layer bi-directional self-attention encoder, and task-specific prediction heads. The prediction heads are split into a classification head for masked POI categories and a regression head for continuous or structured detail vectors. Because the encoder is bi-directional, it uses both left and right context, which is particularly appropriate for infilling missing segments rather than only extrapolating from a prefix.

The training objective is to maximize

$G=(V,E),$4

with the paper writing the MLE objective as

$G=(V,E),$5

The combined loss is

$G=(V,E),$6

where

$G=(V,E),$7

is a Focal Loss with $G=(V,E),$8 and $G=(V,E),$9, and

$x_v=[\text{lat}_v,\text{lon}_v],$0

is MSE over the stop-detail vector. The balancing coefficient is set to $x_v=[\text{lat}_v,\text{lon}_v],$1.

The paper frames the resulting latent structure as “patterns of normalcy”: regularities in daily movement that emerge when the model is required to reconstruct missing state-action tokens from context. A plausible implication is that the model is intended not only for infilling accuracy, but also for learning a semantic prior over routine human mobility.

4. Masking, reconstruction, and what is being inferred

Although both systems are described under the GPS-MTM label, the masked or missing information differs substantially. In the super-resolution system, the input is already a degraded trajectory, often produced by H3-based rounding in which each GPS point is snapped to the center of its H3 cell. The reconstruction target is geometric fidelity: the model must recover finer-grained coordinates from coarse releases or from synthetic trajectories. The end-to-end pipeline is explicitly described as starting with a low-resolution or synthetic trajectory, normalizing coordinates and timestamps, encoding the trajectory with the Transformer encoder, encoding the local road network with the GCN, fusing both embeddings in the decoder, predicting a higher-resolution sequence of GPS points, reversing normalization, and evaluating the result with trajectory similarity metrics (Yonekura et al., 2024).

In the foundation-model system, the missing information is created by random masking over semantically structured tokens rather than by coarsening the coordinates. The model reconstructs masked state/action components from the remaining observed context. Its key downstream use case is trajectory infilling, where GPS traces have gaps due to signal dropout or missing observation windows; however, the same pretraining objective is intended to support destination prediction, anomaly detection, and related mobility tasks (Garg et al., 28 Sep 2025).

The contrast is methodological as well as semantic. The super-resolution formulation couples sequence modeling with a road-network prior through the GCN. The foundation-model formulation instead emphasizes semantic decomposition into POI categories and stop details, with a bi-directional Transformer as the primary learning engine. This suggests that one GPS-MTM line is map-constrained and coordinate-reconstructive, whereas the other is semantically structured and event-reconstructive.

5. Datasets, benchmarks, and reported performance

The super-resolution system is evaluated on the Beijing trajectory dataset from the GeoLife collection. The paper states that trajectories were limited to length $x_v=[\text{lat}_v,\text{lon}_v],$2, that 25,769 virtual GPS trajectories were generated in the Beijing region for training, and that the model was evaluated on real-world trajectory data from the same geographic area. Preprocessing includes H3-based rounding of coordinates, z-score normalization of GPS values, and inverse-distance edge weighting for the graph. The main baselines are a traditional Hidden Markov Model-based map matching approach and an LSTM autoencoder / TrajGAN-style synthetic trajectory generator (Yonekura et al., 2024).

Its principal metric is Fréchet distance. The headline result reported in the abstract is an average Fréchet distance of 0.198 km for GPS-MTM, compared with 0.632 km for map matching and 0.498 km for the LSTM synthetic trajectory method. The paper also reports that 85.7% of reconstructed trajectories fall within a narrow error margin in histogram analysis, that the method remains effective across different H3 resolution levels, and that it recovers fine-grained details that map matching misses. These results are presented as evidence that the combined Transformer encoder-decoder and GCN better captures both temporal dependencies and road-network structure than the listed baselines.

The foundation-model system is evaluated on three datasets: Numosim-LA, Urban Anomalies (UA-Atlanta and UA-Berlin), and Geolife. Numosim-LA has 28 POI categories. Urban Anomalies has 4 primary POI categories and is broken into combined, hunger, interest, social, and work subtasks. Geolife is used as a curated subset with 198 unique POI categories. The evaluation tasks are Forward Dynamics (FD), Inverse Dynamics (ID), Random Masking, and Goal Prediction (Garg et al., 28 Sep 2025).

On Numosim-LA, the reported accuracies are 0.75 for FD, 0.65 for ID, 0.60 for Random, and 0.63 for Goal. For UA-Atlanta, combined, the reported values are 0.55 for ID accuracy, 0.53 for FD accuracy, 0.39 for Random accuracy, and 0.37 for Goal accuracy. For UA-Berlin, combined, the reported values are 0.51 for ID accuracy, 0.46 for FD accuracy, 0.47 for Random accuracy, and 0.43 for Goal accuracy. On Geolife, the raw accuracies are 0.05 for ID, 0.09 for FD, 0.04 for Random, and 0.06 for Goal, while the Bias Ratio remains near 1.0 for some tasks, including 1.06 for ID and 1.36 for FD. The paper interprets this as evidence that the model is not simply collapsing to majority-class predictions despite the large label space.

6. Relationship to JGRM and neighboring trajectory transformers

A recurrent misconception is that any transformer with masking over trajectory data is equivalent to GPS-MTM. JGRM shows why that is not the case. “More Than Routing: Joint GPS and Route Modeling for Refine Trajectory Representation Learning” proposes a joint GPS-and-route framework in which GPS trajectory and route are treated as two modes of a single movement observation, each processed by its own encoder and then fused by a shared transformer for inter-modal information interaction (Ma et al., 2024).

The GPS encoder first uses map matching to build an assignment matrix from GPS points to road segments, groups raw GPS points into sub-trajectories per road segment, and represents each GPS point with seven kinematic features: longitude, latitude, speed, acceleration, angle delta, time delta, and distance. These are encoded hierarchically with an intra-road BiGRU and an inter-road BiGRU. The route encoder takes a sequence of road IDs plus temporal features, uses a GAT layer,

$x_v=[\text{lat}_v,\text{lon}_v],$3

and combines topology with minute-of-day, day-of-week, and learned continuous travel-time embeddings. A Transformer encoder refines the route sequence, and mean pooling yields trajectory embeddings for both modalities.

Before fusion, each token embedding is augmented with a modal embedding and positional embedding,

$x_v=[\text{lat}_v,\text{lon}_v],$4

after which GPS-view and route-view segment- and trajectory-level representations are concatenated and passed through a shared Transformer encoder. The final fused outputs are obtained by averaging the two modalities. Training uses three losses—GPS MLM, Route MLM, and Match—even though the prose sometimes says “two tasks.” The MLM task masks subpaths rather than individual tokens, using a shared mask across GPS and route views so that the same road segments are hidden in both modalities.

The conceptual overlap with GPS-MTM is clear: both use masking, transformer-based sequence modeling, and token-like trajectory elements. The crucial difference is structural. JGRM does not use a single unified masked transformer over one token stream; it uses two modality-specific encoders and a shared transformer for interaction. The paper itself is explicit that JGRM is related in spirit, not identical in form. It is therefore best described as a multimodal masked trajectory representation learner with transformer fusion rather than as a direct instance of a single-stream GPS-MTM architecture.

The same section also clarifies a broader methodological boundary. In the super-resolution GPS-MTM line, the principal issue is recovery of geometric resolution under road-network constraints. In the foundation-model GPS-MTM line, the principal issue is recovery of semantically masked mobility events and the learning of “patterns of normalcy.” JGRM occupies a third position: joint modeling of micro GPS motion cues and macro route semantics through cross-modal interaction. A plausible implication is that these works collectively define a larger design space for trajectory transformers in which masking may serve geometric reconstruction, semantic infilling, or multimodal alignment, depending on how the trajectory is represented and what information is treated as missing.