POIFormer: Transformer for POI Analytics

• The paper introduces POIFormer, a transformer-based framework that integrates multi-modal embeddings and crowd-level density estimates to robustly address noisy POI attribution and next-location recommendation.
• It utilizes spatial, temporal, and semantic feature encodings with contrastive self-supervision to effectively disentangle short-term mobility transitions from long-term user preferences.
• Empirical evaluations on synthetic and real-world datasets demonstrate significant gains over traditional methods, highlighting its scalability and robustness in urban mobility analytics.

POIFormer denotes two distinct but thematically related Transformer-based frameworks for modeling user mobility and point-of-interest (POI) semantics in the context of urban mobility analytics. The first formulation addresses the attribution problem—mapping noisy spatiotemporal “stay points” to true POI visits amid high spatial density and GPS uncertainty. The second version focuses on personalized next-location recommendation by disentangling short-term mobility from long-term user preference, leveraging contrastive self-supervision. Both models exemplify state-of-the-art design for context-rich, noise-resilient, and scalable inference in POI-centric applications.

1. Motivation and Challenges in POI Attribution and Recommendation

Accurate POI attribution and next-location recommendation are foundational for urban mobility research, personalized services, and spatial analytics. Attribution is challenged primarily by two factors: (1) GPS and hybrid localization errors (typically 2–20 meters), which can be commensurate with or exceed POI separation in dense urban areas, and (2) the clustering of tens to hundreds of POIs within small radii (<100 m), rendering proximity-based heuristics unreliable for disambiguation (Saxena et al., 12 Jul 2025). Additionally, for next-location recommendation, user check-in data must be modeled to capture both transient mobility habits and persistent individual preferences (Luo et al., 2023).

Classic methods—e.g., closest-centroid assignment or categorical Bayesian models—either fail to leverage population-level behavioral patterns or impose restrictive assumptions on POI ontologies. Industry pipelines (e.g., SafeGraph’s) require proprietary metadata (building footprints, hierarchies) often unavailable at scale, and typically neglect either user-specific or crowd-level priors. For recommendation, most recurrent/sequential models ineffectively disentangle long-term tastes from local mobility transitions.

POIFormer frameworks address these limitations by (1) jointly modeling spatiotemporal, semantic, individual, and crowd-level features in attribution, and (2) explicitly contrasting user preference representations in recommendation using end-to-end self-attention architectures.

2. Feature Representations and Input Encoding

Both POIFormer variants employ multi-modal, learnable embeddings for the core dimensions of mobility:

• Spatial Features: Geographic coordinates are mapped via Space2Vec [Mai et al. 2020] continuous embeddings, uniquely encoding location in $\mathbb{R}^d$ (Saxena et al., 12 Jul 2025, Luo et al., 2023).
• Temporal Features: Time2Vec [Kazemi et al. 2019] produces per-point encodings for arrival ($t^a$) and departure ($t^d$) times, or discretized check-in times as in next-POI recommendation.
• Semantic Features: A category-embedding matrix $$E^{\text{cat}} \in \mathbb{R}^{|\mathcal C| \times d}$$ maps POI categories (multi-category allowed) to vector space, with per-stay category embeddings summed or averaged.
• Composite Stay Embedding: Stay points are represented as $$\mathbf{x}_i = [e^{\text{space}}_i \| e^{\text{timeA}}_i \| e^{\text{timeD}}_i \| e^{\text{cat}}_i] \in \mathbb{R}^{4d}$$, with positional encodings injected for trajectory ordering.

In the next-POI recommendation POIFormer, each check-in is likewise mapped to $e_i^\rho = e_i^l + e_i^c + e_i^t + \mathrm{pos}_i$.

Crowd-level features in the attribution variant are incorporated via offline-computed kernel density estimates (KDEs), rather than learned as part of the dense input encoding.

3. Transformer-Based Model Architectures

POI Attribution Variant

A bidirectional Transformer encoder processes the sequence of user stays, masking out the current target stay’s input and replacing it with a learned vector (Saxena et al., 12 Jul 2025). All other positions encode the full set of features. Self-attention layers are standard:

• Multi-head, scaled dot-product attention with $Q$, $K$, $V$ projections ($W^Q$, $t^a$0, $t^a$1), with output dimension $t^a$2 per head.
• Residual connections and layer normalization post-attention and post-FFN.
• After $t^a$3 layers, the hidden state $t^a$4 at the masked location serves as a context-dependent summary for POI selection, incorporating both past and future mobility context (explicitly non-causal).

Next-Location Recommendation Variant

This variant employs a three-module encoder–decoder architecture (Luo et al., 2023):

• History Encoder ($t^a$5): Stack of Transformer encoder layers processes historical trajectories to capture sequential dependency and mobility habits.
• Query Generator ($t^a$6): Parameter-shared Transformer encoder takes both original and augmented trajectory views to generate user-specific intention queries. Contrastive InfoNCE loss encourages queries for the same user (augmented/original) to be closer and for different users to be distant.
• Preference Decoder ($t^a$7): Transformer decoder layers fuse the extracted intention query and the encoded trajectory via cross-attention, producing a next-POI representation.

This architecture permits explicit disentanglement and fusion of short-term and long-term behavior.

4. Probabilistic Reasoning and Loss Functions

Attribution Score Computation

POIFormer for attribution combines two sources of probabilistic evidence for each candidate POI $t^a$8:

$t^a$9

where $t^d$0 is derived from the output $t^d$1 via a linear head and LogSoftmax, and $t^d$2 is queried from offline KDEs over spatial–temporal visit distributions for category $t^d$3. The cross-entropy loss is:

$t^d$4

where $t^d$5 is the true visited POI and $t^d$6 is the candidate set (Saxena et al., 12 Jul 2025).

Recommendation Matching and Contrastive Objectives

For the recommendation variant, the matching loss for POI $t^d$7 and predicted embedding $t^d$8 is:

$t^d$9

with $$E^{\text{cat}} \in \mathbb{R}^{|\mathcal C| \times d}$$0 as a learnable temperature and $$E^{\text{cat}} \in \mathbb{R}^{|\mathcal C| \times d}$$1 negative samples. The contrastive loss (InfoNCE) aligns pooled queries from original and augmented trajectories; total loss is a weighted sum:

$$E^{\text{cat}} \in \mathbb{R}^{|\mathcal C| \times d}$$2

(Luo et al., 2023).

5. Empirical Evaluation and Comparative Performance

Datasets

• Attribution: NUMOSIM (synthetic, LA-scale, 200k agents, 16M stays), Breadcrumbs (Lausanne, 81 users, 20k stays, ~500 POIs), both with realistic and injected GPS noise levels.
• Recommendation: Gowalla, SIN/TKY (Foursquare), Brightkite with up to 772k POIs and 4.75M check-ins (Luo et al., 2023).

Baselines

• Closest Centroid, Nishida et al. [2014], SafeGraph proprietary pipeline (attribution)
• RNN, GRU, STRNN, HST-LSTM, DeepMove, ATST-LSTM, STGN, ARNN, LSTPM, GeoSAN, STAN (recommendation)

Results

Attribution (Top-1/Top-3/Top-5 Accuracy; NUMOSIM, noisy settings):

Method	Top-1	Top-3	Top-5
Closest Centroid	0.265	0.591	0.757
Nishida et al.	0.100	0.177	0.201
SafeGraph	0.702	0.794	0.839
POIFormer	0.722	0.909	0.965

Recommendation (Recall@5/Recall@10, Gowalla/SIN/TKY/Brightkite):

Method	Gowalla	SIN	TKY	Brightkite
STAN	0.302/0.400	0.346/0.426	0.375/0.430	0.569/0.673
POIFormer	0.328/0.433	0.406/0.469	0.382/0.470	0.576/0.682

POIFormer demonstrates substantial gains over baselines, especially in dense, noisy attribution scenarios (up to +17 accuracy points in top-5, NUMOSIM) (Saxena et al., 12 Jul 2025). In next-POI recommendation, gains over STAN are +2.6%–5.1% (relative) on Recall@5/10, statistically significant ($$E^{\text{cat}} \in \mathbb{R}^{|\mathcal C| \times d}$$3) (Luo et al., 2023). Ablation studies confirm the critical role of KDE features and category-priors (attribution), as well as explicit query disentanglement and contrastive loss (recommendation).

6. Scalability, Generalization, and Practical Considerations

POIFormer’s computational design enables linear inference complexity in both trajectory length and candidate POI set size. KDEs for crowd-level priors are precomputed offline, leaving only lightweight querying at inference. Transformer computations are batched and GPU-friendly, allowing deployment on large mobility datasets (Saxena et al., 12 Jul 2025). There is no dependence on proprietary building footprints or semantic hierarchies—only GPS, POI location, and category tags are required—enabling broad-scale applicability.

Limitations include fixed context windows (attenuating benefit for extremely long or sparse histories), KDEs with non-adaptive bandwidths (potential over- or under-smoothing in sparse/dense POI regions), and absence of end-to-end density–Transformer co-training. In the recommendation context, only basic augmentation and Transformer blocks are employed; incorporating richer spatial graphs or textual metadata presents a plausible direction for extension. The attribution variant does not currently support memory-augmented or hierarchical context, which could further enhance generalization (Saxena et al., 12 Jul 2025, Luo et al., 2023).

7. Significance and Future Directions

POIFormer represents a unifying paradigm for robust POI modeling under real-world uncertainty, combining crowd-level statistical priors, semantic features, trajectory context, and individualized behavioral history within a scalable self-attention architecture. Its design establishes new benchmarks in both attribution and recommendation, particularly in scenarios of high spatial density and measurement noise.

A plausible implication is that future work may exploit adaptive KDEs, hierarchical Transformers, or end-to-end integration of density estimators. In recommendation, incorporating multi-query intention models or exogenous information (e.g., social ties, textual content) could further advance personalization and resilience. These advances suggest an increasingly holistic approach to modeling human urban mobility, transcending the split between individual and population-level inference demonstrated in POIFormer (Saxena et al., 12 Jul 2025, Luo et al., 2023).