MSRFormer: Multi-Scale Road Embeddings

• MSRFormer is a framework that employs multi-scale feature fusion and contrastive learning to capture heterogeneous spatial interactions in urban road networks.
• It integrates trajectory-informed flow patterns with topological graph structures using spatial flow convolution and graph transformer modules.
• Empirical results from Porto and San Francisco benchmarks show improved classification and prediction performance, validating its effectiveness for traffic analytics.

MSRFormer is a road network representation learning framework that utilizes multi-scale feature fusion to capture heterogeneous spatial interactions within urban road networks. MSRFormer integrates trajectory-informed flow patterns, topological graph structure, and long-distance spatial dependencies using specialized convolutional and transformer-based modules, enabling robust embeddings for road segments suitable for a variety of network analysis and transportation prediction tasks.

1. Architectural Overview

MSRFormer leverages both the static structure of the road network (graph $G$) and vehicle trajectory data ($T$) as inputs. The core workflow consists of the following interconnected modules:

• Preprocessing: Road network $G$ and trajectory dataset $T$ are processed for map-matching and extraction of initial road segment features.
• Spatial Flow Convolution (SFC): Local road embeddings are generated by applying SFC over $k$-order road transfer matrices $P_k$, which are derived from trajectory data and the graph structure.
• Multi-scale Spatial Interaction Module: For each scale $k$ (capturing small, medium, and large spatial ranges), community detection via spectral clustering partitions the network into regions with homogeneous flow interactions.
• Graph Transformer for Feature Extraction: Within each region and scale, a graph transformer performs self-attention, augmented by scale-dependent bias terms encoding flow probabilities from $P_k$.
• Residual Feature Fusion: Outputs from all scales are aggregated using residual connections to preserve and combine multi-scale characteristics.
• Contrastive Learning: The final fused features are refined with a contrastive loss that exploits spatial interaction matrices to identify informative positive and negative pairs.

This pipeline facilitates the extraction and integration of localized and global interaction patterns, producing high-dimensional embedding vectors $R$ for downstream analytics.

2. Spatial Flow Convolution and Road Transfer Matrices

A central methodological innovation lies in spatial flow convolution (SFC), which extends beyond traditional neighborhood aggregation:

• Calculation of Road Transfer Matrix $P_k$: For each road segment node, the $k$-order matrix $P_k$ incorporates both graph connectivity and trajectory transition statistics.
• Convolution Operation: For node features $F$ and trainable matrix $W$, the SFC is formulated as $SSFC = \mathrm{ReLU}(P_k F W)$ (Equation 3), where $P_k$ scales feature propagation according to empirical flow data.
• Interpretation: This design enables the model to assimilate micro-scale traffic flow details, differentiating between heavily and sparsely used segments based on observed transitions.

Community detection using spectral clustering on a derived spatial interaction matrix $S_k$ further segments the graph into regions whose internal interactions share similar flow characteristics, improving the representation’s locality and scale-awareness.

3. Graph Transformer for Scale-dependent Feature Extraction

MSRFormer’s use of graph transformers introduces two important mechanisms:

• Input Processing: Each region is treated as a subgraph with input matrix $H$. Linear projections generate queries $Q = H W_Q$, keys $K = H W_K$, and values $V = H W_V$ (Equation 8).
• Attention Augmentation: The pairwise attention score incorporates both self-attention and a bias term from $P_k$, such that $$A[i,j] = \mathrm{softmax}((QK^T)_{i,j} / \sqrt{d} + b_0(P_k(v_i, v_j)))$$ (Equation 10).
• Significance: Incorporating scale-dependent flow via the bias term ensures that the transformer’s weighting reflects both structural proximity and empirical traffic associations, enhancing its ability to capture non-local dependencies not addressed by GNN message passing.

This architectural choice is essential for modeling the heterogeneity and multi-hop relationships prevalent in urban mobility patterns.

4. Residual Fusion of Multi-scale Features

After feature extraction at all scales, MSRFormer applies residual connections to fuse the resulting representations:

• Fusion Rule: The output of the transformer for scale $k$ is added to its input, i.e. $H' = \mathrm{GraphTransformer}(H) + H$ (Equation 11).
• Implications: This strategy maintains feature continuity and gradient flow during training, enabling the model to balance and integrate fine-grained (local flow) and broad-scale (community and long-range) spatial information.

The fusion mechanism is critical for generating embeddings that are robust to network topology variance and capable of representing both concentrated urban cores and distributed suburban layouts.

5. Contrastive Learning for Representation Refinement

To enhance expressiveness and spatial discrimination in the learned embeddings, MSRFormer employs a contrastive learning algorithm tailored to spatial interaction intensity:

• Pair Construction: Positive samples are node pairs demonstrating strong flow interaction in $S_k$; negative samples lack notable interaction.
• Loss Function: The contrastive loss is defined as $$L = - \mathbb{E}_{+}[\log(\sigma(o(h_i, h_j)))] - \mathbb{E}_{-}[\log(1 - \sigma(o(h_i, h_j)))]$$ (Equation 12), with $h_i$ and $h_j$ as node embeddings, $\sigma$ the sigmoid function, and $o(\cdot, \cdot)$ a similarity measure.
• Result: This training signal compels the embedding space to reflect real multi-scale, heterogeneous spatial and traffic interactions, yielding representations suitable for both unsupervised and supervised network analysis scenarios.

6. Quantitative Evaluation and Empirical Findings

MSRFormer was benchmarked on large-scale datasets from Porto and San Francisco, focusing on two downstream tasks:

• Road Label Classification: Categories include freeways, arterials, and residential streets. The model achieved up to 26% improvement in Macro-F1 compared to competitive baselines (RFN, IRN2Vec, GCN, GAT).
• Traffic Inference: Road segment speed prediction was conducted, with MAE and RMSE as metrics; performance gains reached up to 16% over the strongest competitors.
• Ablation Studies: Removal of multi-scale extraction, residual fusion, or contrastive modules resulted in clear performance degradation, validating each component’s contribution.

The results demonstrate that incorporating trajectory data and multi-scale spatial reasoning is most beneficial in complex road networks, where traditional single-scale or homogeneity-based models are insufficient.

7. Implications and Prospective Applications

MSRFormer advances task-agnostic representation learning for urban road networks by:

• Enhancing Traffic Analytics: Improved embeddings directly benefit congestion analysis, classification, and network matching approaches.
• Supporting Smart Transportation: Representations are applicable to real-time systems for adaptive signaling, routing, and urban planning.
• Generalizability: The model’s scalable construction supports large networks, adaptability across various geographies, and integration into intelligent transportation systems.

A plausible implication is that future extensions could explore joint learning across multiple cities or modalities, and deeper integration with traffic simulation platforms for dynamic prediction and planning. The framework’s methodology embodies a significant step forward in understanding the nuanced interplay of scale effects and flow heterogeneity in urban mobility modeling.