Drive2Vec: Compact Vehicular Embeddings

• Drive2Vec is a deep learning framework that transforms high-dimensional vehicular sensor data into a compact 64-dimensional embedding for efficient, multi-scale prediction.
• It employs stacked GRU encoders and a multi-task prediction head to accurately forecast short-term sensor dynamics and long-term context.
• The unified embedding supports diverse automotive applications including behavior inference, anomaly detection, and driver identification with robust performance metrics.

Drive2Vec is a deep learning framework for embedding high-dimensional vehicular sensor data from CAN bus streams into a compact, actionable, low-dimensional representation suitable for multi-scale prediction and real-world automotive tasks. Developed to support predictive modeling, behavior inference, anomaly detection, and knowledge base generation within resource-constrained onboard systems, Drive2Vec provides a single, unified embedding that is simultaneously predictive of both short-term dynamics and long-term context, outperforming prior baseline representations by substantial margins (Hallac et al., 2018).

1. Motivation and Problem Setting

Modern vehicles output hundreds to thousands of high-frequency signals from their Controller Area Network (CAN) bus—covering engine status, pedal positions, dynamics, digital flags, and more—resulting in a high-dimensional, multi-rate, noisy time series. Compressing this data into a low-dimensional vector ("embedding") has several imperatives:

• Resource Efficiency: Storage and bandwidth constraints on embedded systems demand strong compression. A 64-dimensional embedding is orders of magnitude smaller than the raw 665-channel, 10 Hz timeseries.
• Generalization Across Tasks: Downstream automotive applications (e.g., short-term forecasting, driver style inference, anomaly detection) benefit from representations that retain core behavioral and contextual state, rather than sensor-specific noise.
• Multi-Scale Predictivity: A unified embedding should encode enough information to support both near-term (maneuver prediction) and long-term (road type, driver identity) inference. This design enables shared statistical strength across forecasting, recognition, and unsupervised discovery applications—avoiding the inefficiencies of single-task, model-per-task paradigms (Hallac et al., 2018).

2. Drive2Vec Architecture

The Drive2Vec model ingests a short segment of sensor data and outputs a fixed-size embedding, optimized for multiscale predictive power. The architecture is as follows:

• Input: Recent sensor sequence $X_{t−k+1:t} = [x_{t−k+1}, ..., x_t]$, where $x_i \in \mathbb{R}^{665}$ for $k=10$ (1s at 10 Hz).
• Stacked GRU Encoder: Two layers of Gated Recurrent Units (GRUs), each with hidden size 256, process the temporal sequence. For each timestep $i$:

$$\begin{aligned} r_i &= \sigma(W^{(r)} x_i + U^{(r)} h_{i-1} + b^{(r)}) \ z_i &= \sigma(W^{(z)} x_i + U^{(z)} h_{i-1} + b^{(z)}) \ \tilde{h}_i &= \tanh(W^{(h)} x_i + U^{(h)} (r_i \circ h_{i-1}) + b^{(h)}) \ h_i &= (1-z_i) \circ h_{i-1} + z_i \circ \tilde{h}_i \end{aligned}$$

• Embedding Projection: The final hidden state $h_t^{(2)}$ from the upper GRU is passed through a fully connected layer with ELU activation, yielding the 64-dimensional Drive2Vec embedding $z_t$:

$$z_t = \operatorname{ELU}(W_z h_t^{(2)} + b_z),\qquad z_t \in \mathbb{R}^{64}$$

• Prediction Head: A separate fully connected layer maps $z_t$ to predictions $y_t \in \mathbb{R}^{2660}$, constituting four 665-size blocks corresponding to distinct temporal or statistical prediction tasks:
  1. Exact Sensor Values, 1s: $x_i \in \mathbb{R}^{665}$0
  2. Short-term Average, 1s: $x_i \in \mathbb{R}^{665}$1
  3. Medium-term Average, 10s: $x_i \in \mathbb{R}^{665}$2
  4. Long-term Average, 100s: $x_i \in \mathbb{R}^{665}$3

This modular structure supports simultaneous optimization for fine and coarse-grained forecasting, context inference, and representation learning (Hallac et al., 2018).

3. Multi-Scale Prediction and Training Objectives

Drive2Vec is trained end-to-end in a multi-task regime, encompassing multiple time horizons and auxiliary tasks:

• Short-term Prediction Loss: For near-future sensor values (up to 1s), sum of mean squared error (MSE) for floating channels and binary cross-entropy (BCE) for booleans:

$x_i \in \mathbb{R}^{665}$4

• Long-term Average Loss:

$x_i \in \mathbb{R}^{665}$5

• Context Inference Loss (optional, for tasks like driver ID classification):

$x_i \in \mathbb{R}^{665}$6

The combined loss is $x_i \in \mathbb{R}^{665}$7; typically, $x_i \in \mathbb{R}^{665}$8, $x_i \in \mathbb{R}^{665}$9 set by auxiliary task inclusion. Training uses the Adam optimizer (learning rate tuned via validation), with no dropout in GRUs; regularization is driven implicitly by the multi-task objective (Hallac et al., 2018).

4. Data, Experimental Protocol, and Quantitative Performance

Dataset

• Fleet Composition: 10 instrumented Audi A3s, 64 unique drivers, 2,098 hours of real-road data, 110,023 km driven.
• Sensor Channels: 665 CAN signals (110 normalized floats, 555 booleans), resampled to 10 Hz, floats zero-mean/unit-variance normalized.
• Splitting: 80% train, 10% validation, 10% test; splits by driving session to prevent leakage.

Results

Method	Short-term MSE	Long-term MSE
Drive2Vec	0.020	0.021
Short-only D2V	0.021	0.027
Long-only D2V	0.052	0.021
PCA (64-dim)	0.174	0.053
Last-timestep	0.204	0.069

Drive2Vec reduces short-term MSE by 90.2% and long-term MSE by 69.6% relative to the last-timestep predictor. Multi-scale training provides robust forecasting out to 3 s, outperforming short-only models at all points.

Embedding size ablation shows that 64 dimensions is a critical point: errors decrease steeply from 8 to 64 but plateau thereafter, indicating a good compactness-accuracy tradeoff. For driver classification, micro-F1 is 0.513 for Drive2Vec, outperforming PCA64 (0.387), short-only D2V (0.490), and long-only D2V (0.506).

5. Applications and Case Studies

• Auto-Labeling of Maneuvers: t-SNE applied to embeddings preceding aggressive gas, brake, or steering events yields separable clusters, enabling rapid event detection via proximity in embedding space.
• Risky-State Prediction: Embeddings before "hard brake" events ($k=10$0 units in 0.4 s) nearly perfectly separate from normal driving (AUROC ≈ 0.99998), supporting anticipatory safety interventions.
• Temporal Signature Trace: Projecting $k=10$1 to RGB via PCA and visualizing along trajectories highlights major regime changes (e.g., highway exits), with abrupt embedding shifts corresponding to distinct driving contexts.
• Driver and Context Inference: The same embedding supports driver identification, style analysis, and contextual labeling without additional model overhead.

These findings highlight the practical utility of multiscale embeddings for both supervised and unsupervised downstream analytics in production automotive environments (Hallac et al., 2018).

6. Relation to Driver2vec and Broader Context

"Driver2vec: Driver Identification from Automotive Data" (Yang et al., 2021) adopts a different embedding approach focused on driver identification, utilizing a temporal convolutional network (TCN), triplet loss, and a LightGBM classifier, achieving pairwise identification accuracy of 83.1% for 51 drivers, compared to prior tree-based and RNN baselines. While both Drive2Vec and Driver2vec target representation learning for vehicular sensor streams, Drive2Vec emphasizes multiscale predictive utility and compressive efficiency, whereas Driver2vec explicitly optimizes for discriminative driver features with metric learning and specialized post-classification (Hallac et al., 2018, Yang et al., 2021).

A plausible implication is that embedding models tailored to the compositional structure of vehicle dynamics and driver context can generalize not only to classical prediction tasks but also to emerging needs such as unsupervised behavioral clustering and risk stratification, provided the multi-task training regime and sequence granularity are appropriately aligned with target application requirements.

7. Limitations and Future Directions

Drive2Vec’s key limitations include reliance on labelled data (driver/context tasks), limited fleet scale (10 vehicles, 64 drivers), and omission of architectural regularizers such as explicit dropout. Potential extensions include contrastive/semi-supervised learning on large unlabelled logs, scaling to greater population diversity, and adaptation to event-based or asynchronous sensor streams. The compact, actionable nature of the Drive2Vec representation positions it as a foundation for integrated in-vehicle analytics, auto-labeling, and real-time predictive interventions in next-generation mobility systems (Hallac et al., 2018).