EV-STLLM: Spatio-Temporal LLMs for EV Data

• EV-STLLM is a family of frameworks that fuse multimodal spatio-temporal data from electric vehicles and event sensors to generate actionable insights.
• These systems employ modular architectures like encoder–connector–LLM pipelines, integrating vision, telemetry, and geospatial prompts for real-time forecasting and safety alerts.
• They leverage efficient token merging, graph-aware attention, and multi-frequency fusion to enhance prediction accuracy and operational robustness across diverse EV applications.

Searching arXiv for papers on EV-STLLM and related spatio-temporal LLMs. An EV Spatio-Temporal LLM (EV-STLLM) denotes a LLM framework that ingests multimodal, time-indexed data associated with electric vehicles or event-based sensing and performs reasoning over spatial structure and temporal evolution. Across the literature, the term is not a single canonical architecture but a family of designs that combine a LLM with spatio-temporal encoders, structured prompts, graph or map priors, and, in some cases, event-stream or video representations. In one line of work, an EV-STLLM is described as “a LLM that ingests multimodal, time-indexed EV data—visual perception, telemetry, geolocation, and grid signals—encoded as structured textual and visual tokens” and produces “safety-critical driving alerts,” “energy-aware driving and charging recommendations,” and “fleet and grid-level coordination messages” (Carvalho et al., 2 Oct 2025). In another line, the label refers to electric-vehicle charging forecasting with “multi-frequency and multi-scale information fusion” over a charging-station network (Fan et al., 13 Jul 2025). A broader technical reading also encompasses event-based generative MLLMs for event streams (Li et al., 23 Jan 2025), compact spatio-temporal visual prompting pipelines over frozen LLMs (Jian et al., 2023), and graph- or attention-steered LLMs for spatio-temporal forecasting (Fan et al., 4 Jun 2025, Zhang et al., 12 May 2026, Jeon et al., 17 Sep 2025).

1. Definition and conceptual scope

An EV-STLLM is best understood as a specialization of the spatio-temporal LLM paradigm to EV-related systems, event streams, or both. The literature supports at least three closely related meanings.

First, in EV safety and smart-grid settings, the framework processes camera, GPS + reverse geocoding, CAN bus telemetry, object detection (YOLOv8), and semantic segmentation (Cityscapes-trained model), then converts these signals into structured prompts for a text-only or multimodal LLM that generates context-aware alerts (Carvalho et al., 2 Oct 2025). The operative abstraction is multimodal-to-text fusion for spatio-temporal reasoning over driving scenes, vehicle state, and geospatial context. The same work explicitly positions this as a prototype of an EV-STLLM, with components including “CAN bus telemetry (speed, brake, steering),” “GPS + address,” “symbolic fusion of vision, telemetry, and location,” and “grid integration hooks” such as “fleet coordination, load forecasting, and traffic-aware energy planning” (Carvalho et al., 2 Oct 2025).

Second, in EV charging prediction, EV-STLLM is a task-specific forecasting architecture for charging volume and station occupancy rate over an EV charging network $G=(V,E,\mathbf{A})$, with historical data $\mathbf{X}\in\mathbb{R}^{T\times N\times C}$ and future targets $\mathbf{Y}_S\in\mathbb{R}^{S\times N\times C}$ (Fan et al., 13 Jul 2025). Here the emphasis is not language output but the use of a large pre-trained transformer, partially frozen and adapted with graph-aware attention, to predict future spatio-temporal trajectories.

Third, in event-based vision and spatio-temporal multimodal modeling, EV-STLLM can denote an event-driven or event-camera-based LLM. EventVL is explicitly described as “the first generative event-based MLLM” and as a concrete instantiation of an EV Spatio-Temporal LLM because it consumes event streams $e_i=(x_i,y_i,t_i,p_i)$, constructs an Event Spatiotemporal Representation, aligns event and image semantics, and conditions a frozen LLM for text generation (Li et al., 23 Jan 2025). A plausible implication is that the abbreviation “EV” can denote either electric vehicle or event-based vision depending on context; the surrounding architecture usually disambiguates the intended meaning.

These strands converge on a shared design principle: a frozen or lightly adapted LLM is supplied with compressed representations of spatio-temporal state, rather than raw high-dimensional streams, and is tasked with reasoning, forecasting, or generation conditioned on that state (Jian et al., 2023, Li et al., 23 Jan 2025, Fan et al., 13 Jul 2025).

2. Architectural patterns

The architectural core of EV-STLLM systems differs by application, but recurring motifs are explicit throughout the cited work.

A common formulation is encoder–connector–LLM. In “Expedited Training of Visual Conditioned Language Generation via Redundancy Reduction,” the system uses a frozen ViT as visual encoder $E_{\text{vision}}$, a frozen LLM as decoder $D_{\text{LLM}}$, and a trainable connector $T_{\text{v}\rightarrow\text{l}}$ called TomeFormer, initialized as BERT-base and augmented with Token Merging (ToMe) (Jian et al., 2023). Visual tokens $[v_1,\dots,v_L]$ are projected, merged layer by layer into a shorter sequence $[v'_1,\dots,v'_{L'}]$, aligned to the LLM embedding space by $f_{\text{proj}_2}$, and prepended as soft prompts to the textual input, with a single autoregressive loss

$\mathbf{X}\in\mathbb{R}^{T\times N\times C}$0

The paper’s details explicitly reconstruct this recipe as a basis for a generalized “EV Spatio-Temporal LLM (EV-STLLM)” over “video, event streams, etc.” (Jian et al., 2023).

In EV charging forecasting, EV-STLLM combines heavy preprocessing with a partially adapted large model. The model comprises a data preprocessing module and a forecasting module (Fan et al., 13 Jul 2025). The forecasting module uses Spatio-Temporal-Frequency embedding, Fusion convolution, and a Partially Frozen Graph Attention (PFGA) module built on a GPT-2–like backbone. The first $\mathbf{X}\in\mathbb{R}^{T\times N\times C}$1 layers remain frozen and the last $\mathbf{X}\in\mathbb{R}^{T\times N\times C}$2 layers are unfrozen with graph-aware attention incorporating adjacency matrix $\mathbf{X}\in\mathbb{R}^{T\times N\times C}$3 (Fan et al., 13 Jul 2025). The update in trainable graph-aware layers is written as

$\mathbf{X}\in\mathbb{R}^{T\times N\times C}$4

$\mathbf{X}\in\mathbb{R}^{T\times N\times C}$5

This architecture retains the “sequential feature modeling capabilities of the pre-trained large model while incorporating EV domain knowledge” (Fan et al., 13 Jul 2025).

A third pattern is symbolic fusion + LLM reasoning. In the grid-integrated EV framework, the pipeline is modular: a Perception Layer with YOLOv8 and semantic segmentation, a Telemetry + Geospatial Layer with CAN bus and reverse geocoding, and an LLM Reasoning Layer that receives structured textual prompts or raw image plus prompt (Carvalho et al., 2 Oct 2025). The paper formalizes prompt construction as

$\mathbf{X}\in\mathbb{R}^{T\times N\times C}$6

or, for GPT Vision,

$\mathbf{X}\in\mathbb{R}^{T\times N\times C}$7

This design is not end-to-end multimodal training; rather, neural perception models produce symbolic and numeric descriptors that the LLM interprets (Carvalho et al., 2 Oct 2025).

EventVL exemplifies a fourth pattern: event/image encoder + frozen LLM. It uses an InternViT-300M-448px image encoder, a trainable event encoder initialized from the same architecture, and a frozen InternLM2-1.8B decoder-only LLM (Li et al., 23 Jan 2025). Event and image embeddings are injected as “visual tokens” or prefix embeddings to the LLM. The model is trained with event-only and event+image language modeling losses plus an event–image cosine alignment loss (Li et al., 23 Jan 2025).

Across these variants, a stable encyclopedic characterization is that EV-STLLM systems typically factor into: a spatio-temporal encoder, a compact alignment or connector module, a frozen or partially frozen LLM backbone, and a task head or generative interface (Jian et al., 2023, Li et al., 23 Jan 2025, Fan et al., 13 Jul 2025, Carvalho et al., 2 Oct 2025).

3. Spatial, temporal, and multimodal representations

The defining technical issue in EV-STLLM research is how to encode space and time so that an LLM can process them without prohibitive context cost or semantic mismatch.

In visual spatio-temporal models, redundancy reduction in space and lightweight temporal modeling are central. EVL_Gen reduces 256 ViT tokens to about 28 tokens after 12 layers by merging $\mathbf{X}\in\mathbb{R}^{T\times N\times C}$8 tokens per layer (Jian et al., 2023). For video, it adds a Temporal Attentive Soft Token Contextualizing module inside the ViT. Given video features $\mathbf{X}\in\mathbb{R}^{T\times N\times C}$9 of shape $\mathbf{Y}_S\in\mathbb{R}^{S\times N\times C}$0, the method first applies spatial self-attention per frame, then reshapes to $\mathbf{Y}_S\in\mathbb{R}^{S\times N\times C}$1 and computes temporal attention

$\mathbf{Y}_S\in\mathbb{R}^{S\times N\times C}$2

where $\mathbf{Y}_S\in\mathbb{R}^{S\times N\times C}$3 and $\mathbf{Y}_S\in\mathbb{R}^{S\times N\times C}$4 are learnable linear layers (Jian et al., 2023). This preserves token count temporally while making later spatial merging more effective.

EventVL addresses a different representation problem: event cameras produce asynchronous sparse point streams $\mathbf{Y}_S\in\mathbb{R}^{S\times N\times C}$5 rather than images. Its solution is the Event Spatiotemporal Representation (ESR), consisting of Hierarchical Temporal Split (HTS) and Adaptive Spatial Split (ASS) (Li et al., 23 Jan 2025). HTS builds three temporal levels: $\mathbf{Y}_S\in\mathbb{R}^{S\times N\times C}$6 plus a global aggregation frame $\mathbf{Y}_S\in\mathbb{R}^{S\times N\times C}$7, while ASS selects a spatial tiling from candidate grids and splits the global temporal aggregation into $\mathbf{Y}_S\in\mathbb{R}^{S\times N\times C}$8 high-resolution patches (Li et al., 23 Jan 2025). The final representation is

$\mathbf{Y}_S\in\mathbb{R}^{S\times N\times C}$9

This suggests a general EV-STLLM principle: convert difficult spatio-temporal modalities into structured token sequences that remain compatible with pretrained image or language backbones.

In charging forecasting, the representation problem is handled through multi-frequency and multi-scale information fusion. EV-STLLM first applies Variational Mode Decomposition (VMD) for denoising and ICEEMDAN for refined multi-frequency extraction, then Fuzzy Information Granulation (FIG) for multi-scale abstraction and ReliefF for feature selection (Fan et al., 13 Jul 2025). It then builds three embedding types: token/value embedding $e_i=(x_i,y_i,t_i,p_i)$0, temporal embedding $e_i=(x_i,y_i,t_i,p_i)$1, and spatial embedding

$e_i=(x_i,y_i,t_i,p_i)$2

which are fused by

$e_i=(x_i,y_i,t_i,p_i)$3

The result is a spatio-temporal-frequency representation tailored to EV charging behavior (Fan et al., 13 Jul 2025).

In prompt-centric EV reasoning systems, the representation is textual but still explicitly spatial. The grid-integrated EV framework encodes: left/right region, object class, confidence, distance, semantic segmentation percentages, speed, steering angle, brake state, and reverse-geocoded address (Carvalho et al., 2 Oct 2025). Distances are estimated by a pinhole model,

$e_i=(x_i,y_i,t_i,p_i)$4

and inserted into prompts as object-specific metric signals (Carvalho et al., 2 Oct 2025). Temporal context is mostly implicit in instantaneous speed, braking state, and the fact that each frame is time-aligned with telemetry.

A plausible synthesis is that EV-STLLM research is divided between latent tokenization approaches, which compress spatio-temporal structure into learned embeddings (Jian et al., 2023, Li et al., 23 Jan 2025, Fan et al., 13 Jul 2025), and symbolic prompt approaches, which serialize structured scene state into language-like descriptors (Carvalho et al., 2 Oct 2025). Hybrid systems are also plausible, and some papers explicitly point to them as future directions (Carvalho et al., 2 Oct 2025, Fan et al., 13 Jul 2025).

4. Training regimes and objectives

Training objectives in EV-STLLM systems vary sharply with task, but a recurring theme is to avoid full end-to-end tuning of all parameters.

EVL_Gen is explicitly a one-stage, single-loss framework. It bypasses BLIP-2’s two-stage training and uses only autoregressive caption cross-entropy (Jian et al., 2023). The paper states that this yields practical speedups because there is “one forward pass per step,” no contrastive or image-text matching losses, and a shorter visual prompt after token merging (Jian et al., 2023). The wall-clock comparison on 8×A100-80G with OPT 2.7B reports BLIP-2 at approximately 234 hours, EVL_Gen at approximately 133 hours for 250k steps, and a smaller EVL_Gen variant at approximately 47 hours, alongside the claim of “~5× faster” pre-training for a usable model and competitive performance using only 11M image-text pairs versus 104M (Jian et al., 2023).

EventVL employs a composite objective. The event-only language modeling loss is

$e_i=(x_i,y_i,t_i,p_i)$5

the event+image prior language loss is

$e_i=(x_i,y_i,t_i,p_i)$6

and the alignment loss is

$e_i=(x_i,y_i,t_i,p_i)$7

The total training loss is

$e_i=(x_i,y_i,t_i,p_i)$8

with $e_i=(x_i,y_i,t_i,p_i)$9 by default (Li et al., 23 Jan 2025). The use of the image prior reflects the fact that event streams lack texture and color semantics, and ablations show that combining both image prior and $E_{\text{vision}}$0 improves CIDEr on N-ImageNet validation from 0.466 or 0.524 to 0.573 (Li et al., 23 Jan 2025).

EV-STLLM for charging forecasting uses a customized time–frequency fusion loss. In time domain,

$E_{\text{vision}}$1

while in frequency domain the method computes FFT and defines

$E_{\text{vision}}$2

The total objective is

$E_{\text{vision}}$3

The paper argues that this combats label autocorrelation issues in direct multi-step forecasting and preserves spectral structure (Fan et al., 13 Jul 2025).

The EV safety framework is different: it is evaluated through structured prompt generation and expert comparison rather than end-to-end differentiable training of a new LLM (Carvalho et al., 2 Oct 2025). The model families considered include GPT-5, Gemini, DeepSeek, and GPT Vision, and performance is assessed by correctness vs human expert summaries and latency (Carvalho et al., 2 Oct 2025).

Other spatio-temporal LLM papers suggest additional training paradigms relevant to EV-STLLM. U-STS-LLM uses a unified multi-task objective for forecasting and imputation: $E_{\text{vision}}$4 where $E_{\text{vision}}$5 is MSE over forecast horizons and $E_{\text{vision}}$6 is MSE on masked entries (Zhang et al., 12 May 2026). This suggests a plausible EV-STLLM extension for jointly forecasting demand and reconstructing missing telemetry.

5. Applications and empirical performance

The most concrete EV-STLLM application in the corpus is EV charging forecasting. The EV-STLLM paper uses real-world data from Shenzhen, China, to forecast charging volume (kWh) and occupancy rate (%) in public charging networks (Fan et al., 13 Jul 2025). The dataset includes 1,362 stations and 17,532 piles after processing, with hourly resolution and dynamic features such as charging volume, occupancy, duration, electricity price, service price, and weather (Fan et al., 13 Jul 2025). On Data 1, for 3-step EV volume forecasting, EV-STLLM reports RMSE 26.42, MAE 19.49, and MAPE 0.14, compared with GCN at RMSE 47.40, MAE 37.96, MAPE 0.20; ChatEV at RMSE 44.95, MAE 32.93, MAPE 0.25; and GPT4TS at RMSE 29.27, MAE 21.85, MAPE 0.16 (Fan et al., 13 Jul 2025). On Data 2, 3-step volume forecasting yields RMSE 58.89 versus 120.95 for GCN and 65.25 for GPT4TS (Fan et al., 13 Jul 2025). The paper further reports strong few-shot performance and critical-day robustness during the Chinese Spring Festival holiday (Fan et al., 13 Jul 2025).

A related EV charging forecasting system, EV-LLM, couples GCN-based spatial modeling with LLAMA2-7B and prompt-guided multimodal feature fusion over historical charging data, weather information, and textual descriptions (Fan et al., 4 Jun 2025). On data from 10 charging stations in California, EV-LLM reports MAE 1.7288 and RMSE 2.8960, outperforming ISSA-CNN-GRU, AST-GCN, mRGC-CBi-LSTM, Autoformer, Informer, Transformer, GCN, and LSTM (Fan et al., 4 Jun 2025). An ablation shows degradation without prompt or without GCN, and robustness under 10–20% missing input data (Fan et al., 4 Jun 2025). While this model is not labeled EV-STLLM, it belongs to the same research lineage.

In road safety and driver assistance, the grid-integrated EV framework evaluates three scenarios using real urban driving data from an instrumented Renault Captur on Brazilian urban roads (Carvalho et al., 2 Oct 2025). The system’s text-only GPT-5 latency is approximately 0.9 s, 1.0 s, and 1.3 s across the three scenarios, versus approximately 1.9 s, 2.2 s, and 2.8 s for multimodal GPT Vision (Carvalho et al., 2 Oct 2025). The outputs align with expert judgment in all three cases, including pedestrian proximity, lateral vehicle crowding, and mixed traffic with bicycle and traffic lights (Carvalho et al., 2 Oct 2025). The paper explicitly argues that both latencies are “compatible with real-time driver assistance” (Carvalho et al., 2 Oct 2025).

Event-based EV-STLLM capability is represented by EventVL. On zero-shot event captioning, EventVL-Base (2.3B total parameters) achieves CIDEr 0.823 on N-Caltech101, 2.603 on HARDVS, and 0.573 on N-ImageNet, substantially above image- or video-based MLLM baselines that treat red–blue event representations as RGB inputs (Li et al., 23 Jan 2025). The model is also evaluated on driving scenes in DSEC and produces multi-sentence descriptions of roads, cars, pedestrians, and traffic lights (Li et al., 23 Jan 2025). This suggests that event-camera EV-STLLM variants are already viable for generative scene understanding.

For general spatio-temporal forecasting, the literature gives additional context. U-STS-LLM reports approximately 24% MAE & RMSE improvement over the best baseline in forecasting, 11–13% improvement in high-missing-rate imputation, and 35–40% improvement in zero-shot transfer from Milan to Trento on cellular traffic (Zhang et al., 12 May 2026). ST-LINK reports improvements over OFA and ST-LLM on traffic datasets such as METR-LA and PEMS-BAY, especially at longer horizons and abrupt-change scenarios (Jeon et al., 17 Sep 2025). These are not EV papers, but they supply reusable mechanisms for EV-STLLM design, especially where node graphs, temporal memory, or missing-data robustness are central.

6. Limitations, ambiguities, and open directions

The literature is explicit that EV-STLLM remains a design space rather than a settled architecture. Several limitations recur.

In EV charging forecasting, EV-STLLM depends on a complex preprocessing pipeline involving VMD, ICEEMDAN, FIG, MSSE, and ReliefF (Fan et al., 13 Jul 2025). The paper’s own structure suggests that performance is partly attributable to this carefully engineered front end, not solely to the LLM backbone. It also notes possible dependence on graph quality because PFGA relies on an adjacency matrix encoding spatial proximities among zones (Fan et al., 13 Jul 2025). A plausible implication is that transfer to new cities may require substantial reconstruction of graph and preprocessing pipelines even if the LLM core remains reusable.

The EV safety and grid paper identifies current operation as primarily per-frame reasoning over time-aligned telemetry and location, not full sequence modeling (Carvalho et al., 2 Oct 2025). It explicitly lists as future work: scaling to many scenarios, direct processing of raw video by multimodal LLMs, coordinated fleets, and smart-grid applications such as demand response and grid optimization (Carvalho et al., 2 Oct 2025). Thus, its current EV-STLLM interpretation is more a prototype for structured scene-to-language reasoning than a complete temporal world model.

EventVL identifies limitations in data coverage, model capacity, event representation, failure under noisy or low-semantic events, and real-time deployment on embedded hardware (Li et al., 23 Jan 2025). The use of red–blue pseudo-RGB event frames is acknowledged as convenient rather than necessarily optimal for preserving precise temporal ordering (Li et al., 23 Jan 2025). This matters because event-camera EV-STLLMs intended for robotics or ADAS may require representations closer to continuous space-time.

EVL_Gen, while offering a strong recipe for efficient spatio-temporal visual prompting, also identifies limitations relevant to EV-STLLM design: fixed merging rate $E_{\text{vision}}$7, no text-conditioned visual selection, simple temporal modeling, limited scalability to longer videos, and domain gaps in training data (Jian et al., 2023). These constraints suggest that any EV-STLLM built on aggressive token merging must balance efficiency against task-adaptive detail retention.

Across the broader spatio-temporal LLM literature, two unresolved issues stand out. One is spatial inductive bias: some approaches encode structure via graph-aware attention (Fan et al., 13 Jul 2025, Zhang et al., 12 May 2026), others via rotary spatial embeddings (Jeon et al., 17 Sep 2025), and others by symbolic prompts (Carvalho et al., 2 Oct 2025). Another is the role of language itself: in some systems the LLM is a genuine reasoning layer over multimodal context (Carvalho et al., 2 Oct 2025, Li et al., 23 Jan 2025), whereas in others it functions mainly as a pre-trained sequence processor for numeric tokens (Fan et al., 4 Jun 2025, Li et al., 2024). This suggests that “EV-STLLM” names a spectrum, from language-grounded spatio-temporal reasoning systems to large-transformer forecasting architectures augmented with spatial priors.

A careful synthesis of the field therefore yields a restrained definition. EV-STLLM is not yet a single standard model but a research program centered on coupling LLMs with spatio-temporal encoders for EV systems, event streams, or both. The most stable elements are: compact spatio-temporal tokenization or prompt construction, partial freezing of strong pretrained backbones, explicit handling of spatial structure, and task-specific heads for generation, forecasting, or alerting (Jian et al., 2023, Li et al., 23 Jan 2025, Fan et al., 13 Jul 2025, Carvalho et al., 2 Oct 2025).