Univariate to Multivariate: LLMs as Zero-Shot Predictors for Time-Series Forecasting (2506.02389v1)

Abstract: Time-series prediction or forecasting is critical across many real-world dynamic systems, and recent studies have proposed using LLMs for this task due to their strong generalization capabilities and ability to perform well without extensive pre-training. However, their effectiveness in handling complex, noisy, and multivariate time-series data remains underexplored. To address this, we propose LLMPred which enhances LLM-based time-series prediction by converting time-series sequences into text and feeding them to LLMs for zero shot prediction along with two main data pre-processing techniques. First, we apply time-series sequence decomposition to facilitate accurate prediction on complex and noisy univariate sequences. Second, we extend this univariate prediction capability to multivariate data using a lightweight prompt-processing strategy. Extensive experiments with smaller LLMs such as Llama 2 7B, Llama 3.2 3B, GPT-4o-mini, and DeepSeek 7B demonstrate that LLMPred achieves competitive or superior performance compared to state-of-the-art baselines. Additionally, a thorough ablation study highlights the importance of the key components proposed in LLMPred.

• The paper introduces LLMPred, which transforms time-series data into text and applies LLMs for zero-shot predictions on both univariate and multivariate systems.
• It employs decomposition and tailored prompt engineering to effectively manage noise and capture cross-variable dependencies, significantly reducing MSE.
• The results demonstrate competitive performance on diverse datasets using models like Llama-2 and GPT-4o-mini, highlighting its scalability and practical value.

Univariate to Multivariate: LLMs as Zero-Shot Predictors for Time-Series Forecasting

Introduction to LLMPred

The paper discusses LLMPred, a methodology that leverages LLMs for zero-shot prediction of time-series data, addressing the challenge of modeling complex, noisy, and multivariate systems. LLMPred employs a novel approach where time-series data is transformed into text sequences, allowing LLMs to apply their pre-trained capabilities for prediction tasks. The methodology is distinguished by two key pre-processing techniques: decomposing univariate time-series sequences to enhance prediction accuracy and extending this approach to multivariate data through efficient prompt processing.

Methodology

Time-Series Decomposition

Time-series data is prone to noise and abrupt changes that can obscure trends and patterns. LLMPred mitigates these challenges by decomposing sequences into low-frequency trends and high-frequency fluctuations. This decomposition simplifies the task for LLMs by segregating complex temporal dynamics into manageable components.

The decomposition process utilizes a frequency-based approach, where a range of cut-off frequencies is evaluated to separate low and high-frequency components. The best frequency range is selected based on the similarity metric comprising mean squared error (MSE) and cosine similarity, optimizing the decomposition to preserve the most critical aspects of the data.

Conversion to Text and Tokenization

LLMPred transforms numerical sequences into text, facilitating processing by LLMs. Max normalization ensures uniform value ranges, while specific tokenization techniques adapt to the tokenizers used by different LLM architectures like Llama and GPT models. This approach capitalizes on LLMs' textual predictive capabilities while maintaining numerical integrity.

Multivariate Time-Series Prediction

Extending LLMPred for multivariate time-series requires designing prompts that capture cross-variable dependencies. By mapping features into distinct ranges, the prompt design includes feature distributions in structured textual formats. This technique exploits LLMs' ability to recognize structural patterns and perform efficient multivariate forecasting without extensive fine-tuning.

Post-Processing

Two post-processing strategies further refine predictions: a simple MLP enhances low-frequency component predictions, while a Gaussian transformation aligns high-frequency predictions with historical distributions, ensuring statistical coherence and reducing misalignment artifacts.

Evaluation and Results

The paper provides extensive empirical validations on several datasets using small LLMs such as Llama-2 7B and GPT-4o-mini. Results indicate that LLMPred achieves competitive performance compared to state-of-the-art benchmarks, reducing MSE significantly in univariate generation and maintaining robustness across datasets.

Critically, multivariate predictions benefit from prompt engineering, demonstrating superior adaptability to capture inter-variable dependencies. Ablation studies confirm the importance of decomposition and post-processing components, highlighting critical methodological contributions.

Limitations and Future Directions

While effective, LLMPred faces challenges such as precision loss from decimal place restrictions and potential limitations in frequency decomposition adaptability. Future work aims to improve tokenization to retain higher precision, dynamically adjust frequency parameters, and integrate robust distribution-aware modeling.

Conclusion

LLMPred advances the use of LLMs in time-series forecasting by introducing efficient pre-processing and prompt-processing techniques, thereby extending zero-shot capabilities to complex, multivariate domains. Its results underline the potential for integrating LLMs into predictive tasks across various sectors, offering scalability and adaptability in data-scarce environments. Future enhancements could further refine LLMPred by addressing existing limitations and expanding its applicability.