TsLLM: Teaching LLMs to Reason Over Time Series

Real-world time series analysis rarely occurs in a vacuum. Clinicians interpret electrocardiograms (ECGs) alongside patient symptoms and laboratory results. Financial analysts forecast market trends by synthesizing historical pricing data with breaking news reports. Power grid operators manage energy supply by combining load frequency traces with weather forecasts and outage calendars. In each of these domains, understanding numerical data requires deep contextual reasoning.

Large Language Models (LLMs) possess the contextual reasoning, domain knowledge integration, and interactive capabilities needed for these complex tasks. However, they exhibit a fundamental architectural weakness: they struggle to perceive and process continuous numerical time series data accurately. This structural blindness forces practitioners to rely on specialized models that cannot natively ingest unstructured text or generate natural language explanations.

This article explores how the TsLLM architecture bridges this critical modality gap. By augmenting a pretrained LLM with a specialized, patch-based time series encoder-decoder, TsLLM enables unified reasoning over interleaved text and temporal signals. We will examine the technical mechanisms behind this alignment, the training protocols that make it possible, and why this approach redefines the scope of temporal data analysis in production environments.

The Architectural Bottleneck of Text Tokenization

To understand why LLMs fail at time series analysis, we must first look at how they ingest data. The core issue lies in text tokenization. Tokenizers are optimized for sub-word units in natural language, not for continuous, high-variance numerical values.

When a standard text tokenizer processes a numerical sequence, it often splits individual data points into multiple, arbitrary tokens. This awkward fragmentation destroys the inherent numerical relationships and severely inflates the sequence length. For instance, analyzing a standard 10-second electrocardiogram (with 12 leads recorded at 100 Hz) requires approximately 84,000 tokens when using a conventional LLM tokenizer. This massive token footprint immediately saturates the maximum context window of many models, leaving virtually no room for the unstructured contextual information—such as clinical notes or few-shot examples—that makes LLMs useful in the first place.

Beyond the mechanical inefficiency of tokenization, LLMs suffer from a severe exposure deficit. A thorough analysis of standard LLM pretraining corpora reveals a drastic lack of high-quality, dense numerical time series data. Because these models are not exposed to the underlying dynamics of temporal signals during their foundational training, they fail to develop the inductive biases required to recognize high-frequency dynamics, causal relationships, or precise numerical patterns.

Bridging Modalities with Patch-Based Encoding

To circumvent the tokenization bottleneck, TsLLM replaces text-based numerical representation with a dedicated temporal perception module. Instead of processing individual data points, the architecture segments the continuous time series into fixed-length patches.

Each patch (typically comprising 32 consecutive time points) is transformed into a single, dense vector representation through a learned Multilayer Perceptron (MLP). This patch-based encoding preserves fine-grained local temporal patterns while achieving dramatic sequence compression. The LLM processes these highly compressed, information-dense representations rather than tens of thousands of fragmented text tokens. In multivariate time series, each component is encoded independently, allowing the LLM to dynamically learn the cross-channel relationships through its native self-attention mechanism.

Decoupling Scale and Shape for Numerical Stability

Real-world time series span vastly different scales. A neurological recording might measure fluctuations in microvolts, while a financial dataset tracks billions of dollars. Traditional time series models handle this variance by normalizing inputs globally and rescaling outputs statically. However, an LLM needs to understand the absolute scale to reason semantically—knowing that a heart rate of 200 bpm indicates severe tachycardia is critical clinical context that is lost if the data is purely normalized.

TsLLM solves this by decoupling the time series into two distinct components: shape and scale. The input sequence is first normalized using sequence-level z-score normalization to extract the pure temporal pattern (the shape). Simultaneously, the normalization parameters (the mean and standard deviation) are subjected to a log transformation to compress extreme ranges, and then encoded through dedicated MLPs.

During the model's forward pass, these scale embeddings are added directly to the shape representations. This scale-aware encoding ensures that the model can reason about absolute magnitudes while maintaining the mathematical stability required for neural network training.

The Regularized Latent Space and Training Objective

To guarantee that these dense patch representations are both highly compressed and semantically meaningful, the time series encoder and decoder are pretrained using a (beta)-Variational Autoencoder (VAE) objective.

The goal of this pretraining is to balance reconstruction fidelity (ensuring no critical signal is lost) with representation quality (ensuring the LLM can easily interpret the latent space).

The training objective can be written as:

\[ \mathcal{L}_{VAE} = \mathbb{E}_{q_\phi(z|x)}[\log p_\theta(x|z)] - \beta \cdot KL(q_\phi(z|x)||p(z)) \]

Here, \(x\) represents the input time series patch, \(z\) the latent representation, \(q_\phi(z|x)\) the encoder network, \(p_\theta(x|z)\) the decoder network, and \(\beta\) the weight of the Kullback-Leibler (KL) divergence term.

The reconstruction term (Mean Squared Error) ensures that the decoded signal matches the original time series accurately. The KL divergence term regularizes the latent space, forcing it to be smooth and well-structured. The researchers employed a sigmoid-based schedule for the (beta) parameter, starting at 0.5 to prioritize reconstruction early in the process, and gradually ramping up to 1.25 to encourage disentangled, robust representations. This scheduling effectively prevents the posterior collapse commonly seen in VAE training.

Modality Fusion via Cross-Attention Adapters

With the time series compressed into a stable latent space, the final architectural challenge is bridging the domain gap between the time series encoder and the pretrained LLM backbone.

TsLLM utilizes a lightweight cross-attention adapter layer. This module maps the learned time series representations directly into the LLM’s token embedding space. To facilitate seamless integration, the LLM’s vocabulary is extended with a special <|ts|> token, which acts as a placeholder for time series patches. During processing, the embeddings of the <|ts|> tokens are dynamically replaced with their corresponding, adapted time series representations.

Because this process works in reverse through a decoder adapter, the model treats time series analysis as an autoregressive generation problem over interleaved sequences. It can read a text prompt, analyze a block of time series data, output natural language reasoning, and generate a new time series forecast—all within a single, unified generation loop.

Experimental Protocol: Multi-Stage Alignment and Training

Training a model to handle both nuanced text and raw numerical data without catastrophic forgetting requires a carefully staged curriculum. The TsLLM architecture is trained on a massive multimodal corpus comprising over 27 billion tokens.

1. Alignment (1 Billion Tokens): The time series encoder, decoder, and the LLM backbone are completely frozen. Only the adapter layers and scale encodings are trained. This stage uses raw sequences and synthetic tasks to establish a strict bridge between the modalities without disrupting the LLM’s pretrained weights.
2. Continued Pretraining (9 Billion Tokens): All parameters are unfrozen. The model is exposed to vast amounts of unstructured, interleaved text and time series data to build comprehensive multimodal understanding.
3. Supervised Fine-Tuning (20 Billion Tokens): The model shifts to an instruction-following format. It trains on prompt-response pairs, learning to execute specific analytical tasks specified via natural language instructions, enabling strong zero-shot and few-shot capabilities.

Evidence and Results: Rethinking Contextual Forecasting

The empirical results demonstrate the unique advantages of this interleaved architecture, particularly in tasks where pure numerical prediction is insufficient.

Time Series Question-Answering

To evaluate complex reasoning, the researchers tested TsLLM on the ECG-QA benchmark, a dataset requiring models to verify conditions, choose between diagnoses, and answer open-ended queries about electrocardiograms. TsLLM significantly outperformed zero-shot frontier LLMs. More notably, it matched or exceeded the exact-match accuracy of specialized, domain-specific systems trained exclusively on ECG data, achieving an average accuracy of 65.0%.

This suggests that a general-purpose LLM, once granted temporal perception, can leverage its broad reasoning capabilities to perform expert-level medical analysis. Furthermore, on the synthetic TimeSeriesExam benchmark, TsLLM achieved an 86.4% accuracy rate, drastically outperforming massive frontier models like Gemini 2.5 Pro and GPT-4o-mini, which hovered between 30% and 50%.

Contextual Forecasting

Traditional machine learning methods fail when forecasting requires unstructured textual context. On the Context Is Key (CiK) benchmark—a dataset containing realistic forecasting tasks explicitly conditioned on text, such as predicting stock prices based on news reports—TsLLM demonstrated clear superiority. It achieved a Symmetric Mean Absolute Percentage Error (SMAPE) of 64.5%, significantly beating traditional models like XGBoost (76.8%) and ARIMA (90.7%), as well as text-only LLMs that lacked numerical perception.

Crucially, rigorous ablations and evaluations on standard text benchmarks (including MMLU-Pro, GPQA, and GSM8K) confirmed that integrating time series perception caused virtually zero degradation to the base LLM’s foundational language and math capabilities.

Limitations and Open Questions

Despite its breakthroughs in contextual reasoning, the TsLLM architecture comes with notable tradeoffs.

The primary limitation is computational overhead. Both training the multimodal adapters and running inference with a full LLM backbone require substantially more GPU resources than deploying traditional, lightweight time series models (such as ARIMA or dedicated forecasting transformers).

Additionally, the results show that while TsLLM is highly competitive, it does not consistently achieve state-of-the-art performance on pure, numerical-only tasks like anomaly detection. On the MSL anomaly dataset, it scored an F1 of 73.3, trailing behind strictly numerical models like UniTS-FT (81.2). This performance gap indicates that for rigidly defined numerical predictions lacking contextual variables, specialized models remain the most efficient choice.

An open question remains regarding evaluation standards. The field currently lacks robust, expert-validated benchmarks specifically designed to assess multimodal time-series-and-language tasks, making it difficult to fully quantify the generalization limits of models like TsLLM across diverse industrial applications.

Conclusion: Expanding the Scope of Temporal Analysis

The challenge of scale and context in time series analysis can no longer be ignored by the AI community. The TsLLM architecture proves that the tokenization bottleneck is solvable. By implementing a high-fidelity, patch-based encoder-decoder and decoupling shape from scale, practitioners can map continuous numerical signals directly into the cognitive space of Large Language Models.

The true value of this breakthrough is not in replacing specialized algorithms for pure numerical forecasting. Instead, it lies in opening up an entirely new paradigm of interactive temporal analysis. By giving LLMs the ability to natively perceive time series data, we move closer to enterprise AI systems that can seamlessly read a financial chart, correlate it with global news events, generate a reasoned forecast, and explain its logic—all within a single, unified workflow.

Source: Parker, F., Chan, N., Zhang, C., & Ghobadi, K. (2026). TsLLM: Augmenting LLMs for General Time Series Understanding and Prediction. Proceedings of the 43rd International Conference on Machine Learning (ICML 2026).