LETS-C: Leveraging Language Embedding for Time Series Classification
Rachneet Kaur, Zhen Zeng, Tucker Balch, Manuela Veloso
2024-07-10
Summary
This paper talks about LETS-C, a new method that improves how we classify time series data (like stock prices or weather patterns) by using language embedding models instead of traditional large language models (LLMs).
What's the problem?
The main problem is that while LLMs have been successful in various tasks, they are very large and require a lot of computing power and memory. This makes them difficult to use, especially for tasks like time series classification where faster and lighter solutions are needed.
What's the solution?
To solve this problem, the authors propose LETS-C, which uses a language embedding model to convert time series data into a format that can be easily classified. Instead of fine-tuning a large model, LETS-C pairs the embeddings from the language model with a simpler classification system made up of convolutional neural networks (CNNs) and multilayer perceptrons (MLPs). This approach not only reduces the number of parameters needed for training but also improves classification accuracy compared to existing methods.
Why it matters?
This research is important because it offers a more efficient way to analyze time series data, making it accessible for more applications without needing extensive computational resources. By achieving high performance with fewer resources, LETS-C can help in fields like finance, healthcare, and environmental monitoring where timely and accurate data analysis is crucial.
Abstract
Recent advancements in language modeling have shown promising results when applied to time series data. In particular, fine-tuning pre-trained large language models (LLMs) for time series classification tasks has achieved state-of-the-art (SOTA) performance on standard benchmarks. However, these LLM-based models have a significant drawback due to the large model size, with the number of trainable parameters in the millions. In this paper, we propose an alternative approach to leveraging the success of language modeling in the time series domain. Instead of fine-tuning LLMs, we utilize a language embedding model to embed time series and then pair the embeddings with a simple classification head composed of convolutional neural networks (CNN) and multilayer perceptron (MLP). We conducted extensive experiments on well-established time series classification benchmark datasets. We demonstrated LETS-C not only outperforms the current SOTA in classification accuracy but also offers a lightweight solution, using only 14.5% of the trainable parameters on average compared to the SOTA model. Our findings suggest that leveraging language encoders to embed time series data, combined with a simple yet effective classification head, offers a promising direction for achieving high-performance time series classification while maintaining a lightweight model architecture.