Chimera: Effectively Modeling Multivariate Time Series with 2-Dimensional State Space Models
Ali Behrouz, Michele Santacatterina, Ramin Zabih
2024-06-13
Summary
This paper introduces a new model called Chimera, which is designed to effectively analyze and predict multivariate time series data. This model improves upon traditional methods by using a two-dimensional approach to better capture complex patterns over time.
What's the problem?
Modeling multivariate time series data—like tracking multiple stock prices or health metrics over time—can be challenging. Traditional State Space Models (SSMs) are good for simple, single-variable data but struggle with more complex relationships and patterns. They often cannot handle non-linear dependencies, which are common in real-world data, and can be slow and inefficient when processing large datasets.
What's the solution?
Chimera addresses these issues by using two input-dependent two-dimensional SSM heads that allow it to learn from both long-term trends and seasonal patterns in the data. This means it can better understand how different variables interact over time. The model also includes a new training method that speeds up the process, making it more efficient. Through experiments, Chimera has shown superior performance in various tasks, such as classifying ECG signals, forecasting future values, and detecting anomalies in time series data.
Why it matters?
This research is significant because it provides a more powerful tool for analyzing complex time series data across different fields like healthcare and finance. By improving how we model these kinds of data, Chimera can lead to better predictions and insights, helping professionals make informed decisions based on accurate analyses.
Abstract
Modeling multivariate time series is a well-established problem with a wide range of applications from healthcare to financial markets. Traditional State Space Models (SSMs) are classical approaches for univariate time series modeling due to their simplicity and expressive power to represent linear dependencies. They, however, have fundamentally limited expressive power to capture non-linear dependencies, are slow in practice, and fail to model the inter-variate information flow. Despite recent attempts to improve the expressive power of SSMs by using deep structured SSMs, the existing methods are either limited to univariate time series, fail to model complex patterns (e.g., seasonal patterns), fail to dynamically model the dependencies of variate and time dimensions, and/or are input-independent. We present Chimera that uses two input-dependent 2-D SSM heads with different discretization processes to learn long-term progression and seasonal patterns. To improve the efficiency of complex 2D recurrence, we present a fast training using a new 2-dimensional parallel selective scan. We further present and discuss 2-dimensional Mamba and Mamba-2 as the spacial cases of our 2D SSM. Our experimental evaluation shows the superior performance of Chimera on extensive and diverse benchmarks, including ECG and speech time series classification, long-term and short-term time series forecasting, and time series anomaly detection.