BlackGoose Rimer: Harnessing RWKV-7 as a Simple yet Superior Replacement for Transformers in Large-Scale Time Series Modeling
Li weile, Liu Xiao
2025-03-11
Summary
This paper talks about BlackGoose Rimer, a new AI tool that handles large amounts of time-based data (like stock prices or weather patterns) much faster and better than older models by using a smarter design.
What's the problem?
Existing AI models for time-based data are slow, need lots of computer power, and struggle to handle really big datasets efficiently.
What's the solution?
BlackGoose Rimer uses a special setup called RWKV-7 that mixes old and new data cleverly, making it way faster and more accurate while using fewer resources.
Why it matters?
This helps businesses and scientists predict trends (like sales or climate changes) more reliably without needing supercomputers, saving time and money.
Abstract
Time series models face significant challenges in scaling to handle large and complex datasets, akin to the scaling achieved by large language models (LLMs). The unique characteristics of time series data and the computational demands of model scaling necessitate innovative approaches. While researchers have explored various architectures such as Transformers, LSTMs, and GRUs to address these challenges, we propose a novel solution using RWKV-7, which incorporates meta-learning into its state update mechanism. By integrating RWKV-7's time mix and channel mix components into the transformer-based time series model Timer, we achieve a substantial performance improvement of approximately 1.13 to 43.3x and a 4.5x reduction in training time with 1/23 parameters, all while utilizing fewer parameters. Our code and model weights are publicly available for further research and development at https://github.com/Alic-Li/BlackGoose_Rimer.