RecLGB: Enhancing LightGBM using Recursive VAE with Mixed Attention for Time-Series Forecasting

Time-series forecasting demands efficient modeling of long-range dependencies while maintaining computational practicality. Current methods, particularly deep learning approaches, often struggle with complexity and scalability when handling extensive historical sequences. We propose RecLGB, a hybrid framework that synergizes LightGBM's efficiency with a memory-augmented deep architecture. At its core, RecLGB integrates a recursive Variational Autoencoder (VAE) enhanced by a mixed attention mechanism, which preserves temporal order through linear inductive biases while dynamically capturing dependencies via self-attention. The recursive VAE compresses lengthy historical sequences into compact hierarchical representations, serving as an external memory for LightGBM to leverage without computational overload. RecLGB is evaluated on five real-world datasets, and experiments demonstrate RecLGB's superiority, achieving superior accuracy and faster inference than Transformer-based baselines. This work bridges deep sequential modeling with gradient-boosted trees, offering a scalable, interpretable solution for resource-constrained forecasting. Code is available at: https://github.com/Mayer-myx/RecLGB.