MixRecLGB: Language-Enhanced Mixed Attention for Temporal Context Modeling in Time Series Forecasting

Accurate time series forecasting demands effective integration of temporal dynamics and contextual semantics. While existing attention mechanisms capture numerical patterns effectively, they often neglect domain-specific temporal knowledge. We propose MixRecLGB, a novel framework that synergizes LightGBM with a language-enhanced mixed attention mechanism. Our key contributions include: 1) A recursive VAE architecture (RecLGB) that compresses long historical sequences into hierarchical memory features through progressive latent space learning; 2) A temporal-semantic fusion mechanism that injects frozen language model embeddings into both static linear attention and dynamic self-attention components, preserving temporal order while incorporating contextual knowledge; 3) A parameter-efficient integration strategy that enhances attention computation through adaptive bias injection and feature fusion, requiring minimal architectural modifications. Evaluated on five real-world datasets, MixRecLGB reduces forecasting errors while maintaining computational efficiency. This work establishes a new paradigm for combining deep temporal modeling with efficient gradient boosting, particularly effective for long-term forecasting scenarios.