Knowledge Aware State-Space Capsule Network for Multivariate Time Series Classification

Multivariate time series classification (MTSC) requires a model capable of capturing both localized temporal patterns and long-range dependencies while effectively modeling complex inter-variable relationships. Existing convolutional neural network (CNN)-based capsule models suffer from limited receptive fields, constraining their ability to model long-range dependencies, while transformer-based capsule models rely solely on self-attention, which, despite its effectiveness in capturing global features, struggles with preserving local structures and efficiently processing long sequences. To overcome these limitations, we propose KACapMamba, a Knowledge-Aware State-Space Capsule Network, which integrates three attentive Mamba blocks with a routing layer to achieve hierarchical temporal modeling. Unlike conventional transformer-based methods that primarily depend on self-attention for global dependency modeling, each attentive Mamba block in KACapMamba fuses 1-dimensional CNNs, self-attention, state-space module (SSM), and mutual cross-attention, enabling a more structured and adaptive feature representation. Self-attention ensures effective long-range dependency modeling, while SSM provides a recurrent-state mechanism, inherently better suited for sequential processing compared to purely attention-based architectures, thereby enhancing temporal continuity and long-term pattern retention. Notably, mutual cross-attention addresses the limitations of traditional fusion strategies such as element-wise addition or multiplication, which lack the capacity to selectively enhance relevant features. By dynamically reweighting interactions between features, mutual cross-attention enables more expressive, context-aware representations, leading to improved feature disentanglement and inter-variable modeling. Additionally, the routing layer further enhances hierarchical feature disentanglement by refining capsule activations, reinforcing structural coherence and feature selectivity. Experiments conducted across the UEA benchmark archive demonstrate that KACapMamba consistently achieves the highest ‘win’/‘tie’/‘lose’/‘best’ ratios when evaluated against 10 leading transformer and Mamba architectures under both $Accuracy$ and $F_{1}$ metrics. Moreover, in comparison with 22 state-of-the-art MTSC models, it again secures the most favorable performance profile, demonstrating a clear and statistically supported advantage across both evaluation measures.