Resting-state functional magnetic resonance imaging (rs-fMRI) measures intrinsic neural activity, and analyzing its frequency-domain characteristics provides insights into brain dynamics. Owing to these properties, rs-fMRI is widely used to investigate brain disorders such as autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD). Conventional frequency-domain analyses typically rely on the Fourier transform, which lacks flexibility in capturing non-stationary neural signals due to its fixed resolution. Furthermore, these methods primarily utilize only real-valued features, such as the magnitude or phase, derived from complex-valued spectral representations. Consequently, direct modeling of the real and imaginary components, particularly within fMRI analyses, remains largely unexplored, overlooking the distinct and complementary spectral information encoded in these components. To address these limitations, we propose a novel Transformer-based framework that explicitly models the real and imaginary components of continuous wavelet transform (CWT) coefficients from rs-fMRI signals. Our architecture integrates spectral, temporal, and spatial attention modules, employing self- and cross-attention mechanisms to jointly capture intra- and inter-component relationships. Applied to the Autism Brain Imaging Data Exchange (ABIDE)-I and ADHD-200 datasets, our approach achieved state-of-the-art classification performance compared to existing baselines. Comprehensive ablation studies demonstrated the advantages of directly utilizing real and imaginary components over conventional frequency-domain features and validate each module's contribution. Moreover, attention-based analyses revealed frequency- and region-specific patterns consistent with known neurobiological alterations in ASD and ADHD. These findings highlight that preserving and jointly leveraging the real and imaginary components of CWT-based representations not only enhances diagnostic performance but also provides interpretable insights into neurodevelopmental disorders.