Abstract Sleep apnea/hypopnea syndrome (SAHS) is a widespread disorder associated with serious health risks. Although polysomnography is the diagnostic gold standard, its use is limited by high costs, extended duration, and labor-intensive procedures. To provide a more efficient solution, we propose DSENet, a novel dual-stream encoder-based network that enables automatic SAHS detection using only single-channel respiratory signals. DSENet employs a hierarchical partitioning strategy, dividing input signals into patches and units to effectively capture both coarse-grained and fine-grained respiratory patterns. Furthermore, the integration of the multi-scale feature aggregation module facilitates effective extraction of multi-scale global and local features, while an adversarial decoupling classifier increases robustness across different populations. Extensive experiments demonstrate that DSENet not only efficiently models complex temporal dependencies, but also consistently outperforms existing methods. In particular, using the nasal airflow signal, DSENet achieved 86.29% accuracy and 76.74% F1-score on Multi-Ethnic Study of Atherosclerosis, and 98.18% accuracy and 97.79% F1-score on Apnea-ECG. Code is publicly available at https://github.com/eecdxu/Respiratory-Event-Detection .