Bi<sub>2</sub>Te<sub>3</sub>-based compounds are the only commercially available thermoelectric materials, yet their low-temperature performance below 300 K remains underdeveloped. This study systematically explores the effects of Bi/Sb ratio modulation and Se substitution on the electrical and thermal transport properties of Bi<sub>x</sub>Sb<sub>2-x</sub>Te<sub>3</sub> and Bi<sub>0.4</sub>Sb<sub>1.6</sub>Te<sub>3-y</sub>Se<sub>y</sub> materials. The research aims to optimize their thermoelectric performance in cryogenic regions through combined bandgap tuning and defect engineering. Materials were synthesized using a melt-quenching and spark plasma sintering process, followed by phase analysis via X-ray diffraction and microstructural characterization by electron probe microanalysis. First-principles calculations and Hall effect measurements were employed to investigate defect formation mechanisms and carrier transport behavior. In the Bi<sub>x</sub>Sb<sub>2-x</sub>Te<sub>3</sub> system, increasing Bi content reduced the bandgap from 0.168 eV for Bi<sub>0.4</sub>Sb<sub>1.6</sub>Te<sub>3</sub> to 0.113 eV for Bi<sub>0.58</sub>Sb<sub>1.42</sub>Te<sub>3</sub>, shifting the peak <i>ZT</i> temperature to lower ranges. However, enhanced alloy scattering degraded carrier mobility from 332 to 109 cm<sup>2</sup>/(V·s) and power factor from 4.58 to 1.12 mW/(m·K<sup>2</sup>). To address this, Se was substituted into the Te lattice of Bi<sub>0.4</sub>Sb<sub>1.6</sub>Te<sub>3</sub>. First-principles calculations revealed that Se substitution reduced the formation energy of Se<sub>Te</sub> + Bi<sub>Sb</sub> complexes, effectively suppressing Sb<sub>Te</sub> antisite defects. This resulted in a carrier concentration decrease from 3.32×10<sup>19</sup> to 2.64×10<sup>19</sup> cm<sup>-3</sup> while maintaining high mobility at 279 cm<sup>2</sup>/(V·s). Concurrently, Se-induced point defects enhanced phonon scattering, lowering lattice thermal conductivity by 17 % from 0.46 to 0.38 W/(m·K). Bi<sub>0.4</sub>Sb<sub>1.6</sub>Te<sub>2.97</sub>Se<sub>0.03</sub> sample achieved a <i>ZT</i> value of 0.93 at 220 K, representing a 16 % improvement over the pristine Bi<sub>0.4</sub>Sb<sub>1.6</sub>Te<sub>3</sub> sample with a <i>ZT</i> value of 0.80. The peak <i>ZT</i> increased from 1.17 to 1.31 at 350 K, demonstrating a 12 % enhancement. These improvements arise from the synergistic effects of band engineering, where flattened valence band edges increased effective mass, and defect engineering, which suppressed antisite defects and strengthened phonon scattering. This work provides a dual optimization strategy for BiSbTe-based materials, balancing bandgap reduction with defect control to improve cryogenic performance. The findings are particularly relevant for applications in infrared detectors and multistage thermoelectric cooling systems.