Understanding the role of structural variants such as chromosomal inversions in local adaptation among small, isolated populations is an important addition to robust conservation strategies, as most studies investigating inversions to date have been conducted on high gene flow systems. Brook Trout (Salvelinus fontinalis), an economically important top sportfish, is extremely vulnerable to thermal stress. Local adaptation with respect to this trait warrants investigation as climate change accelerates the loss of cold-stream ecosystems. We performed low-coverage whole-genome sequencing on 192 Brook Trout from 9 small, isolated streams in Nova Scotia, Canada, to assess genetic diversity within and among stream populations. We detected four structural variants in the three westernmost populations, which differ from all other streams in water temperature, streamflow, and surficial geology. The structural variants appear to be chromosomal inversions. These genomic regions exhibit high linkage disequilibrium, and PCA revealed the presence of three karyotypes. Redundancy analysis provides support for potential local adaptation, with temperature, pH, and streamflow being important predictors of genomic variance and statistically significant SNPs falling within potential inverted regions. Mitogenome analyses suggest that a single glacial lineage recolonized the region. Individuals carrying the potential chromosomal inversions exhibited one of five related mitochondrial DNA haplotypes, but these haplotypes were found also in individuals without the potential inversions, suggesting they arose post-recolonization. The genetic differentiation among the nine surveyed Brook Trout populations persists even after exclusion of the potentially inverted regions, suggesting these regions do not control the population structure of these Brook Trout populations.