Crop rotation is widely practiced to improve agricultural sustainability, yet its impact on microbial diversity remains unclear. We conduct a global meta-analysis of 2406 paired observations to examine the effects of crop rotation on microbial diversity based on high-throughput sequencing data. We show that crop rotation significantly increases bacterial Shannon diversity and species richness, but has no effect on bacterial beta diversity. In contrast, crop rotation significantly increases fungal beta diversity without affecting fungal Shannon diversity and species richness. Changes in microbial communities are linked to soil pH, and available nitrogen and phosphorus. Notably, legume vs. non-legume, arbuscular mycorrhiza vs. non- arbuscular mycorrhiza, C3 vs. C4, and annual vs. perennial crop transitions, as well as climate and soil factors, affect the response ratios of microbial metics to crop rotation. Furthermore, the response ratios of bacterial Shannon diversity, bacterial species richness, and fungal species richness are positively related to the response ratio of crop yield. Our study reveals positive but differential effects of crop rotation on bacterial and fungal diversities, which are linked to improved crop productivity. Our findings thus have implications for using crop rotation to conserve soil microbial biodiversity, which is related to soil health and function, and enhance global food security.