Abstract The feature pyramid network is an effective feature enhancement network. It is proposed and proved that its success is due to a divide-and-conquer solution to the optimization problem in object detection rather than multi-scale feature fusion. Divide-and-conquer refers to using feature layers of different scales as prediction feature layers instead of using one single object detection head. In this paper, we propose the hierarchical detection structures on YOLOv3-5 baseline models, which have good improvements. Firstly, we add a 7×7 grid as a prediction feature layer on YOLOv3, achieving 67.8% mAP@.5 and 46.3% mAP@.5:.95 on VOC and COCO datasets, which are 1.9% and 0.8% higher than the original. Then, we add 10×10 and 5×5 grids simultaneously on YOLOv4-5 as prediction feature layers. On VOC and COCO datasets, YOLOv4 achieves 61.8% mAP@.5 and 36.4% mAP@.5:.95, which are 1.6% and 1.1% higher than the original. YOLOv5 has four different models of S, M, L, and X. Finally, we do comparative experiments on all four models. For VOC dataset, 61.3%, 66.6%, 69.9% and 70.9% mAP@.5:.95 are achieved in YOLOv5_S-X, which are 4.4%, 1.4%, 1.6% and 1.1% higher than the original, respectively. The results of comparative experiments on each model show that the hierarchical detection structure based on anchors is effective for YOLO series models.