Accurate and efficient roll mark detection on the strip steel surfaces is a fundamental but "hard" ultra-tiny target detection problem due to its small pixel occupation in low-contrast images. By fully exploiting the prior information of roll marks, this article proposed a Prior-Guided YOLOX network (PG-YOLOX). First, inspired by the prior that the horizontal distribution of the roll marks is more uneven than the vertical direction, an orthogonal context attention (OCA) is carefully designed between the backbone and neck to better capture tiny target features by enhancing context representations. Besides, a cross-adaptive aggregation (CAA) module is constructed that adopts a cross-layer semantic prior during feature fusion to improve feature selection. Notably, a fresh tiny object detection dataset collected in an industrial scenario, Steel-Tiny, is released to the public. Based on experiments on the Steel-Tiny, our proposed PG-YOLOX has the highest mean average precision (mAP) (71.7%) for detecting roll marks, outperforming state-of-the-art methods. The generalization ability of our PG-YOLOX is demonstrated on the public remote sensing dataset VEDAI. The data will be publicly available at https://www.ilove-cv.com/steel-tiny-2/ .