Analysis of data crawling and visualization methods for recruitment industry information

Abstract With the popularity of the Internet, major job boards have become the main way for job seekers to obtain information, but the current job boards generally lack clear information display, overall industry trend analysis and data visualization, which makes job seekers fall into confusion. In this paper, we first crawl the job boards based on Scrapy framework, store the data in EXCEL after cleaning, explore the potential connection between different data through Apriori algorithm, and finally visualize the different job information in a targeted way. In this paper, we take the job information of printing industry as an example for implementation, extract key information, and propose a suitable visualization method for different types of job information. The method proposed in this paper can not only help job seekers understand the current talent demand of the industry in a simple, intuitive and fast way, but also has some guiding significance for the talent training program of universities.