Research on Anti-crawler and Anti-Anti-crawler Technology
作者
Xiaolong Zhang,Yuan Chen,Jiawei Liang,Jiawei Zhang,Tianyang Lu,Chuanqing Wang
标识
DOI:10.1109/icinc58035.2022.00015
摘要
In the era of big data, the extensive application of crawler technology makes the operation of the website difficult, and the anti-crawler technology of the website also emerges as the times require. How to develop anti-anti-crawler technology to effectively capture data according to anti-crawler Technology is of great significance for obtaining data. This paper introduces the basic architecture process and corresponding coping strategies of the commonly used website anti-crawler technology and crawler, analyzes the technical methods of anti-crawler, and puts forward some suggestions for anti-anti-crawler technology.