Threat Miner - A Text Analysis Engine for Threat Identification Using Dark Web Data

Cyber threats continue to grow with novel methods to attack computing systems, highlighting the need for sophisticated mechanisms and techniques to protect against such dynamic threats. Contemporary cyber defence mechanisms utilise a range of methods which rely on monitoring network or system-level events. However, with the growing use of the dark web by mal-actors to share exploits, breaches, and data leaks, the use of such information to strengthen defence mechanisms becomes an intriguing prospect. In this paper, we present our efforts to develop a text mining engine (Threat Miner) which analyses data from dark web forums and transforms it into actionable intelligence. Leveraging cutting-edge machine learning techniques and utilising a bespoke threat dictionary, Threat Miner extracts useful information from dark web forums into STIX form, enabling it to be used with threat intelligence platforms. We also present the results of a thorough evaluation \nof our scheme which was conducted with the CrimeBB dataset to understand the feasibility of the approach as well as its effectiveness in strengthening defence capability against cyber threats.