System-Level Data Management for Endpoint Advanced Persistent Threat Detection: Issues, Challenges and Trends

Advanced persistent threat (APT) attacks pose significant security threats to governments and large enterprises. Endpoint detection and response (EDR) methods, which are standard solutions to combat APT attacks, can efficaciously respond to associated security threats by leveraging the semantic richness of provenance graphs and clear causality relations to resist illegal tampering. However, the large number of audit logs produced over time, which provide key supporting information for EDR methods, lead to substantial computational overhead and increased storage costs. Therefore, a robust data management framework must be developed. However, most existing reviews discuss data collection, compression, and storage methods independently. Due to the lack of a comprehensive, structured survey of data management strategies, current data management analyses tend to be separated into individual modules, making it difficult to obtain prompt and precise guidance for higher-level security analysis tasks from these analyses. In this paper, a comprehensive and structured survey of data management strategies based on provenance graphs is conducted, the core ideas of the mainstream approaches to each aspect of data management are summarized, and existing approaches are systematically classified and compared. Then, the problems with individual data management modules are investigated, and potential complementary and collaborative strategies are examined based on the insights and challenges of existing work as a basis for recommending best practices for practical deployment. Finally, an ideal data management framework is described to guide future research.