A survey of strategy-driven evasion methods for PE malware: Transformation, concealment, and attack

The continuous proliferation of malware poses a formidable threat to the cyberspace landscape. Researchers have proffered a multitude of sophisticated defense mechanisms aimed at its detection and mitigation. Nevertheless, malware writers persistently pursue pioneering and innovative methods to evade detection by security software, thereby presenting an ever-evolving and dynamic threat to computer systems. Malware evasion refers to the use of certain strategies by malware to evade the detection of security software. Despite numerous surveys on malware evasion techniques, the existing surveys were fragmented and focused on specific types of evasion methods, leading to a lack of systematic and comprehensive research on malware evasion approaches. To fill this gap, this paper proposed a strategy-driven framework from the perspective of malware writers. Based on this framework, we categorize existing evasion detection techniques into transformation (alter the structural and behavioral pattern of the malware), concealment (conceal the behavior of the malware), and attack-based (engage in an attack on the detector to render it inoperable) methods and conduct a comprehensive survey of the relevant research works. In addition, we demonstrate how to integrate existing evasion strategies in the process of generating malware from the perspective of malware writers to subvert the multiple defenses of defenders. Our investigation indicates that: 1) evasion techniques such as packer and code obfuscation remain the foremost selection for attackers, no fewer than 10 off-the-shelf tools provide great assistance to them, 2) environment analysis is the primary concealment-based strategy used by the attacker (48% of the reviewed concealment-based strategy), defenders need greater efforts to counter them, 3) only 3 works discussed techniques for evasion attacks by leveraging fragilities in antivirus engines, meaning that direct attack on the detector is no longer as effective, 4) reinforcement learning algorithm serves as the most popular adversarial attack-based methods and 50% of works based on reinforcement learning are effective against real-world antivirus engines. Furthermore, this paper delves into the development trends in evasive malware and open issues for defenders. The primary objective of this survey is to furnish researchers and practitioners with a thorough comprehension of malware evasion strategies and techniques, thereby fostering the advancement of more potent and efficient approaches to detect and thwart malware.