YAMME: a YAra-byte-signatures Metamorphic Mutation Engine
Antonio Coscia, Vincenzo Dentamaro, Stefano Galantucci, Antonio Maci, Giuseppe Pirlo
Abstract
Recognition of known malicious patterns through signature-based systems is unsuccessful against malware for which no known signature exists to identify them. These include not only zero-day but also known malicious software able to self-replicate rewriting its own code leaving unaffected its execution, namely metamorphic malware. YARA is a popular malware analysis tool that uses the so-called YARA-rules, which are built to match malicious contents within files or network packets analyzed by an Anti-Virus engine. Sometimes such content is expressed in the form of a byte-signature, i.e., a sequence of operational machine-level code. However, these can be bypassed since malware obfuscation techniques can change these sequences, rewriting them in several equivalent forms. This paper presents YAMME, a YARA-byte-signatures Metamorphic Mutation Engine to strengthen rules against some malware obfuscation techniques deployed in metamorphic mutation engines. First, it rewrites YARA-bye-signatures in several equivalent ways, as a metamorphic mutation engine would do. Second, an optimization phase exploits the YARA-rules syntax constructs to provide several rules formats, making them suitable for different real-world application requirements. YAMME rules have been evaluated on MWOR, G2, NGVCK, and MetaNG datasets, resulting in a better detection rate than that achieved by YARA-rules generated through AutoYara. Furthermore, an analysis of computational overhead required by different YAMME rules formats validates the low impact introduced by the mutation engine at the YARA-rules level.