Sala Riunioni 1 - Computer Science Department - 2.00 pm

Title
"Code obfuscation and malware detection by abstract interpretation"


Abstract

A key concern of software developers is to defend their programs against malicious host attacks, that usually aim at stealing, modifying or tampering with the code in order to take (economic) advantages over it. Besides, a related security issue involves the execution of malicious software, called malware, on host machines.
In recent years code obfuscation has been used by both software developers in order to protect secret properties of their programs, and malicious code writers in order to avoid detection. The basic idea of code obfuscation is to transform programs in order to make them more difficult to understand and analyze while preserving their functionality. In the software protection scenario, the lack of a rigorous theoretical framework for code obfuscation makes it difficult to formally analyze and certify the effectiveness of obfuscation. We face this problem by providing a formal definition of code obfuscation based on program semantics and abstract interpretation. This allow us to study and relate the effectiveness of commonly used obfuscations. Recent developments in malware technology have led to the so called metamorphic malware. The basic idea of metamorphism is that malicious code changes (through obfuscation) during execution: each successive generation of a malware changes the syntax while leaving the semantics almost unchanged. The reason why code obfuscation is able to foil most of the existing detection scheme lies in the syntactic nature of these schemes that often ignore program functionality. Thus, addressing the malware detection problem from a semantic point of view could lead to more robust detection systems. Based on the semantic definition of code obfuscation, we developed a formal framework for proving soundness and completeness of existing malware detectors. Moreover, we are working on the development of a formal model for describing the behaviour of metamorphic malware.