New Article in Elsevier Computer & Security
Our colleague Sebastian Raubitzek, researcher at SBA Research and a member of the Security and Privacy Research Group at the University of Vienna, has published a journal article titled “Obfuscation Detection using Matrix Complexity Features of Binary Grayscale Images” in Elsevier’s Journal Computers & Security in collaboration with the CD lab AsTra.
© Niklas Schnaubelt
Abstract
Malware that conceals its behaviour through code obfuscation remains a central challenge for automated detection. This work introduced a novel approach for detecting the presence of obfuscation and identifying specific techniques. We transform binary code into grayscale images by mapping its bytes to a pixel intensity and apply singular value decomposition (SVD) to extract 18 matrix-complexity metrics that reflect structural changes introduced by an obfuscation. Using this approach, we evaluate eight Tigress obfuscation techniques on whether they leave a distinct spectral signature that can be classified. To obtain statistically robust results, we employ an ensemble of 100 independently tuned ExtraTrees models trained on different stratified 80/20 splits. The ensemble achieves average accuracies of 0.99 for detecting obfuscation, 0.94 for obfuscation type attribution, and 0.93 for identifying specific techniques. Feature-importance rankings and per-metric distribution plots make the results interpretable and transferable. The contributions of this study are (i) a reproducible pipeline for classifying obfuscated binaries, (ii) a detailed analysis of how obfuscation alters binary structure and its image representation, and (iii) actionable insight into which SVD metrics are most indicative of each transformation.
Authors: Sebastian Raubitzek, Sebastian Schrittwieser, Caroline König, Patrick Felbauer, Kevin Mallinger, Andreas Ekelhart, Edgar Weippl