How Anti-virus Heuristic Detection Works

Metamorphic coding of a virus will not automatically protect against detection by heuristic analysis, though in some cases it is enough to prevent discovery by these methods. Where polymorphic code is susceptible to detection by heuristic anti-virus software capturing a screen-shot on its execution (if this is timed precisely to acquire enough code, then the AV can deal appropriately with the infection), a metamorphic virus does not offer this opportunity. The capability of the scanning software and its environment is pitted against the coding of the author and the encryption that is used. This genus of virus carries the metamorphic encryption engine with it (which can comprise of up to 80% of the total malware coding). Other more practical evasion techniques like making random loops/interruptions/delays in execution and processes can be combined with its genetic, evolving encryption to enable evasion and escape. Signature-based recognition using hashes is of course of no use in identifying these ever-mutating infections.

Static heuristic detection uses string/advanced string-scanning techniques and code analysis subroutines to catalog and determine between malicious code and genuine/unaltered system files. This depends on recognizing commands in the coding. If a readable string can revealed and a self-replicating process is discovered by the scanner, then it is obvious immediately that the program is a self-replicating virus. But first, the problem of finding the questionable file for scrutiny and then of decrypting the code must be undertaken before the sample can be algorithmically scored to assess if it is likely to be malware. Definitive identification by this algorithm recognition (or Static analysis), is rendered useless when confronted by indecipherable encryption – so if found, the file becomes ‘questionable’ at best, unless there is a flaw in the coding (if the author has used a weak, generic form), or the scanner has a very proficient emulator for decryption. A metamorphic virus can encode its binary representation differently each time it infects a new host or file. This changes the representation of the whole infection as the metamorphic engine element changes also. This helps the virus to hide from this particular heuristic process.

The other identification method used in heuristics – Dynamic, or behavior-based – can only be carried out if the file is already under scrutiny as suspect. If the scanner can identify a suspected infection before execution (by regularly scanning files for alterations or unprompted access or deletions, &c), then this suspect file then be encouraged to run and be analyzed in a virtual CPU/ Sandbox environment for confirmation (having been tricked into thinking that it was in a safe environment to decrypt and execute). This detection may only occur after the virus has decrypted back to machine language and executed in the host system.

It has been recommended on blogs about theoretically defeating AV systems that by tactics such as inserting ‘quiet bits’ into the code (pauses that create delays in processes) that some sandboxes will time-out before malware detection/recognition is complete. Another anti-detection suggestion is to program the malware to execute at random times rather than automatically on first opportunity, and that this would perhaps enable start-up and replication to be accomplished before any detection by routine static heuristic scanning.

In concept it is simple to identify a virus heuristically – if enough code can be untangled to demonstrate the replication process. The technical problem is not so much the AV software that uses static/dynamic heuristics – if this was combined with hash/signature-based detection, with reference to commercial and public databases, almost any malware would probably be detected. The main element that can protect a virus from heuristic analysis is system limitations/user expectations, not solely its metamorphism. Performance versus results is the main challenge of the AV software of today, and it is the biggest element that protects a complex metamorphic virus. The scanner needs to be efficient, though fast (modern people like fast!), and if it isn’t quick enough, then it won’t sell. Greater speed can be achieved with bigger memory and a more powerful CPU – so mass-produced software technology has certain limits that the malware authors can exploit. If the average P.C user had enough resources to run a virtual machine on their home computer and emulate a complete Windows O/S sandbox on top of their actual O/S – at the same time operating as normal… well, no infection would have a cat’s chance in Hell of going undetected. As things are, these technical limitations combine with metamorphism to protect a virus from heuristic detection.

Leave a Comment

Your email address will not be published.

Time limit is exhausted. Please reload CAPTCHA.