Re: Efficiently detecting obfuscated shell code

["Don Parker" <dparker () rigelksecurity com>]

Well the thing of it is if you have someone who knows how to swap out 0x90 for some 
other 1 byte function that renders most of the hex matching useless. What remains? 
Possible content matching ie: MEOW in the recent RPC DCOM exploits. That too however can 
be removed. All of this is only good anyways for that period of time where the exploit 
is made public and the time it takes for the companies to patch. As I know from personal 
experience that can sometimes be a lengthy procedure. The experimentation I have done in 
my home lab has only reinforced the fact that it can be done, and relatively simple. As 
you say however there is no sure fire cure. I do have some idea's on how to improve 
detectioin but they of course involve time, and having the proper in-house talent to do 
so :-)

Cheers

Don

-------------------------------------------
Don Parker, GCIA
Intrusion Detection Specialist
Rigel Kent Security & Advisory Services Inc
www.rigelksecurity.com
ph :613.249.8340
fax:613.249.8319
--------------------------------------------

On Feb 4 , Paul Robertson <proberts () patriot net> wrote:

On Wed, 4 Feb 2004, Don Parker wrote:

> Hey guys/gals, I have been sending this question around some of the lists, and have had
> little real discussion on it. Question being; is it possible to reliably detect an
> obfuscated egg? Many of the ids signatures I have seen are a little loose, and always 
go
> for the nop sled with some port matching.

I think it depends on the obfuscation.  Just like with polymorphic
viruses, or indeed packed viruses, it's possible to grab the thing and
sometimes flag on the packer or the extraction engine, and sometimes on
payload characteristics, but sometimes you have to go to the exectution point.

The packer/extractor will be detectable, it's just a question of if it's
detecable without too high a false positive rate (I've been meaning to
look at those things, but real work keeps getting in the way...)

> The problem though is that it is a relatively trivial matter to sub the nop with an
> ascii character. Or someone who has a little more skill can insert another 1 byte
> function that won't affect the egg itself. These ids evasion attempts are becoming more
> widely known. With the prevalence of such programs as ADMutate and phiral.c simplifying
> the task as it were this will probably become more prevalent.

In detecting malcode, you have to go after the indcator, not the
instantiation if you want good generic detection.  "This unwraps itself"
is always a good leading indicator.

> Its not every company which has layered defences which includes application level
> firewalls, and a properly tuned ids with good signatures. This is not even taking into
> account an analyst who will recognize what they are seeing. Snort's fnord does a good
> job of detecting shell code actually, and known obfuscated variants too. Any thoughts 
on
> this?

To really do things right, you want to be at the point of execution.  I
wrote an interesting proof-of-concept kernel module a couple of years ago
that trapped execve calls to about 20 things, including /bin/sh for
processes who had a listening socket (or who's parents had a listening
socket) -- you could opt out a process with a signal, but kill() was also
wrappered- it was pretty good for "this stops 99% of "tunnel in/tunnel
out" trojans and shellcode in the wild (for the primary shellcode vectors
such as Apache, Sendmail...)- but there's nothing that's going to be 100%.

Obviously MAC in the kernel is the "right way" to do this, but my module
was a drop-in- I just never got the time to take it to where it really
needed to be.

I don't think you can do good detection on the wire for obfuscation
without significant slowdown and either a sandboxed execution or emulation
engine when you're talking actual executable content, but the indicators
of it probably have enough similarity that you could get pretty darned
close, but that many depend heavily on what it is you're trying to
protect.

If you run 1000 samples through an obfuscator, and can't get a reliable
detector for more than 40% of them, it's probably a lost cause.  The good
news is that most of the bad guys use the same tools, so if you can get
"today's obfuscator," it's probably good enough for prime time, even
though you might not get generic detection.

I have been meaning to go through the common shellcode libs and see if
they code match with any distribution binaries- that would give a good
starting point for on-machine protection.

Paul
-----------------------------------------------------------------------------
Paul D. Robertson      "My statements in this message are personal opinions
proberts () patriot net      which may have no basis whatsoever in fact."
probertson () trusecure com Director of Risk Assessment TruSecure Corporation
_______________________________________________
firewall-wizards mailing list
firewall-wizards () honor icsalabs com
<a href='http://honor.icsalabs.com/mailman/listinfo/firewall-
wizards'>http://honor.icsalabs.com/mailman/listinfo/firewall-wizards</a>

_______________________________________________
firewall-wizards mailing list
firewall-wizards () honor icsalabs com
http://honor.icsalabs.com/mailman/listinfo/firewall-wizards