Re: looking for recursion stack overflow exploit

[Valdis.Kletnieks () vt edu]

On Wed, 20 Nov 2002 07:27:21 EST, bukys () cs rochester edu said:
> While a recursion-induced stack overflow can obviously lead to a
> denial-of-service attack, are there any examples of it being turned
> into an opportunity for remote execution?

The only possibility I can see here is if you can find some way to subvert
the "stack size exceeded" error handler when the recursion finally runs out
of stack.  However, it's probably not productive, since most programs don't
include recursive code to start with, and if you are able to subvert an error
handler, it's a lot faster/easier to hijack whatever your system's moral
equivalent of the Unix SIGSEGV, and then reference non-existent memory and
exploit quickly.

On the other hand, the Unix libc usually contains the qsort() and ftw()
routines, which might be interesting.  ftw() is prone to race conditions,
and it *might* be possible to feed qsort() a specially crafted array of
values that would give it indigestion at an inconvenient time (the place
to start would probably be an out-of-memory condition in the compare() function
passed to qsort()).
-- 
                                Valdis Kletnieks
                                Computer Systems Senior Engineer
                                Virginia Tech

Attachment: _bin
Description: