Core Dump as an Intrusion Event

[Crispin Cowan <crispin () WIREX COM>]

Background:  StackGuard 2.0 (as released this summer) does not provide
secure resistance to format bugs.  However, because StackGuard changes
some data layouts, it does tend to change the offsets that are required
to make the exploit work.  As a result, exploits tuned for the
"standard" instance of a vulnerable program tend to just cause the
victim program to dump core without giving up the shell prompt.

This leads me to conjecture that "core dump" makes a good intrusion
detection event.  Server apps. ("services", e.g. Apache, ftpd, fingerd
;-) should not be dumping core, so you could treat a core dump as an
indication that an attacker is rattling your door.  StackGuard enhances
this effect, by making it unlikely that the first attack attempt will
work.  Other factors may also be used to enhance this effect.

In theory, theory is just like practice, but in practice it's different.

Anyone have practical comments on this hypothesis?  In practice, how
often do services dump core for non-security reasons?  If services dump
core for non-security reasons even just a little, then the
false-positive rate of intrusion detection from this clue gets out of
control.

Caveat:  I know that this is a bad heuristic for Windows machines :-)

Thanks,
    Crispin

--
Crispin Cowan, Ph.D.
Chief Research Scientist, WireX Communications, Inc. http://wirex.com
Free Hardened Linux Distribution:                    http://immunix.org