Fwd: x-forwarded-for an IDS capability

["Arian J. Evans" <arian.evans () anachronic com>]

inline. re-send as plaintext.


On Wed, Apr 29, 2009 at 7:55 AM, Hellman, Matthew
<Hellman.Matthew () principal com> wrote:
> That's a nice idea, I personally haven't seen or heard of it being implemented.
>  If you can get a trace with the alert you might see it there. Also, a SIM should
> be able to do this for you (by means of including the firewall/router/proxy logs
> and correlating them).

Actually, no. Firewall / router / and some proxy logs are usually
useless. SIMs cannot usually do this for you (though they could, if
you fed them the right data). This is not a fault of SIMs so much as
it is a fault of their implementation (what data sources you feed
them).

Few Router / FW / IDS / IPS/ SIM situations analyze more than GET /
URI data structure. Occasionally you will find some partial HTTP
header logging, maybe cookies and such. The RFC and W3C specifications
for HTTP logging specify NOT to log POST requests. POST is supposed to
be used for both non-idempotent data (volatile or state-changing) and
sensitive data, and neither is supposed to be captured or logged. So
most WWW logging systems do not allow you to capture POSTdata
(including even some HTTP proxies).

The main attack surface of most web vulnerabilities are in POST requests.

Most attacks today (SQLi Bots, XSS worms) occur over POST requests,
AKA Form-submits in POSTdata strings.

The reduction in attack surface via GET is for a variety of reasons,
including both changes in modern programming techniques, and increased
automatic input validation by "scrubbers" (IIS URLscan), and more
robust URI escaping by browsers.

---

Long and short: Yes, this could be done with a SIM if you have a
robust proxy or WAF AND are capturing *everything* in the Request
object.

But.....Most cannot AND do not capture the entire Request object
today. So != visibility.

I still work with many fine folks wondering why they are getting
hacked or have gotten hacked and why they cannot see what is going on.

Betwixt their firewall / router / WWW logs / Web Server Agents
(URLScan type stuff) / syslog / splunk / etc. they cannot find the
activity, which is where most traditional traffic analysis guidelines
tell them to go find this sort of activity.

The reason for the inability to find the data, in summary, is because
the vast majority of successful attacks are passed in POST Requests
(POSTdata) and a smaller percentage in Header values, the former of
which is almost never logged, and the latter of which are rarely to
partially logged.

On a secondary note, a surprising number of folks still don't decrypt
and inspect their SSL traffic (usually the most interesting HTTP
traffic) at the network layers (via firewall, IDS, IPS) and do not
seem aware that they have this obvious blind spot. This is so 2001. :)

Perhaps the current generation of technologies that can share SSL
certs will solve this whether or not people realize it is getting
solved.

That or everyone will buy a WAF and implement it correctly.

Or start blocking all this Port 80 and 443 craziness.

Cheerio,

--
Arian Evans

"There's of course been the hookers and the cocaine, there's been a
lot of mistakes, and a lot of lessons we had to learn, and most of all
there's been a lot of World of Warcraft."

George Broussard, 2009