Re: [EXTERNAL] WAF Metrics

[Greg Frazier via Dailydave <dailydave () lists aitelfoundation org>]

I'm not parsing your argument. If you knew the bug was there, you would fix
the bug. The WAF is there to mitigate the bugs that you are not aware of.
Further, web accesses that are out of scope of your intended functionality
but do not trigger a bug may be information gathering attacks that you
would, in hindsight, have wished your WAF had blocked. I would argue that
the WAF is not a stop-gap at all--it is an integral part of your site's
functionality.

A questionfor the masses. You note that "Being attacked constantly is a
side-effect of being on the Internet." Do you find it disturbing that that
is the case? Would you be interested in a p2p protocol stack that blocked
(e.g.,) port-scanning? I find it odd that we all put up with the constant
presence of malicious behavior. It can be stopped--why do we not invest in
the technology to stop it?

On Wed, Jul 15, 2020 at 6:14 PM Don Ankney via Dailydave <
dailydave () lists aitelfoundation org> wrote:

> So far, this conversation focuses on how effectively WAFs block malicious
> HTTP requests. I'd argue that this is both a red herring and an abuse of
> WAF technology.  A WAF only protects the enterprise when it blocks a
> request that would trigger an actual bug. If there's no bug present, all
> that's really happening is that likely malicious requests are being logged
> at a much higher costs than if it were simply allowed to sit in the web
> logs. Being attacked constantly is a side-effect of being on the Internet.
> You should expect to be constantly scanned and probed. Ideally, action
> should only be necessary if an attack is actually practical, otherwise
> you're simply reacting to the background chatter of the Internet.
>
> If there is a bug present and the WAF blocks the attempt, then it is a
> legitimate protective action has occurred. The WAF is simply a stopgap,
> however. If the bug is present, then the WAF will eventually be evaded
> allowing an attack to succeed. As a result, I'd argue that success metrics
> for a WAF are effectively twofold:
>
> 1) How clearly does it signal that there's an actual bug in the software
> behind it?
> 2) How much time can it provide for a fix to that software?
>
> The problems you are protecting against aren't in the WAF or it's tuning
> -- it's in the underlying software. A WAF is simply a stopgap that buys
> time while code is corrected or vendors ship a patch. Relying on a WAF as
> anything more than a defense-in-depth mechanism is a losing proposition
> over time. Metrics need to reflect that.
> ------------------------------
> *From:* Chuck McAuley via Dailydave <dailydave () lists aitelfoundation org>
> *Sent:* Tuesday, July 14, 2020 7:14 AM
> *To:* John Lampe <jlampe () tenable com>; Rafal Los <Rafal () ishackingyou com>
> *Cc:* dailydave () lists aitelfoundation org <
> dailydave () lists aitelfoundation org>
> *Subject:* [Dailydave] Re: [EXTERNAL] WAF Metrics
>
>
> This isn’t directly related to John’s observation below, but it got me
> motivated to further clarify some of the challenges involved in testing
> WAFs.
>
>
>
> I’ve seen many implementations over the years that try to determine the
> decision making process of an IPS, WAF, or similar device by simply
> interrogating it from the client side only. The realities of test of
> measurement is that it requires the user to implement both a client and
> server side process of whatever it is you are testing to validate that not
> only was malicious content allowed or blocked, but that IF it was allowed,
> it was allowed through without a modification that would impact the intent
> of the vulnerability.
>
>
>
> That’s a mouthful of a sentence. Let me make it briefer:
>
> You can’t tell if the bad stuff got through unless you are wrapped around
> the bad stuff blocking thing.
>
>
>
> This gets even harder when you are testing WAF’s. They very often
> interrupt the connection, marshal the request, then issue a new HTTP
> request to the server. This means that while your malicious  request might
> have altered headers, or Content-Transfer-Encoding changes the actual
> little nugget of maliciousness is left alone.
>
>
>
> “Fine”, you say, “I’ll just examine the URL request, because I’m only
> interested in testing items that are impacted there.” And then
> CVE-2017-5638 comes along and you need to accommodate for deserialization
> vulnerabilities in other headers. The WAF catches it. But your current test
> implementation can’t.
>
>
>
> And that’s an easy example.
>
>
>
> You can have a chain like this:
>
> Client POST multipart http2 -> proxy -> TLS 1_1 -> Internet -> IPS
> Performing TLS inspection -> Different handshake TLS 1_0 -> WAF/LB ->
> http1.1 cleartext/chunked -> server.
>
>
>
> I just came up with that off the top of my head, so please don’t challenge
> me on the technical limitations of said example :).
>
>
>
> The point is that testing in isolation is very different from testing a
> deployed system under test and for each new technology deployed, the number
> of permutations increases dramatically. This list has discussed many times
> the merits and problems with “defense in depth” strategies, so I won’t
> belabor that point. Only to say “it’s hard to get an accurate read on if
> you are secure or not.” Lunchtime doubly so.
>
>
>
> -chuck
>
>
>
> *From: *John Lampe via Dailydave <dailydave () lists aitelfoundation org>
> *Reply-To: *John Lampe <jlampe () tenable com>
> *Date: *Monday, July 13, 2020 at 8:07 PM
> *To: *Rafal Los <Rafal () ishackingyou com>
> *Cc: *"dailydave () lists aitelfoundation org" <
> dailydave () lists aitelfoundation org>
> *Subject: *[Dailydave] Re: [EXTERNAL] WAF Metrics
>
>
>
> [EXTERNAL]
>
> Yeah, I guess the way I would envision it going would be:
>
>
>
> 1) web app scanner sees XSS vuln on /path/to/foo.php
>
> 2) my integration ties that web app scan into a format to pass to WAF
>
> 3) WAF sets up anti-xss rules on /path/to/foo.php (we had to actually
> create a static mapping for this step)
>
> 4) measure how many hits the waf blocks to that endpoint for the XSS
>
>
>
> John
>
>
>
>
>
>
>
>
>
>
>
> On Mon, Jul 13, 2020 at 10:46 AM Rafal Los <Rafal () ishackingyou com> wrote:
>
> *** CAUTION: This email was sent from an EXTERNAL source. Think before
> clicking links or opening attachments. ***
>
>
> ------------------------------
>
> John,
>
> Can you expand on #2? How do you measure the number of attacks stifled?
>
>
>
> _
> *-- **Rafal*
> *_Mobile:*
> * (404) 606-6056 **_Email**: Rafal.Los@Seventy7.Consulting
> <Rafal.Los@Seventy7.Consulting>*
>
>
>
>
>
> *From: *John Lampe via Dailydave <dailydave () lists aitelfoundation org>
> *Reply-To: *John Lampe <jlampe () tenable com>
> *Date: *Saturday, July 11, 2020 at 9:52 PM
> *To: *Dave Aitel <dave.aitel () gmail com>
> *Cc: *"dailydave () lists aitelfoundation org" <
> dailydave () lists aitelfoundation org>
> *Subject: *[Dailydave] Re: [EXTERNAL] WAF Metrics
>
>
>
> So, I recently did an integration for a company that took their web app
> scanner results and mapped those to existing WAF rules. I can think of 2
> metrics based off that
>
>
>
> 1) How many real-world vulns have a corresponding check in the WAF? and
>
> 2) Once the WAF rules have been put in place to protect
> actually-vulnerable endpoints, how many attacks were actually stifled?
>
>
>
> John
>
>
>
>
>
> On Sat, Jul 11, 2020 at 12:51 PM Dave Aitel via Dailydave <
> dailydave () lists aitelfoundation org> wrote:
>
> *** CAUTION: This email was sent from an EXTERNAL source. Think before
> clicking links or opening attachments. ***
>
>
> ------------------------------
>
> So I'm making a video on metrics, of all things, and I wanted to post both this
> question
> <https://urldefense.com/v3/__https:/twitter.com/daveaitel/status/1281629327776522242?s=20__;!!I5pVk4LIGAfnvw!1DZZL1viGJTRH-H2akN1jntqUUjdEe6Oa7-HctTc9IePgQzC3DN13JryFgb8Id0i$>and
> the best answer so far to the list to see if anyone had any other ideas or
> followups.
>
>
>
> -dave
>
>
>
>
>
>
>
> _______________________________________________
> Dailydave mailing list -- dailydave () lists aitelfoundation org
> To unsubscribe send an email to dailydave-leave () lists aitelfoundation org
>
> _______________________________________________
> Dailydave mailing list -- dailydave () lists aitelfoundation org
> To unsubscribe send an email to dailydave-leave () lists aitelfoundation org

_______________________________________________
Dailydave mailing list -- dailydave () lists aitelfoundation org
To unsubscribe send an email to dailydave-leave () lists aitelfoundation org