Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ?

[Crusty Saint <saintcrusty () gmail com>]

Hi firewalZ

I could not agree more, but you'de have to have the time&budget to do this.

My work at this Service-Oriented Company is Cost-driven, not result-driven.
Whatever they might claim. Sad to say so but we have zero money-budget and a
rather tight time-budget.  Setting up such a test environment is not
something you'd be doing overnight and getting all the equipment could in
itself prove time-consuming, could.

My skepsis is there has not been an efficiënt recommended profile or default
profile available for years. Suddenly NSS Labs somehow made this possible.
Or the vendors tweak their profiles favourably, which would not be a unique
case in itself.

On the other hand, most the mid-size companies will indeed match a
predictable profile ( Microsoft-only, Microsoft-Oracle,
Microsoft-(Oracle)-Apache, for example ) so setting up an efficiënt profile
AND a meaningfull test-environment would be far from impossible. And
meaningfull.

The attack-surfaces we're protecting is very different from one customer to
the next or the other. As such we have less use for these magical
recommended profiles, though they could work. And by no means are they
perfect, despite what NSS Labs might suggest with their reports.


Greetings,

S-C.

2011/5/9 firewalZ <firewalz () gmail com>

> I personally am a bit sceptical of NSS and other similar so-called
> third party tests. From a user perspective, you can not get any
> information from them without paying big $. From a vendor perspective,
> you cannot participate without paying big $ (i have heard this same
> story from more than one vendor). Smells a bit like a racket to me.
> Bottom line is that if you are able, build a small test lab (many open
> source options), look into getting demo loaners from various vendors
> and test for your self. Understand your environment, potential attack
> entry points and unique vulnerabilities. I feel this would be a far
> greater use of the time and money that an NSS report would cost.
>
> FirewalZ
>
>
>
> On Thu, May 5, 2011 at 12:18 PM, Crusty Saint <saintcrusty () gmail com>
> wrote:
> > Hi,
> >
> > For a while now i've been stumped by the NSS Labs IPS resulsts for
> > checkpoint during the latest ISP Test.
> http://www.checkpoint.com/campaigns/intrusion-prevention-system/index.html
> >
> http://www.checkpoint.com/images/campaigns/intrusion-prevention-system/nss-block-rate-ips.png
> > for the picture which dates January 11th 2011
> >
> > Now my question is to what extent does this claim hold any water ?
> >
> > I've seen and evaluated this recommended profile and it is sparse. IMHO
> not
> > really what i would take for a profile that realistically holds back
> 97.3%
> > of all tests.
> >
> > So to make things worse for this intermediate IPS Engineer i've laid my
> > hands on the NSS Labs testing methodology to better understand if
> checkpoint
> > either tweaked it's profile to nss-labs testing or they really did
> deliver
> > close to the Holy Grail for what IPS's are concerned.
> >
> >
> > I'm begging for your input and knowledgeable comments. Such trickery
> bothers
> > me. And if it is not trickery i definitely need to soup up my skills and
> > mojo fast.
> >
> >
> > Best Regards,
> >
> > S-C
> >
> > --
> > - - -
> > Security Engineer - Tags: Analyst Systems Security Linux Firewall Network
> > Web Troubleshooting - If you think I deserve a rant, write me off-list
> ------------------------------------------------------------------------------
> > WhatsUp Gold - Download Free Network Management Software
> > The most intuitive, comprehensive, and cost-effective network
> > management toolset available today.  Delivers lowest initial
> > acquisition cost and overall TCO of any competing solution.
> > http://p.sf.net/sfu/whatsupgold-sd
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://www.geocrawler.com/redir-sf.php3?list=snort-users
>
>
> ------------------------------------------------------------------------------
> WhatsUp Gold - Download Free Network Management Software
> The most intuitive, comprehensive, and cost-effective network
> management toolset available today.  Delivers lowest initial
> acquisition cost and overall TCO of any competing solution.
> http://p.sf.net/sfu/whatsupgold-sd
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users



-- 
- - -
Security Engineer - Tags: Analyst Systems Security Linux Firewall Network
Web Troubleshooting - If you think I deserve a rant, write me off-list

------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network 
management toolset available today.  Delivers lowest initial 
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users