Re: Next Generation Firewalls

[Brandon Dick <bdick () MURRAYSTATE EDU>]

Thanks for the info Michael.  I'm surprised to hear the 1500D performed 
at close to 10GB according to NSS.  Granted, that's with only IPS, but 
still, for the price they were asking, that's a good deal.   The fact 
that the reseller pulled out of the bid led me to think they were not 
going to be able to deliver on our requirements, but it sounds like they 
might have come close.

I will definitely check out NSS in the future.

On 6/21/2015 12:02 PM, Ferguson, Michael wrote:
> Brandon,
>
> I can say the lesson we learned from our POC and the Breaking Point tests that we did is that you can't rely on the data sheet of any vendor.  If Mark Twain were around 
> today, he might say there are "Lies, Statistics and Vendor Performance Metrics".  When you see performance numbers of any Data Sheet and think how impressive they 
> are, there could be some truth to it.  But it's also because they're testing against optimal traffic conditions to get the best-possible numbers.  So similar to 
> Statistics, you can convincingly argue a point with Vendor Performance Metrics that don't match up to real life.  As you start adding a mix of real-world TCP traffic and 
> turn on more than just one NextGen firewall feature, that's when performance significantly drops compared to the data sheet (and I mean significantly).  Some vendors 
> also talk about IMIX traffic and give separate metrics for it, but the results of our stress tests didn't match vendor claims for IMIX traffic when tested for real-world 
> traffic.  Moreover, when you turn on two or more NextGen firewall features (ie. Application Awareness/Control, URL Filtering, Anti-Virus, and Logging), you're completely 
> without bearing on how to spec out equipment as the numbers aren't there to tell you.  I can think of 2 vendors that give distinct performance metrics with at least 2 
> features turned on:  Palo Alto with AppID + IPS and Checkpoint with Production IPS (IPS + Logging).  But most vendors don't state the cumulative impact of turning on 2 
> nextgen firewall features, and none tell you the impact of turning on 3 features or more.
>
> The most reliable performance metrics I've seen for real-world traffic mix are from NSS Labs.  For the Fortigate 1500D, NSS Labs rated it 
> capable of 9,597 Mbps in Sept 2014.  This rating is calculated as an average of 5 different tests--4 protocol mix tests (Enterprise Perimeter, 
> Financial, Education and Datacenter) and a test for 21KB Http response rates.  Fortinet's data sheet claims the 1500D is capable of 11G of 
> IPS.  So on the face of it, the vendor's data sheet number is pretty close to the likely real-world performance tested by NSS Labs, only 
> over-estimating performance by 1.4G (the Financial test that NSS Labs does can really bring down the data sheet numbers).  However, the NSS Labs 
> rating is only for IPS inspection.  It doesn't include any performance impact when turning on other features like logging, App Control, 
> etc, which will certainly drop the performance to much less than 9.6G depending on what features you turn on.  It's too bad that NSS Labs 
> only rated IPS performance in the NGFW report, which is a report that should let you know the performance impact of more features than just IPS. 
>  As a matter of fact, Application Inspection is discussed in their NextGen Firewall reports as an important criteria.  But in 2014 and earlier, 
> the rated performance by NSS Labs for NGFW tests is only for IPS.
>
> We asked NSS Labs about this discrepancy during our POC and they recognized this as a limitation in their report.  They mentioned 
> they're looking to add cumulative performance numbers for Application Control + IPS in the next set of NGFW tests, which are expected 
> to be released in late Summer/early Fall.  However, I don't believe their NGFW tests will include the cumulative impact of turning on 
> other features like URL Filtering, A/V or Logging.  This is why the Breaking Point tests were so valuable to us.  We could cut skip 
> worrying about the data sheet and really see what the cumulative impact of turning on additional features was.  We tested for IPS, 
> AppControl and Logging, then added URL Filtering and A/V (we weren't able to fully test SSL decryption because of cipher problems with 
> the Breaking Point).  Also, for our tests, we ran them against a Fortigate 3700D; so I couldn't tell you what to expect for a 1500D.
>
> But what I can say is that for relatively, similar priced models from different vendors (and I use relatively in a general sense 
> as some vendors were well more than others), the Fortinet 3700D really stood out in performance compared to others we tested in 
> our POC, recognizing that the results of our Breaking Point tests didn't match up to the data sheet numbers from Fortinet.  
> Also, one other thing to be mindful about is that performance numbers are based on aggregate bandwidth--adding inbound + 
> outbound.  This means if you're 1.4G down and 400M up on your overall bandwidth, you should test for 1.8G now, then factor 
> in what you expect 3 to 5 years from now as an aggregate.
>
> Last, I'd like to just comment on the word NGFW, or NextGen Firewall.  I admit it is an awkward term, particularly when we start discussing firewalls 
> 10-15 years from now and try to distinguish the Nextest Gen Firewalls then against the old NextGen Firewalls of today.  I just don't know of any 
> elegant term to use to distinguish the current brand of firewalls that have Application Control, which to me is what makes them NextGen Firewalls.  When 
> IDC coined the term UTM, the UTM firewalls at that time didn't have Application Control as part of their capability.  But since then, they've 
> added it.  And by adding Application Control, I think UTM devices have become more than UTM devices because they not only combine network security 
> features in one box, but they have layer7 visibility that is fundamentally new.  Subconsciously, this is why I think Gartner phrased the term Next-Gen 
> Firewall to highlight that they're firewalls that do UTM + Application Control.  Unfortunately, Gartner's kind of confused the discussion by 
> indicating that firewalls with Application Control and other security features are UTM for small to middle-sized organizations and firewalls with the same 
> features for larger organizations are NextGen firewalls.  Really, there all NextGen firewalls if they have Application Control along with other features 
> like URL Inspection, A/V and the optional AntiSpam.  Do we call this new brand of firewalls AUTM or ATM devices--Application UTM or Application Threat 
> Management?  Probably not!  But what term to refer to our current generation of firewalls is still open for discussion and we use NextGen because we 
> don't have a better term.
>
> It might end up like when our campus had been building a new building a few years ago before it got an official name.  We 
> called it New Hall during construction, which was all fine and good.  But then New Hall became Named Hall and the new New 
> Hall starts getting built without a name yet.  We learned our lesson the second time not to name the new New Hall as New 
> Hall, but give it a different name during construction until it becomes Named Hall.  This might be how we address the 
> NextGen Firewall question.  When the second generation of NextGen Firewalls comes out with their own revolutionary 
> technology (like inline sand-boxing to be ridiculous), we'll actually call that type of firewall by a name that 
> distinguishes its capability instead of as NextGen.
>
>
> --
> Mike Ferguson
> Chapman University
> Network Operations Manager
> 714-744-7873
>
>
> -----Original Message-----
> From: The EDUCAUSE Security Constituent Group Listserv [mailto:SECURITY () LISTSERV EDUCAUSE EDU] On Behalf Of Brandon 
> Dick
> Sent: Friday, June 19, 2015 5:32 PM
> To: SECURITY () LISTSERV EDUCAUSE EDU
> Subject: Re: [SECURITY] Next Generation Firewalls
>
> For you all that went with Fortinet:
>
> When we were doing our bid, we put in requirements that the Firewall be
> 10GB capable and someone put in a bid for the Fortinet 1500D. According
> to the specs, the 1500D would put a Palo Alto 5050 to shame, but the
> cost difference wasn't even close, with Fortinet being the clear winner
> in price.  I kept digging into it with the reseller and requesting
> guarantees that this would actually perform as well as they said, and
> eventually the reseller pulled out of the bid.   So I never really got
> the full story on that one.
>
> So my question is, how many of you think the 1500D would've actually
> handled 10GB of traffic?  I noticed in your comparison below, they chose
> the 3600C, which makes me think even more that the 1500D wouldn't have
> performed to spec.  It was just a really odd situation...
>
> On 6/19/2015 2:45 PM, Kumar, Shashank wrote:
> > Hello All,
> >
> > We recently went through a bake-off ourselves and evaluated the Palo Alto 5050s, Fortinet 3600C, ASA5585-SSP-20 and the 
> > SRX 3600.
> > PA and Fortinet impressed us the most with flexible deployment options (routed mode, transparent mode, tap mode, 
> > virtual systems), good hardware specs and throughput, intuitive management consoles and good vendor interaction. 
> > Positive feedback from universities that had deployed PA and Fortinet was reassuring.
> >
> > Our evaluation process involved hitting the test unit with constant 2Gig of traffic and measuring the throughput with 
> > various features enabled.  We also planned for virtual systems and doubling our throughput within the next 3 years or 
> > so.  Let me know if you would like to see the test template that we used.
> >
> > Hope this helps.
> >
> > Best Regards,
> > Shashank
> > FGCU Network Services |Tel: 239-590-7448
> >
> > Florida has a very broad public records law.  As a result, any written communication created or received by Florida 
> > Gulf Coast University employees is subject to disclosure to the public and the media, upon request, unless otherwise 
> > exempt.  Under Florida law, e-mail addresses are public records.  If you do not want your email address released in 
> > response to a public records request, do not send electronic mail to this entity.  Instead, contact this office by 
> > phone or in writing.
> >
> > -----Original Message-----
> > From: The EDUCAUSE Security Constituent Group Listserv [mailto:SECURITY () LISTSERV EDUCAUSE EDU] On Behalf Of Gramke, 
> > Jim
> > Sent: Friday, June 19, 2015 2:12 PM
> > To: SECURITY () LISTSERV EDUCAUSE EDU
> > Subject: Re: [SECURITY] Next Generation Firewalls
> >
> > I've got to second the Fortigate recommendation.  We've had a High Availability pair of them for a number of iterations 
> > now.   Recently PA made a push, and we looked, but just undoable because of cost and even performance differences.  Bang for 
> > buck, Fortigate is a formidable competitor.
> >
> >
> > -----Original Message-----
> > From: Ferguson, Michael [mailto:mferguson () CHAPMAN EDU]
> > Sent: Thursday, June 18, 2015 11:17
> > Subject: Re: Next Generation Firewalls
> >
> > We ourselves just completed a Firewall POC solution.  I would recommend also being open to Fortinet as part of your 
> > consideration as this is what we selected.   It’s too early to say how well we like the solution as we’re in the 
> > process of implementation.  But at least during our POC,  it distinguished itself the most in a couple key categories 
> > with an emphasis on making sure we have a NGFW firewall that not only works well today, but also 4-5 years from now.  
> > Like you, we saw our selection of NGFW as a significant investment.  I’ll refrain from mentioning the other solutions 
> > we considered, but suffice it to say that we considered all the top solutions that are performing well in NSS Labs’ new 
> > Cyber Advanced Warning System.
> >
> > Our testing included Ixia Breaking Point tests, which I would recommend you consider as part of your evaluation if you 
> > have time to do it.  We also captured live traffic from our Production environment and sent it to several other 
> > solutions simultaneously using a Gigamon.  This was valuable for seeing the manageability and effectiveness of each of 
> > the solutions against each other, but not very useful when considering performance.  We also ran other security tests 
> > outside the Breaking Point to look at the effectiveness of each tool.  But by far, the results of the Breaking Point 
> > tests revealed the most distinction of the products we evaluated.
> >
> > I know Ixia offers Test Consulting for a relatively modest fee, as well as some security consulting firms.  There might 
> > be other ways to get a Breaking Point or a different stress-testing tool from Spirent or others.  But needless to say, 
> > it was very enlightening to see the differences between each of the solutions under heavy stress when all inspection 
> > (including Application awareness) and logging was turned on.
> >
> > --
> > Mike Ferguson
> > Chapman University
> > Network Operations Manager
> > 714-744-7873
> >
> >
> > -----Original Message-----
> > From: The EDUCAUSE Security Constituent Group Listserv [mailto:SECURITY () LISTSERV EDUCAUSE EDU] On Behalf Of Carroll, 
> > Tim
> > Sent: Thursday, June 18, 2015 7:00 AM
> > To: SECURITY () LISTSERV EDUCAUSE EDU
> > Subject: [SECURITY] Next Generation Firewalls
> >
> > All,
> >
> > Roane State Community College is in the process of reviewing next generation firewalls.  Since this is a significant 
> > investment, I would be interested in hearing from the community what you are using, your experience, why you made the 
> > choice and your satisfaction with the vendor chosen.
> >
> > Thanks in advance for any feedback.
> >
> > Regards,
> >
> > Tim Carroll
> > Assistant Vice President and Chief Information Officer Information Technology Roane State Community College
> >
> > ________________________________
> >
> > This email is intended for the addressee and may contain privileged information. If you are not the addressee, you are 
> > not permitted to use or copy this email or its attachments nor may you disclose the same to any third party. If this 
> > has been sent to you in error, please delete the email and notify us by replying to this email immediately.
> >
> > ________________________________
> >
> > Never give out your username or password to anyone. This includes any accounts you have such as: FGCU, bank and credit 
> > card accounts, and other personal accounts.

--
Brandon Dick
Network Engineer
Information Systems
Murray State University
Phone: (270) 809-3694
Fax:   (270) 809-3465

 

MSU Information Systems staff will never ask for your password or other confidential information via email.