nanog mailing list archives

Re: Checkpoint IPS


From: Patrick Tracanelli <eksffa () freebsdbrasil com br>
Date: Fri, 6 Feb 2015 12:52:35 -0200

Hello,

On 06/02/2015, at 11:08, Ray Soucy <rps () maine edu> wrote:

An IPS doesn't have to be in line.

AFAIK this is basically what defines an IPS.

It can be something watching a tap and scripted to use something else
to block traffic (e.g. hardware filtering options on a router that can
handle it).

You are naming IPS on what is actually an IDS+Active Response as I mentioned before (my #1 option of choice). Not been 
online won’t for instance prevent the so called single packet attacks and therefore what should be the advantage of an 
IPS over an IDS is just thrown away. Which, again, I accept what’s missed for what I still have. But I don’t think it 
can be named IPS acting passively+actively, since it’s not actually not preventing.

An IDS tied into an internal RTBH setup to leverage uRPF filtering in
hardware can be pretty effective at detecting and blocking the typical
UDP attacks out there before they reach systems that don't handle that
as gracefully (e.g. firewalls or host systems).

That’s exactly the point I agree and adopt. Maybe a first packet will pass, but it takes longer anyway to be 
deterministic whether the traffic is common or attack traffic. It’s an IPs there and exausting/starving won’t cause 
issues to the network, only to it’s own capability.

Even if you keep it
from being automated and just have it be an IDS that you can have a
human respond to is pretty valuable.

Not a coincidence that Bejtlich’s NSM101and TCP/IP Weapons School courses are usually sold out on Black Hat. Valuable 
humans behind the tools will always provide better benefits than what vendors may generically sell/deliver. 



On Thu, Feb 5, 2015 at 6:40 PM, Patrick Tracanelli
<eksffa () freebsdbrasil com br> wrote:

On 05/02/2015, at 12:31, Terry Baranski <terry.baranski.list () gmail com>
wrote:

On Thu, Feb 5, 2015 at 8:34 AM, Roland Dobbins <rdobbins () arbor net> wrote:

I've never heard a plausible anecdote, much less seen meaningful

statistics,

of these devices actually 'preventing' anything.


People tend to hear what they want to hear. Surely your claim can't be
that
an IPS has never, in the history of Earth, prevented an attack or exploit.
So it's unclear to me what you're actually trying to say here.

And the fact that well-known evasion techniques still work against these
devices today, coupled with the undeniable proliferation of compromised
hosts residing within networks supposedly 'protected' by these devices,
militates against your proposition.


Your tendency of making blanket statements is somewhat baffling given the
multitude of intricacies, details, and varying circumstances involved in a
complex topic like this. To me, it's indicative of an overly-simplified
and/or biased way of looking at things.

In any case, go ahead and stick with your router ACLs and (stateful!)
proxies. Different strokes.

-Terry


There's room for a good engineered strategy for protection which won't turn
into a point of failure in the overall networking topology.

For decades, since first rainbow series books were written and military
"strategy" started to be added to information security, it's always been
about defense in depth and layered defense. Today those buzzwords became an
incredibly "bullshit bingo" on sales force strategy on selling magical boxes
and people tend to forget the basics. Layers and the "depth" is not a theory
just to be added to drawings and keynote presentations.

Considering a simplistic topology for 3-tier (4 if you count T0) depth
protection strategy:

(Internet)--[Tier-0]--(Core Router)--[Tier1]--(core
switch)--[Tier2]--(DMZ)--[Tier3]--(Golden Secret)

One security layer (tier, whatever) is there to try to fill the gap from the
previous one.

How deep you have to dig depends on who you are. If you are the end
organization willing to protect the golden secret, how complex is your
topology, or if you are the carrier, the telecom the company worried only
about BGP, PPS, BPS and availability other than the actual value for what's
the real target for the attack (if not availability)

In summary, in my experience what will (not) work is:

1) Tier 0 & Tier 1
On border, core, (Tier0) or on Tier-1 protection layers (typical
"firewall/dmz" topological position)

- Memory and CPU exaustion will shut down your operations (increase latency
and decrease availability) easily, if you have a Protecting Inbound Proxy
(Web Application Firewall, for the sales jargon), Stateful Firewall or IPS.

The thing here is, you are insane or naive if you believe a finite state
machine with finite resources can protect you from a virtually infinite
(unlimited) source of attacks. No matter how much RAM you have, how much CPU
cycles you have, they are finite, and since those "amazing stateful w/ deep
inspection, scrub, normalization and reassembly" features on a firewall will
demand at least 4K RAM per session and a couple CPU cycles per test, you
have a whole "line rate" (1.4Mpps / 14Mpps for 1GbE 10GbE ports) attack
potential, and come on, if a single or a 3-way packets sequence will cost
you a state, it's an easy math count to find out you are in a bad position
trying to "secure" those Tier0 and Tier1 topological locations. It's just
easier and cheaper for the one attacking, than for your amazing firewall or
IPS.

So what to do here? Try to get rid of most automated/scripted/simple attack
you can. You can do ingress filtering, a lot of BCP, protection against
SNMP/NTP/DNS amplification, verify reverse path, (verrevpath, antipsoof,
verrevreachability), and many good / effective (but limited) protection with
ACL, data plane protection mechanisms, BGP blackholing, Null Routing
(sending stuff to disc0 on BSD, null0 on Cisco, etc) and Stateles
Firewalling.

Also, an IDS sensor might fit here, because if CPU/Memory starvation happens
on an IDS sensor, the worst thing will happen is some packets getting routed
without proper processing. But still, they will get routed.

Always stateles, always simple tests. No Layer7 inspection of any form, no
load balancer, WAF, whatever.  No regex, hell no! No memory pointers that
can point to dark processor/memory locations (again, no regex).

2) On Tier 2 protection (A defense depth that comes after core protection
and after Tier-1):

- Will miserably fail if you use stateful firewall in excess
- Will fail even quicker if you use a WAF or IPS

Many "security engineers" and "security experts" (or worse, security tools
vendors and developers) excessively use stateful tests on transport
protocols that are simply... stateles.

What good you have on stateful firewalling... ICMP? UDP? DDP? IGMP? come on,
all those state timers are worthless and your memory limits will reach very
soon. Remember, in the average, 4K RAM at least, per stateful session. Much
more resources are needed for IPS, much much more for inbound proxy (WAF),
not to mention CPU.

Will you add an IPS here? What for, other than easing putting down and
making your network and services unavailable?

Proxy? Mod Security? Hell no! Not here. Did you check how many regex and
rules you have in the "base_rules" collection for a Top 10 OWASP protection
on Mod Security? What about vendor specifics rules, or Sans Top 25
specifics.

This is just not the place.

Also, let's rationale, what is your real benefit on deep inspection stateful
filtering on those SSH sessions allowing for your trusted locations only?
Really, what good will it make? Drop it stateles, allow it stateles! If you
really believe stateful worths something for protocols such as SSH, do it
somewhere else (Tier3 or host-based).

Focus on stateful protection only for what really needs some protection of
this kind. Packets related, fragile services for packets arbitrarily
assembled. Maybe SIP (SIP, not RTP), HTTPS, HTTP. Layer3/Layer4 fragile and
public services, after some stateles inspection was already done on Tier1,
should deserve stateful protection only. Not your whole network! Not your
whole set of services or services.

And no ICMP, no UDP do deserve stateful.

Also, pick up a good tool for this.

Most sysadmins, security guys or network operators never notice how their
tools may betray 'em hardly (by deault).

For example one use OpenBSD PF firewall, every rule you make will
"automagically" become an stateful rule. And this is not good! This is
terrible! This "auto" behavior makes the security engineer blind, since the
rules he is writing is not the rules he is adding to kernel.

OpenBSD's PF has a "no state" option and nobody uses it! It means you are
doing it wrong... UDP/ICMP/TCP rules always have "keep state" added, if you
don't explicitly set "no state". Beware!

The same is valid for Linux Netfilter, and 9 in 10 commercial firewalls
(checkpoint, mikrotik, fortinet, whatever...).

FreeBSD's ipfw on the other hand is stateles by default. It means if you
don't explicitly add "keep-state" there, it's stateles. Which is good,
unless you explicitly want a rule to be stateful. It's excellent as a
default behavior for protection layers not close to your "golden egg
provider". On the other hand, ipfw by default has a limit of 4096 states
which is TOO LOW. Beware too!

Regarding IPS/WAF/Proxy or "Next Generation" firewalls, this is not the
place to add it.

On the other hand you need some level of extra protection firewalls will not
provide, and some Layer7 inspection on Tier2 will be good.

But not Proxy! Not mod security! and not IPS! (no WAF)

IMHO, an IDS will fit good here.

IDS, not IPS, not IDP. Not a magical solution...

Why, from my past experiences, an IDS approach here will fit? Due it's
passive nature. If your IDS (say, Suricata, Bro, keeping on open source, or
your commercial option of choice) starves on CPU or memory, what's the worst
thing to happen?

You will have a high number of PPS on that perimeter port passing without
getting processed/inspected. Your overall security will decrease, but you
still have Tier3 (and maybe other tiers) to fulfill the gap! Packets that
still can be processed should have active response in place taking care of a
lot of attacks.

What I mean is, if you have limited (and you do have limited) processing
and memory power, say you can IDS inspect 400Kpps but a 1Mpps attack is
ongoing, you have 40% inspection power, but packets not processed still get
routed! Without latency and without packet loss because it's an IDS and not
an IPS. It's not inline. It's passive, sitting there. Limited resources,
priority and lower kernel CPU scheduler relevance.

And for those 40% (very bad scenario I am drawing here) you still have
active response in place, rerouting, reconfiguring stateles firewall,
stateles ACLs and null routes on upper tiers (tier 0, tier 1, routers,
switches) and lower tiers (tier 3 reconfigured upon tier 2 decision).

What you will do is try to fill the gap on next tiers, or increase your
filtering capabilities that are still stateles or passive.

For many business, this is the end for added protection layers. A big ISP,
transport provider, content delivery business... more protection from this
point is something for the specific end business, and completely related to
their needs, their business model, process, responsibilities and overall
evaluated requirements. Not a general model to fit.

But for everyone else up to this tier you have relevant security mechanisms
and tecnology added, and still low starvation/exaustion risks due to the
stateles and passive nature of the approach.

3) On Tier 3, a protection strategy for your "golden eggs" provider, the
golden secret, your business secret and intelligence

Usually, this protection level is for the corporate strategy. The business,
not the carrier, the service provider or the network operator. And is a
business specific requirement. Meaning it may not exist at all!

Now, for me, here is where you add more stateful inspection (still, only
what is actually efficient, not that god damn echo reply/request wasting
memory to be tracked down - useless!).

Here is a good place for a WAF, after firewall and IDS protection took
place. WAF is a not a panaceia for anything, it's aimed for specific attacks
against applications and protocols, is not resilient to coward attacks
(volumetric, L3/L4 exaustion, etc). And a proxy in the end is always a web
server, so inherits all it's fragility, and therefore it must be protected
as well, before it can actually protect anything else.

Host Intrusion Detection central servers (receiving information gathered
from HIDS on the actual hosts) also fits Tier3. As well as other host-based
controls that may add telemetry information to a central location.

Talking about telemetry, it's important everywhere, and while generating
flows or snmp info grabbing may impact processing usage on critical core
devices, those extra added boxes should also passively help telemetry, with
flow generation or minimum capability for snmp servings.

Nothing here is new. I am talking about, again, basic stuff discussed for
decades on colored cover military books, drafts, discussions and BCPs and
really old stuff discussed for people who may be already dead, sometimes
(Itojun and other samurais' missed). I'm only mentioning the basic 3 tech
domains (firewall, ids, proxy), but the other two basic (pen test, vuln
scan) that are more process than technology are important as well.

For me, and again this is very personal opinion, I never run an IPS unless
the customer "really wants to" (or a stupid compliance requirement really
requires to). An IDS+Active Response is as good as IPS, and the only extra
benefit an IPS will add compared to IDS is related to "single packet
attacks", that ones that will pass quickly enough before the active response
blocks it. But "single packet" attacks are related to poorly written
software (or unpatched / not fixed software) since it's not really an
attack, it's a trigger to bad code misbehavior and should be addressed on
the host.

This is a very simple model, and easy to understand. However how many
situations we've seen big companies getting completely unavailable or AAA
getting broken because people insist to buy (and sell) "miracle boxes" added
to core locations, and those miracle boxes will have amazing deep inspection
firewalls or IPS or DLP (whatever it means, Data Loss, Data Leak, it means
whatever you want to buy)...

There may be a place for those stuff, but it's not on core. Nor on second
level protection layers.

In the end, ISO27002, PCI-DSS, CIA and AAA triads, what are they worth if
you add an IPS to your core? When memory is exausted, processing is starved,
you will have packet loss, latency, or you will be completely offline. And
what's CIA if you "security features" are breaking Integrity due to missing
packets, or breaking full Availability at all? What you have from CIA? Only
confidentiality? Better take that plug off. Same for AAA, if Authorization
and Authentication are broken or failed due to exaustion/starvation what you
get? Accounting/Auditing? So you will sit and read the logs to find out what
went wrong?

Discouraging firewall/IDS/proxy protection layers is as bad as over
leveraging it.

Dosage is what distinguishes the poison from the vaccine.

--
Patrick Tracanelli
FreeBSD Brasil
Tel.: (31) 3516-0800
http://www.freebsdbrasil.com.br
"Long live Hanin Elias, Kim Deal!"





-- 
Ray Patrick Soucy
Network Engineer
University of Maine System

T: 207-561-3526
F: 207-561-3531

MaineREN, Maine's Research and Education Network
www.maineren.net

--
Patrick Tracanelli

FreeBSD Brasil LTDA.
Tel.: (31) 3516-0800
316601 () sip freebsdbrasil com br
http://www.freebsdbrasil.com.br
"Long live Hanin Elias, Kim Deal!"


Current thread: