Re: Important Comments re: INtrusion Detection

[tqbf () secnet com]

To: Kurt Ziegler, AbirNet Inc. 
Regarding: Your response to "Re: Important comments re: ID"

Mr. Ziegler, before I address the contents of your response to my 
message on Firewall-Wizards, I'd like to thank you for taking the time
to publically address the issues we're bringing forth. Given the 
fact that this discussion so directly affects your organization's 
product, I can appreciate that this is a delicate and somewhat dangerous
debate to step into.

That said, allow me to get the non-technical issues out of the way
before I drown myself in acronyms. 

You state that SecNet is being "self-serving" both in our paper
(particularly the introduction, in which we state that the basic
technology used by your product is flawed), and in our message to security
lists regarding that paper. While I can understand where you might get
that impression, I find it necessary to clarify something.

My company is not a vendor of intrusion detection systems. I am aware of
no current plans within my organization to start selling intrusion
detection systems. Most importantly, even though I've sent many, many 
messages to this and other lists explaining why I think the proxy approach
is better than the passive analysis approach, neither me nor any employee
at SecNet has any financial interest in seeing a proxy IDS deployed.

I like intrusion detection systems. I think the technology and research
work that goes into building a system that can detect attacks against
computers is fascinating. I have had an interest in intrusion detection
for many years, and have the greatest possible respect for ID research. 
One thing I am specifically not trying to do is to prevent intrusion 
detection systems from being deployed.

I do not think ID systems and security scanners are competitive
technologies. I think they serve two very different purposes, and are
appropriate in different circumstances. Since my company currently sells
only a static configuration analysis security scanning tool, I cannot
see how we serve any internal interest at all in pointing out limitations
in intrusion detection systems. However, our primary competitor does sell
the current market-leading intrusion detection system, and thus I can only
ask that you take my word for it when I say that our intensions behind
publishing this research were not self-serving in this respect.

The next claim I feel the need to address is that we're being
"sensationalistic" in our press releases, messages, and publications by
stating that passive network ID has "fundemental" flaws. I do not agree
with this in the least, Mr. Ziegler.

Both my reputation and the reputation of Secure Networks Inc. stand behind
each and every claim we have made about your intrusion detection
technology. We published a 60 page technical report explaining why we work
from the perspective that passive network ID is flawed. It would waste the
time of most of the readers of this list for me to explain over again why
I believe we're justified in making the claims that we do; I will thus
restrict myself to addressing the specific points you raise in your 
message. 

Now, for the technical points.

Excuse me (and correct me) if I'm not summarizing your points accurately
or if I'm omitting anything significant.

Point 1: Proxy ID is limited in that it cannot detect attacks in traffic 
         that does not flow through the monitor.

Of course, I agree with this point. I cite my messages to firewall-wizards
as an example of how much I agree with. I think you have misunderstood the
intended meaning of my messages with respect to the word "unobtrusive"
(your response complains that our statement that ID is not unobtrusive).

We are not saying that passive ID is obtrusive. When we discuss ID systems
ceasing to be unobtrusive, we refer to our own proposed fixes to the
problems, and, particularly, to our proposal of proxy "active" network
intrusion detection as an alternative to the flawed passive approach. I
believe we have acknowledged repeatedly the (obvious) point that active 
network ID cannot detect attacks in traffic that doesn't pass through the
proxy.

Point 2: Proxy ID is limited in that it adds a bottleneck to the network.

Again, this is a point we have acknowledged repeatedly on this list; in
fact, I believe we were the ones who initially brought this fact up. We
have continually stated that proxy ID has its share of problems, and have
acknowledged that we do not have all the answers. The question we're
saddling the readers of this list is simple:

Given the choice between unreliable passive network ID and reliable active
proxy ID, do you want to trade security for speed?

Point 3: Intrusion detection systems detect anomalies, so if we can
         send an alarm whenever we see odd traffic, we will have solved
         the problem.

First off, intrusion detection systems do not detect anomalies. ID systems
detect misuse of computing resources. Many of the problems detected by
SessionWall-3 are not anomolous at the TCP and IP protocol level; for
instance, a "phf" attack can be carried out in a stream of packets that is
completely normal looking from the headers.

A subset of ID systems detect misuse of computing resources by looking for
use of the system that is "out of the ordinary". A classification of what
"out of the ordinary" is can be made via many different techniques. Some
systems use statistical analysis, some use graph theory, and some use
neural networks. 

Another subset of ID systems relies on what ID researchers call "misuse
detection". In "misuse detection", the system does not detect statistical
abnormality --- instead, it looks for specific patterns of traffic that
indicate known attacks. Many of these systems, including yours, use
pattern matching to detect "signatures" of attack, such as the "phf" in an
HTTP request.

One possible way to address the problems we mention is to apply your
misuse-detection methods to IP and TCP, and detect specifically attacks
against the ID systems. In other words, you could view the problems we
point out as nothing more than a new set of attack signatures that need to
be added to your ID system.

This is a valid approach. If you do this reliably, you will maintain the
ability to detect all attacks on the network. However, you will not have
regained the ability to detect specific types of attacks; rather, you will
be reduced to detecting only attacks on the IDS. These attacks can hide
more significant ones.

Point 4: Since passive ID systems "see the same packets" as active
         active systems, there can be no difference in analysis 
         capability.

This is a point we disagree strongly about. 

Before I continue to address this claim, I'd like to make very clear my
organization's repeated claims about the validity of passive network ID.
The specific claim we are making with regards to the "fundemental flaws"
of passive ID is that it cannot be a reliable technology for analyzing 
traffic for specific patterns of attacks unless systems employing it are
equipped with secondary sources of information that resolve ambiguities in
the meaning of network traffic. 

When I refer to "specific patterns of attack", I refer to the marketing
claims of IDS vendors which state that their systems are capable of
picking out, say, a "phf" in a stream of HTTP traffic, or a wu-ftpd signal
race attempt in FTP traffic. Our public claims regarding intrusion
detection have continually acknowledged the fact that we do believe that
ID systems are capable of detecting suspicious traffic. What we do not
believe is that an ID system such as SessionWall-3 is capable of detecting
specific attacks against endpoints.

The issue of the benefits of proxy ID over passive ID is useful because it
underscores the basic problems we're identifying in passive ID. We are not
saying that proxy ID is the only solution to these problems; however,
pointing to an alternative technology that addresses the vulnerabilities
in your session helps us make our points.

That said, let's get the basic source of our disagreement out of the way:

A proxy IDS and a passive IDS dealing with the same network traffic do in
fact see the exact same packets on the wire. You are correct in pointing
this out.

Our contention is that a given stream of packets has different meaning,
at the TCP and IP protocol levels of analysis, depending on several
conditions that are not measureable by the traffic alone. These conditions
include:

        - destination host operating system type and revision
        - destination host configuration
        - number of hops to destination
        - MTU of each hop to destination

The reason that we claim this information is critical to accurate protocol
analysis and session reconstruction is simple. Two different operating
systems may process and reconstruct the same stream of traffic
differently. For instance, Windows NT 4.0 and FreeBSD 2.2 will reconstruct
two entirely different IP datagrams given the exact same stream of IP
fragments. A passive ID system cannot possibly know which reassembly is
correct without either knowing the operating system is running at the
destination or analyzing every possible interpretation of those fragments.

Another example involves TCP stream reconstruction. A TCP stream is ended
by an RST packet. The TCP RST message indicates to the recipient that the
other end of the connection has closed the connection and is no longer
using the parameters of the connection (the TCP service selectors - source
and destination port) for that particular connection.

We assert that it is vital for an ID system to correctly interpret RST
messages. This is because the parameters of a connection can be re-used
once the connection is closed to communicate streams of packets with
entirely different sequencing. An ID system (such as RealSecure in its
most recent public release) that ignores RST messages can be
"desynchronized" from a stream of traffic it ignores RST messages; if I
can silently tear down a connection and create a valid new one without the
IDS knowing, the data in the new connection will appear to have bad
sequence numbers to the ID system. 

SessionWall does not ignore RST messages. This is a good thing. Now, think
on this: an RST message is only valid when the sequence numbers in the
segment are valid. If SessionWall accepted RST messages with invalid
sequence numbers, it would incorrectly "drop" the connection it was
monitoring. 

Unless it was trying to monitor a 4.4BSD system. Most deployed versions of
FreeBSD, NetBSD, and OpenBSD have a bug, initially identified by Darren
Reed (I believe), that causes them to accept RST messages that have
invalid sequence numbers. 

See the problem here? If you process the stream of packets "correctly",
reconstructing a TCP session as indicated by the RFC, you will allow an
attacker to silently attack all the 4.4BSD machines on your network. If
you duplicate the 4.4BSD bug, you will allow an attacker to silently
attack systems without this bug. If you simply issue an alarm when you
detect an unsequenced RST, you're detecting an attack, but not the
specific one that is actually occuring. Your only real option is to
attempt to duplicate the TCP/IP driver bugs of every system you could ever
want to monitor.

This is, to say the least, a difficult task. We don't believe that it is
computationally feasable in a real-time passive ID system. 

This is one of the "fundemental problems" we are identifying in your
technology. Packets on a network have ambiguous meaning. There is, we
believe, no feasable solution for this problem that doesn't involve adding
a secondary source of information to your system that identifies the
correct OS version of every system on the network.

The basic problem here is that the ID system and the end-systems it
watches can perform "protocol analysis" inconsistantly. This is due not
only to bugs in the ID system, which you CAN fix, but to bugs in ALL of
the end-systems, which you CANNOT fix. Even if your system obeyed the
RFCs to the letter, you would not have fixed the problem. 

How does a proxy solve this problem? Good question. A proxy is not a
passive monitor. It is an ENDPOINT. Proxies do not simply forward packets
from one interface to the other. They perform protocol analysis, and, in
effect, forward the RESULTS OF THIS ANALYSIS (NOT the actual packets!) to
it's intended destination. 

In other words, a properly implemented proxy cannot allow an end-system to
see traffic that it does not understand (at least at the IP and TCP
layers). When confronted with a stream of overlapping fragments, it
reconstruct the datagram it believes is being sent, and sends the
reconstructed datagram (NOT the fragments themselves) to the end-system.
When sent an unsequenced RST, the proxy ignores it, and doesn't forward it
to the end-system. 

The traffic forwarded by an IDS proxy is, at the IP and TCP levels,
innocuous and easily understood by both the IDS and the end-system. 

A passive ID system can't do this. It doesn't control the traffic that is
being sent to the end-system. A proxy dictates the meaning of a series of
packets: it allows the end-system to see only traffic that it understands.
A passive ID system must guess as to the meaning of traffic. 

Now, your message states that passive ID systems and proxy systems detect
the exact same abnormalities, and that proxy systems cannot detect more
than a passive system. You're right. The difference is not that a proxy
can detect MORE than a passive system. It's that it does not allow traffic
that contains attacks that AREN'T recognized by the system to make it to
the end-system. 

In passive systems, there is a great potential for an attacker to slip an
attack past the monitor. In an active system, there is none, because only
traffic that is understood by the proxy ever makes it past the proxy. 

Point 5: Passive systems are chosen over proxy systems because they
         do not bottleneck the network and are unobtrusive

Passive systems are chosen over proxy systems because there are, to my
knowledge, no commercially available proxy ID systems. 

-----------------------------------------------------------------------------
Thomas H. Ptacek                                        Secure Networks, Inc.
-----------------------------------------------------------------------------
http://www.enteract.com/~tqbf                           "mmm... sacrilicious"