IDS mailing list archives
RE: Belaboring the point of FPs (haha!)
From: "Golomb, Gary" <GGolomb () enterasys com>
Date: Tue, 19 Aug 2003 11:45:17 -0400
Yes, this is the perfect platform to spin a sales pitch. I'm surprised it took this long for someone else to jump in the fray with said intentions... However there's a bigger issue here, and not something new either. I remember spouting-off a bit about this a couple of years ago on this list... That is, the QUALITY and INTEGRITY of the signatures being written. No matter who the IDS vendor is (ISS, Sourcefire, Enterasys, or otherwise) and no matter what kind of "restaurant you order at," if the signatures are poorly written, you're going to *increase* your chance of false positives ***AND*** false negatives. In the case below, we see an example of a signature increasing the chance of false positives. The signature is a good idea and a great way to generically detect a compromised system via looking for a Windows shell, but it should definitely be tuned. Banners like this appear at the beginning of a session and payload - not in the middle. Maybe a "Banner Field" protocol decoder can be developed so this problem doesn't affect trons. I could show you just as many examples of bad signatures increasing your chance of getting false negatives because of the same simple mistakes. And these examples are MUCH more interesting! On the flip side, traffic changes, so signature tuning is an evolving process. In addition to that, what might be suspicious in one environment, might not be in another. That's why the ability to fully edit **ALL** signatures from your vendor is imperative. Detection mechanisms are developed around the most common forms of an attack, but depending on the protocols and custom applications in use on your network, there could be an annoying overlap (read: false positive) between a common attack profile, and normal traffic for you. That's why not being able to fully view and edit signatures from your vendor can quickly become a pain. This takes us down a whole new path, but I'm trying to keep this email under 15 pages... Anyways, this is the one thing that cracks me up about industry analysis and testing. There's tons of focus on performance, stability, look and feel, protocol decoding, etc. in IDS tests being published. But what about the reason you buy an IDS in the first place??? Who's *really* testing how well an IDS can detect the attacks it says it will. Not the decoders and the normalizers, I mean the signatures. Surprisingly VERY few, and everyone else just blindly accepts that. My hat's off to Blade Software for raising this issue in the first place. At any rate, whether it's Bob, or Marty, or whoever... If you make mistakes writing signatures, then it's not the IDSs fault - it's the signature's fault. Protocol decoding (like in Bob's example below) and all these other sales pitches we've been getting on this list recently are band-aids for this problem. And think about what it means to higher-layer analysis (ie: correlation) methodologies that need to make decisions based on data that was flagged by bad signatures in the first place... Nice, huh? Just some off-the-cuff thoughts... -gary
-----Original Message----- From: Graham, Robert (ISS Atlanta) [mailto:rgraham () iss net] Sent: Friday, August 15, 2003 2:25 PM To: Paul Schmehl; focus-ids () securityfocus com Subject: RE: Belaboring the point of FPs There is a lot of good points Marty makes about how Snort gives you
the
exact results that you ask for. I don't mean to take this analogy too far, but I'd like to point out
the
stories of genies-in-bottles who give you exactly what you ask for,
but
not quite what you want. We all know the story of king Midas who
wished
that everything he touched turned to gold -- but then realized the
folly
of his ways when he hugged his daughter, which turned her to gold. I'm not saying that this is a BIG problem for Snort, please don't read too much into the analogy. It is indeed a good thing that Snort gives you what you ask for -- I'm just trying to point that it isn't 100% good. The basic problem is that the Snort rules language is not expressive enough to give you what you want. It's like going to a French
restaurant
and ordering only those things on the menu that you can pronounce. I have friends that order bagels at the morning because they are embarrassed by their poor pronunciation of the French word
"croissant".
It's true that the coffee shop is not at fault for delivering what the customer asked for, but it doesn't mean the customer is completely
happy
with his bagel. I write lots of signatures for my IDS (RealSecure 7). I have written a clone of Snort (Trons). Most of the signatures that I write cannot be expressed in the Snort rules language. For example, I put an IMAP protocol-decode on port 143 that explicitly recognizes what an e-mail message is, and therefore won't match any patterns inside it (unless,
of
course, those patterns are supposed to be for e-mail messages). You could certainly extend the Snort rules languages with plugins. The 'uricontent' keyword is a good example of a limitation with pattern-matching that had to be resolved. You could certainly add a plugin for IMAP that resolves the false-positive discussed below, but the issue is that nobody has. Such problems can easily be solved
within
the Snort architecture, it's just that when you get Snort today, such problems are not solved. Again, I'd like to point out that when you use my IDS, you'll get a
set
of signatures that I wanted to give you. You can certainly add your
own
with the Trons feature and other "protocol-field" capabilities we give you, and you can sometimes adjust the signatures, but you DON'T have
the
complete ability (like Snort) to arbitrarily change the signatures
that
I wrote for you. As you can expect, since I wrote the signatures in a specific way, I believe that you'll get what you 'want' better out of
my
IDS than Snort, but it's certainly true that you have less ability to 'ask' my IDS to do something slightly different. -----Original Message----- From: Paul Schmehl [mailto:pauls () utdallas edu] Sent: Monday, August 11, 2003 10:29 PM To: focus-ids () securityfocus com Subject: Belaboring the point of FPs Marty, I'm not picking on you, honest I'm not. I'm sitting here at home, monitoring our DMZ snort, waiting for the RPC worm to hit, and sure enough, I get a hit on sid 2123 - successful admin, cmd.exe. So I think, yep, there's the first box to get infected. Here's rule 2123: alert tcp $HOME_NET !21:23 -> $EXTERNAL_NET any (msg:"ATTACK-RESPONSES Microsoft cmd.exe banner"; flow:from_server,established; content:"Microsoft Windows"; content:"(C) Copyright 1985-"; distance:0;
content:"Microsoft
Corp."; distance:0; reference:nessus,11633;
classtype:successful-admin;
sid:2123; rev:1;) Looks good, but....analysis of the three packets shows very quickly
that
it's an FP. The traffic is *from* our imap server on port 143 *to* an off campus site. Right direction, wrong alert. The payload? A bugtraq post someone was reading about the worm. I recognized it right away,
because
I had just read the same post myself. (No, the off campus address was
not
me.) An anomaly? Not really. I see these *every* time some new exploit shows up. List traffic triggers alerts, because the attack ports are either not specified or by default include mail ports (POP3, IMAP and SMTP). Now surely you will admit *those* are false positives? Here's the payload (yeah, I know, more alerts :( ): 000 : 2A 20 36 39 31 36 20 46 45 54 43 48 20 28 46 4C * 6916 FETCH
(FL
010 : 41 47 53 20 28 5C 53 65 65 6E 29 20 42 4F 44 59 AGS (\Seen)
BODY
020 : 5B 31 5D 20 7B 33 32 32 38 7D 0D 0A 0D 0A 6D 75 [1]
{3228}....mu
030 : 6C 74 69 74 68 72 65 61 64 69 6E 67 20 26 6F 73 ltithreading
&os
040 : 20 64 65 74 65 63 74 69 6F 6E 20 26 26 20 6D 61 detection &&
ma
050 : 63 72 6F 73 20 73 75 70 70 6F 72 74 2E 2E 2E 0D cros
support....
060 : 0A 0D 0A 65 78 70 6C 6F 69 74 20 63 61 6E 20 62 ...exploit can
b
070 : 65 20 66 6F 75 6E 64 20 68 65 72 65 3A 20 20 77 e found here:
w
080 : 77 77 2E 63 72 6F 75 6C 64 65 72 2E 63 6F 6D 2F
ww.croulder.com/
090 : 68 61 78 6F 72 63 69 74 6F 73 2F 6B 61 68 74 32
haxorcitos/kaht2
0a0 : 2E 7A 69 70 0D 0A 0D 0A 0D 0A 65 78 61 6D 70 6C
.zip......exampl
0b0 : 65 3A 20 4B 61 48 54 2E 65 78 65 20 31 30 2E 31 e: KaHT.exe
10.1
0c0 : 30 2E 34 30 2E 30 20 31 30 2E 31 30 2E 32 35 35 0.40.0
10.10.255
0d0 : 2E 32 35 35 20 33 30 30 0D 0A 5F 5F 5F 5F 5F 5F .255
300..______
0e0 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F
________________
0f0 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F
________________
100 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 0D 0A 20 20 20 ___________.. 110 : 20 20 20 20 20 20 20 20 4B 41 48 54 20 49 49 20 KAHT
II
120 : 2D 20 4D 41 53 53 49 56 45 20 52 50 43 20 45 58 - MASSIVE RPC
EX
130 : 50 4C 4F 49 54 0D 0A 20 20 44 43 4F 4D 20 52 50 PLOIT.. DCOM
RP
140 : 43 20 65 78 70 6C 6F 69 74 2E 20 4D 6F 64 69 66 C exploit.
Modif
150 : 69 65 64 20 62 79 20 61 54 34 72 40 33 77 64 65 ied by
aT4r@3wde
160 : 73 69 67 6E 2E 65 73 0D 0A 20 20 23 68 61 78 6F sign.es..
#haxo
170 : 72 63 69 74 6F 73 20 26 26 20 23 6C 6F 63 61 6C rcitos &&
#local
180 : 68 6F 73 74 20 20 40 45 66 6E 65 74 20 4F 77 6E host @Efnet
Own
190 : 7A 20 79 6F 75 21 21 21 0D 0A 5F 5F 5F 5F 5F 5F z
you!!!..______
1a0 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F
________________
1b0 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F
________________
1c0 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 0D 0A 0D 0A 20 5B __________....
[
1d0 : 2B 5D 20 54 61 72 67 65 74 73 3A 20 31 30 2E 31 +] Targets:
10.1
1e0 : 30 2E 34 30 2E 30 2D 31 30 2E 31 30 2E 32 35 35
0.40.0-10.10.255
1f0 : 2E 32 35 35 20 77 69 74 68 20 33 30 30 20 54 68 .255 with 300
Th
200 : 72 65 61 64 73 0D 0A 20 5B 2B 5D 20 53 63 61 6E reads.. [+]
Scan
210 : 20 49 6E 20 50 72 6F 67 72 65 73 73 2E 2E 2E 0D In
Progress....
220 : 0A 2D 20 43 6F 6E 6E 65 63 74 69 6E 67 20 74 6F .- Connecting
to
230 : 20 31 30 2E 31 30 2E 34 30 2E 34 0D 0A 20 20 20 10.10.40.4.. 240 : 53 65 6E 64 69 6E 67 20 45 78 70 6C 6F 69 74 20 Sending
Exploit
250 : 74 6F 20 61 20 5B 57 69 6E 32 6B 5D 20 53 65 72 to a [Win2k]
Ser
260 : 76 65 72 2E 2E 2E 2E 20 46 41 49 4C 45 44 0D 0A ver....
FAILED..
270 : 20 20 2D 20 43 6F 6E 6E 65 63 74 69 6E 67 20 74 - Connecting
t
280 : 6F 20 31 30 2E 31 30 2E 34 30 2E 39 0D 0A 20 20 o 10.10.40.9.. 290 : 20 53 65 6E 64 69 6E 67 20 45 78 70 6C 6F 69 74 Sending
Exploit
2a0 : 20 74 6F 20 61 20 5B 57 69 6E 58 50 5D 20 53 65 to a [WinXP]
Se
2b0 : 72 76 65 72 2E 2E 2E 2E 20 46 41 49 4C 45 44 0D rver....
FAILED.
2c0 : 0A 20 20 2D 20 43 6F 6E 6E 65 63 74 69 6E 67 20 . -
Connecting
2d0 : 74 6F 20 31 30 2E 31 30 2E 34 30 2E 31 32 0D 0A to
10.10.40.12..
2e0 : 20 20 20 53 65 6E 64 69 6E 67 20 45 78 70 6C 6F Sending
Explo
2f0 : 69 74 20 74 6F 20 61 20 5B 57 69 6E 58 50 5D 20 it to a
[WinXP]
300 : 53 65 72 76 65 72 2E 2E 2E 2E 20 46 41 49 4C 45 Server....
FAILE
310 : 44 0D 0A 20 20 2D 20 43 6F 6E 6E 65 63 74 69 6E D.. -
Connectin
320 : 67 20 74 6F 20 31 30 2E 31 30 2E 34 30 2E 32 31 g to
10.10.40.21
330 : 0D 0A 20 20 20 53 65 6E 64 69 6E 67 20 45 78 70 .. Sending
Exp
340 : 6C 6F 69 74 20 74 6F 20 61 20 5B 57 69 6E 58 50 loit to a
[WinXP
350 : 5D 20 53 65 72 76 65 72 2E 2E 2E 0D 0A 20 2D 20 ] Server.....
-
360 : 43 6F 6E 65 63 74 61 6E 64 6F 20 63 6F 6E 20 6C Conectando con
l
370 : 61 20 53 68 65 6C 6C 20 52 65 6D 6F 74 61 2E 2E a Shell
Remota..
380 : 2E 0D 0A 0D 0A 4D 69 63 72 6F 73 6F 66 74 20 57 .....Microsoft
W
390 : 69 6E 64 6F 77 73 20 58 50 20 5B 56 65 72 73 69 indows XP
[Versi
3a0 : 3D 46 33 6E 20 35 2E 31 2E 32 36 30 30 5D 0D 0A =F3n
5.1.2600]..
3b0 : 28 43 29 20 43 6F 70 79 72 69 67 68 74 20 31 39 (C) Copyright
19
3c0 : 38 35 2D 32 30 30 31 20 4D 69 63 72 6F 73 6F 66 85-2001
Microsof
3d0 : 74 20 43 6F 72 70 2E 0D 0A 0D 0A 43 3A 5C 57 49 t
Corp.....C:\WI
3e0 : 4E 44 4F 57 53 5C 73 79 73 74 65 6D 33 32 3E 2E
NDOWS\system32>.
3f0 : 0D 0A 20 2D 20 43 6F 6E 6E 65 63 74 69 6F 6E 20 .. -
Connection
400 : 43 6C 6F 73 65 64 0D 0A 20 2D 20 43 6F 6E 6E 65 Closed.. -
Conne
410 : 63 74 69 6E 67 20 74 6F 20 31 30 2E 31 30 2E 34 cting to
10.10.4
420 : 30 2E 35 32 0D 0A 20 20 20 53 65 6E 64 69 6E 67 0.52..
Sending
430 : 20 45 78 70 6C 6F 69 74 20 74 6F 20 61 20 5B 57 Exploit to a
[W
440 : 69 6E 58 50 5D 20 53 65 72 76 65 72 2E 2E 2E 20 inXP]
Server...
450 : 46 41 49 4C 45 44 0D 0A 20 2E 20 2D 20 43 6F 6E FAILED.. . -
Con
460 : 6E 65 63 74 69 6E 67 20 74 6F 20 31 30 2E 31 30 necting to
10.10
470 : 2E 34 30 2E 35 30 0D 0A 20 20 20 53 65 6E 64 69 .40.50..
Sendi
480 : 6E 67 20 45 78 70 6C 6F 69 74 20 74 6F 20 61 20 ng Exploit to
a
490 : 5B 57 69 6E 32 6B 5D 20 53 65 72 76 65 72 2E 2E [Win2k]
Server..
4a0 : 2E 0D 0A 20 2D 20 43 6F 6E 65 63 74 61 6E 64 6F ... -
Conectando
4b0 : 20 63 6F 6E 20 6C 61 20 53 68 65 6C 6C 20 52 65 con la Shell
Re
4c0 : 6D 6F 74 61 2E 2E 2E 0D 0A 0D 0A 4D 69 63 72 6F
mota.......Micro
4d0 : 73 6F 66 74 20 57 69 6E 64 6F 77 73 20 32 30 30 soft Windows
200
4e0 : 30 20 5B 56 65 72 73 69 3D 46 33 6E 20 35 2E 30 0 [Versi=F3n
5.0
4f0 : 30 2E 32 31 39 35 5D 0D 0A 28 43 29 20 43 6F 70 0.2195]..(C)
Cop
500 : 79 72 69 67 68 74 20 31 39 38 35 2D 32 30 30 30 yright
1985-2000
510 : 20 4D 69 63 72 6F 73 6F 66 74 20 43 6F 72 70 2E Microsoft
Corp.
520 : 0D 0A 0D 0A 43 3A 5C 57 49 4E 4E 54 5C 73 79 73
....C:\WINNT\sys
530 : 74 65 6D 33 32 3E 65 78 69 74 0D 0A 0D 0A 20 2D tem32>exit....
-
540 : 20 43 6F 6E 6E 65 63 74 69 6F 6E 20 43 6C 6F 73 Connection
Clos
550 : 65 64 0D 0A 20 2D 20 43 ed.. - C Paul Schmehl (pauls () utdallas edu) Adjunct Information Security Officer The University of Texas at Dallas AVIEN Founding Member http://www.utdallas.edu
--------------------------------------------------------------------------- Attend Black Hat Briefings & Training Federal, September 29-30 (Training), October 1-2 (Briefings) in Tysons Corner, VA; the worldÂs premier technical IT security event. Modeled after the famous Black Hat event in Las Vegas! 6 tracks, 12 training sessions, top speakers and sponsors. Symanetc is the Diamond sponsor. Early-bird registration ends September 6 Visit: www.blackhat.com ---------------------------------------------------------------------------
Current thread:
- RE: Belaboring the point of FPs (haha!) Golomb, Gary (Aug 21)