Snort mailing list archives

Previous By Date Next
Previous By Thread Next

Re: [Snort-users] How snort handles several copies of the same packet?

From: Russ Combs <rcombs () sourcefire com>
Date: Fri, 26 Oct 2012 15:23:02 -0400
On Fri, Oct 26, 2012 at 9:48 AM, <elof () sentor se> wrote:


(I'm adding snort-devel to the cc)


1)
Perfect! That was the short and simple explaination I was looking for. :-)
Thanks.


2)
Do you know if it is an easy task to add a counter to count the number of
"redundant" packets that are received by snort, but skipped due to the fact
that they are considered duplicates and therefore redundant?
(I.e. the packets we've been discussing in this thread.)

Like adding something simillar to this:

Packet I/O Totals:
  Received:    3716022
  Analyzed:    3716022 (100.000%)
  Dropped:     0 ( 0.000%)
  Filtered:    0 ( 0.000%)
  Outstanding: 0 ( 0.000%)
  Injected:    0
  Redundant:   1855011 ( 49.919%)

It would be very interesting to have a counter that states how many
percent of the incoming packets is actually redundant to snort and is
therefore discarded.
If one has a really high Redundant count, this indicates that the SPAN
should probably be reviewed and reconfigured to get rid of those duplicates.
(...or if this is not possible place a smart tap in front of the sensor,
and have the tap remove the duplicates)
The above example value, almost 50% should indicate that half the mirrored
traffic is mirrored twice to the destination port.

In general, that is not a trivial task and would probably be performance

prohibitive.



3)
Under "Stream5 statistics:", would it be hard to also add a counter for
the amount of detected TCP Retransmissions?



That can be done.



If one take the total amount of Redundant packets (see section 2 above)
and subtract the amount of detected TCP Retransmissions, one should have an
estimate of how much unneccessary traffic snort is actually burdened with.


4)
Under "Stream5 statistics:" I see
  TCP Discards: 12345
  TCP Gaps:     2222

I guess that "TCP Gaps" is a counter of missing packets in the SPAN, in an
otherwise working tcp flow.



Yes, where missing means data not seen was acked.  When Snort is seeing
both directions (as it should) and nothing is being dropped, gaps should
not happen.



What are "TCP Discards"?

Stream5 can discard packets for a variety of reasons, including repeated

SYNs, bad sequence numbers or timestamps, reassembly issues.  Many of these
can be discovered by enabling Stream5 preprocessor rules.



5)
Given the following example counters...

Breakdown by protocol (includes rebuilt packets):
  Eth:       100000 (100%)
  TCP:       80000  (80%)
Packet I/O Totals:
  Dropped:   1000   (1%)
Stream5 statistics:
  TCP Gaps:  2222

Could I estimate the amount of EXTERNAL drops in the mirrored traffic by
doing something like this? :

There are 2222 detected TCP gaps.
1% of all traffic is randomly dropped internally on the sensor itself.
1% of 80 000 tcp packets = 800.
So roughly 800 of the 2222 tcp gaps are due to the sensor itself.
The rest, roughly 1400 gaps, should therefore be EXTERNAL drops.

That seems reasonable.  However, note that we have an open bug on gap

counting, and I suspect many of those gap counts are bogus.

Am I thinking correctly?


/Elof



On Wed, 24 Oct 2012, Russ Combs wrote:

 On Wed, Oct 24, 2012 at 11:35 AM, <elof () sentor se> wrote:




Open Source Snort.
No global threshold.

I'm logging directly to ascii from snort and to unified2 where barnyard2
then take over.
In both my ascii alert log and in my postgres I only get one alert.

I think this is great! ...but I'm curious and would like to know how and
where in the process this filtering/aggregation is done.



There are a couple things going on here.  Stream5 basically processes data
the same way a receiving host would.  For example, retransmitted or
duplicated data that falls to the left of the window (ie it was already
acknowledged) will just be discarded.

And Snort will not queue events that have already fired on the same
session
(or fragment).  These non-events are counted and output at shutdown as
Limits::Alert:

Limits:
     Match:            0
     Queue:            0
       Log:            0
     Event:            0
     Alert:            0

Hope that helps.



/Elof


On Wed, 24 Oct 2012, Joel Esler wrote:

 Are you talking about Open Source Snort?  Or Sourcefire product?


Do you have a global threshold in place?


On Oct 24, 2012, at 10:42 AM, elof () sentor se wrote:



Yes, there's an performance impact. That is expected.

But what about the alerting? Somewhere snort must be


filtering/aggregating the packets, understanding that the "duplicates"

are
actually the same packet, and only generate ONE alert for its bad payload
data.


I'm asking for a description of this part.


How does snort detect and filter out these "duplicates"?
Which packets are disregarded and which are kept?

Like if the packet in my example contain malicious code, will the


logged packet be





routing)
The first packet with TTL 60?

retransmission)
The first packet with ipid 3333?

duplicate SPAN)
Simply the first packet?
Another question: Are true duplicates seen as retransmissions and


processed as such?







Perhaps the answer is that the logging system simply detects that the


next received, analyzed and logged packet is the same as the one just

logged, and silently supresses it.


I don't think this filtering/aggregation happen this late in the



process though.



Some clarification of how this works would be appreciated.


/Elof


On Wed, 24 Oct 2012, Joel Esler wrote:

 On Oct 24, 2012, at 4:48 AM, elof () sentor se wrote:



I know that snort only generates ONE alert even if the mirrored


traffic



see the same packet twice or more:


...like before and after a router:
x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60
y:y:y:y:y:y z:z:z:z:z:z 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 59
^^^^^^^^^^^^^^^^^^^^^^^                                           ^^

...or tcp retransmissions:
x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60
x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3334, TTL 60
x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3335, TTL 60
                                                       ^^^^

...or two *exact* duplicates of every packet due to faulty SPAN:
x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60
x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60


Only having one alert in the above cases is really nice, but I
wonder:

Can someone describe how this is done and what is happening in snort,


both



on the individual packet level, and in stream5?


How does snort detect and filter out these "duplicates"?
Which packets are disregarded and which are kept?



Everything is analyzed independently.  I've seen the problem commonly


at many sites.  Filtering out the duplicate traffic on a span is

important
for optimum performance.




--
Joel Esler
Senior Research Engineer, VRT
OpenSource Community Manager
Sourcefire







------------------------------**------------------------------**
------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_**sfd2d_oct<http://p.sf.net/sfu/appdyn_sfd2d_oct>
______________________________**_________________
Snort-users mailing list
Snort-users@lists.sourceforge.**net <Snort-users () lists sourceforge net>
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/**lists/listinfo/snort-users<https://lists.sourceforge.net/lists/listinfo/snort-users>
Snort-users list archive:
http://sourceforge.net/**mailarchive/forum.php?forum_**name=snort-users<http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users>

Please visit http://blog.snort.org to stay current on all the latest
Snort news!





------------------------------------------------------------------------------
WINDOWS 8 is here. 
Millions of people.  Your app in 30 days.
Visit The Windows 8 Center at Sourceforge for all your go to resources.
http://windows8center.sourceforge.net/
join-generation-app-and-make-money-coding-fast/
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!
Previous By Date Next
Previous By Thread Next

Current thread: