Snort mailing list archives
Re: [Snort-users] How snort handles several copies of the same packet?
From: Russ Combs <rcombs () sourcefire com>
Date: Fri, 26 Oct 2012 15:23:02 -0400
On Fri, Oct 26, 2012 at 9:48 AM, <elof () sentor se> wrote:
(I'm adding snort-devel to the cc) 1) Perfect! That was the short and simple explaination I was looking for. :-) Thanks. 2) Do you know if it is an easy task to add a counter to count the number of "redundant" packets that are received by snort, but skipped due to the fact that they are considered duplicates and therefore redundant? (I.e. the packets we've been discussing in this thread.) Like adding something simillar to this: Packet I/O Totals: Received: 3716022 Analyzed: 3716022 (100.000%) Dropped: 0 ( 0.000%) Filtered: 0 ( 0.000%) Outstanding: 0 ( 0.000%) Injected: 0 Redundant: 1855011 ( 49.919%) It would be very interesting to have a counter that states how many percent of the incoming packets is actually redundant to snort and is therefore discarded. If one has a really high Redundant count, this indicates that the SPAN should probably be reviewed and reconfigured to get rid of those duplicates. (...or if this is not possible place a smart tap in front of the sensor, and have the tap remove the duplicates) The above example value, almost 50% should indicate that half the mirrored traffic is mirrored twice to the destination port. In general, that is not a trivial task and would probably be performance
prohibitive.
3) Under "Stream5 statistics:", would it be hard to also add a counter for the amount of detected TCP Retransmissions?
That can be done.
If one take the total amount of Redundant packets (see section 2 above) and subtract the amount of detected TCP Retransmissions, one should have an estimate of how much unneccessary traffic snort is actually burdened with. 4) Under "Stream5 statistics:" I see TCP Discards: 12345 TCP Gaps: 2222 I guess that "TCP Gaps" is a counter of missing packets in the SPAN, in an otherwise working tcp flow.
Yes, where missing means data not seen was acked. When Snort is seeing both directions (as it should) and nothing is being dropped, gaps should not happen.
What are "TCP Discards"? Stream5 can discard packets for a variety of reasons, including repeated
SYNs, bad sequence numbers or timestamps, reassembly issues. Many of these can be discovered by enabling Stream5 preprocessor rules.
5) Given the following example counters... Breakdown by protocol (includes rebuilt packets): Eth: 100000 (100%) TCP: 80000 (80%) Packet I/O Totals: Dropped: 1000 (1%) Stream5 statistics: TCP Gaps: 2222 Could I estimate the amount of EXTERNAL drops in the mirrored traffic by doing something like this? : There are 2222 detected TCP gaps. 1% of all traffic is randomly dropped internally on the sensor itself. 1% of 80 000 tcp packets = 800. So roughly 800 of the 2222 tcp gaps are due to the sensor itself. The rest, roughly 1400 gaps, should therefore be EXTERNAL drops. That seems reasonable. However, note that we have an open bug on gap
counting, and I suspect many of those gap counts are bogus. Am I thinking correctly?
/Elof On Wed, 24 Oct 2012, Russ Combs wrote: On Wed, Oct 24, 2012 at 11:35 AM, <elof () sentor se> wrote:Open Source Snort. No global threshold. I'm logging directly to ascii from snort and to unified2 where barnyard2 then take over. In both my ascii alert log and in my postgres I only get one alert. I think this is great! ...but I'm curious and would like to know how and where in the process this filtering/aggregation is done.There are a couple things going on here. Stream5 basically processes data the same way a receiving host would. For example, retransmitted or duplicated data that falls to the left of the window (ie it was already acknowledged) will just be discarded. And Snort will not queue events that have already fired on the same session (or fragment). These non-events are counted and output at shutdown as Limits::Alert: Limits: Match: 0 Queue: 0 Log: 0 Event: 0 Alert: 0 Hope that helps./Elof On Wed, 24 Oct 2012, Joel Esler wrote: Are you talking about Open Source Snort? Or Sourcefire product?Do you have a global threshold in place? On Oct 24, 2012, at 10:42 AM, elof () sentor se wrote:Yes, there's an performance impact. That is expected. But what about the alerting? Somewhere snort must befiltering/aggregating the packets, understanding that the "duplicates"are actually the same packet, and only generate ONE alert for its bad payload data.I'm asking for a description of this part.How does snort detect and filter out these "duplicates"? Which packets are disregarded and which are kept? Like if the packet in my example contain malicious code, will thelogged packet berouting) The first packet with TTL 60? retransmission) The first packet with ipid 3333? duplicate SPAN) Simply the first packet? Another question: Are true duplicates seen as retransmissions andprocessed as such?Perhaps the answer is that the logging system simply detects that thenext received, analyzed and logged packet is the same as the one justlogged, and silently supresses it.I don't think this filtering/aggregation happen this late in theprocess though.Some clarification of how this works would be appreciated./Elof On Wed, 24 Oct 2012, Joel Esler wrote: On Oct 24, 2012, at 4:48 AM, elof () sentor se wrote:I know that snort only generates ONE alert even if the mirroredtrafficsee the same packet twice or more:...like before and after a router: x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60 y:y:y:y:y:y z:z:z:z:z:z 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 59 ^^^^^^^^^^^^^^^^^^^^^^^ ^^ ...or tcp retransmissions: x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60 x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3334, TTL 60 x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3335, TTL 60 ^^^^ ...or two *exact* duplicates of every packet due to faulty SPAN: x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60 x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60 Only having one alert in the above cases is really nice, but I wonder: Can someone describe how this is done and what is happening in snort,bothon the individual packet level, and in stream5?How does snort detect and filter out these "duplicates"? Which packets are disregarded and which are kept?Everything is analyzed independently. I've seen the problem commonlyat many sites. Filtering out the duplicate traffic on a span isimportant for optimum performance.-- Joel Esler Senior Research Engineer, VRT OpenSource Community Manager Sourcefire------------------------------**------------------------------** ------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_**sfd2d_oct<http://p.sf.net/sfu/appdyn_sfd2d_oct> ______________________________**_________________ Snort-users mailing list Snort-users@lists.sourceforge.**net <Snort-users () lists sourceforge net> Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/**lists/listinfo/snort-users<https://lists.sourceforge.net/lists/listinfo/snort-users> Snort-users list archive: http://sourceforge.net/**mailarchive/forum.php?forum_**name=snort-users<http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users> Please visit http://blog.snort.org to stay current on all the latest Snort news!
------------------------------------------------------------------------------ WINDOWS 8 is here. Millions of people. Your app in 30 days. Visit The Windows 8 Center at Sourceforge for all your go to resources. http://windows8center.sourceforge.net/ join-generation-app-and-make-money-coding-fast/
_______________________________________________ Snort-devel mailing list Snort-devel () lists sourceforge net https://lists.sourceforge.net/lists/listinfo/snort-devel Archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel Please visit http://blog.snort.org for the latest news about Snort!
Current thread:
- How snort handles several copies of the same packet? elof (Oct 24)
- Re: How snort handles several copies of the same packet? Joel Esler (Oct 24)
- Re: How snort handles several copies of the same packet? elof (Oct 24)
- Re: How snort handles several copies of the same packet? Joel Esler (Oct 24)
- Re: How snort handles several copies of the same packet? elof (Oct 24)
- Re: How snort handles several copies of the same packet? Russ Combs (Oct 24)
- Re: [Snort-users] How snort handles several copies of the same packet? elof (Oct 26)
- Re: [Snort-users] How snort handles several copies of the same packet? Russ Combs (Oct 26)
- Re: How snort handles several copies of the same packet? elof (Oct 24)
- Re: How snort handles several copies of the same packet? Joel Esler (Oct 24)