Re: Issues with the Snort Manual (Patch)

[<Joshua.Kinard () us-cert gov>]

Hi Ryan,

Thank you very much for the below explanations!  They shed a lot of light on things.

> > - session: What is the purpose or function of the 'binary' parameter?
[snip]
> Since rule options are run on *both* individual and stream-reassembled
> packets, this rule option will end up logging data twice into the same
> file. Data may also appear out-of-order.

This sounds like something that can be boiled down into one of those notification blocks for the session keyword, so 
that those using it are aware of this.


> > - General: Is a space after a colon in a Snort rule option the 
> > preferred format, or is no space preferred?  Snort parses either-or, 
> > but as I mentioned, the main SourceFire product has choked on 
> > instances of these (specifically, importing a text file with a rule 
> > whose "sid" parameter contains such a space).
>
> First off, Snort by itself will parse the rule regardless of spaces
> after the colon. I have filed a bug with our UI team to get your import
> issue cleared up.
>
> In practice, all of our rules get written without the space. Thanks for
> changing the manual to reflect that.

Sounds good to me.  The patch only addresses rule examples in the rule options section of the manual, as that's what 
I'm more knowledgeable on.  I didn't want to tread on doing similar for the various configuration directives, because 
even though they can resemble a rule option syntaxically, they're much more flexible.  Might be worth a review for 
consistency.


> > - General: Are spaces between parameters in a rule option also the 
> > preferred format, or are no spaces preferred?  Similar to the last 
> > question.  I haven't noticed SourceFire mistreating options formatted 
> > as such, but the possibility probably exists.
>
> I find rules to be more readable if there are spaces in between the rule
> options. Those spaces have been kept in the manual to make it easier to
> read. VRT tends to leave out those spaces for compactness, especially if
> those parameters are short numbers.
>
> For example:
> byte_test:4,>,200,36;    vs.    byte_test:4, >, 200, 36;
>
> This is just a matter of personal style; Snort should parse it both ways.
> For documentation purposes, we'll use the spaces from here on out.

This makes sense, thanks for the clarification.


> > I had an additional thought over the weekend, that is not addressed in 
> > the manual nor reflected in my patch.  The manual states that HTTP 
> > content modifiers for pcre (U, I, P, H, D, M, C, K, S, & Y) cannot be 
> > used in combination with the 'R' relative modifier (or 'B' rawbytes).
> > Content, however, disallows use of the 'rawbytes' option with the HTTP 
> > modifiers, but speaks nothing of the two relative modifiers, 'distance'
> > or 'within'.
[snip]
> I'll combine this into one explanation:
>
> The HTTP Inspect preprocessor breaks up an HTTP request into multiple
> distinct buffers. Those "http_cookie", "http_method", and other http
> modifiers specify which buffer to use when looking for that content.
> "rawbytes" means to use the original payload buffer, not any of these
> normalized buffers.
>
> When doing one content match relative to another, both contents have to
> be in the same buffer. You can combine distance/within with an HTTP modifier
> if the same HTTP modifier is used for both contents. If the contents are
> in different buffers, the match will fail. "rawbytes" cannot be used
> with an HTTP modifier because that would specify two buffers for the
> same content.
>
> For the same reason, distance/within are allowed with uricontent, but
> only work when used in conjunction with another uricontent. You can't
> have a uricontent match relative to an http_method content, for example.*

This is extremely useful information.  Possible to distill this down into a format for inclusion in the Snort manual?  
Might also be worth for the SourceFire documentation team to review as well for inclusion in the next Analyst Guide 
update.

Has there been any thought given to organizing the various content modifiers a bit differently?  Rough brainstorming 
suggests an organization based on modifier function/buffer type:

Standard modifiers:
nocase, rawbytes

Positional modifiers:
depth, offset

Relative modifiers:
distance, within

Normalized HTTP Buffer modifiers:
http_client_body, http_cookie, http_header, http_method, http_uri, http_stat_code, http_stat_msg

Unnormalized HTTP Buffer modifiers:
http_raw_cookie, http_raw_header, http_raw_uri

Detection Engine modifier(s):
fast_pattern


I also think mentioning, in reference to content and its family, that rule options with a "relative" type parameter, 
act as a modifier (of sorts), and provide PDF anchor links (or whatever they're called in LaTeX) to jump to that 
specific rule option's explanation in the manual.

Conditional modifiers (that test a condition in a relative buffer):
byte_test, isdataat, pcre

Operational modifiers (that take action on data in a relative buffer):
asn1, base64_decode, byte_extract, byte_jump




On the same/similar topic, "uricontent" could use some fleshing out.  Its explanation in Snort is vague and I think is 
a source of confusion.  It doesn't even exist in SourceFire (one mention in the manual, in a rule example only, but not 
available in the UI), since it's just an anagram for content with an http_uri modifier.

Probably the most important/confusing line is this:
> For a description of the parameters to this function, see the content rule options in Section 3.5.1.

I think that can lead people to think that all (or most) of content's modifiers are available to uricontent.  While a 
mention is made about uricontent and rawbytes as being incompatible, uricontent and http_stat_code would equally be 
incompatible, since they are modifiers that point to different buffers.

Based on your explanation, I can draw the following conclusion regarding modifier compatibility with uricontent:

Always valid:
nocase, depth, offset

ONLY valid IF relative to an _existing_ uricontent:
distance, within

NOT valid:
http_client_body, http_cookie, http_raw_cookie, http_header, http_raw_header, http_method, http_uri, http_raw_uri, 
http_stat_code, http_stat_msg, & rawbytes.

Uncertain about fast_pattern.  I've seen examples of 'content:"/foobar"; http_uri; fast_pattern;', which would imply 
that 'uricontent:"foobar"; fast_pattern;' is valid.  Do any of fast_pattern's parameters affect (or are incompatible 
with) the operation of uricontent?

And what about the other options that contain a "relative" type parameter (but are not necessarily chained to 
content/uricontent)?  Is that relative to the primary pattern buffer (i.e., content w/ rawbytes), or the last used 
buffer (such as pointed at by http_uri/uricontent)?


Almost in a sense, a dependency graph/chart/matrix would be useful to describe the compatible/incompatible mix of these 
modifiers/rule options.  Snort doesn't look to have the facilities to test every possible combination and toss back an 
error, so enshrining it in the manual might be worthwhile so users don't inadvently write bad or 
ineffective/inefficient rules.  The fast_pattern option especially deserves its own compatibility matrix just about.




Lastly, an an issue raised by someone on my internal team today stated that Snort will ignore the "flow" option when 
applied to UDP streams.  I knew this was inaccurate, however, the explanation in the Snort manual for the "flow" option 
doesn't address UDP at all.  The SourceFire manual DOES address this on Page 1035 (System Analyst Guide, 4.9.1):

>       UDP stream preprocessing occurs when the rules engine processes packets
>       against a UDP rule that includes the flow keyword using any of the following
>       arguments:
>               * Established
>               * To Client
>               * From Client
>               * To Server
>               * From Server
>
>       UDP is a connectionless protocol that does not provide a means for two
>       endpoints to establish a communication channel, exchange data, and close the
>       channel. UDP data streams are not typically thought of in terms of sessions.
>
>       However, the stream preprocessor uses the source and destination IP address
>       fields in the encapsulating IP datagram header and the port fields in the UDP
>       header to determine the direction of flow and identify a session. A session ends
>       when a configurable timer is exceeded, or when either endpoint receives an
>       ICMP message that the other endpoint is unreachable or the requested service is
>       unavailable.

Adding this to flow's description (in some form) would be really helpful, especially highlighting which flow parameters 
are valid for TCP vs. UDP.  Maybe even ICMP, since it has a stream5 configuration directive?


Thanks!,

--J

------------------------------------------------------------------------------
This SF Dev2Dev email is sponsored by:

WikiLeaks The End of the Free Internet
http://p.sf.net/sfu/therealnews-com
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel