Re: Fast-pattern matcher does not honor ignore_data in the SMTP Preprocessor

[Joel Esler <jesler () sourcefire com>]

It's currently slated for 2.9.3, yes.  

As for your suggestions, they are good.  We currently are doing a complete revamp on a couple critical pieces of Snort, 
(smtp not being one at this time).  But we'll take a look at this.

J

On Feb 21, 2012, at 8:22 PM, Joshua Kinard wrote:

> On 11/28/2011 08:24, Joel Esler wrote:
>
> > This has been entered into the system as well, just FYI.  It's been a crazy week around here with trying to get 
> > 2.9.2 out and the Holiday.
> >
> > Joel
> >
> > On Nov 20, 2011, at 9:05 PM, Joshua Kinard wrote:
> >
> > > Hi snort-devel,
> > >
> > > I discovered a bug between the SMTP preprocessor and the fast_pattern
> > > matcher when 'ignore_data' is turned on.  The issue stems from using a a
> > > rule that has only a single content match with a pattern found only in the
> > > e-mail body.  It seems that the fast-pattern matcher will find this pattern
> > > in the overall packet and flag an event before the SMTP preprocessor has a
> > > chance to fully normalize/decode the packet and apply the 'ignore_data' option.
> > >
> > > I've attached an SMTP example session from twin Postfix MTAs running on a
> > > Linux VM (over IANA reserved IPs on loopback, all local).  The sender is
> > > "jack" from host "formenos.aman.arda" to "jill" at "tirion.aman.arda" (props
> > > to anyone who gets the references).  The message body uses text generated
> > > from the great Lorem Ipsum generator.  The file attachment is 768K of
> > > /dev/urandom data to simulate a bog-standard SMTP session of message + MIME
> > > attachment.
> > >
> > > This is my sample SMTP Preprocessor config on snort-2.9.2.  Assume I have
> > > the other preprocessors set to sane defaults:
> > >
> > > preprocessor smtp:                  \
> > >       normalize all               \
> > >       inspection_type stateful    \
> > >       ports { 25 587 }            \
> > >       ignore_data                 \
> > >       ignore_tls_data             \
> > >       no_alerts                   \
> > >       b64_decode_depth 0          \
> > >       max_mime_mem 2097152
> > >
> > >
> > > Given the above, this rule below WILL alert on the specified pattern (found
> > > in the third Lorem Ipsum paragraph, last sentence), regardless of the
> > > 'ignore_data' directive:
> > >
> > > alert tcp any any -> any 25 (                    \
> > >       msg:"Lorem Ipsum?";                      \
> > >       flow:established,to_server;              \
> > >       content:"sollicitudin"; nocase;          \
> > >       sid:42000009; rev:1; priority:1;         \
> > >       classtype:bad-unknown;                   \
> > > )
> > >
> > > This is because the fast-pattern matcher finds "sollicitudin" in the
> > > unreassembled packets.  I believe the SMTP preprocessor has some done kind
> > > of processing by this point (I stepped through the code using gdb/ddd, and
> > > the alert happens after the 8th TCP packet), but I guess a check somewhere
> > > misses or fast-pattern simply doesn't handle the 'ignore_data' directive
> > > properly.
> > >
> > >
> > > A workaround I found was to short-circuit the fast-pattern matcher with a
> > > content match you ALWAYS expect to be in an SMTP packet, like "Received:":
> > >
> > > alert tcp any any -> any 25 (                    \
> > >       msg:"Lorem Ipsum?";                      \
> > >       flow:established,to_server;              \
> > >       content:"Received|3a|"; fast_pattern;    \
> > >       content:"sollicitudin"; nocase;          \
> > >       sid:42000009; rev:1; priority:1;         \
> > >       classtype:bad-unknown;                   \
> > > )
> > >
> > > With this, we short-circuit fast-pattern because it will find "Received:",
> > > then the SMTP preprocessor does what it does, and the second content match
> > > fails as expected because we want to ignore the DATA portion of the e-mail.
> > >
> > > It's a hack, though, and you're giving up performance by the short-circuit.
> > > I honestly don't know a good way around this with the current design of the
> > > SMTP preprocessor.  Other dynamic preprocessors, like SIP, DCE, the new GTP
> > > ones, all provide functions to tell the engine where to look.  My guess is
> > > an equivalent 'smtp_header' and 'smtp_body' set of rule options might be
> > > needed (while 'file_data' continues to point at decoded MIME data), so that
> > > the fast-pattern matcher can be pointed to the appropriate decode buffer and
> > > not generate potential false positives.
>
>
> Curious to know if there's been any updates on this bug.  Is this slated to
> be fixed in 2.9.3?  Or has any further thought been given to crating
> 'smtp_header' and 'smtp_body' rule options to better control the
> fast-pattern matcher?
>
> I've looked at the code in the SMTP preprocessor and isolated the specific
> checks that indicate why fast-pattern doesn't honor ignore_data.  I suspect
> that, basing off of the new modbus_data and dnp3_data options, replacing the
> bits for ignore_data with instructions to copy the SMTP header to one decode
> buffer and the body to another for use within in a rule might not be too
> much of an issue if I get some free time.  Any tips on other code points to
> look at (outside of adding the actual rule option hooks and such)?
>
> Example -- we avoid the fast-pattern bug and positional checks:
> alert tcp any any -> any 25 (msg:"Spam E-mail from Bob";
> flow:established,to_server; smtp_header; content:"From|3a 20|Bob Bob
> <bob () bob com>"; smtp_body;
> content:"http|3a|//clicktracker.bob.com/spam?id="; sid:xxxxxxxx; rev:1;
> classtype:policy-violation; )
>
> Arguably, having these two options would make the SMTP Preprocessor far more
> useful.  They would move the doe_ptr, so they would go before a content, not
> after.  Even better would be creating buffers for From, To, and Subject,
> then dump the rest of the header into the smtp_header buffer.
>
>
> Thanks!
>
> -- 
> Joshua Kinard
> Gentoo/MIPS
> kumba () gentoo org
> 4096R/D25D95E3 2011-03-28
>
> "The past tempts us, the present confuses us, the future frightens us.  And
> our lives slip away, moment by moment, lost in that vast, terrible in-between."
>
> --Emperor Turhan, Centauri Republic


------------------------------------------------------------------------------
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing 
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!