Snort mailing list archives
Re: Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features
From: Joel Esler <jesler () sourcefire com>
Date: Sat, 17 Sep 2011 12:52:20 -0400
If there is a demand for it, I'm sure we could add it to a future release. We started with HTTP and SMTP because those were the two that had the biggest amount of people asking for it. J On Sep 16, 2011, at 9:51 PM, Jason Haar wrote:
Hi Joel Great feature - are you going to generalize this to other protocols too? I'd love to see filenames and authentication details for CIFS for starters? Keep up the good work Jason On 17/09/11 10:01, Joel Esler wrote:I thought this one was of special interest to our community, so forgive the cross post. This post is by: Bhagyashree Bantwal, one oelof our Snort Developers! ------ http://blog.snort.org/2011/09/snort-291-http-and-smtp-logging.html Snort 2.9.1 HTTP and SMTP logging features To provide better context to the alerts generated by Snort, Snort version 2.9.1 introduced some new logging features for HTTP and SMTP which will help the user to better analyze the alerts. This logging of extra data by HTTP Inspect and SMTP preprocessors is similar to the X-Forwarded-For/True-Client-IP logging introduced prior to Snort 2.9.1. HTTP Logging: Let's talk about HTTP URI and Hostname logging. How to enable logging of HTTP URI and Hostname? To turn on the logging of the HTTP Request URI, the config option to use is "log_uri" and to enable the logging of hostnames, use the config option "log_hostname" as shown in the example below: preprocessor http_inspect: global memcap preprocessor http_inspect_server: log_uri \ log_hostname The memcap here will determine the maximum amount of memory the HTTP Inspect preprocessor will use for logging the URI and Hostname data. You can refer to the Snort Manual for further details. When these features are turned on in HTTP Inspect, the HTTP Request URI and HTTP Request hostname headers are extracted and logged to unified2 as an "extra data event" with each alert generated for that particular session. It is recommended to turn on stream5 reassembly with PAF on HTTP ports to ensure correctness and accuracy. When a HTTP Request URI is greater than 2048 or when a HTTP hostname (specified in the "Host" Request header) is greater than 256, Snort will log the truncated the URI and/or hostname. A preprocessor alert with GID:119 and SID:25 is generated when hostname exceeds 256 bytes. There is also a preprocessor alert with GID:119 and SID:24 for multiple "Host" headers in one HTTP request. Please note that the URI and hostname are only logged in Unified2 mode and not logged with -A cmg. So, how do you read the output? Unified2 can be read using the Snort tool u2spewfoo. You can find it under the tools/u2spewfoo directory in the Snort tarball. Example of the Output: (Event) sensor id: 0 event id: 2 event second: 1299776137 event microsecond: 355217 sig id: 1 gen id: 1 revision: 0 classification: 0 priority: 0 ip source: 10.1.2.3 ip destination: 10.9.8.7 src port: 60710 dest port: 80 protocol: 6 impact_flag: 0 blocked: 0 Packet sensor id: 0 event id: 2 event second: 1299776137 packet second: 1299776137 packet microsecond: 355217 linktype: 1 packet_length: 214 [ 0] 02 09 08 07 06 05 02 01 02 03 04 05 08 00 45 00 ..............E. [ 16] 00 C8 00 04 00 00 40 06 5C 19 0A 01 02 03 0A 09 ......@.\....... [ 32] 08 07 ED 26 00 50 00 00 00 02 00 00 00 02 50 10 ...&.P........P. [ 48] 01 00 0C 19 00 00 47 45 54 20 2F 20 48 54 54 50 ......GET / HTTP [ 64] 2F 31 2E 30 0D 0A 48 6F 73 74 3A 20 77 77 77 2E /1.0..Host: www. [ 80] 77 33 2E 0D 0A 20 6F 72 67 20 0D 0A 55 73 65 72 w3... org ..User [ 96] 2D 41 67 65 6E 74 3A 20 4D 6F 7A 69 6C 6C 61 2F -Agent: Mozilla/ [ 112] 35 2E 30 20 28 58 31 31 3B 20 55 3B 20 4C 69 6E 5.0 (X11; U; Lin [ 128] 75 78 20 78 38 36 5F 36 34 3B 20 65 6E 2D 55 53 ux x86_64; en-US [ 144] 3B 20 72 76 3A 31 2E 39 2E 32 2E 31 35 29 20 47 ; rv:1.9.2.15) G [ 160] 65 63 6B 6F 2F 32 30 31 31 30 33 30 33 20 55 62 ecko/20110303 Ub [ 176] 75 6E 74 75 2F 31 30 2E 31 30 20 28 6D 61 76 65 untu/10.10 (mave [ 192] 72 69 63 6B 29 20 46 69 72 65 66 6F 78 2F 33 2E rick) Firefox/3. [ 208] 36 2E 31 35 0D 0A 6.15.. (Event) sensor id: 0 event id: 3 event second: 1299776137 event microsecond: 355254 sig id: 1 gen id: 1 revision: 0 classification: 0 priority: 0 ip source: 10.1.2.3 ip destination: 10.9.8.7 src port: 60710 dest port: 80 protocol: 6 impact_flag: 0 blocked: 0 Packet sensor id: 0 event id: 3 event second: 1299776137 packet second: 1299776137 packet microsecond: 355254 linktype: 1 packet_length: 176 [ 0] 02 09 08 07 06 05 02 01 02 03 04 05 08 00 45 00 ..............E. [ 16] 00 A2 00 05 00 00 40 06 5C 3E 0A 01 02 03 0A 09 ......@.\>...... [ 32] 08 07 ED 26 00 50 00 00 00 A2 00 00 00 02 50 10 ...&.P........P. [ 48] 01 00 8F E5 00 00 41 63 63 65 70 74 2D 45 6E 63 ......Accept-Enc [ 64] 6F 64 69 6E 67 3A 20 67 7A 69 70 2C 64 65 66 6C oding: gzip,defl [ 80] 61 74 65 0D 0A 41 63 63 65 70 74 2D 43 68 61 72 ate..Accept-Char [ 96] 73 65 74 3A 20 49 53 4F 2D 38 38 35 39 2D 31 2C set: ISO-8859-1, [ 112] 75 74 66 2D 38 3B 71 3D 30 2E 37 2C 2A 3B 71 3D utf-8;q=0.7,*;q= [ 128] 30 2E 37 0D 0A 4B 65 65 70 2D 41 6C 69 76 65 3A 0.7..Keep-Alive: [ 144] 20 31 31 35 0D 0A 43 6F 6E 6E 65 63 74 69 6F 6E 115..Connection [ 160] 3A 20 6B 65 65 70 2D 61 6C 69 76 65 0D 0A 0D 0A : keep-alive.... (ExtraDataHdr) event type: 4 event length: 33 (ExtraData) sensor id: 0 event id: 2 event second: 1299776137 type: 9 datatype: 1 bloblength: 9 HTTP URI: / (ExtraDataHdr) event type: 4 event length: 43 (ExtraData) sensor id: 0 event id: 2 event second: 1299776137 type: 10 datatype: 1 bloblength: 19 HTTP Hostname: www.w3.org In the output above, two extra data events are generated for the event with GID:1 and SID: 1. The first extra data event displays the URI and the second displays the hostname associated with that session. It is important to note that the unique "event id" and "event second" are used to correlate the alerts with their extra data. The "event type" indicates the presence of an "extra data" record and the unique "type" will determine the type of the extra data event logged. Here is a list of extra data types used in Snort. Type 1: True-Client-IP/XFF IPv4 address Type 2: True-Client-IP/XFF IPv6 address Type 4: HTTP Gzip decompressed data Type 5: SMTP filename Type 6: SMTP MAIL FROM addresses Type 7: SMTP RCPT TO addresses Type 8: SMTP Email headers Type 9: HTTP Request URI Type 10: HTTP Request Hostname Type 11: Packet's IPv6 Source IP Address Type 12: Packet's IPv6 Destination IP Address SMTP Logging: The SMTP preprocessor, similar to HTTP Inspect preprocessor, can log extra data associated with each alert. The data that SMTP logs are as follows: 1. The email addresses in the MAIL FROM command. 2. The email addresses in the RCPT TO command. When there are multiple RCPT TO headers, the email addresses are concatenated using commas. 3. The filename of the MIME attachment extracted from the "Content-Disposition" header. Multiple filenames are appended with commas. 4. SMTP email headers. How to enable logging of SMTP "extra data"? SMTP preprocessor can log the extra data mentioned above using the following config options. preprocessor smtp: memcap 838860 \ log_mailfrom \ log_rcptto \ log_filename \ log_email_hdrs \ email_hdrs_log_depth 56 The config options "log_mailfrom", "log_rcptto", "log_filename" and "log_email_hdrs" do not take any arguments. However, email_hdrs_log_depth requires the user to pass a number between 0-20480, which will determine in bytes the amount of email headers logged. The config option "memcap" is used to determine the maximum memory used for logging the above mentioned data. Again, stream5 reassembly needs to be turned on for SMTP ports for this feature to work correctly. Without stream5 reassembly, the SMTP extra data won't be logged. How to read the Output? Like HTTP extra data, SMTP extra data can be read using the u2spewfoo and "event id" and "event second" are used to correlate the alerts with their extra data events. Example of the Output: (ExtraData) sensor id: 0 event id: 2 event second: 1273194706 type: 6 datatype: 1 bloblength: 33 SMTP MAIL FROM Addresses: (ExtraData) sensor id: 0 event id: 2 event second: 1273194706 type: 7 datatype: 1 bloblength: 57 SMTP RCPT TO Addresses: , (ExtraData) sensor id: 0 event id: 2 event second: 1273194706 type: 5 datatype: 1 bloblength: 26 SMTP Attachment Filename: sample.txt,foo.txt (ExtraData) sensor id: 0 event id: 2 event second: 1273194706 type: 8 datatype: 1 bloblength: 323 SMTP EMAIL HEADERS: MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: multipart/mixed; boundary="_----------=_12731947069000" Date: Thu, 6 May 2010 21:11:46 -0400 From: bbantwal () sourcefire com To: bbantwal () sourcefire com Subject: Email Sent via Perl X-Mailer: MIME::Lite 3.027 (F2.74; T1.29; A2.06; B3.07; Q3.07 <bbantwal () sourcefire com><bbantwal () sourcefire com><foo-barfoo () gmail com> We will not be updating the spo_database output method to insert this information into the database. As a reminder, this output method will be removed in a future Snort release in 2012. Sourcefire has turned over the maintenance of the "ACID" database schema to the Barnyard2 group, and will also, no longer be maintaing it either. -- Joel Esler Senior Research Engineer, VRT OpenSource Community Manager Sourcefire ------------------------------------------------------------------------------ BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA http://p.sf.net/sfu/rim-devcon-copy2 _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!-- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +64 3 9635 377 Fax: +64 3 9635 417 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 ------------------------------------------------------------------------------ BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA http://p.sf.net/sfu/rim-devcon-copy2_______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!
------------------------------------------------------------------------------ BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA http://p.sf.net/sfu/rim-devcon-copy2
_______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!
Current thread:
- Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Joel Esler (Sep 16)
- Re: Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Jason Haar (Sep 16)
- Re: Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Joel Esler (Sep 17)
- Re: Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Jason Haar (Sep 17)
- Re: Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Richard Bejtlich (Sep 17)
- Re: Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Joel Esler (Sep 17)
- Re: [Snort-Users] Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Bamm Visscher (Sep 19)
- Re: [Snort-Users] Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Jason Brvenik (Sep 19)
- Re: [Snort-Users] Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Bamm Visscher (Sep 19)
- Message not available
- Re: [Snort-Users] Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Bamm Visscher (Sep 19)
- Re: [Snort-Users] Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Joel Esler (Sep 20)
- Re: [Snort-Users] Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Jason Brvenik (Sep 19)
- Re: Snort.org Blog: Snort 2.9.1 HTTP and SMTP logging features Jason Haar (Sep 16)