Re: Patch to correct the way Snort names output files

["Tom Peters \(thopeter\) via Snort-devel" <snort-devel () lists snort org>]

Noah,

Thank you for this contribution. We are looking at it right now.

Tom


From: Snort-devel <snort-devel-bounces () lists snort org<mailto:snort-devel-bounces () lists snort org>> on behalf of 
Noah Dietrich <noah_dietrich () 86penny org<mailto:noah_dietrich () 86penny org>>
Date: Monday, December 31, 2018 at 1:27 PM
To: "snort-devel () lists snort org<mailto:snort-devel () lists snort org>" <snort-devel () lists snort 
org<mailto:snort-devel () lists snort org>>
Subject: [Snort-devel] Patch to correct the way Snort names output files

Snort team:

Attached is a patch that fixes the issue that I reported regarding the way that Snort was naming and re-naming the 
output alert files. The issue was that snort created the initial file without the unixtime in the name, and then 
re-named the file by appending the unixtime when the file size limit was reached. This causes issues with Splunk and 
the ELK stack, because they have to wait for the file to be re-named before the file can be indexed (otherwise you risk 
duplicating or missing events, or waiting until the log file rolls over, which could be a long time).

This patch fixes the issue by modifying the get_instance_file() in main/thread.cc to append the unixtime to all 
filenames by default (the unixtime will indicate when the file was created).  A side-effect of this change is that i 
have removed the RollAlertFile() function in log/log.cc, as it is no longer needed.

If you re-start Snort, events will not be written to a half-full alert file, instead alerts will be written to a new 
file (I supposed you could modify the code to continue filling the most recent alert file, but I don't think that's 
necessary, and i can't think of a reason you'd need that functionality).

This is my first time submitting a patch to a project, so please let me know if there is anything I should be doing 
differently. I'm also not a professional C coder, so it's very possible that my code will need to be implemented 
differently to handle issues I am not aware of.

I have tested this patch successfully with the following loggers:
alert_csv
alert_fast
alert_full
log_codecs
log_hext

 and I tested the output with the following options (to make sure that this patch doesn't screw up more complex output 
options):
  --run-prefix
  --id-zero
  --id-subdir

Note: this will also append the unixtime to the appid_stats.log file

this patch can be installed by copying it to the snort3 folder, navigating to that folder, and running:
patch -p1 < unixtime-filenames.diff

The real benefit of this patch is that your file-based output will be created in a way that Splunk or the ELK stack (or 
other log-collecting software) can easily, quickly, and correctly ingest Snort alerts and other outputted information.  
I have written a Splunk plugin that takes advantage of the functionality this patch enables, and will make ingesting 
Snort log data much easier. Basically this new method of file naming works the way most log-collecting software 
expects, which should make it easier to load Snort alerts into those tools.

Thanks, and happy new year.
Noah

_______________________________________________
Snort-devel mailing list
Snort-devel () lists snort org
https://lists.snort.org/mailman/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!