Re: Best script to pre-load signature metadata into a database

[elof () sentor se]

On Thu, 7 Oct 2010, Joel Esler wrote:
> On Oct 7, 2010, at 10:19 AM, elof () sentor se wrote:
>
> > I want to pre-load my Postgres database with all the signature metadata
> > (titles, references, prios, etc) from my rules.
> >
> > I fould the script 'rules.pl' in an old FLoP tarball...
> > ...but if there is a script that import the Generator signatures (from
> > gen-msg.map) as well, that would be even better.
> >
> > What am I looking for?
>
> I have no idea.
> If you take barnyard2 (if that outputs to postgres), and point it at your sid-msg.map file when barnyard2 starts up, 
> it will insert all that stuff into the db when you have an alert.
> Otherwise, can you clarify what you are trying to do?

There is a bug, or rather a race condition, when you have more than one 
sensor that see the same traffic.

Both sensors log the same alert.
Both barnyard2 processes ask the database: Do there already exist metadata 
in the database for sid 1234?
The database check and answer No to both querys.
Both barnyard2 processes will then insert the metadata.

This results in TWO inserts for the same metadata for sid 1234, so the 
next time this sid trigger an alert, Barnyard2 ask the database "Do there 
already exist metadata in the database for sid 1234?", and the database 
respond with TWO references. The database output module in barnyard2 can't 
handle this since there must be one unique reference.
So no sid 1234 events are logged to the database, and simultaneously the 
syslog is flooded with warnings about:

database: warning (SELECT sig_id   FROM signature  WHERE sig_name = 'SNMP 
private access udp '    AND sig_rev = 13    AND sig_sid = 1413    AND 
sig_gid = 1 ) returned more than one result

barnyard2[19021]: database: warning (SELECT ref_id   FROM reference  WHERE 
ref_system_id = 1    AND ref_tag = '4088') returned more than one result



So...

By pre-loading the database with all metadata before snort and barnyard2 
is even started, this race condition should never appear, because when 
both sensors ask "Do there already exist metadata in the database for sid 
1234?" both answers will be Yes, and there won't be duplicate 
metadata inserts.
As a positive side effect, barnyard2 will log alerts faster, since it will 
never have to deal with inserting metadata before inserting the actual 
event.


What I'm looking for is the best and most updated script for inserting all 
this metadata (from sid-msg.map, reference.config, classification.config 
and gen-msg.map).


What am I looking for?

/Elof

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users