Re: rules downloads and scalability

[Martin Roesch <roesch () sourcefire com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

FYI, every once in a while we were getting people who didn't know how  
to configure cron who were trying to download rule updates every  
second.  Since we update rules typically on a daily basis at best, 15  
minutes ought to work pretty well for everyone...

      -Marty

On Sep 18, 2006, at 9:21 AM, Eric Hines wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I do want to add to my comment. I do understand Sourcefire's reasoning
> for doing this. With the number of times Snort has been downloaded and
> half that number of people were checking our web site multiple times a
> day (I hear its as excessive as every 10 mins), I too would have put a
> mechanism in place to prevent it.
>
> Also, I took a closer look at the Sourcefire message for download
> limiting. It seems to be every 15 minutes. I think if anyone downloads
> new rules more often than every 15 minutes, something needs to be  
> changed :)
>
> - -------------- snip -------------
>
> Next download available at: 2006-09-18 09:33:54 (Currently: 2006-09-18
> 09:18:55)
>
> You don't have permission to access
> /pub-bin/downloads.cgi/Download/vrt_os/snortrules- 
> snapshot-2.4.tar.gz on
> this server.
>
> - -------------- snap -------------
>
>
> Best Regards,
>
> Eric S. Hines, GCIA, CISSP
> CEO, President, Chairman
> Applied Watch Technologies, LLC
>
>
> - --------------------------------------------------
>
> Eric S. Hines, GCIA, CISSP
> CEO, President, Chairman
> Applied Watch Technologies, LLC
>
> - --------------------------------------------------
>
> Email:   eric.hines () appliedwatch com
> Address: 1095 Pingree Road
>          Suite 221
>          Crystal Lake, IL
>          60014
> Tel:     (877) 262-7593 ext:327
> Local:   (847) 854-5831
> Fax:     (847) 854-5106
> Web:     http://www.appliedwatch.com
>
> - --------------------------------------------------
> Security Management for the Open Source Enterprise
>
>
>
>
>
> Eric Hines wrote:
> > Jason,
> >
> > Its not limiting specific to Oinkmaster. Applied Watch began  
> > seeing this
> > a few weeks ago through regular rule downloads with our Command  
> > Center
> > using specific Oink Code. Sourcefire seems to be limiting user- 
> > specific
> > Oink Code to download rules only once a day.
> >
> > Eric Hines, GCIA, CISSP
> > CEO, President
> > Applied Watch Technologies, LLC
> > 1095 Pingree Road
> > Suite 221
> > Crystal Lake, IL 60014
> > Tel: (877) 262-7593
> > Web: http://www.appliedwatch.com
> >
> > Jason Haar wrote:
> > > I notice the "www.snort.org/pub-bin/oinkmaster.cgi" script has  
> > > some form
> > > of download limiting component (to stop people like me repeatably
> > > downloading the same live data while editing/updating local  
> > > scripts - ahem).
> > >
> > > Anyway, such scaling issues happen. I'd like to suggest that  
> > > Sourcefire
> > > look to ClamAV to see how they handled people hammering their  
> > > servers
> > > looking for updates that didn't exist (i.e. they were already up to
> > > date). Their rules basically have a serial number and they put  
> > > that into
> > > a DNS record, and then their freshclam update daemon looks to  
> > > that DNS
> > > record before deciding to actually do a HTTP connection to  
> > > download an
> > > update. Than plus some time-of-day randomization and load sharing  
> > > should
> > > go a loooong way on the scalability side...
> > >
> > > Just an idea.
> >
> >
> > --------------------------------------------------------------------- 
> > ----
> > Using Tomcat but need to do more? Need to support web services,  
> > security?
> > Get stuff done quickly with pre-integrated technology to make your  
> > job easier
> > Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> > Geronimo
> > http://sel.as-us.falkag.net/sel? 
> > cmd=lnk&kid=120709&bid=263057&dat=121642
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://www.geocrawler.com/redir-sf.php3?list=snort-users
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.4 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFFDp1u1va6QYTV0EMRAnAhAJ4zWwA9A9cllGydztaCGnxM4pBPDACcDC6E
> HxZN2OTS2R1ZwYTGXCSWvLM=
> =h5NC
> -----END PGP SIGNATURE-----
> <eric.hines.vcf>
> ---------------------------------------------------------------------- 
> ---
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your  
> job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642______________________________ 
> _________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users

- --
Martin Roesch - Founder/CTO, Sourcefire Inc. - +1-410-290-1616
Sourcefire - Security for the Real World - http://www.sourcefire.com
Snort: Open Source IDP - http://www.snort.org


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)

iD8DBQFFDrjBqj0FAQQ3KOARApjcAJ0Whha6kOjETlSUNG57l6I9gj/mAACfRR5v
TwB7ei/tB75RoRtL7gOEJ9o=
=qC5G
-----END PGP SIGNATURE-----

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users