Re: Updating Rules with PulledPork and no outside connection

[JJC <cummingsj () gmail com>]

That works perfectly well, PP doesn't care about the opensource.gz file...
I added that as a feature to handle based on popular request.....  I'm
hoping to have a faster extraction method in the upcoming release... but as
it stands it's much faster to externalize the opensource.gz extraction to
PP...

JJC

On Wed, Sep 19, 2012 at 2:35 PM, Michael Steele <michaels () winsnort com>wrote:

> I’m thinking there has to be a quicker way to do this. ****
>
> ** **
>
> If PulledPork is only extracting the *.txt files to the signature folder,
> maybe there is a way to run a Windows, or Unix command line unzip from the
> perl script?****
>
> Iuse, and it takes about 30 seconds: unzip -j -qq opensource.zip *.txt -d
> d:\winids\apache\htdocs\base\signature****
>
> Can you answer this because I’m not real clear:****
>
> I’m thinking of processing the rules file offline using PulledPork, and
> then extracting the opensource.gz file later to the signature folder, does
> that work or does PulledPork need to do anything with them, at the same
> time it’s processing the rules?****
>
> ** **
>
> Kindest regards,****
>
> Michael...****
>
> ** **
>
> *From:* JJC [mailto:cummingsj () gmail com]
> *Sent:* Wednesday, September 19, 2012 3:33 PM
>
> *To:* Michael Steele
> *Cc:* snort-users () lists sourceforge net
> *Subject:* Re: [Snort-users] Updating Rules with PulledPork and no
> outside connection****
>
> ** **
>
> You are correct, unfortunately the perl module that does the extracting is
> slow when it comes to that specific tarball, has to do with reading the
> index that contains a metric ton of .txt files and then pulling them out...
> you might be better off just scripting a cron run of tar against that
> file....****
>
> ** **
>
> JJC****
>
> On Wed, Sep 19, 2012 at 1:06 PM, Michael Steele <michaels () winsnort com>
> wrote:****
>
> That fixed it.****
>
>  ****
>
> Not sure what the final solution is here, as its painfully slow processing.
> ****
>
>  ****
>
> There isn’t any real processing of the opensource.gz file, other the
> extracting the signatures and moving them to the designated folder?****
>
>  ****
>
> Kindest regards,****
>
> Michael...****
>
>  ****
>
> *From:* JJC [mailto:cummingsj () gmail com]
> *Sent:* Wednesday, September 19, 2012 2:09 PM****
>
>
> *To:* Michael Steele
> *Cc:* <snort-users () lists sourceforge net>
> *Subject:* Re: [Snort-users] Updating Rules with PulledPork and no
> outside connection****
>
>  ****
>
> Ok, so around line 1787, in the condition ($NoDownload && !$grabonly)
> there should be a chunk that reads:****
>
>  ****
>
> *unless ( $rule_file =~ /snortrules-snapshot-\d{4}\.tar\.gz/ ) {*****
>
>  ****
>
> you will want to change it to the following and see what happens:****
>
>  ****
>
> *unless ( $rule_file =~ /snortrules-snapshot-\d{4}\.tar\.gz/
>                     || $rule_file =~ /opensource\.gz/ )
> {*****
>
>  ****
>
> On Wed, Sep 19, 2012 at 8:30 AM, JJ Cummings <cummingsj () gmail com> wrote:*
> ***
>
> Im gonna have a look shortly
>
> Sent from the iRoad****
>
>
> On Sep 19, 2012, at 6:50, "Michael Steele" <michaels () winsnort com> wrote:*
> ***
>
> I’m no expert here at all, but is there a chance that there is NO code
> even built into PulledPork that deals with the  ‘opensource.gz’ file in
> ‘NoDownload’ routine?****
>
>  ****
>
> Also is the NoDownload routine processing the rules twice?****
>
>  ****
>
> Kindest regards,****
>
> Michael...****
>
>  ****
>
> *From:* Michael Steele [mailto:michaels () winsnort com]
> *Sent:* Tuesday, September 18, 2012 6:30 PM
> *To:* 'JJC'
> *Cc:* snort-users () lists sourceforge net
> *Subject:* Re: [Snort-users] Updating Rules with PulledPork and no
> outside connection****
>
>  ****
>
> Here is another run using –vvnT****
>
>  ****
>
> Looking at the log it seems to be processing the rules twice.****
>
>  ****
>
> Kindest regards,****
>
> Michael...****
>
>  ****
>
> *From:* JJC [mailto:cummingsj () gmail com]
> *Sent:* Tuesday, September 18, 2012 1:47 PM
> *To:* Michael Steele
> *Cc:* snort-users () lists sourceforge net
> *Subject:* Re: [Snort-users] Updating Rules with PulledPork and no
> outside connection****
>
>  ****
>
> Interesting, can you do a run with -vv and send the results?****
>
>  ****
>
> JJC****
>
> On Tue, Sep 18, 2012 at 6:19 AM, Michael Steele <michaels () winsnort com>
> wrote:****
>
> Attached is a log of the run. Attached is my pulledpork.conf and I'm
> looking
> for something that is causing PulledPork to not process the opensource.gz
> file in offline mode.
>
> It appears to be a problem with PulledPork only processing the
> snortrules-snapshot-2931.tar.gz  in offline mode as PulledPork processes
> both files (opensource.gz and snortrules-snapshot-2931.tar.gz ) if you are
> processing in online mode.
>
> My run line includes switches: -nvT
>
> Both files have been place in the: temp_path=
>
> I'm assuming that PulledPork should process both files exactly as it does
> in
> offline mode as it does in online mode, minus the file downloading, and as
> long as the two files reside in the designated temp folder.
>
> I'm not sure about checksums in offline mode as  PulledPork seems to
> process
> the snortrules-snapshot-2931.tar.gz every time its ran in offline mode,
> regardless of any checksum. I believe it does the same thing in in online
> mode. The checksum only prevents PullePork  from downloading the file/s
> again in online mode.****
>
>
> Kindest regards,
> Michael...
>
> -----Original Message-----
> From: JJ Cummings [mailto:cummingsj () gmail com]
> Sent: Monday, September 17, 2012 12:48 PM
> To: Michael Steele
> Cc: <snort-users () lists sourceforge net>
> Subject: Re: [Snort-users] Updating Rules with PulledPork and no outside
> connection****
>
> Place the tarballs in the defined temp path that you have in your
> pulledpork.conf.. You will want to tell pp not to download and not to
> validate checksums...
>
> JJC
>
> Sent from the iRoad
>
> On Sep 17, 2012, at 7:02, "Michael Steele" <michaels () winsnort com> wrote:
>
> > I've looked through the list archive and was unable to find any
> > specifics on how to do this.
> >
> > I need to run PulledPork on a closed network.
> >
> > The run line I have is:
> > 'perl d:\winids\pulledpork\pulledpork.pl -c
> > d:\winids\pulledpork\etc\pulledpork.conf -v -T -n'
> >
> > I'm pretty sure the -n tells PulledPork to process locally?
> >
> > There are two files that need to be used and I'm not sure what to do
> > with them?
> > 1) snortrules-snapshot-2931.tar.gz
> > 2) opensource.gz
> >
> >
> > Do these lines need to be hashed out?
> > rule_url=https://www.snort.org/reg-rules/|snortrules-snapshot.tar.gz|<
> > oinkco
> > de>
> > rule_url=https://www.snort.org/reg-rules/|opensource.gz|<oinkcode>
> >
> >
> > Just to verify; using the -T in the run line means I don't have to
> > hash out the so_rules section below?
> >
> > sorule_path=/usr/local/lib/snort_dynamicrules/
> > snort_path=/usr/local/bin/snort
> > config_path=/usr/local/etc/snort/snort.conf
> > sostub_path=/usr/local/etc/snort/rules/so_rules.rules
> > distro=FreeBSD-8.1
> >
> > Kindest regards,
> > Michael...
> >
> >
> >
> >
> >
> >
> > ----------------------------------------------------------------------
> > --------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's security and
> > threat landscape has changed and how IT managers can respond.
> > Discussions will include endpoint security, mobile security and the
> > latest in malware threats.
> > http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://www.geocrawler.com/redir-sf.php3?list=snort-users
> >
> > Please visit http://blog.snort.org to stay current on all the latest
> Snort
> news!****
>
>
> ----------------------------------------------------------------------------
> ****
>
> --
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and threat
> landscape has changed and how IT managers can respond. Discussions will
> include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest Snort
> news!****
>
>  ****
>
>  ****
>
> ** **
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!