Re: WebTrends Alternative

[Siglite <siglite () criticalstop com>]

We also considered (very briefly) webtrends.  And the salt-mines (perl scripts
ect...) to filter the logs were by far the most cost effective solution for us
as well.  I wound up writing a perl script that filtered the logs into a format
that was easily imported into MS SQL Server.  From there, I wrote my own custom
browser based reports front-ended by IIS.  I did this purely for archival
purposes as I felt checkpoint's log viewer was sufficient for filtering active
logs.  But the concept of my firewall housing several hundred megabytes of
archived log files did not appeal to me.

I've been maintaining a six month log rotation on the SQL box for about a year
now.  A six month archive entails about a million records in the database for
our company, and I've been considering moving to Oracle since SQL 6.5 strains
badly under the queries.

Maybe we should talk about this 'starting a company' concept offline heheh.
I'm fairly certain I could write a product (with some help) that would undercut
webtrends pricing by 3/4.

I think the most difficult aspect of this would be the question 'how do you
automate moving the logs off of th box' be it windows NT, Solaris, Nokia, or
whatever platform checkpoint's currently running on.  Network transfer from the
client-side implys some sort of listening service running on the firewall.
Bad.  I haven't looked into a viable cross-platform solution for this
firewall-side.  I've just continued to trudge through the logs with my perl
script and my sql server.

Perhaps I, or someone else, or maybe a group of us could consider writing a C
(or other) executable that launched from cron and connected directly to a SQL
or Oracle server and dumped the logs in using the same calls as the microsoft
BCP, or whatever bulk-import utilities are available through oracle.  Once the
data is in an RDBMS, reporting becomes GREATLY simplified IMHO.

-KT Morgan
CCSA/CCSE
siglite () criticalstop com


"Burden, James" wrote:

> I am running several firewalls at our site from several different vendors.
> Being from the 'old school' and use to writing shell scripts to parse the
> logs to get to the point where a human has to look at them.  Modification to
> the scripts is usually fairly constant as you are continually trying to
> refine what you want to view, or report.
>
> Recently a product was mentioned in the firewalls list that would perform
> this function called "WebTrends" - http://www.webtrends.com  I looked into
> this company as a possible way to cut down on our labor while keeping our
> firewall (server) log reviews current.  What I wanted to do was to download
> all of our firewall logs to a UNIX platform with a dedicated NIC to an NT
> box running WebTrends.  Many of the firewalls on the market are supported by
> WebTrends, and they even give configuration on how to push syslog (UDP 514)
> from the firewalls to the server.  I asked for a possible solution of:
>
> Firewall <--IPSEC (syslog) --> UNIX box <-cross-over-cable-WebTrends
>
> He replied to my five question email with four URLs pointing at his site.
> Thus not answering any question, beyond what I already knew, except for the
> following question:
> *  When/if WebTrends would ever support a complete UNIX solution?
> (WebTrends is currently geared towards NT....)
>
> Unfortunately, WebTrends licensing strategy makes it financially infeasible
> for a large shop.  Basically, you license each firewall ($1497) or you can
> purchase additional licenses in groups of 4 ($4497).  In all fairness, the
> sales guy did state that he would work with the price with us.  If you
> wanted support for the product then you were required to pay $1798 for each
> firewall per year.  When I inquired about a site license he stated that it
> was not available.  He made the comment that they have to make money also.
> While I am not against someone making money, I do not think that WebTrends
> is the way to go for a large shop with several firewalls.  For instance,
> lets say you have 40 firewalls:
>
> 10 (4 licenses in 1) x $4497 = $44,970
>
> And if you want support:
>
> 40 x $1798 = $71,920 per year
>
> Now, I still have to hire the people to read the logs/reports and act on
> them.  I have received estimates for outsourcing this entire process between
> $900-$3000 per firewall.  Plus, I would get change management, someone to
> make the changes, patch the firewalls, and etceteras.
>
> 40 x $900 = $36,000 per year
>
> What I am looking for is a product with a bit more of a "progressive" view.
> Does anyone know of one?  Or is it back to the shell scripting salt mines?
> Another idea that comes to mind is to hire a couple of developers to write
> my own.
>
> As a last idea, is anyone interested in setting up a new business?  ;-)
>
> Happy Hunting,
> Jim
>
> James L. Burden, Security Engineer and Architect
> California Independent System Operator
> Phone: 916.351.2243 http://www.caiso.com
> 41DF 0E4C 26E0 2FD3 8C81  A260 5C40 280E B4AE 7420
> _____________________________________
>   Know yourself, Know your enemy
>      in a hundred battles you will never be in danger,
>   Know the ground, Know the weather,
>      and your victory will be total.    - Sun Tzu
> _____________________________________
>
> Disclaimer:  The above represents my personal opinions and not an
> official endorsement or position by the California ISO, my current
> employer.  I reserve the right to disavow them at my convenience.