RE: Tool for large log file reporting? (was: RE: WebSens e)

["Behm, Jeffrey L." <BehmJL () bvsg com>]

> From: Joe Ippolito
> There are active server pages included with MS Proxy 
> that will create tables and DSN's to make MS Proxy
> log to MS SQL server.  See MS Proxy help pages for 
> how to.  Real easy to setup but does slow down the 
> proxy a bit. Your queries then become SQL queries 
> and log size can get quite large.  P.S. MS Proxy
> is much faster on IIS 5 (Win2K).

Thanks for the response!

We tried Sql server, agreed, it is easy to set up, slows down the proxy a
bit, etc., but the show stopper for us was when we tried to remove the older
entries from the db. At that point, the Proxy server hung until the
(pseudo-sql command follows) "delete from x where date <  [today - 7 days
(or whatever)]" would finish. At least our prognosis was that it would
continue once the query finished, but we couldn't let the query run longer
than 10 minutes, as all http traffic stopped through the proxy server. We
tried creating indexes on the logdate/time field, no difference. Perhaps
some setting in Sql Server would do the trick, but of course, this is the FW
group, not Sql-server, so no flames please.

In short, The sql server functionality seemed to be the holy grail, as the
queries we ran came back in seconds instead of minutes. All seemed great
until we needed to truncate the older, no-longer-needed data.

The amount of data this client generates _just in MSProxy log files_ is
incredible (400MB/day and growing) and we haven't found anything that will
resonably manage (i.e. inserts, queries, _and_ deletes) the larger amount of
data.

Thoughts?

Jeff

_______________________________________________
Firewall-wizards mailing list
Firewall-wizards () nfr net
http://www.nfr.net/mailman/listinfo/firewall-wizards