Re: Logs Bank

[Andrew Mulholland <andy-nanog () bash sh>]

To answer your question.

"yes"

However, with almost everything I can think of, there will be an element of
development required in order to achieve the results you're after. - at a
previous work place a few years ago we fed all event logs into hadoop, from
where we produced reports, initially just into excel files,  and then later
created a webapp which produced near realtime stats/reports/graphs.

I've not looked recently at LogStash, or 8pussy, but primary concern would
be how well they deal with huge log volumes, how they scale when one server
is not big enough to hold all the logs any more, how they deal with many
users searching at the same time etc.

If you want to actually just get on with crunching logs, and drawing graphs
in a timely fashion, Splunk is proven, and works well up to big scale (we
were feeding almost 1TB/day of logs into it at my last company)...


Splunk is not cheap, but when considering the cost of development +
suppport if you went down the route of task of rolling something equivalent
in capabilities, its not bad value.

thanks

Andrew


On Tue, Nov 8, 2011 at 7:59 PM, <joshua.klubi () gmail com> wrote:

> Hi,
>
> If I may ask, is there any OSS that can serve as a log bank or log server,
> where it aggregate logs from  different sources , and the logs can be
> accessed using the web from any location on the network and can do
> graphical presentations based on.the frequency or content os the logs.
>
> Thank you
>
> Joshua
>
> --
> Sent from my Nokia N9