nanog mailing list archives
Re: improving signal to noise ratio from centralized network syslogs
From: Michael Loftis <mloftis () wgops com>
Date: Fri, 26 Jan 2018 06:01:11 +0000
On Thu, Jan 25, 2018 at 8:11 PM Joe Maimon <jmaimon () jmaimon com> wrote:
Hey All, Centralized logging is a good thing. However, what happens is that every repetitive, annoying but not (usually) important thing fills up the log with reams of what you are not looking for. Networks are a noisy place and silencing every logged condition is impractical and sometimes undesirable. What I am interested in is an automated zoom-in zoom-out tool to mask the repetition of "normal" events and allow the unusual to stand out. Add to that an ability to identify gaps in the background noise. (The dog that didnt bark) What I am not interested in are solutions based upon preconfigured filters and definitions and built in analysis for supported (prepopulated definitions) platforms, this is all about pattern mining/masking and should be self discoverable. Ideally a command tool to generate static versions of the analysis coupled with a web platform (with zoom +- buttons) for realtime. I made a crude run of it with SLCT, using its generated patterns to grep -v, and that in and of itself was useful, but needs a bit of work. Also, its not quite real time. Any ideas would be greatly appreciated.
Not cheap, but Splunk comes to mind.
Joe
-- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler
Current thread:
- improving signal to noise ratio from centralized network syslogs Joe Maimon (Jan 25)
- Re: improving signal to noise ratio from centralized network syslogs Michael Loftis (Jan 25)
- Re: improving signal to noise ratio from centralized network syslogs Steven Miano (Jan 26)
- RE: improving signal to noise ratio from centralized network syslogs Edwin Pers (Jan 26)
- Re: improving signal to noise ratio from centralized network syslogs Alain Hebert (Jan 26)
- Re: improving signal to noise ratio from centralized network syslogs Casey Russell (Jan 26)
- Re: improving signal to noise ratio from centralized network syslogs Steven Miano (Jan 26)
- Re: improving signal to noise ratio from centralized network syslogs Rich Kulawiec (Jan 31)
- Re: improving signal to noise ratio from centralized network syslogs George William Herbert (Jan 31)
- Re: improving signal to noise ratio from centralized network syslogs Michael Loftis (Jan 25)