nanog mailing list archives
Re: improving signal to noise ratio from centralized network syslogs
From: Shane Short <shane () short id au>
Date: Mon, 5 Feb 2018 10:25:36 +0800
In addition to that, you can use some fancy awk colour coding, so you can make it highlight certain lines based on content.. I use this for my e-mail logs, but I’m sure it could be adapted: tail -n 1000 -f /var/log/mail-submission.log | grep smtp.*relay | awk ' /sent/ {print "\033[32m" $0 "\033[39m"} /bounced/ {print "\033[31m" $0 "\033[39m"} /deferred/ {print "\033[33m" $0 "\033[39m"} '
On 4 Feb 2018, at 5:49 am, Scott Weeks <surfer () mauigateway com> wrote: --- jmaimon () jmaimon com wrote: Centralized logging is a good thing. However, what happens is that every repetitive, annoying but not (usually) important thing fills up the log with reams of what you are not looking for. --------------------------------------- Apologies, I'm late to the party. But I just want to add one thing for the archives. It's along with what Rich Kulawiec said, "it forces you to look at your own data, which is really helpful. You'll be surprised at what you find if you've never done it before." This is accurate. It's fun to see what your network is putting out. This is all from memory (I've done it so many times it's in there permanently... :-) as I don't have a unix server or a router in front of me to use, so don't hold me to exact details... And it's mainly for the newbies. Have all the routers send to one syslog file, switches to another and other devices to a third on a *nix box: For example, send the router messages to /var/log/router.log and the switch messages to /var/log/switch.log This is done with the 'logging facility' command on the devices: After defining your syslog server's IP address and the level of messaging you want (I set it to debug because I want to see everything): on the routers: logging facility local0 on the switches: logging facility local1 on the logging server in: /etc/rsyslog.conf local0.* /var/log/router.log local1.* /var/log/switch.log Use logrotate to manage the log files as they can get quite large. Then, you can watch your network in real time like so (below is all one line): tail -f /var/log/router.log /var/log/switch.log | egrep -vi 'term1|term2|termN' 'egrep -v' takes out all the lines you don't want to see while the syslog messages scroll across the screen. Say there is a battery condition on router1 and a duplex mismatch on a switch I don't want to see: tail -f /var/log/router.log /var/log/switch.log | egrep -vi 'router1.*battery|switch1.*duplex.*mismatch' For me, N can get to 40-50 sometimes, so I put it into a shell script like so: vi log.sh --------------------------- #! /bin/sh tail -f /var/log/router.log /var/log/switch.log | egrep -v 'term1|term2|termN' --------------------------- then, run it like so: ./log.sh It's all netgeek fun-n-games from there on. :) scott
Current thread:
- Re: improving signal to noise ratio from centralized network syslogs Scott Weeks (Feb 03)
- Re: improving signal to noise ratio from centralized network syslogs Tarko Tikan (Feb 04)
- Re: improving signal to noise ratio from centralized network syslogs Shane Short (Feb 04)
- Re: improving signal to noise ratio from centralized network syslogs Brian Knight (Feb 05)
- <Possible follow-ups>
- Re: improving signal to noise ratio from centralized network syslogs Scott Weeks (Feb 05)
- Re: improving signal to noise ratio from centralized network syslogs valdis . kletnieks (Feb 05)
- Re: improving signal to noise ratio from centralized network syslogs James Bensley (Feb 05)
- Re: improving signal to noise ratio from centralized network syslogs valdis . kletnieks (Feb 05)
- Re: improving signal to noise ratio from centralized network syslogs John Kougoulos (Feb 06)
- Re: improving signal to noise ratio from centralized network syslogs valdis . kletnieks (Feb 05)
- Re: improving signal to noise ratio from centralized network syslogs Scott Weeks (Feb 05)