Re: syslog and network management

[david () lang hm]

On Mon, 3 Mar 2008, Darden, Patrick S. wrote:

> UDP is a LOT faster than TCP.  No ECC so it uses less cpu, less memory, and has less of a memory footprint.  If you 
> were dropping a lot of UDP, then TCP would not help at all--you would receive less, just more reliably.
>
> NG is a great app.  Not sure why it failed you.  Good idea to try a different syslogd.

ng does a lot of things, however all I need is a very trivial syslog 
implementation. I don't need it to do any filtering (not by apps, not by 
servers, not even by facility or severity), all I need it to do is to 
recieve logs (checking to see if it needs to add host and timestamp to the 
message), write them to disk, and relay a copy to the next log server.

> You state that you switched to regular syslogd with async file io--was the file io set to async with NG also?  If it 
> starts happening again try:

as I remember, -ng didn't let me just disable sync writes, instead it had 
me give a max number of messages to buffer. I set this value at a few 
hundred.

> vmstat 5 (show disk activity every 5 seconds, io contention, # writes, etc.)
> top (let you check cpu activity, ram, top apps, etc.)
> sar
> netstat -i (check for errors, overruns, and overall activity)

I'll do so in the future if/when I revisit the choice. the macine didn't 
have sar on it, but I did check the other things and didn't find any 
smoking guns. since switching to a different syslogd solved the problem I 
didn't dig that hard.

David LAng

> --p
>
> --p
>
> -----Original Message-----
> From: firewall-wizards-bounces () listserv icsalabs com
> [mailto:firewall-wizards-bounces () listserv icsalabs com]On Behalf Of
> david () lang hm
> Sent: Saturday, March 01, 2008 2:07 AM
> To: Firewall Wizards Security Mailing List
> Subject: Re: [fw-wiz] syslog and network management
>
>
> On Thu, 28 Feb 2008, ArkanoiD wrote:
>
> > Hmm, did you try tcp transport (if your router does support it)?
> > It might be better..
>
> the sending devices did not support tcp transport, but there is not much
> of an excuse for a program who's purpose is receiving logs to do so poorly
> at it. if it's missing so many UDP packets that the OS is overflowing it's
> buffers and dropping them than it's going to do bad things to the tcp
> dataflow as well. the difference is that now you are able to rely on the
> sender to act as a buffer as well. but that leaves your logs where you
> don't want them, eating up resources on the sender while being vunerable
> to disruption.
>
> David Lang
>
> > On Tue, Feb 26, 2008 at 02:12:51PM -0800, david () lang hm wrote:
> > > > We were logging 6 PIXen as well as many switches and routers (and a
> > > > much lesser level). We never "noticed" a great loss of messages... I
> > > > guess I can assume you did, and maybe I could learn from how you did!
> > > > :)
> > > >
> > > > What daemon do you use?
> > >
> > > we tried to use syslog-ng to receive activity from our border router and
> > > write a copy locally (in large chunks) and relay the logs to another
> > > syslog server inside.
> > >
> > > we noticed a LOT of missing logs, when we changed to the default debian
> > > syslogd we were able to handle an order of magnatude more logs without any
> > > sign of missing logs (from around 100/sec to >1000/sec)
> >
> > _______________________________________________
> > firewall-wizards mailing list
> > firewall-wizards () listserv icsalabs com
> > https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards
> _______________________________________________
> firewall-wizards mailing list
> firewall-wizards () listserv icsalabs com
> https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards
> _______________________________________________
> firewall-wizards mailing list
> firewall-wizards () listserv icsalabs com
> https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards
_______________________________________________
firewall-wizards mailing list
firewall-wizards () listserv icsalabs com
https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards