Re: syslog and network management

[david () lang hm]

On Wed, 27 Feb 2008, Timothy Shea wrote:

> > we noticed a LOT of missing logs, when we changed to the default
> > debian
> > syslogd we were able to handle an order of magnatude more logs
> > without any
> > sign of missing logs (from around 100/sec to >1000/sec)
>
> I am also perplexed by this.  syslog-ng has many (many) flaws but in
> terms of dropping packets it has always out-performed every syslogd
> implementation I have run across ("performance" as being defined as
> receiving the highest percentage of packets - this is UDP after
> all.)   So I have to question how it was implemented.  How did you
> validate the drop count?  How was syslog-ng implemented?  Which debian
> version?

this was on debian 3.0, I allowed syslog-ng a substantial buffer (100 or 
so messages IIRC) before writing to disk.

we noticed that a cron job running once a min was loosing ~30% of it's 
reports. we then switched to the normal syslog (with async writes to disk) 
and not only stopped loosing the reports from the cron job, but also found 
that we were getting close to 10 times as many logs from the other 
sources. this was a low-end box, but it was doing nothing else.

David Lang

> t.s
>
>
> On Feb 26, 2008, at 4:12 PM, david () lang hm wrote:
>
> > On Mon, 25 Feb 2008, Brian Loe wrote:
> >
> > > On Fri, Feb 22, 2008 at 8:06 PM,  <david () lang hm> wrote:
> > >
> > > > > I've found that if you utilize, for instance, syslog-ng, you can
> > > > > split
> > > > > up the log files based on whatever (device type, network, etc.).
> > > > > Searching those smaller files is a lot less CPU intensive.
> > > >
> > > > true, but I found that syslog-ng was far less effective at the more
> > > > important job of receiving syslog messages from the wire and
> > > > writing them
> > > > to disk
> > >
> > > Really? How so?
> > >
> > > We were logging 6 PIXen as well as many switches and routers (and a
> > > much lesser level). We never "noticed" a great loss of messages... I
> > > guess I can assume you did, and maybe I could learn from how you did!
> > > :)
> > >
> > > What daemon do you use?
> >
> > we tried to use syslog-ng to receive activity from our border router
> > and
> > write a copy locally (in large chunks) and relay the logs to another
> > syslog server inside.
> >
> > we noticed a LOT of missing logs, when we changed to the default
> > debian
> > syslogd we were able to handle an order of magnatude more logs
> > without any
> > sign of missing logs (from around 100/sec to >1000/sec)
> >
> > David Lang
> > _______________________________________________
> > firewall-wizards mailing list
> > firewall-wizards () listserv icsalabs com
> > https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards
>
> _______________________________________________
> firewall-wizards mailing list
> firewall-wizards () listserv icsalabs com
> https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards
_______________________________________________
firewall-wizards mailing list
firewall-wizards () listserv icsalabs com
https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards