Re: pcap_next_ex() and pcap_dump() performance decreases over time...

[barcaroller <barcaroller () gmail com>]

On 2015-08-22 23:21:55 +0000, Guy Harris said:

> > I've been running performance tests against pcap_next_ex() and 
> > pcap_dump().  I've placed micro-second timers around both functions
>
> So those are presumably real-time timers rather than CPU timers.

Using the Boost timer functions.  I could be wrong but I think they are 
based on gettimeofday().


> > and sent millions of packets to my test programs.
> >
> > Both functions performed admirably over a short period of time (up to 8 
> > Gbits/sec) but, as time progressed, both functions became increasingly 
> > slower until the data rate dropped to 200 Mbits/sec.
>
> How short is "short"?  Tens of seconds?  Minutes?

Minutes.  If I run my test programs for two minutes, the average rates 
are great.  If I run them for 15 minutes, the average rates drop 
dramatically.

I also keep track of the maximum times for individual pcap_next_ex() 
and pcap_dump() calls.  For the two minute test, the maximum (not 
average) for both functions is <10 microseconds.  For the 15 minute 
test, the maximum (not average) for both functions can be as high as 
2,000,000 microseconds (2 seconds).


> > Please note that the two functions become slower even when I read/wrote 
> > new pcap files (i.e. I would close a pcap file and start 
> > reading/writing a new PCAP file, but the performance still remained 
> > low).
>
> What happens if you have a test program that doesn't write the packets out?

Currently the test programs read and write, but not at the same time.  
The first phase writes out the packets; the second phase reads them in.


> > Does anyone know how to explain this?
>
> Not without at least knowing on what OS you're doing this.  Those 
> routines make system calls, so there might be a *kernel* resource 
> involved.

CentOs 6.5, 8 cores, 16GB RAM, lots of free disk space


_______________________________________________
tcpdump-workers mailing list
tcpdump-workers () lists tcpdump org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers