Re: Wireshark memory handling

[Jeff Morriss <jeff.morriss.ws () gmail com>]

Erlend Hamberg wrote:
> On Friday 9. October 2009 03.47.16 didier wrote:
> > I don't see what you would get with mmaped files vs enough swap. But if
> > you are using wireshark, ie working interactively, it'd be slow, slow as
> > in unusable.

One advantage of using memory mapped files instead of swap is that if 
your OS is swapping, *everything* is slow.  If only Wireshark is, er, 
swapping, only Wireshark is slow.

(It had been a while since I had experienced real swapping--then I left 
tshark running on a buildbot fuzz failure.  After tshark used half my 
swap, it took a good 10-15 minutes for the process to die after I killed 
it--during which time my machine was completely useless.)

> > > But we never use wireshark if it needs to hit harddisks (for us roughly
> > 3 times the file size), it's too slow.
>
> Too slow, full stop? Our experience in using disk-cached data in interactive 
> programs is very limited, but our naïve assumptions were that it that data is 
> sequential and the operating system's disk buffering system does its job, it 
> should be possible to work with this solution. It is of course hard to put 
> exact numbers on how fast something has to be, but if the speed dropped below 
> a level where it's not possible to use the program interactively at all, this 
> solution is of no use.

Give it a try.  Here's what I just did (for fun :-)):

Starting condition: a whole bunch of RAM free (forget how much, probably 
around 2-2.5 Gb), no swap in use.

Start wireshark loading a 7.6 million packet capture file.
Linux appears to keep free RAM in the ~20-30 Mb range, but swap usage 
grows to 1.5 Gb.  Wireshark consumes 4120 Mb.

Scrolling around the packet list is actually not too slow now.  But 
doing even something simple (that does not involve going through the 
packet list) like Statistics->Summary takes 1-2 minutes.  (Normally the 
summary page is basically instantaneous.)  I didn't even try filtering.

Using memory mapped files would probably help quite a bit with keeping 
the UI responsive because only Wireshark's, for example, packet data 
would be on disk but the executable pages and "core" memory like the 
statistics could be kept in RAM (or at least whatever the OS gives us).

One could imagine having different allocators (like ep_ and se_ today) 
that use malloc & friends (for "core" stuff) or go to the mmap'd pool 
(for packet data, etc.).
___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org>
Archives:    http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request () wireshark org?subject=unsubscribe