Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.)

[Evan Huus <eapache () gmail com>]

On Fri, Jul 11, 2014 at 5:12 PM, Bálint Réczey <balint () balintreczey hu>
wrote:

> Hi All,
>
> Please provide the input data for letting others reproduce the results
> or perform the performance tests on pcap files already available to
> the public.
>
> I'm not a fan of implementing custom memory management methods because
> partly because I highly doubt we can beat jemalloc easily on
> performance


The only place we reliably beat jemalloc (or even glib) is when we have a
large number of allocations that live together, and can be freed with
free_all. Anything else is basically noise. As Jakub's test noted, the main
block allocator is actually slightly slower than g_slice_* if the frees are
done manually.


> and custom allocation methods can also have nasty bugs
> like the one observed in OpenSSL:
> http://www.tedunangst.com/flak/post/analysis-of-openssl-freelist-reuse
>
> I wrote a short post about making all programs in Debian resistant to
> malloc() related attacks using ASAN and wmem in its current form
> prevents implementing the protection:
>
> http://balintreczey.hu/blog/proposing-amd64-hardened-architecture-for-debian/

It's not clear to me from reading the blog post or the mail to debian what
the actual protections would be, or why wmem would prevent them from
working. Could you clarify please? Glib has its own allocator logic
internally for g_slice_*, does this also cause problems?


> Please don't sacrifice protection for 2% speedup. Please keep wmem
> usage for cases where it is used for garbage collecting (free() after
> end of frame/capture file) not when the allocation and deallocation
> are already done properly.
>
> Thanks,
> Balint
>
> 2014-07-11 8:58 GMT+02:00 Jakub Zawadzki <darkjames-ws () darkjames pl>:
> > Hi,
> >
> > On Sat, Jun 21, 2014 at 10:12:48PM +0200, Jakub Zawadzki wrote:
> > > If we're in topic of optimizing 'slower' [de]allocations in common
> functions:
> > > - tvb allocation/deallocation (2.5%, or 3.4% when no filtering)
> > >
> > >    243,931,671  *  ???:tvb_new
> [/tmp/wireshark/epan/.libs/libwireshark.so.0.0.0]
> > >    202,052,290  >   ???:g_slice_alloc (2463493x)
> [/usr/lib64/libglib-2.0.so.0.3600.4]
> > >    291,765,126  *  ???:tvb_free_chain
> [/tmp/wireshark/epan/.libs/libwireshark.so.0.0.0]
> > >    256,390,635  >   ???:g_slice_free1 (2435843x)
> [/usr/lib64/libglib-2.0.so.0.3600.4]
> > > This, or next week I'll try to do tvb.
> >
> > ... or maybe this week:
> >
> > ver0 | 18,055,719,820 (-----------) | Base version
> 96f0585268f1cc4e820767c4038c10ed4915c12a
> > ver1 | 18,185,185,838 (0.6% slower) | Change tvb allocator g_slice_* to
> wmem with file scope
> > ver2 | 17,809,433,204 (1.4% faster) | Change tvb allocator g_slice_* to
> wmem with file/packet scope
> > ver3 | 17,812,128,887 (1.3% faster) | Change tvb allocator g_slice_* to
> simple object allocator with epan scope
> > ver4 | 17,704,132,561 (2.0% faster) | Change tvb allocator g_slice_* to
> simple object allocator with file scope
> > I have uploaded patches & profiler outputs to:
> http://www.wireshark.org/~darkjames/tvb-opt-allocator/
> > Please review, and check what version is OK to be applied.
> >
> >
> > P.S: I'll might find some time to do ver5 with slab allocator, but it'll
> look like object allocator API with epan scope.
> > Cheers,
> > Jakub.
> ___________________________________________________________________________
> > Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org>
> > Archives:    http://www.wireshark.org/lists/wireshark-dev
> > Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
> >              mailto:wireshark-dev-request () wireshark org
> ?subject=unsubscribe
> ___________________________________________________________________________
> Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org>
> Archives:    http://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
>              mailto:wireshark-dev-request () wireshark org
> ?subject=unsubscribe
___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org>
Archives:    http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request () wireshark org?subject=unsubscribe