Wireshark mailing list archives
Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.)
From: Bálint Réczey <balint () balintreczey hu>
Date: Sat, 12 Jul 2014 02:42:33 +0200
2014-07-12 0:07 GMT+02:00 Anders Broman <a.broman58 () gmail com>:
Den 11 jul 2014 23:13 skrev "Bálint Réczey" <balint () balintreczey hu>:Hi All, Please provide the input data for letting others reproduce the results or perform the performance tests on pcap files already available to the publicOk I'll see if we can use something from the wiki instead.I'm not a fan of implementing custom memory management methods because partly because I highly doubt we can beat jemalloc easily on performance and custom allocation methods can also have nasty bugs like the one observed in OpenSSL: http://www.tedunangst.com/flak/post/analysis-of-openssl-freelist-reuseWe have gone through a set of memory allocation schemes already to try to improve performance (g_slice, emen and now wmem) are you saying that you are opposed to that?
No, IMO there is need for wmem for being able to keep memory allocated during scopes, but I would prefer seeing it only tracking and one-by-one freeing the memory areas instead of doing bulk alloc()-s for optimizing for speed. I tried to detail this in my answer to Evan's email.
As wmem seems to be the faster scheme for packet scope memory allocation /free, why not use it in all possible places where the scope is "packet"?Please don't sacrifice protection for 2% speedup. Please keep wmem usage for cases where it is used for garbage collecting (free() after end of frame/capture file) not when the allocation and deallocation are already done properly.? A slow scheme might be working well but that in it self does not warrant to not replace it with a faster one. With this reasoning we shouldn't have introduced ep memory in the first place. What percentage if improvement do you think makes a change worthwhile? The set of improvements Jacub and I have done lately has given a reduction of 40-50 percent compared to 1.10 measuring with the sample file. The problem is that each improvement only yeald a percent or 2. Agreed that we shouldn't complicate the code for a small speed gain.
40-50% reduction is great and congratulations for such a speedup! I hope memory allocation handling is responsible for only small fraction of it because I would like to keep the possibility of detecting memory allocation related errors and I would prefer using tools implemented outside of Wireshark. For example I would avoid bulk allocations to make us able to use ASAN and Valgrind, even if we would implement our canaries.
In your blog you say that people would accept double the execution time with increased security - I'm not so sure. If say the reformatting of a video takes one hour instead of 30 minutes.
In Debian you would be able to pick a slow and secure video player for streaming from the untrusted Internet and a fast and less secure video encoder. I expect people to install Wireshark from the hardened-amd64 repository if they want to monitor a hostile network, while others pick the fast Wireshark for using it in their safe labs. So it depends and there will be good options to choose from. Cheers, Balint
Just my 2 cents AndersThanks, Balint 2014-07-11 8:58 GMT+02:00 Jakub Zawadzki <darkjames-ws () darkjames pl>:Hi, On Sat, Jun 21, 2014 at 10:12:48PM +0200, Jakub Zawadzki wrote:If we're in topic of optimizing 'slower' [de]allocations in common functions: - tvb allocation/deallocation (2.5%, or 3.4% when no filtering) 243,931,671 * ???:tvb_new [/tmp/wireshark/epan/.libs/libwireshark.so.0.0.0] 202,052,290 > ???:g_slice_alloc (2463493x) [/usr/lib64/libglib-2.0.so.0.3600.4] 291,765,126 * ???:tvb_free_chain [/tmp/wireshark/epan/.libs/libwireshark.so.0.0.0] 256,390,635 > ???:g_slice_free1 (2435843x) [/usr/lib64/libglib-2.0.so.0.3600.4]This, or next week I'll try to do tvb.... or maybe this week: ver0 | 18,055,719,820 (-----------) | Base version 96f0585268f1cc4e820767c4038c10ed4915c12a ver1 | 18,185,185,838 (0.6% slower) | Change tvb allocator g_slice_* to wmem with file scope ver2 | 17,809,433,204 (1.4% faster) | Change tvb allocator g_slice_* to wmem with file/packet scope ver3 | 17,812,128,887 (1.3% faster) | Change tvb allocator g_slice_* to simple object allocator with epan scope ver4 | 17,704,132,561 (2.0% faster) | Change tvb allocator g_slice_* to simple object allocator with file scope I have uploaded patches & profiler outputs to: http://www.wireshark.org/~darkjames/tvb-opt-allocator/ Please review, and check what version is OK to be applied. P.S: I'll might find some time to do ver5 with slab allocator, but it'll look like object allocator API with epan scope. Cheers, Jakub. ___________________________________________________________________________ Sent via: Wireshark-dev mailing list <wireshark-dev () wireshark org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-request () wireshark org?subject=unsubscribe___________________________________________________________________________ Sent via: Wireshark-dev mailing list <wireshark-dev () wireshark org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-request () wireshark org?subject=unsubscribe
___________________________________________________________________________ Sent via: Wireshark-dev mailing list <wireshark-dev () wireshark org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-request () wireshark org?subject=unsubscribe
Current thread:
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.), (continued)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Bálint Réczey (Jul 11)
- Message not available
- Fwd: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) darkjames-ws (Jul 21)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Bálint Réczey (Jul 24)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 24)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Bálint Réczey (Jul 24)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Evan Huus (Jul 11)
- Re: tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting epan_dissect_t when filtering.) Bálint Réczey (Jul 11)