Snort mailing list archives

Previous By Date Next
Previous By Thread Next

Re: Segfault building Snort3 b245 with --enable-tcmalloc flag

From: Noah Dietrich <noah_dietrich () 86penny org>
Date: Fri, 15 Jun 2018 12:19:14 +0200
Thank you Michael for the deep-dive analysis of this issue. It looks that
installing gperftools 2.7 from github fixes the problem, although I haven't
done more than just run snort with the -V flag. The steps it took for this
are below.
Has anyone done an analysis of the performance benefits of using tcmalloc?
I'd assume that you're getting more speed for the trade-off of increased
memory usage, but it would be interesting if someone has some results to
look at.

Thanks,
Noah


*# remove gperftools 2.5 from system:*
sudo apt-get remove libgoogle-perftools-dev
sudo apt-get autoremove

*$ install gperftools 2.7 from github from source:*
cd ~
wget
https://github.com/gperftools/gperftools/releases/download/gperftools-2.7/gperftools-2.7.tar.gz
tar xzvf gperftools-2.7.tar.gz
cd gperftools-2.7/
./configure
make
sudo make install

*# get, build, and install Snort3*
cd ~/snort_src/
git clone https://github.com/snort3/snort3.git
cd snort3/
./configure_cmake.sh --prefix=/opt/snort3a4 --enable-tcmalloc
cd build/
make
sudo make install

*# Verify libraries linked and that snort3 runs:*
noah@snort18x:~$ ldd /opt/snort3a4/bin/snort
...
libtcmalloc.so.4 => /usr/local/lib/libtcmalloc.so.4 (0x00007fc7ce334000)
...

noah@snort18x~$ /opt/snort3a4/bin/snort -V

   ,,_     -*> Snort++ <*-
  o"  )~   Version 3.0.0 (Build 245) from 2.9.11
   ''''    By Martin Roesch & The Snort Team
           http://snort.org/contact#team
           Copyright (C) 2014-2018 Cisco and/or its affiliates. All rights
reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using DAQ version 2.2.2
           Using LuaJIT version 2.1.0-beta3
           Using OpenSSL 1.1.0g  2 Nov 2017
           Using libpcap version 1.8.1
           Using PCRE version 8.39 2016-06-14
           Using ZLIB version 1.2.11
           Using FlatBuffers 1.9.0
           Using Hyperscan version 4.7.0 2018-06-13
           Using LZMA version 5.2.2


On Thu, Jun 14, 2018 at 6:16 AM, Michael Altizer via Snort-devel <
snort-devel () lists snort org> wrote:


I can reproduce it on my Ubuntu 18.04 system, but it doesn't look like a
bug in Snort.  From combing through the backtrace and stepping through the
code, it looks like a messy bug in the interaction between tcmalloc,
libhwloc, and libltdl.  libhwloc uses libltdl to iterate over and load all
of its plugins.  Somewhere in the middle of "canonicalizing" the paths of
the plugins it is evaluating, libltdl's loop gets horribly confused and
starts writing over itself forever until get gets a SEGV.  In stepping
through it in a debugger, it appears that the malloc it calls to create the
new character array to write into returns an address 16 bytes into the
original path.  I didn't dig into the assembly, but it might actually be a
compiler issue where it optimized libltdl around assumptions using builtin
malloc and then it gets hosed by tcmalloc doing the actual allocation.
Alternatively, something in what it's doing is buggily stomping on
tcmalloc's internal metadata, allowing it to give out the "path" memory
again even while it's being used and read.  Or, who knows, maybe there's a
bug in the gperftools 2.5 that Ubuntu is packing (we tested with 2.7)?  Not
that it's a great solution, but I can make the crash go away by
uninstalling the hwloc-plugins package.

On 06/13/2018 12:50 AM, Noah Dietrich wrote:

Hello,

thanks for tracking that down.  I have verified that the segfault issue
also occurs when building on Ubuntu 16 x64.

Changing the command to:
*      ./configure_cmake.sh --prefix=/opt/snort --enable-tcmalloc
--disable-docs*
does not generate any errors and allows for successful build and
installation.

however, running snort3 when* -enable-tcmalloc *is included as above
generates a segfault;
   noah@snort18x:~/snort_src/snort3/build$* /opt/snort/bin/snort -V*
   Segmentation fault (core dumped)

Please let me know if any further info is needed.
thanks
Noah




On Tue, Jun 12, 2018 at 8:32 PM, Y M via Snort-devel <
snort-devel () lists snort org> wrote:


After looking at the errors, which I received too, the build fails at
generating the text documentation at the target prefix. Using the
--disable-text-docs alone did not stop the documentation generation.
However, disabling all types of documentation generation resulted in a
successful build.

# ./configure_cmake.sh --prefix=/usr/local/snort --enable-tcmalloc
--disable-text-docs --disable-html-docs --disable-pdf-docs --disable-docs

...
[ 99%] Built target preprocessor_states
Scanning dependencies of target snort2lua
[100%] Building CXX object tools/snort2lua/CMakeFiles/sno
rt2lua.dir/init_state.cc.o
[100%] Building CXX object tools/snort2lua/CMakeFiles/sno
rt2lua.dir/snort2lua.cc.o
[100%] Linking CXX executable snort
[100%] Linking CXX executable snort2lua
[100%] Built target snort2lua
[100%] Built target snort

This doesn't particularly answers your question, but hopefully can help
you move along.

Thanks.
YM

------------------------------
*From:* Snort-devel <snort-devel-bounces () lists snort org> on behalf of Y
M via Snort-devel <snort-devel () lists snort org>
*Sent:* Tuesday, June 12, 2018 8:25 PM
*To:* snort-devel () lists snort org
*Subject:* Re: [Snort-devel] Segfault building Snort3 b245 with
--enable-tcmalloc flag

Interesting. Snort build 245 with tcmalloc over CetnOS 7 and FreeBSD 11
was successful. There have been recent changes in Ubuntu 18 that are making
several apps too unhappy. I will do some testing on my side and report back
if I come up with anything useful.

YM

------------------------------
*From:* Snort-devel <snort-devel-bounces () lists snort org> on behalf of
Noah Dietrich <noah_dietrich () 86penny org>
*Sent:* Tuesday, June 12, 2018 5:26 PM
*To:* snort-devel () lists snort org
*Subject:* [Snort-devel] Segfault building Snort3 b245 with
--enable-tcmalloc flag

Hello,

when building Snort 3 245 with the --enable-tcmalloc flag, snort throws a
segfault.

*Environment: *git clone of the latest Snort3 (b245) on Ubuntu 18 x64 in
a VMware virtual Machine.

*Commands:*
cd ~/snort_src
git clone https://github.com/snort3/snort3.git
cd ./snort3
./configure_cmake.sh --prefix=/usr/local  --enable-tcmalloc
cd build
make

*Error:*
...
Scanning dependencies of target all_built_sources
[ 97%] Documents: building commands.txt with
Segmentation fault (core dumped)
doc/CMakeFiles/all_built_sources.dir/build.make:80: recipe for target
'doc/commands.txt' failed
make[2]: *** [doc/commands.txt] Error 139
make[2]: *** Deleting file 'doc/commands.txt'
CMakeFiles/Makefile2:6091: recipe for target
'doc/CMakeFiles/all_built_sources.dir/all' failed
make[1]: *** [doc/CMakeFiles/all_built_sources.dir/all] Error 2
Makefile:151: recipe for target 'all' failed
make: *** [all] Error 2



When you remove the --enable-tcmalloc flag, snort compiles and runs
without error:

   ,,_     -*> Snort++ <*-
  o"  )~   Version 3.0.0 (Build 245) from 2.9.11
   ''''    By Martin Roesch & The Snort Team
           http://snort.org/contact#team
           Copyright (C) 2014-2018 Cisco and/or its affiliates. All
rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using DAQ version 2.2.2
           Using LuaJIT version 2.1.0-beta3
           Using OpenSSL 1.1.0g  2 Nov 2017
           Using libpcap version 1.8.1
           Using PCRE version 8.39 2016-06-14
           Using ZLIB version 1.2.11
           Using FlatBuffers 1.9.0
           Using Hyperscan version 4.7.0 2018-06-12
           Using LZMA version 5.2.2


please let me know if you need any other information.
thank you,
Noah



_______________________________________________
Snort-devel mailing list
Snort-devel () lists snort org
https://lists.snort.org/mailman/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!





_______________________________________________
Snort-devel mailing listSnort-devel@lists.snort.orghttps://lists.snort.org/mailman/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!



_______________________________________________
Snort-devel mailing list
Snort-devel () lists snort org
https://lists.snort.org/mailman/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!



_______________________________________________
Snort-devel mailing list
Snort-devel () lists snort org
https://lists.snort.org/mailman/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!
Previous By Date Next
Previous By Thread Next

Current thread: