Re: Snort++ crashes abruptly

[Russ <rucombs () cisco com>]

Awesome, thanks!

On 12/14/16 7:04 PM, João Soares wrote:
> Hi Russ,
>
> Here it goes:
>
> snort:
> /usr/local/src/snort3/src/service_inspectors/http_inspect/http_stream_splitter_reassemble.cc:208:
> virtual const StreamBuffer* HttpStreamSplitter::reassemble(Flow*,
> unsigned int, unsigned int, const uint8_t*, unsigned int, uint32_t,
> unsigned int&): Assertion `total <= MAX_OCTETS' failed.
>
> Program received signal SIGABRT, Aborted.
> [Switching to Thread 0x7fff922b6700 (LWP 65469)]
> 0x00007ffff58671d7 in raise () from /lib64/libc.so.6
> Missing separate debuginfos, use: debuginfo-install
> glibc-2.17-157.el7_3.1.x86_64 hwloc-libs-1.11.2-1.el7.x86_64
> libdnet-1.12-13.1.el7.x86_64 libgcc-4.8.5-11.el7.x86_64
> libpcap-1.5.3-8.el7.x86_64 libstdc++-4.8.5-11.el7.x86_64
> libtool-ltdl-2.4.2-21.el7_2.x86_64 luajit-2.0.4-3.el7.x86_64
> numactl-libs-2.0.9-6.el7_2.x86_64 openssl-libs-1.0.1e-60.el7.x86_64
> pcre-8.32-15.el7_2.1.x86_64 xz-libs-5.2.2-1.el7.x86_64
> zlib-1.2.7-17.el7.x86_64
> (gdb) bt
> #0  0x00007ffff58671d7 in raise () from /lib64/libc.so.6
> #1  0x00007ffff58688c8 in abort () from /lib64/libc.so.6
> #2  0x00007ffff5860146 in __assert_fail_base () from /lib64/libc.so.6
> #3  0x00007ffff58601f2 in __assert_fail () from /lib64/libc.so.6
> #4  0x0000000000532d51 in HttpStreamSplitter::reassemble
> (this=0x7ffef2bbfdd0, flow=0x7fff4c140f90, total=66912,
>      data=0x7ffef01dade0 "GET
> /uploads/2016/05/11/Fotolia_108635123_Subscription_XXL.690x460.60x60.jpg
> HTTP/1.1\r\nHost: www.universal.org\r\nConnection:
> keep-alive\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
> AppleWebKi"..., len=1360, flags=256, copied=@0x7fff920e15ac: 1360) at
> /usr/local/src/snort3/src/service_inspectors/http_inspect/http_stream_splitter_reassemble.cc:208
> #5  0x0000000000560ccb in TcpReassembler::flush_data_segments
> (this=0x7ffef3322b10, p=0x7fff74147110, toSeq=2441337851) at
> /usr/local/src/snort3/src/stream/tcp/tcp_reassembler.cc:484
> #6  0x0000000000561518 in TcpReassembler::_flush_to_seq
> (this=0x7ffef3322b10, bytes=4061, p=0x7fff74147110, pkt_flags=128) at
> /usr/local/src/snort3/src/stream/tcp/tcp_reassembler.cc:641
> #7  0x0000000000561a72 in TcpReassembler::flush_to_seq
> (this=0x7ffef3322b10, bytes=4061, p=0x7fff74147110, pkt_flags=128) at
> /usr/local/src/snort3/src/stream/tcp/tcp_reassembler.cc:743
> #8  0x0000000000561cae in TcpReassembler::flush_stream
> (this=0x7ffef3322b10, p=0x7fff74147110, dir=128) at
> /usr/local/src/snort3/src/stream/tcp/tcp_reassembler.cc:814
> #9  0x0000000000561d58 in TcpReassembler::final_flush
> (this=0x7ffef3322b10, p=0x7fff74147110, peg=@0x7fff9222d540: 1137,
> dir=128) at /usr/local/src/snort3/src/stream/tcp/tcp_reassembler.cc:833
> #10 0x0000000000561ebf in TcpReassembler::flush_queued_segments
> (this=0x7ffef3322b10, flow=0x7fff4c140f90, clear=true, p=0x0) at
> /usr/local/src/snort3/src/stream/tcp/tcp_reassembler.cc:847
> #11 0x000000000054cd8b in TcpSession::clear_session
> (this=0x7ffef0c5a760, free_flow_data=true, flush_segments=true,
> restart=false, p=0x0) at
> /usr/local/src/snort3/src/stream/tcp/tcp_session.cc:170
> #12 0x000000000056589d in TcpStreamSession::cleanup
> (this=0x7ffef0c5a760) at
> /usr/local/src/snort3/src/stream/libtcp/tcp_stream_session.cc:432
> #13 0x00000000005c5243 in Flow::reset (this=0x7fff4c140f90,
> do_cleanup=true) at /usr/local/src/snort3/src/flow/flow.cc:130
> #14 0x00000000005cddf0 in FlowCache::release (this=0x7fff74e6ffa0,
> flow=0x7fff4c140f90, reason=IDLE, do_cleanup=true) at
> /usr/local/src/snort3/src/flow/flow_cache.cc:149
> #15 0x00000000005ce3fd in FlowCache::timeout (this=0x7fff74e6ffa0,
> num_flows=1, thetime=1481759993) at
> /usr/local/src/snort3/src/flow/flow_cache.cc:317
> #16 0x00000000005c66db in FlowControl::timeout_flows
> (this=0x7fff743cf780, cur_time=1481759993) at
> /usr/local/src/snort3/src/flow/flow_control.cc:233
> #17 0x000000000053e472 in Stream::timeout_flows (cur_time=1481759993) at
> /usr/local/src/snort3/src/stream/stream.cc:379
> #18 0x00000000005a7ecd in Snort::packet_callback (pkthdr=0x7fff920e1a50,
> pkt=0x7fff724ee042 "") at /usr/local/src/snort3/src/main/snort.cc:855
> #19 0x0000000000651261 in pcap_process_loop (user=0x7fff74000a50
> "\300\b", pkth=<optimized out>, data=0x7fff724ee042 "") at daq_pcap.c:370
> #20 0x00007ffff797d99e in pcap_handle_packet_mmap () from
> /lib64/libpcap.so.1
> #21 0x00007ffff7981ae1 in pcap_read_linux_mmap_v2 () from
> /lib64/libpcap.so.1
> #22 0x000000000065138b in pcap_daq_acquire (handle=0x7fff74000a50,
> cnt=0, callback=<optimized out>, metaback=<optimized out>,
> user=<optimized out>) at daq_pcap.c:388
> #23 0x00000000006263a4 in SFDAQInstance::acquire (this=0x7fff74000980,
> max=0, callback=0x5a7d38 <Snort::packet_callback(void*, _daq_pkthdr
> const*, unsigned char const*)>)
>      at /usr/local/src/snort3/src/packet_io/sfdaq.cc:492
> #24 0x000000000059db64 in Analyzer::analyze (this=0x7fff95c1c9f0) at
> /usr/local/src/snort3/src/main/analyzer.cc:219
> #25 0x000000000059d789 in Analyzer::operator() (this=0x7fff95c1c9f0,
> ps=0x7fff95c1cbb0) at /usr/local/src/snort3/src/main/analyzer.cc:112
> #26 0x000000000047c635 in std::__invoke<Analyzer<Swapper*> > (__f=...)
> at /usr/include/c++/4.8.2/functional:234
> #27 0x000000000047c5ef in
> std::reference_wrapper<Analyzer>::operator()<Swapper*>(Swapper*&&) const
> (this=0x7fff95780558) at /usr/include/c++/4.8.2/functional:467
> #28 0x000000000047c56d in
> std::_Bind_simple<std::reference_wrapper<Analyzer>
> (Swapper*)>::_M_invoke<0ul>(std::_Index_tuple<0ul>)
> (this=0x7fff95780550) at /usr/include/c++/4.8.2/functional:1732
> #29 0x000000000047c475 in
> std::_Bind_simple<std::reference_wrapper<Analyzer>
> (Swapper*)>::operator()() (this=0x7fff95780550) at
> /usr/include/c++/4.8.2/functional:1720
> #30 0x000000000047c40e in
> std::thread::_Impl<std::_Bind_simple<std::reference_wrapper<Analyzer>
> (Swapper*)> >::_M_run() (this=0x7fff95780538) at
> /usr/include/c++/4.8.2/thread:115
> #31 0x00007ffff61c0230 in ?? () from /lib64/libstdc++.so.6
> #32 0x00007ffff734bdc5 in start_thread () from /lib64/libpthread.so.0
> #33 0x00007ffff592973d in clone () from /lib64/libc.so.6
>
> If you need anything else, I'll do my best.
>
> Best regards
>
> On 12/14/2016 03:53 PM, Russ wrote:
> > If you configure with --enable-debug and run in a debugger you should
> > get the full call stack.
> >
> > On 12/14/16 10:39 AM, João Soares wrote:
> > > Thanks for your fast reply.
> > >
> > > Is there any built-in option that does what you are asking? By stracing
> > > snort I got these results:
> > >
> > > ... (thousands and thousands of nanosleeps)
> > > nanosleep({0, 1000000}, NULL)           = 0
> > > nanosleep({0, 1000000}, NULL)           = 0
> > > nanosleep({0, 1000000}, NULL)           = 0
> > > nanosleep({0, 1000000}, NULL)           = 0
> > > nanosleep({0, 1000000}, NULL)           = 0
> > > nanosleep({0, 1000000}, NULL)           = 0
> > > nanosleep({0, 1000000},  <unfinished ...>
> > > +++ killed by SIGABRT +++
> > >
> > > Executing snort with -v, doesn't give me any more info other than what I
> > > already provided:
> > >
> > > snort:
> > > /usr/local/src/snort3/src/service_inspectors/http_inspect/http_stream_splitter_reassemble.cc:208:
> > >
> > > virtual const StreamBuffer* HttpStreamSplitter::reassemble(Flow*,
> > > unsigned int, unsigned int, const uint8_t*, unsigned int, uint32_t,
> > > unsigned int&): Assertion `total <= MAX_OCTETS' failed.
> > > Aborted
> > >
> > > On 12/14/2016 02:23 PM, Russ wrote:
> > > > Ouch.  Thanks for reporting this.  Can you provide a full backtrace?
> > > >
> > > > On 12/14/16 9:15 AM, João Soares wrote:
> > > > > Hi everyone,
> > > > >
> > > > > I've just updated Snort++ to Version 3.0.0-a4 (Build 221) and it is
> > > > > crashing from time to time. I've collected the following errors:
> > > > >
> > > > > AppIdDbg failed to create a related flow for xxx.xx.xx.xx-0 ->
> > > > > yyy.yy.yy.yy-52094 17
> > > > >
> > > > > (The crash does not happen here)
> > > > >
> > > > > snort:
> > > > > /usr/local/src/snort3/src/service_inspectors/http_inspect/http_stream_splitter_reassemble.cc:208:
> > > > >
> > > > > virtual const StreamBuffer* HttpStreamSplitter::reassemble(Flow*,
> > > > > unsigned int, unsigned int, const uint8_t*, unsigned int, uint32_t,
> > > > > unsigned int&): Assertion `total <= MAX_OCTETS' failed.
> > > > >
> > > > > (It crashes here)
> > > > >
> > > > > Does anyone have any idea why this is happening? If you need additional
> > > > > info, please reply, I will provide it ASAP.
> > > > >
> > > > > Best regards,
> > > > > João Soares
> > > > > ------------------------------------------------------------------------------
> > > > >
> > > > > Check out the vibrant tech community on one of the world's most
> > > > > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > > > > _______________________________________________
> > > > > Snort-users mailing list
> > > > > Snort-users () lists sourceforge net
> > > > > Go to this URL to change user options or unsubscribe:
> > > > > https://lists.sourceforge.net/lists/listinfo/snort-users
> > > > > Snort-users list archive:
> > > > > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
> > > > >
> > > > > Please visit http://blog.snort.org to stay current on all the latest
> > > > > Snort news!
> > > > ------------------------------------------------------------------------------
> > > >
> > > > Check out the vibrant tech community on one of the world's most
> > > > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > > > _______________________________________________
> > > > Snort-users mailing list
> > > > Snort-users () lists sourceforge net
> > > > Go to this URL to change user options or unsubscribe:
> > > > https://lists.sourceforge.net/lists/listinfo/snort-users
> > > > Snort-users list archive:
> > > > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
> > > >
> > > > Please visit http://blog.snort.org to stay current on all the latest
> > > > Snort news!


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!