Re: Question involving segmentation fault

[Phil Wood <cpw () lanl gov>]

Folks,

Here is an update on the problem I reported involving CleanExit and
segmentation fault.  It still could be all my problem since I use a modified
libpcap.  However, tcpdump does not appear to have the problem.  And snort
only seems to have it with a specific mix of traffic and only after I
cause it to quit with a TERM siganl, or in this recent case, by causing
pcap_loop to "error" by using an option I have put in libpcap to "ETIMEDOUT"
once the packet time is greater that a value I supplied (in this case,
midnight).  By doing so I've ruled out the "signal" handling mechanisms.

I checked some of the global variables in CleanExit and found that I got
there and set pv.pid_filename[0] = 0;  However, snort never called
DropStats.  So, I'm thinking somewhere in

    while(idx != NULL)
    {
        idx->func(sig, idx->arg);
        idx = idx->next;
    }

it bombed.

I'm thinking that some part of the stack is being clobbered.  Here is some
debug information:

(gdb) where
  #0  0x4f452525 in ?? ()
  #1  0x0804ed98 in InterfaceThread (arg=0x0) at snort.c:1720
  #2  0x0804b284 in main (argc=19, argv=0xbffff614) at snort.c:480
  #3  0x401a765f in __libc_start_main () from /lib/libc.so.6

(gdb) up
  #1  0x0804ed98 in InterfaceThread (arg=0x0) at snort.c:1720
  1720            CleanExit(SIGQUIT);
  (gdb) list 1694
  1689                                * any interfaces, and no threads
  1690                                * would start on the same interface
  1691                                * simultaneously */
  1692    #endif
  1693
  1694        myint = intnum;
  1695        intnum++;
  1696
  1697    #ifdef USE_PTHREADS
  1698        pthread_mutex_unlock(&pt_lock);
  1699    #endif
  1700
  1701        if(pv.test_mode_flag)
  1702        {
  1703       LogMessage("\nSnort sucessfully loaded all ...
  1704            CleanExit(SIGQUIT);
  1705        }
  1706
  1707        if(pv.daemon_flag)
  1708        {
  1709            LogMessage("Snort initialization completed successfully ...
  1710        }
  1711
  1712    /* Read all packets on the device.  Continue until cnt packets read */
  1713    if(pcap_loop(pds[myint], pv.pkt_cnt,
                                    (pcap_handler) ProcessPacket, NULL) < 0)
  1714        {
  1715            if(pv.daemon_flag)
  1716          syslog(LOG_CONS | LOG_DAEMON, "pcap_loop: %s", pcap_geterr(pd));
  1717            else
  1718                ErrorMessage("pcap_loop: %s", pcap_geterr(pd));
**     the following message was printed by the pcap library:
         pcap_loop: User specified timeout occured
       which is what I expect for this run of snort.
  1719
**1720            CleanExit(SIGQUIT);
  1721        }
  1722        CleanExit(SIGQUIT);
  1723
  1724        return NULL;                /* avoid warnings */
  1725    }
  1726
  1727
  1728
(gdb) print pv.daemon_flag
  $15 = 0           <- correct not running in daemon mode
(gdb) print intnum  <- a static int
  $13 = 1
(gdb) print myint   <- just an int  (should == 0)
  $14 = 135252968
** for what its worth, if you 
(gdb) x/s 135252968
  0x80fcbe8:       "%%EOF\r"
** which is part of strings found in the general area like so:
(gdb) x/s 0x80fca00
0x80fca00:       "\f 
ÍÖf\227°\223ØL&º\f(A\204ÐfÞ\203\b4\"!\234\027\210\206mÿ\006\020dpNî\030!\rl¾G\006\fIÝ\236³zh\030&\177÷nÐí\006\203\013Ò\220Ìm0±\021\0340A¡\021\020Á\020Àä\r¡¬9\003Á\225Ã\210\210°\230T#³a\200\203#\201\202>¦\030XgÐ\"\rqä\016a\225\002\"#\210\210\217¯\204\210\034Ãdï
 ¶\006&\222\aÕx\217\217ÿÿÿÿä2@àr\a&ä\017\003\207ªö@ðh\034\206\233\220<6 
\206H2\016A¯\210\036\017ä2\003YÈ\026E\220,»!\220£ÓL\020d4 \2060\203á\006 1 00"...
(gdb) x/s
0x80fcac8:       "000 n \r0000505655 00000 n \r0000506438 00000 n \r0000506665 00000 n \r0000511633 00000 n 
\r0000512018 00000 n \r0000512943 00000 n \r0000513090 00000 n \rtrailer\r<<\r/Size 115\r/Info 3 0 R \r/Root 1 0 R 
\r/ID[<3"...
(gdb) x/s
0x80fcb90:       "67c1346077949b560c4094b5d5ace24><367c1346077949b560c4094b5d5ace24>]\r>>\rstartxref\r513227\r%%EOF\r"

Does anyone have ideas about what to poke at?  I guess I could start backing
off on the preprocessors.

Thanks in advance,

Phil


On Wed, Feb 06, 2002 at 10:07:39AM -0700, Phil Wood wrote:
> Folks,
>
> I've got a re-occuring problem to do with signal handling in snort.
> It has happened at one time or another with ever version up to the current
> (1.8.4-beta1).  And that is, sometimes when I send a TERM signal to
> the snort process, it will segmentation fault.  This doesn't occur always,
> but usually after a realtively long run (24 hours).  I restart the
> snort process every 24 hours by issuing the kill command.  The process
> is encapsulated in a shell script which packages up the results, moves
> any core files to a unique name, and restarts the snort process. 
>
> I even tried using the tcpdump signal handling stuff to see if it might
> be some bollixed up linux libc library problem. However, nothing I've done
> has made any difference.  It happens on redhat? and debian potato linux 
> systems.  It could be some problem I've introduced through my own munging.
> But, I just thought I'd ask and see if anyone else is seeing this problem.
>
> Thanks,
>
> Phil
>
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users

-- 
Phil Wood, cpw () lanl gov


_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users