Snort mailing list archives

Previous By Date Next
Previous By Thread Next

Re: File extraction during http/ftp transaction

From: "Joel Esler (jesler)" <jesler () cisco com>
Date: Wed, 11 Mar 2015 17:38:27 +0000
The name and extension can be faked would be my response.


On Mar 11, 2015, at 1:21 PM, Rishabh Shah <rishabh420 () gmail com<mailto:rishabh420 () gmail com>> wrote:

Hi Hui,

I missed creating the directory(assumed that snort would create one). It is working now. Thanks a ton Hui.

One minor query regarding the new files:
-rw------- 1 root root  7091 Mar 11 22:48 9D29C44863C6A27D45F8621E6A636DF0746245C5F436DB9CA488252A7FF76579
-rw------- 1 root root 22016 Mar 11 22:49 67792ACE824606664CE51973800D6B952CA4733CAF6F03CCF5F636768EFB39B1

Can it not retain the name/extension of the file?

Thanks,
Rishabh.

On Wed, Mar 11, 2015 at 10:12 PM, Hui cao <huica () cisco com<mailto:huica () cisco com>> wrote:
Sorry. Don't change the conf, but check whether you have permission "write" on the folder /home/file_capture/tmp/

Best,
Hui.

On 03/11/2015 12:37 PM, Rishabh Shah wrote:
Hi Hui,

I removed signature and transferred two pcap files, but no luck:

File Preprocessor Statistics
  Total file type callbacks:            2
  Total file signature callbacks:       2
  Total files would saved to disk:      2
  Total files saved to disk:            0
  Total file data saved to disk:        0         bytes
  Total files duplicated:               0
  Total files reserving failed:         0
  Total file capture min:               0
  Total file capture max:               0
  Total file capture memcap:            0
  Total files reading failed:           0
  Total file agent memcap failures:     0
  Total files sent:                     0
  Total file data sent:                 0
  Total file transfer failures:         0
===============================================================================
File type stats:
         Type              Download   (Bytes)      Upload     (Bytes)
        PCAP(145)          2          3870         0          0
            Total          2          3870         0          0

File signature stats:
         Type              Download   Upload
        PCAP(145)          2          0
            Total          2          0

File type verdicts:
        UNKNOWN:           2
            LOG:           0
           STOP:           0
          BLOCK:           0
         REJECT:           0
        PENDING:           0
   STOP CAPTURE:           0
          Total:           2

File signature verdicts:
        UNKNOWN:           2
            LOG:           0
           STOP:           0
          BLOCK:           0
         REJECT:           0
        PENDING:           0
   STOP CAPTURE:           0
          Total:           2

Total files processed:             2
Total files data processed:        3870      bytes
Total files buffered:              2
Total files released:              2
Total files freed:                 0
Total files captured:              2
Total files within one packet:     2
Total buffers allocated:           2
Total buffers freed:               0
Total buffers released:            2
Maximum file buffers used:         1
Total buffers free errors:         0
Total buffers release errors:      0
Total memcap failures:             0
Total memcap failures at reserve:  0
Total reserve failures:            0
Total file capture size min:       0
Total file capture size max:       0
Total capture max before reserve:  0
Total file signature max:          0
Maximum buffers can allocate:      3196
Number of buffers in use:          0
Number of buffers in free list:    3194
Number of buffers in release list: 2



On Wed, Mar 11, 2015 at 10:02 PM, Hui cao <huica () cisco com<mailto:huica () cisco com>> wrote:
Can you remove signature? If this is enabled, it only captures file that matches to a signature list.

preprocessor file_inspect: type_id, capture_disk /home/file_capture/tmp/, capture_queue_size 5000

Best,
Hui.


On 03/11/2015 12:24 PM, Rishabh Shah wrote:
Hi Hui,

I included file_magic.conf in my snort configuration file. After starting the snort process, I transferred 3 files and 
this is the output after stopping snort:

File Preprocessor Statistics
  Total file type callbacks:            1
  Total file signature callbacks:       1
  Total files would saved to disk:      1
  Total files saved to disk:            0
  Total file data saved to disk:        0         bytes
  Total files duplicated:               0
  Total files reserving failed:         0
  Total file capture min:               0
  Total file capture max:               0
  Total file capture memcap:            0
  Total files reading failed:           0
  Total file agent memcap failures:     0
  Total files sent:                     0
  Total file data sent:                 0
  Total file transfer failures:         0
===============================================================================
File type stats:
         Type              Download   (Bytes)      Upload     (Bytes)
        PCAP(145)          1          1935         0          0
            Total          1          1935         0          0

File signature stats:
         Type              Download   Upload
        PCAP(145)          1          0
            Total          1          0

File type verdicts:
        UNKNOWN:           1
            LOG:           0
           STOP:           0
          BLOCK:           0
         REJECT:           0
        PENDING:           0
   STOP CAPTURE:           0
          Total:           1

File signature verdicts:
        UNKNOWN:           1
            LOG:           0
           STOP:           0
          BLOCK:           0
         REJECT:           0
        PENDING:           0
   STOP CAPTURE:           0
          Total:           1

Total files processed:             3
Total files data processed:        8124      bytes
Total files buffered:              1
Total files released:              1
Total files freed:                 0
Total files captured:              1
Total files within one packet:     1
Total buffers allocated:           1
Total buffers freed:               0
Total buffers released:            1
Maximum file buffers used:         1
Total buffers free errors:         0
Total buffers release errors:      0
Total memcap failures:             0
Total memcap failures at reserve:  0
Total reserve failures:            0
Total file capture size min:       0
Total file capture size max:       0
Total capture max before reserve:  0
Total file signature max:          0
Maximum buffers can allocate:      3196
Number of buffers in use:          0
Number of buffers in free list:    3195
Number of buffers in release list: 1
===============================================================================


On Wed, Mar 11, 2015 at 9:34 PM, Hui cao <huica () cisco com<mailto:huica () cisco com>> wrote:
In READMe.file:

Pre-packaged file magic rules:

A set of file magic rules is packaged with Snort. They can be located at
"etc/file_magic.conf". To use this feature, it is recommended that the
these pre-packaged rules are used; doing so requires that you include
the file in your Snort configuration as such:

  include etc/filemagic.conf

On 03/11/2015 12:01 PM, Hui cao wrote:
Have you added file magic into your configuration. What's the snort output?

Best,
Hui.

On 03/11/2015 11:56 AM, Rishabh Shah wrote:
Thanks Hui. That worked for me!
Now I started snort after adding file_inspect preprocessor.
preprocessor file_inspect: type_id, signature, capture_disk /home/file_capture/tmp/, capture_queue_size 5000

(Got the following console logs to confirm that file_inspect has started)

File config:
    file type: ENABLED
    file signature: ENABLED
    file capture: ENABLED
    file capture directory: /home/file_capture/tmp/
    file capture disk size: 300 (Default) megabytes
    file sent to host: DISABLED (Default), port number: 0

File service: file type enabled.
File service: file signature enabled.
File service: file capture enabled.
File capture thread started tid=0x7f0aaa783700 (pid=19354)


I initiated file transfer via HTTP/FTP as shown below:

rishab%ftp 192.168.2.200
Connected to 192.168.2.200:21<http://192.168.2.200:21/>.
220 (vsFTPd 2.0.5)
Name (192.168.2.200:21:fwdevtest1): fwuser
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> get new.pcap
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for new.pcap (1555 bytes).
226 File send OK.
1555 bytes received in 0.4 seconds (3887 bytes/s)
ftp>
ftp> quit
221 Goodbye.
rishab%wget 192.168.2.200/dns.pcap<http://192.168.2.200/dns.pcap>
--2015-03-11 21:23:16--  http://192.168.2.200/dns.pcap
Connecting to 192.168.2.200:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1935 (1.9K) [text/plain]
Saving to: ?dns.pcap?

100%[======================================================================================================================================================================================>]
 1,935       9.39KB/s   in 0.2s

2015-03-11 21:23:19 (9.39 KB/s) - ?dns.pcap? saved [1935/1935]


After killing the snort process, I do not see any file created in that location:

root@fwuser-virtual-machine:/home# ls
fwuser

Am I missing anything?


On Wed, Mar 11, 2015 at 9:09 PM, Hui cao <huica () cisco com<mailto:huica () cisco com>> wrote:
Have you done make clean before you do a make?

Best,
Hui.


On 03/11/2015 11:38 AM, Rishabh Shah wrote:
Hi Hui,

I am hitting the same issue while executing make. These are the commands that I issued:
root@fwuser-virtual-machine:~/snort_src/snort-2.9.7.0#<mailto:root@fwuser-virtual-machine:%7E/snort_src/snort-2.9.7.0#> 
./configure --enable-file-inspect --enable-open-appid --enable-sourcefire

root@fwuser-virtual-machine:~/snort_src/snort-2.9.7.0#<mailto:root@fwuser-virtual-machine:%7E/snort_src/snort-2.9.7.0#> 
make


/root/snort_src/snort-2.9.7.0/src/plugbase.c:216: undefined reference to `SetupAppId'
detection-plugins/libspd.a(detection_options.o): In function `detection_hash_free_func':
/root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:553: undefined reference to `optionAppIdFree'
detection-plugins/libspd.a(detection_options.o): In function `detection_option_hash_func':
/root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:252: undefined reference to `optionAppIdHash'
detection-plugins/libspd.a(detection_options.o): In function `detection_option_key_compare_func':
/root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:409: undefined reference to `optionAppIdCompare'
collect2: error: ld returned 1 exit status
make[3]: *** [snort] Error 1
make[3]: Leaving directory `/root/snort_src/snort-2.9.7.0/src'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/root/snort_src/snort-2.9.7.0/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/root/snort_src/snort-2.9.7.0'
make: *** [all] Error 2


On Wed, Mar 11, 2015 at 8:40 PM, Hui cao <huica () cisco com<mailto:huica () cisco com>> wrote:
Hi Rishabh,

You need to add —enable-open-appid to you ./configure.

./configure --enable-file-inspect —enable-open-appid

Best,
Hui.

On 03/11/2015 10:33 AM, Rishabh Shah wrote:
Hi Joel,

Thanks for your prompt reply. I did a ./configure --enable-file-inspect and while executing make, I saw the following 
error messages:

/root/snort_src/snort-2.9.7.0/src/plugbase.c:216: undefined reference to `SetupAppId'
detection-plugins/libspd.a(detection_options.o): In function `detection_hash_free_func':
/root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:553: undefined reference to `optionAppIdFree'
detection-plugins/libspd.a(detection_options.o): In function `detection_option_hash_func':
/root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:252: undefined reference to `optionAppIdHash'
detection-plugins/libspd.a(detection_options.o): In function `detection_option_key_compare_func':
/root/snort_src/snort-2.9.7.0/src/detection-plugins/detection_options.c:409: undefined reference to `optionAppIdCompare'
collect2: error: ld returned 1 exit status
make[3]: *** [snort] Error 1
make[3]: Leaving directory `/root/snort_src/snort-2.9.7.0/src'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/root/snort_src/snort-2.9.7.0/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/root/snort_src/snort-2.9.7.0'
make: *** [all] Error 2

I am not sure why am I seeing those messages as I see a reference to the above errors:

root@fwuser-virtual-machine:~/snort_src/snort-2.9.7.0/src#<mailto:root@fwuser-virtual-machine:%7E/snort_src/snort-2.9.7.0/src#>
 grep -r "optionAppIdFree" .
Binary file ./detection-plugins/detection_options.o matches
Binary file ./detection-plugins/sp_appid.o matches
./detection-plugins/sp_appid.c:void optionAppIdFree(AppIdOptionData *optData)
./detection-plugins/sp_appid.c:        optionAppIdFree(optData);
Binary file ./detection-plugins/libspd.a matches
./detection-plugins/detection_options.c:            optionAppIdFree(key->option_data);
./detection-plugins/sp_appid.h:void optionAppIdFree(AppIdOptionData *optData);


I appended the following line in snort.conf:
preprocessor file_inspect: type_id, signature, capture_disk /home/file_capture/tmp/, capture_queue_size 5000

While executing snort process, I got a core file with the following message:

File config:
    file type: ENABLED
    file signature: ENABLED
    file capture: ENABLED
    file capture directory: /home/file_capture/tmp/
    file capture disk size: 300 (Default) megabytes
    file sent to host: DISABLED (Default), port number: 0

Segmentation fault (core dumped)

The traceback of the core file points to:

root@fwuser-virtual-machine:~/snort_src# gdb snort -c core
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from snort...done.

warning: exec file is newer than core file.
[New LWP 10904]

warning: .dynamic section for "/usr/local/lib/snort_dynamicengine/libsf_engine.so" is not at the expected address 
(wrong library or version mismatch?)

warning: .dynamic section for "/usr/local/lib/snort_dynamicpreprocessor//libsf_reputation_preproc.so" is not at the 
expected address (wrong library or version mismatch?)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/bin/snort -c /etc/snort/snort.conf -Q -i eth1:eth2 -l /var/log/snort'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  strlen () at ../sysdeps/x86_64/strlen.S:106
106     ../sysdeps/x86_64/strlen.S: No such file or directory.
(gdb) bt
#0  strlen () at ../sysdeps/x86_64/strlen.S:106
#1  0x00007f6ab63050a6 in appIdStatsInit (appFileName=0x7f6ab6628170 <config+16> "appstats-unified.log", 
statsPeriod=10, rolloverSize=20971520, rolloverPeriod=86400) at appIdStats.c:264
#2  0x00007f6ab62fa2d0 in AppIdCommonInit (memcap=268435456) at commonAppMatcher.c:297
#3  0x00007f6ab6303798 in AppIdInit (sc=0x1eb9770, args=0x1f516e0 "app_stats_filename appstats-unified.log, 
app_stats_period 10, app_detector_dir /usr/local/lib/openappid") at spp_appid.c:157
#4  0x000000000042048e in InitVarTables (p=0x1eb9770) at parser.c:5728
#5  0x000000000046c3d0 in CheckAppId (option_data=0x0, p=0x0) at sp_appid.c:342
#6  0x0000000000000000 in ?? ()
(gdb) Quit

I had installed openappid as well.


On Wed, Mar 11, 2015 at 7:00 PM, Joel Esler (jesler) <jesler () cisco com<mailto:jesler () cisco com>> wrote:

On Mar 11, 2015, at 9:23 AM, Rishabh Shah <rishabh420 () gmail com<mailto:rishabh420 () gmail com>> wrote:

Hi Snort Team,

Is it possible to extract any file during http/ftp transactions? The HTTP preprocessor makes it possible to read the 
HTTP URI/content. Does snort have the intelligence to extract the file during any transfer?


Beginning with 2.9.6.0, Snort has had the ability to extract files from streams and write them to disk.

Check out the README: https://www.snort.org/faq/readme-file

--
Joel Esler
Open Source Manager
Threat Intelligence Team Lead
Talos Group




--
Regards,
Rishabh Shah.



------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/



_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net<mailto:Snort-users () lists sourceforge net>
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org<http://blog.snort.org/> to stay current on all the latest Snort news!


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net<mailto:Snort-users () lists sourceforge net>
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org<http://blog.snort.org/> to stay current on all the latest Snort news!



--
Regards,
Rishabh Shah.




--
Regards,
Rishabh Shah.




------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/



_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net<mailto:Snort-users () lists sourceforge net>
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org<http://blog.snort.org/> to stay current on all the latest Snort news!


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net<mailto:Snort-users () lists sourceforge net>
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org<http://blog.snort.org/> to stay current on all the latest Snort news!



--
Regards,
Rishabh Shah.




--
Regards,
Rishabh Shah.




--
Regards,
Rishabh Shah.
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net<mailto:Snort-users () lists sourceforge net>
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!
Previous By Date Next
Previous By Thread Next

Current thread: