tcpdump mailing list archives
Re: [PATCH] compress savefiles after each rotation
From: Hannes Gredler <hannes () juniper net>
Date: Thu, 23 Mar 2006 18:33:14 +0100
checked in - tx for your submission; - /hannes Sebastien Raveau wrote:
Hello everybody,I am submitting this patch for tcpdump that adds the -z flag (to be used in conjunction with -C or -G) which can be used to specify a command tcpdump should execute on each savefile after it's been rotated.For example, running: tcpdump -G 3600 -z bzip2 -w "capture%Y%m%d%H%M%S.pcap"will make tcpdump close its current savefile and open a new one every hour, continue dumping packets in the new savefile while bzip2 compresses (with minimum priority so that this does not disturb the capture) the previous savefile and renames it to something like capture20060312153936.pcap.bz2This might sound like a crazy idea, but check it, sometimes the size of compressed savefiles is down to 30% of the size of original savefiles! :-)It has been thoroughly tested on Linux and should work on all Unices as I always pay attention to writing code that is compliant with the Single UNIX Specification, but since I don't use Windows, I don't know if this patch might break Windows compatibility or not... Could somebody please check that for me?Kind regards, ------------------------------------------------------------------------ diff -ur tcpdump-2006.03.12/CREDITS tcpdump/CREDITS --- tcpdump-2006.03.12/CREDITS 2006-02-03 09:32:39.000000000 +0100 +++ tcpdump/CREDITS 2006-03-12 14:59:10.000000000 +0100 @@ -123,6 +123,7 @@ Sami Farin <safari () iki fi> Scott Rose <syberpunk () users sourceforge net> Sebastian Krahmer <krahmer () cs uni-potsdam de> + Sebastien Raveau <sebastien.raveau () epita fr> Sebastien Vincent <svincent () idems fr> Seth Webster <swebster () sst ll mit edu> Shinsuke Suzuki <suz () kame net> Only in tcpdump: Makefile Only in tcpdump: addrtoname.o Only in tcpdump: af.o Only in tcpdump: checksum.o Only in tcpdump: config.h Only in tcpdump: config.log Only in tcpdump: config.status Only in tcpdump: cpack.o Only in tcpdump: gmpls.o Only in tcpdump: gmt2local.o Only in tcpdump: ipproto.o Only in tcpdump: l2vpn.o Only in tcpdump: machdep.o Only in tcpdump: nlpid.o Only in tcpdump: oui.o Only in tcpdump: parsenfsfh.o Only in tcpdump: pcap_dump_ftell.o Only in tcpdump: print-802_11.o Only in tcpdump: print-ah.o Only in tcpdump: print-aodv.o Only in tcpdump: print-ap1394.o Only in tcpdump: print-arcnet.o Only in tcpdump: print-arp.o Only in tcpdump: print-ascii.o Only in tcpdump: print-atalk.o Only in tcpdump: print-atm.o Only in tcpdump: print-beep.o Only in tcpdump: print-bfd.o Only in tcpdump: print-bgp.o Only in tcpdump: print-bootp.o Only in tcpdump: print-cdp.o Only in tcpdump: print-chdlc.o Only in tcpdump: print-cip.o Only in tcpdump: print-cnfp.o Only in tcpdump: print-dccp.o Only in tcpdump: print-decnet.o Only in tcpdump: print-domain.o Only in tcpdump: print-dvmrp.o Only in tcpdump: print-eap.o Only in tcpdump: print-egp.o Only in tcpdump: print-eigrp.o Only in tcpdump: print-enc.o Only in tcpdump: print-esp.o Only in tcpdump: print-ether.o Only in tcpdump: print-fddi.o Only in tcpdump: print-fr.o Only in tcpdump: print-gre.o Only in tcpdump: print-hsrp.o Only in tcpdump: print-icmp.o Only in tcpdump: print-igmp.o Only in tcpdump: print-igrp.o Only in tcpdump: print-ip.o Only in tcpdump: print-ipcomp.o Only in tcpdump: print-ipfc.o Only in tcpdump: print-ipx.o Only in tcpdump: print-isakmp.o Only in tcpdump: print-isoclns.o Only in tcpdump: print-juniper.o Only in tcpdump: print-krb.o Only in tcpdump: print-l2tp.o Only in tcpdump: print-lane.o Only in tcpdump: print-ldp.o Only in tcpdump: print-llc.o Only in tcpdump: print-lmp.o Only in tcpdump: print-lspping.o Only in tcpdump: print-lwres.o Only in tcpdump: print-mobile.o Only in tcpdump: print-mpcp.o Only in tcpdump: print-mpls.o Only in tcpdump: print-msdp.o Only in tcpdump: print-nfs.o Only in tcpdump: print-ntp.o Only in tcpdump: print-null.o Only in tcpdump: print-ospf.o Only in tcpdump: print-pflog.o Only in tcpdump: print-pgm.o Only in tcpdump: print-pim.o Only in tcpdump: print-ppp.o Only in tcpdump: print-pppoe.o Only in tcpdump: print-pptp.o Only in tcpdump: print-radius.o Only in tcpdump: print-raw.o Only in tcpdump: print-rip.o Only in tcpdump: print-rsvp.o Only in tcpdump: print-rx.o Only in tcpdump: print-sctp.o Only in tcpdump: print-sip.o Only in tcpdump: print-sl.o Only in tcpdump: print-sll.o Only in tcpdump: print-slow.o Only in tcpdump: print-smb.o Only in tcpdump: print-snmp.o Only in tcpdump: print-stp.o Only in tcpdump: print-sunatm.o Only in tcpdump: print-sunrpc.o Only in tcpdump: print-symantec.o Only in tcpdump: print-syslog.o Only in tcpdump: print-tcp.o Only in tcpdump: print-telnet.o Only in tcpdump: print-tftp.o Only in tcpdump: print-timed.o Only in tcpdump: print-token.o Only in tcpdump: print-udp.o Only in tcpdump: print-vjc.o Only in tcpdump: print-vqp.o Only in tcpdump: print-vrrp.o Only in tcpdump: print-wb.o Only in tcpdump: print-zephyr.o Only in tcpdump: setsignal.o Only in tcpdump: smbutil.o Only in tcpdump: strlcat.o Only in tcpdump: strlcpy.o Only in tcpdump: tcpdump diff -ur tcpdump-2006.03.12/tcpdump.1 tcpdump/tcpdump.1 --- tcpdump-2006.03.12/tcpdump.1 2005-12-13 09:37:23.000000000 +0100 +++ tcpdump/tcpdump.1 2006-03-12 15:34:19.000000000 +0100 @@ -40,6 +40,9 @@ .B \-C .I file_size ] [ +.B \-G +.I rotate_seconds +] [ .B \-F .I file ] @@ -90,14 +93,14 @@ .br .ti +8 [ -.B \-G -.I rotate_seconds -] -[ .B \-y .I datalinktype ] [ +.B \-z +.I command +] +[ .B \-Z .I user ] @@ -608,6 +611,31 @@ .B \-y Set the data link type to use while capturing packets to \fIdatalinktype\fP. .TP +.B \-z +Used in conjunction with the +.B -C +or +.B -G +options, this will make +.I tcpslice +run " +.I command file +" where +.I file +is the savefile being closed after each rotation. For example, specifying +.B \-z gzip +or +.B \-z bzip2 +will compress each savefile using gzip or bzip2. +.IP +Note that tcpslice will run the command in parallel to the capture, using +the lowest priority so that this doesn't disturb the capture process. +.IP +And in case you would like to use a command that itself takes flags or +different arguments, you can always write a shell script that will take the +savefile name as the only argument, make the flags & arguments arrangements +and execute the command that you want. +.TP .B \-Z Drops privileges (if root) and changes user ID to .I user diff -ur tcpdump-2006.03.12/tcpdump.c tcpdump/tcpdump.c --- tcpdump-2006.03.12/tcpdump.c 2006-02-09 21:33:49.000000000 +0100 +++ tcpdump/tcpdump.c 2006-03-12 15:19:50.000000000 +0100 @@ -65,6 +65,9 @@ #include <stdio.h> #include <stdlib.h> #include <string.h> +#include <limits.h> +#include <sys/resource.h> +#include <sys/wait.h> #ifndef WIN32 #include <pwd.h> #include <grp.h> @@ -88,6 +91,7 @@int dflag; /* print filter code */int Lflag; /* list available data link types and exit */ +char *zflag = NULL; /* compress each savefile using a specified command (like gzip or bzip2) */static int infodelay;static int infoprint; @@ -98,6 +102,7 @@/* Forwards */static RETSIGTYPE cleanup(int); +static RETSIGTYPE child_cleanup(int); static void usage(void) __attribute__((noreturn)); static void show_dlts_and_exit(pcap_t *pd) __attribute__((noreturn));@@ -292,6 +297,7 @@ struct dump_info {char *WFileName; + char *CurrentFileName; pcap_t *pd; pcap_dumper_t *p; }; @@ -466,7 +472,7 @@ { register int cnt, op, i; bpf_u_int32 localnet, netmask; - register char *cp, *infile, *cmdbuf, *device, *RFileName, *WFileName, *WFileNameAlt; + register char *cp, *infile, *cmdbuf, *device, *RFileName, *WFileName; pcap_handler callback; int type; struct bpf_program fcode; @@ -517,7 +523,7 @@opterr = 0;while ( - (op = getopt(argc, argv, "aA" B_FLAG "c:C:d" D_FLAG "eE:fF:G:i:lLm:M:nNOpqr:Rs:StT:u" U_FLAG "vw:W:xXy:YZ:")) != -1) + (op = getopt(argc, argv, "aA" B_FLAG "c:C:d" D_FLAG "eE:fF:G:i:lLm:M:nNOpqr:Rs:StT:u" U_FLAG "vw:W:xXy:Yz:Z:")) != -1) switch (op) {case 'a':@@ -812,6 +818,15 @@ } break; #endif + case 'z': + if (optarg) { + zflag = strdup(optarg); + } else { + usage(); + /* NOTREACHED */ + } + break; + case 'Z': if (optarg) { username = strdup(optarg); @@ -987,6 +1002,7 @@ #endif /* WIN32 */ (void)setsignal(SIGTERM, cleanup); (void)setsignal(SIGINT, cleanup); + (void)setsignal(SIGCHLD, child_cleanup); /* Cooperate with nohup(1) */ #ifndef WIN32 if ((oldhandler = setsignal(SIGHUP, cleanup)) != SIG_DFL) @@ -998,19 +1014,18 @@ if (WFileName) { pcap_dumper_t *p; /* Do not exceed the default NAME_MAX for files. */ - WFileNameAlt = (char *)malloc(NAME_MAX + 1); + dumpinfo.CurrentFileName = (char *)malloc(NAME_MAX + 1);- if (WFileNameAlt == NULL)- error("malloc of WFileNameAlt"); + if (dumpinfo.CurrentFileName == NULL) + error("malloc of dumpinfo.CurrentFileName");/* We do not need numbering for dumpfiles if Cflag isn't set. */if (Cflag != 0) - MakeFilename(WFileNameAlt, WFileName, 0, WflagChars); + MakeFilename(dumpinfo.CurrentFileName, WFileName, 0, WflagChars); else - MakeFilename(WFileNameAlt, WFileName, 0, 0); + MakeFilename(dumpinfo.CurrentFileName, WFileName, 0, 0);- p = pcap_dump_open(pd, WFileNameAlt);- free(WFileNameAlt); + p = pcap_dump_open(pd, dumpinfo.CurrentFileName); if (p == NULL) error("%s", pcap_geterr(pd)); if (Cflag != 0 || Gflag != 0) { @@ -1166,6 +1181,12 @@ #endif }+static RETSIGTYPE+child_cleanup(int signo _U_) +{ + wait(NULL); +} + static void info(register int verbose) { @@ -1194,10 +1215,30 @@ }static void+compress_savefile(const char *filename) +{ + if (fork()) + return; + /* + * Set to lowest priority so that this doesn't disturb the capture + */ +#ifdef NZERO + setpriority(PRIO_PROCESS, 0, NZERO - 1); +#else + setpriority(PRIO_PROCESS, 0, 19); +#endif + if (execlp(zflag, zflag, filename, NULL) == -1) + fprintf(stderr, + "compress_savefile:execlp(%s, %s): %s\n", + zflag, + filename, + strerror(errno)); +} + +static void dump_packet_and_trunc(u_char *user, const struct pcap_pkthdr *h, const u_char *sp) { struct dump_info *dump_info; - char *name;++packets_captured; @@ -1236,6 +1277,12 @@pcap_dump_close(dump_info->p);/*+ * Compress the file we just closed, if the user asked for it + */ + if (zflag != NULL) + compress_savefile(dump_info->CurrentFileName); + + /* * Check to see if we've exceeded the Wflag (when * not using Cflag). */ @@ -1245,9 +1292,11 @@ exit(0); /* NOTREACHED */ } + if (dump_info->CurrentFileName != NULL) + free(dump_info->CurrentFileName); /* Allocate space for max filename + \0. */ - name = (char *)malloc(NAME_MAX + 1); - if (name == NULL) + dump_info->CurrentFileName = (char *)malloc(NAME_MAX + 1); + if (dump_info->CurrentFileName == NULL) error("dump_packet_and_trunc: malloc"); /* * This is always the first file in the Cflag @@ -1255,13 +1304,12 @@ * We also don't need numbering if Cflag is not set. */ if (Cflag != 0) - MakeFilename(name, dump_info->WFileName, 0, + MakeFilename(dump_info->CurrentFileName, dump_info->WFileName, 0, WflagChars); else - MakeFilename(name, dump_info->WFileName, 0, 0); + MakeFilename(dump_info->CurrentFileName, dump_info->WFileName, 0, 0);- dump_info->p = pcap_dump_open(dump_info->pd, name);- free(name); + dump_info->p = pcap_dump_open(dump_info->pd, dump_info->CurrentFileName); if (dump_info->p == NULL) error("%s", pcap_geterr(pd)); } @@ -1277,17 +1325,25 @@ * Close the current file and open a new one. */ pcap_dump_close(dump_info->p); + + /* + * Compress the file we just closed, if the user asked for it + */ + if (zflag != NULL) + compress_savefile(dump_info->CurrentFileName); + Cflag_count++; if (Wflag > 0) { if (Cflag_count >= Wflag) Cflag_count = 0; } - name = (char *)malloc(NAME_MAX + 1); - if (name == NULL) + if (dump_info->CurrentFileName != NULL) + free(dump_info->CurrentFileName); + dump_info->CurrentFileName = (char *)malloc(NAME_MAX + 1); + if (dump_info->CurrentFileName == NULL) error("dump_packet_and_trunc: malloc"); - MakeFilename(name, dump_info->WFileName, Cflag_count, WflagChars); - dump_info->p = pcap_dump_open(dump_info->pd, name); - free(name); + MakeFilename(dump_info->CurrentFileName, dump_info->WFileName, Cflag_count, WflagChars); + dump_info->p = pcap_dump_open(dump_info->pd, dump_info->CurrentFileName); if (dump_info->p == NULL) error("%s", pcap_geterr(pd)); } @@ -1517,7 +1573,7 @@ (void)fprintf(stderr, "\t\t[ -M secret ] [ -r file ] [ -s snaplen ] [ -T type ] [ -w file ]\n"); (void)fprintf(stderr, -"\t\t[ -W filecount ] [ -y datalinktype ] [ -Z user ]\n"); +"\t\t[ -W filecount ] [ -y datalinktype ] [ -z command ] [ -Z user ]\n"); (void)fprintf(stderr, "\t\t[ expression ]\n"); exit(1); Only in tcpdump: tcpdump.o Only in tcpdump: util.o Only in tcpdump: version.c Only in tcpdump: version.o
- This is the tcpdump-workers list. Visit https://lists.sandelman.ca/ to unsubscribe.
Current thread:
- [PATCH] compress savefiles after each rotation Sebastien Raveau (Mar 12)
- Re: [PATCH] compress savefiles after each rotation Hannes Gredler (Mar 23)