Re: Fw: new file format

["Gianluca Varenni" <gianluca.varenni () polito it>]

There is another issue related to these block types.

Fulvio's proposal:
a shb (even corrupted by the ftp transfer) can begin with the following
strings:
\r\n\r\x1A  -> 1 reserved block type
\r\n\n\r    -> 1 reserved block type
\n\r\x1a\?? -> 256 reserved block types
\x1a\r\n\r  -> 1 reserver block type
\x1a\r\r\n  -> 1 reserved block type
\x1a\n\r\?? -> 256 reserved block types
As a consequence we have 516 reserved block types (and we need at least 6
checks to test if a block has one of these 516 special types). With 6 check
we are able to see if the block is a proper section (and its byte order), a
ftp transfer error, or a normal block.


Gianluca's proposal:
a shb (even corrupted by the ftp transfer) can begin with the following
strings:
\r\n\n\r   -> 1 reserved block type
\n\n\r\??  -> 256 reserved block types
\r\r\n\r(\n\r) -> 1 reserved block type
As a consequence we have 258 reserved block types (and we need at least 3
checks to test if a block has one of these 258 special types).
If the block type is "\r\n\n\r" we need at most two other tests to look for
the byte order.
If the block type is "\n\n\r\??" or "\r\r\n\r" we need some tests (I do not
know how many of them) to look for the byte order in the file and find if
it's an invalid block or a ftp tranfer error.

Considering that a usual file will have a lot of blocks, but very few
sections, I think that my solution is better, since it takes at most 3
checks to see if a block is NOT a shb (and this very common), although it
will take some more tests to look for the byte order if one of these
reserved block types is found (and this not very common).

Any final decision for this issue??

Have a nice day
GV


----- Original Message ----- 
From: "Fulvio Risso" <fulvio.risso () polito it>
To: <tcpdump-workers () lists tcpdump org>
Cc: "Gianluca Varenni" <gianluca.varenni () polito it>
Sent: Friday, July 30, 2004 12:48 AM
Subject: RE: [tcpdump-workers] Fw: new file format


I would like to point out advatanges / disadvatanges of the
two proposals:

Fulvio's proposal:
+ byte order and correct ascii transfer are checked with a
  single string
- there is the possibility not to detect faulty ftp trasfers
  (in case of ftp trasfers from Unix to Win, the last byte is
  '?'. If it happens that the following byte in the file is
  '\r' or '\n', we may not able to detect an incorrect ascii
  transfer)

Gianluca's proposal:
- byte order and correct ascii transfer are checked with two
  different strings
+ incorrect faulty ftp trasfers are always detected
- this mechanism is a bit more complicated because of the two
  strings required

Am I missing something?
Cheers,

fulvio


> -----Original Message-----
> From: tcpdump-workers-owner () lists tcpdump org
> [mailto:tcpdump-workers-owner () lists tcpdump org]On Behalf Of Gianluca
> Varenni
> Sent: giovedì 29 luglio 2004 22.13
> To: tcpdump-workers () lists tcpdump org
> Cc: Guy Harris
> Subject: Re: [tcpdump-workers] Fw: new file format
>
>
> Hi.
>
> I haven't seen any reply to Fulvio's proposal about removing the
> byte order
> magic and using the string "\r\n\r\x1A".
>
> Personally, I prefer my original proposal of leaving the byte order magic
> and the block type "\r\n\n\r" (just because we have 8 bytes to
> check if the
> file is correct and understand its byte order).
>
> Can we decide what is the final block type for the section header block?
>
> Have a nice day
> GV
>
> ----- Original Message -----
> From: "Gianluca Varenni" <gianluca.varenni () polito it>
> To: <tcpdump-workers () tcpdump org>
> Sent: Monday, July 26, 2004 5:04 PM
> Subject: [tcpdump-workers] Fw: new file format
>
>
>
> ----- Original Message -----
> From: "Fulvio Risso" <fulvio.risso () polito it>
> To: "Gianluca Varenni" <gianluca.varenni () polito it>; "Guy Harris"
> <guy () alum mit edu>
> Cc: "Loris Degioanni" <ldegioanni () ucdavis edu>; "Michael Richardson"
> <mcr () sandelman ottawa on ca>
> Sent: Friday, July 23, 2004 11:22 PM
> Subject: RE: new file format
>
>
> I would like to propose the string "\r\n\r\x1A", so that:
>
> - correct byte order, correct ftp transfer  ==> "\r\n\r\x1A"
> - correct byte order, wrong ftp UNIX to Win ==> "\r\n\n\r\"
> - correct byte order, wrong ftp Win to UNIX ==> "\n\r\x1A\?"
> - changed byte order, correct ftp transfer  ==> "\x1A\r\n\r"
> - changed byte order, wrong ftp UNIX to Win ==> "\x1A\r\r\n"
> - changed byte order, wrong ftp Win to UNIX ==> "\x1A\n\r\?"
>
> If I'm not making any further mistake, this string should recognize both
> byte order and correct ascii transfer.
>
> Cheers,
>
> fulvio
>
> > -----Original Message-----
> > From: Gianluca Varenni [mailto:gianluca.varenni () polito it]
> > Sent: venerdì 23 luglio 2004 21.34
> > To: Guy Harris
> > Cc: Loris Degioanni; Fulvio Risso; Michael Richardson
> > Subject: Re: new file format
> >
> >
> >
> > ----- Original Message -----
> > From: "Guy Harris" <guy () alum mit edu>
> > To: "Gianluca Varenni" <gianluca.varenni () polito it>
> > Cc: "Loris Degioanni" <ldegioanni () ucdavis edu>; "Fulvio Risso"
> > <fulvio.risso () polito it>; "Michael Richardson"
> > <mcr () sandelman ottawa on ca>
> > Sent: Friday, July 23, 2004 12:14 PM
> > Subject: Re: new file format
> >
> >
> > > On Jul 23, 2004, at 11:57 AM, Gianluca Varenni wrote:
> > >
> > > > If the file is transfered from win to unix in ASCII mode, the file
> > > > should
> > > > become
> > > >
> > > > \n\n\r .......... In this case we recognize the first three
> characters
> > > > "\n\n\r", try to convert the first 12 bytes from unix-ascii to
> > > > win-ascii,
> > > > and check the byte order magic at bytes 8-11.
> > > >
> > > > If the file is transfered from unix to win in ascii mode, the file
> > > > should
> > > > become
> > > > \r\r\n\r\n\r ....... In this case we recognize (for
> example) the first
> > > > three
> > > > chars "\r\r\n" and try to convert the first n characters
> (24 chars??)
> > > > from
> > > > win-ascii to unix-ascii, and check the byte order magic at
> bytes 8-11.
> > > Is there always only one way of converting between UN*X format and
> > > Windows format?  Or could information be lost, even for this
> particular
> > > case, in a way that prevents it from being reclaimed?
> >
> > I don't know...
> >
> > > One possibility would be to assume that if the byte-order
> magic appears
> > > within some number of bytes of the proper position the file
> was a valid
> > > dump file transferred in ASCII order.  I suspect that, in a UN*X to
> > > Windows transfer, at most every \n would become a \r, so that'd mean
> >
> > Not sure about that.
> >
> > > that the magic number would be at most 6 bytes after the correct
> > > position (if the block size *happened* to be 168430090, i.e.
> > > 0x0a0a0a0a, or \n\n\n\n) and, in a Windows to UN*X transfer, at most
> > > every \r would be deleted, so that'd mean that the magic number would
> > > be at most 6 bytes before the correct position (if the block size
> > > *happened* to be 218959117, i.e. 0x0d0d0d0d, or \r\r\r\r), so if the
> > > byte-order magic appears (in either byte order) in an offset between 2
> > > (8-6) and 14 (8+6), assume the file was a valid file transferred in
> > > ASCII mode.
> > >
> > > I.e., don't bother trying to recover the block type and block size,
> > > just check for a byte-order magic in any of the places where it could
> > > possibly be.
> >
> > Neither I want to recover the file.
> >
> > I like the solution you propose: if the \r\n\n\r is not found at the
> > beginning of the file, we look for the magic number from byte 2
> to byte 14
> > of the file. If we find, we report an "ASCII format conversion mismatch"
> > error (or something like that).
> >
> > I think this is the solution to the problem.
> >
> > Have a nice day
> > GV
> >
> >
> >
> >
> > > (We'd do the same with subsequent section header blocks, and say
> > > "sorry, some part of this file was transferred in ASCII mode" or
> > > "somebody concatenated to a file another file transferred in ASCII
> > > mode".)
>
>
>
>
> -
> This is the tcpdump-workers list.
> Visit https://lists.sandelman.ca/ to unsubscribe.
>
>
>
> -
> This is the tcpdump-workers list.
> Visit https://lists.sandelman.ca/ to unsubscribe.



-
This is the tcpdump-workers list.
Visit https://lists.sandelman.ca/ to unsubscribe.