Re: Metamako DLT Request

[David Mirabito <davidm () metamako com>]

On 24 July 2018 at 11:46, Guy Harris <gharris () sonic net> wrote:

> On Jul 23, 2018, at 6:36 PM, David Mirabito <davidm () metamako com> wrote:
>
> > We'd like to request a new DLT_ value for Metamako's packet capture
> trailer
> > that is generated by our MetaWatch network application.
> >
> >      https://www.metamako.com/applications/metawatch-app.html
> >
> > It is a variable length data trailer, appended immediately after an
> > original Ethernet Payload. It is backwards-compatible with DLT_EN10MB but
> > adds additional fields after the original frame payload until the
> > end-of-frame marker.
> >
> > We were thinking something like this
> >
> >    LINKTYPE_METAMAKO_TRAILER --> DLT_METAMAKO_TRAILER
> >
> > Is there anything in particular that you require?
>
> A detailed description of the format of the trailer.
>
> Presumably, for Ethernet packets with a type field, the trailer can't be
> found until the payload is dissected, because there's no other way to
> determine how large the payload is.
>
> Also, presumably, the trailer comes after any padding, and there's no FCS
> in these packets.



Hi Guy,

I have attached a more detailed description as a text file (lest email
mangle the ASCII-art), but in short, a packet would look on the wire like:

* SOF
* Destination address
* Source Address
* EtherType/Length
* Payload
* FCS . <--- everything up to, and including, here is unmodified by
MetaWatch. Typically our appending a trailer may allow this FCS to be
captured by NICs which would otherwise have stripped it
* Optional Extension TLVs
* Timestamp: Seconds
* Timestamp: Nanoseconds
* Flags (currently whether FCS was correct at ingress, and presence of any
extension TLVs)
* Device ID (by default unique portion of the device's serial number)
* Port ID
* NewFCS (calculated so that metawatch emits a valid frame, not always
captured by capturing NICs)
* EOF

The intent when parsing is that the base trailer is always the last 12
bytes so given a complete capture one may simply skip directly to "wlen -
12", so there is no need to interpret EtherLen, or do a full dissection to
locate the trailer. I agree that the case where caplen < wirelen may be
problematic, but then you're guaranteed a corrupted or missing trailer
anyway so it might not be worth even trying.

Our current Wireshark integration does indeed register as a heuristic
Ethernet Trailer, and as such does rely on a correct prior dissection to
identify any trailer and give us a chance to match. We tend ot make use of
the "Fixed ethernet trailer length" feature in the cases where prior
dissection fails to identify any trailer.

The main motivation to request a DLT that this allows us first-chance look
at each frame in capture known to contain trailers, in part to eliminate
such reliance. We then may scrape the trailer off the end, and then pass
the remainder down the usual dissector stack. To me it seemed similar to
the pseudo-header notion, except working backwards from the buffer's end.
It also allows us to poke our timestamp in as "the" time used for display
and any measurements (i.e RTT) done by sub-dissectors when opening such a
trace file.

Regarding padding:

* In a compliant frame, the original sender would have padded and then
added an FCS (covering the padding) on the wire. To us this looks like any
other 64-byte frame and we still immediately stick the trailer immediately
after this original FCS
* In the case we are presented with an undersized frame, we will add enough
padding ourselves such that "whatever came in + padding + tailer" will be a
legal frame. Presumably in this case the original FCS would not have been
correct, but the one we apply at the end will be correct, regardless.

Either way this would preserve the parsing scheme. The "original FCS" may
be flagged as incorrect in the case of runts or other bit errors prior to
our ingress, but we still "wrap" in a trailer + good new-FCS allowing such
frames to be captured on commodity HW.

I hope this helps, please let us know any more info might be required.

Thanks,
David M

Attachment: MetamakoTrailer.txt
Description:

_______________________________________________
tcpdump-workers mailing list
tcpdump-workers () lists tcpdump org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers