Re: Request for a new LINKTYPE_/DLT_ type.

["Dave Barach (dbarach)" <dbarach () cisco com>]

Inline, see >>> Thanks... Dave

-----Original Message-----
From: Guy Harris <guy () alum mit edu> 
Sent: Monday, November 26, 2018 3:01 PM
To: Dave Barach (dbarach) <dbarach () cisco com>
Cc: tcpdump-workers () lists tcpdump org
Subject: Re: [tcpdump-workers] Request for a new LINKTYPE_/DLT_ type.

On Nov 26, 2018, at 6:03 AM, Dave Barach (dbarach) <dbarach () cisco com> wrote:

> I've built a wireshark dissector for fd.io vpp graph dispatcher pcap traces. Please see 
> https://fdio-vpp.readthedocs.io/en/latest/ for a description of the code base / project, etc. 
>
> For development purposes, I borrowed one of the USERxxx encap types. Please allocate a LINKTYPE_/DLT_ type for this 
> file format, so I can upstream the dissector.
>
> Thanks... Dave Barach
> Fd.io vpp PTL
>
> Trace Record format
> -------------------
>
> VPP graph dispatch trace record description, in network byte order. Integers wider than 8 bits are in little endian 
> byte order.

"Byte order" doesn't apply to 8-bit fields; if all fields are in little-endian byte order, what, if anything, is in 
network byte order (big-endian)?

And is everything guaranteed to be in little-endian byte order *even if the tracing code is running on, for example, a 
Power ISA processor running in big-endian mode, or on z/Architecture processor (which *only* runs big-endian)?

> > > Good point. It would be easy to trace the 1 x 32-bit and 1 x 16 bit quantities in for-real network byte order. I'll 
> > > just go do that. Frankly, we haven't run the code base on a PowerPC or other big-endian processor in years. I'm 
> > > fairly sure that the dispatch trace code would be the least of anyone's problems if/when we go there again.  

>    0                   1                   2                   3
>    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>   |Major Version  |Minor Version  |Buffer index high 16 bits      |
>   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>   |Buffer index low 16 bits       |Node Name Len  | Node name ... |
>   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>   + Node name cont'd...     ...                   | NULL octet    |
>   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>   | Primary buffer metadata (64 octets)                           |
>   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>   | [Secondary buffer metadata (64 octets, major version > 1)]    |
>   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>   | ASCII trace length 16 bits    |  ASCII trace ...              |
>   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>   | ASCII trace cont'd ...    ...                 | NULL octet    |
>   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>   | Packet data (up to 16K)                                       |
>   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Is there a page at any fd.io or VPP Web site that describes the header, to which we could point?

> Notes: as of this writing, major version = 1, minor version = 0.

Presumably any code that can read major version M, minor version N will also be able to read major version M, minor 
version K, for all values of K <= N.

> > > That's the goal, but since the paint is barely dry on v1.0 it would be slightly rash of me to make that claim...

> See below for pro forma definitions of the primary buffer metadata and 
> primary opaque data. Please refer to fd.io vpp source code before you invest, send money, or write code: "git clone 
> https://gerrit.fd.io/r/vpp";
>
> Trace records are generated by code in .../src/vlib/main.c:dispatch_pcap_trace(...).
>
> The secondary buffer metadata shown in the diagram above is NOT present in version 1 traces.

So if some future version 2 of the trace is defined, an update will be sent to tcpdump-workers, describing the 
secondary buffer metadata?

For the fields defined in that header:

What is the buffer index?

> > > A 32-bit buffer handle which can be rapidly converted into either a virtual address or a physical addresses. It's 
> > > highly useful as a filter in WS: since we trace e.g. 100 packets in ethernet-input, then 100 packets in ip4-input, 
> > > etc.  

Does the node name length include the terminating NUL?  (Presumably anything writing those files MUST, in the RFC 2119 
sense, null-terminate strings, and anything writing those files MUST not assume that the strings are null-terminated; a 
count *and* a terminating NUL is redundant.)

Does the ASCII trace length include the terminating NUL?  Is that just an opaque string to display to the user, or are 
there any ways in which an application can parse it?

> > > Yes, the NULL is included in the count. Yes, it's slightly redundant. Yes, it keeps people from shooting themselves 
> > > in the foot when processing the data.

In an earlier mail on another list you said:

> Packet data can be anything: L2, L3, L4 or above. The vpp dissector knows from the node name what to expect. I have a 
> [seriously incomplete as of this writing] table of the form:
>
> #define foreach_node_to_dissector_handle                        \
> _("ip6-lookup", "ipv6", ip6_dissector_handle)                   \
> _("ip4-input-no-checksum", "ip", ip4_dissector_handle)          \
> _("ip4-lookup", "ip", ip4_dissector_handle)                     \
> _("ip4-local", "ip", ip4_dissector_handle)                      \
> _("ip4-udp-lookup", "ip", udp_dissector_handle)                 \
> _("ip4-icmp-error", "ip", ip4_dissector_handle)                 \
> _("ip4-glean", "ip", ip4_dissector_handle)                      \
> _("ethernet-input", "eth_maybefcs", eth_dissector_handle)

Presumably, once a node name is used for a particular type of payload, it will always indicate that particular payload 
type.

> > > vpp uses a "current_data" (signed) index, to point at the current layer being processed. On input, all nodes know 
> > > where the "index finger" is supposed to point. 

Could new node names be added in the future?

> > > Nodes are added constantly. There are over 400 at the moment. The current dissector tries for an exact node name 
> > > match, and falls back to guessing that data[current_data] points at either an ip4 header or an ip6 header. Failing 
> > > that, it dissects data as if it were an ethernet frame. 

The "error-drop" node is problematic: there is no good way to guess what's being dropped.   

Is there a page at any fd.io or VPP Web site that gives the current list of node names, showing what payload type each 
node name indicates?

Yes, see https://wiki.fd.io/view/VPP/CurrentData - I'm about to spin up a community project to populate it.


> Pro forma structure definitions:

So which of those structures describes the primary metadata?

> > > vlib_buffer_t. The key fields are flags, current_data, and current_length. 
_______________________________________________
tcpdump-workers mailing list
tcpdump-workers () lists tcpdump org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers