Re: hf_http_response_code in packet-http.c

[Pascal Quantin <pascal.quantin () gmail com>]

Le 13 juil. 2017 22:00, "Sultan, Hassan via Wireshark-dev" <
wireshark-dev () wireshark org> a écrit :



> -----Original Message-----
> From: Wireshark-dev [mailto:wireshark-dev-bounces () wireshark org] On Behalf
> Of Erik de Jong
> Sent: Thursday, July 13, 2017 11:58 AM
> To: Developer support list for Wireshark <wireshark-dev () wireshark org>
> Subject: Re: [Wireshark-dev] hf_http_response_code in packet-http.c
>
>
>
> On Thu, Jul 13, 2017 at 8:47 PM, Sultan, Hassan via Wireshark-dev
<wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> > wrote:
>
>
>
>
>       > -----Original Message-----
>       > From: Pascal Quantin [mailto:pascal.quantin () gmail com
> <mailto:pascal.quantin () gmail com> ]
>       > Sent: Thursday, July 13, 2017 11:26 AM
>       > To: Sultan, Hassan <sultah () amazon com
> <mailto:sultah () amazon com> >
>       > Cc: Developer support list for Wireshark <wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> >
>       > Subject: RE: [Wireshark-dev] hf_http_response_code in
packet-http.c
>       >
>       >
>       >
>       > Le 13 juil. 2017 19:54, "Sultan, Hassan" <sultah () amazon com
> <mailto:sultah () amazon com>
>       > <mailto:sultah () amazon com <mailto:sultah () amazon com> > > a
> écrit :
>       >
>       >
>       >
>       >
>       >       > -----Original Message-----
>       >       > From: Pascal Quantin [mailto:pascal.quantin () gmail com
> <mailto:pascal.quantin () gmail com>
>       > <mailto:pascal.quantin () gmail com
> <mailto:pascal.quantin () gmail com> > ]
>       >       > Sent: Thursday, July 13, 2017 10:39 AM
>       >       > To: Developer support list for Wireshark <wireshark-
>       > dev () wireshark org <mailto:dev () wireshark org>  <mailto:wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> > >
>       >
>       >       > Cc: Sultan, Hassan <sultah () amazon com
> <mailto:sultah () amazon com>
>       > <mailto:sultah () amazon com <mailto:sultah () amazon com> > >
>       >       > Subject: Re: [Wireshark-dev] hf_http_response_code in
packet-
> http.c
>       >       >
>       >
>       >       > Hi Hassan,
>       >       >
>       >       >
>       >       > 2017-07-13 19:25 GMT+02:00 Sultan, Hassan via
Wireshark-dev
>       > <wireshark-
>       >
>       >       > dev () wireshark org <mailto:dev () wireshark org>
> <mailto:dev () wireshark org <mailto:dev () wireshark org> >  <mailto:wireshark-
> <mailto:wireshark->
>       > dev () wireshark org <mailto:dev () wireshark org>  <mailto:wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> > > >:
>       >
>       >       >
>       >       >
>       >       >
>       >       >
>       >       >       > -----Original Message-----
>       >       >       > From: Wireshark-dev [mailto:wireshark-dev-
> <mailto:wireshark-dev->
>       > bounces () wireshark org <mailto:bounces () wireshark org>
> <mailto:wireshark-dev-bounces () wireshark org <mailto:wireshark-dev-
> bounces () wireshark org> >
>       >       > <mailto:wireshark-dev-bounces () wireshark org
> <mailto:wireshark-dev-bounces () wireshark org>  <mailto:wireshark-
> <mailto:wireshark->
>       > dev-bounces () wireshark org <mailto:dev-bounces () wireshark org> >
> > ] On Behalf
>       >       >       > Of Erik de Jong
>       >       >       > Sent: Wednesday, July 12, 2017 10:12 PM
>       >       >       > To: Developer support list for Wireshark
<wireshark-
>       >
>       >       > dev () wireshark org <mailto:dev () wireshark org>
> <mailto:dev () wireshark org <mailto:dev () wireshark org> >  <mailto:wireshark-
> <mailto:wireshark->
>       > dev () wireshark org <mailto:dev () wireshark org>  <mailto:wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> > > >
>       >       >       > Subject: Re: [Wireshark-dev]
hf_http_response_code in
> packet-
>       > http.c
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       > On Thu, Jul 13, 2017 at 1:12 AM, Sultan, Hassan
via
> Wireshark-
>       > dev
>       >       > <wireshark-
>       >
>       >       >       > dev () wireshark org <mailto:dev () wireshark org>
> <mailto:dev () wireshark org <mailto:dev () wireshark org> >
>       > <mailto:dev () wireshark org <mailto:dev () wireshark org>
> <mailto:dev () wireshark org <mailto:dev () wireshark org> > >
> <mailto:wireshark- <mailto:wireshark->
>       > <mailto:wireshark- <mailto:wireshark-> >
>
>       >
>       >       > dev () wireshark org <mailto:dev () wireshark org>
> <mailto:dev () wireshark org <mailto:dev () wireshark org> >  <mailto:wireshark-
> <mailto:wireshark->
>       > dev () wireshark org <mailto:dev () wireshark org>  <mailto:wireshark-
> dev () wireshark org <mailto:wireshark-dev () wireshark org> > > > > wrote:
>       >       >       >
>       >       >       >
>       >       >       >       Hi,
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       >       I am starting to learn the Wireshark code
base, and one
> thing
>       >       > puzzles
>       >       >       > me…
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       >       Why is hf_http_response_code defined as a
FT_UINT16
> with
>       >       > BASE_DEC
>       >       >       > rather than an FT_STRING ?
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       >       It’s a text field… not an integer.
>       >       >       >
>       >       >       >
>       >       >       > Presenting it as a number allows for filtering
like:
>       >       >       > http.response.code > 200
>       >       >       >
>       >       >       > Which would not be possible when presented as a
string.
>       >       >
>       >       >       Thanks for the info, but in that case would it not
be more
>       > appropriate to
>       >       > have the normal field as an FT_STRING and add a generated
field
> as
>       > FT_UINT16
>       >       > ? My understanding of generated fields is that this is
their
> purpose :
>       > represent
>       >       > data that doesn't exactly correspond to the packet data.
>       >       >       We could still keep the field named as is today
(hence
> ensuring all
>       >       > existing filters still work), but simply make it a
generated field,
> and add
>       > an
>       >       > FT_STRING to represent the actual data as it is in the
packet.
>       >       >
>       >       >       Thoughts ?
>       >       >
>       >       >
>       >       >
>       >       >
>       >       > Does having the field as it is designed today generating
you any
> issue?
>       > For fields
>       >       > having numerical values only, it makes much more sense to
have
> the
>       > digits
>       >       > directly instead of the string and that's already what we
are doing
> in
>       > numerous
>       >       > places. And personally I do not see any valid reason to
change
> that.
>       > Your
>       >       > suggestion implies that many fields would start to be
duplicated
>       > without any
>       >       > added value.
>       >
>       >
>       >       My reason is that anyone wanting to build automation based
on
>       > Wireshark's parsing will not be able to rely on Wireshark's
reporting of
> field type
>       > & format to interpret the packet data.
>       >       The automation would walk through parsed fields and be told
that
> the
>       > http response code is a 3-bytes UINT16 with value say 200, so
bytes 00
> 00 c8,
>       > which is wrong. Right now the way these fields are setup works for
> human
>       > interpretation, by people looking at the rendered value on a
screen,
> but it is far
>       > from ideal for automation.
>       >
>       >
>       > Thanks fact that HTTP is text based does not change the fact that
the
> code is
>       > supposed to be a number. So I'm not sure to get your point.
>       > Moreover tons of people use tshark for automation and I do not
> remember
>       > someone complaining about the current output. So maybe I simply
did
> not
>       > understand your use case.
>       >
>       >
>       >       One could argue it's the main purpose of Wireshark, but I
think it
>       > needlessly prevent a whole class of use-cases from being achieved
> with
>       > Wireshark's technology.
>       >
>       >
>       >
>       > Could you elaborate? What use case do you have in mind exactly?
>
>
>       Imagine automation that attempts to perform modification on packets
> using the parsed information provided by Wireshark, or somehow goes and
> directly reads fields in the packet data based on what Wireshark's parsed
> information says (type/offset/length/...). Wireshark reporting the field
as
> FT_UINT16 instead of say a UTF-8 FT_STRING throws off the automation as it
> would either attempt to write an FT_UINT16 at the offset of the field,
writing
> the wrong format, or be confused by seeing a 3 bytes FT_UINT16 and give
up.
> But in that case it's more like you're doing a part of the parsing in
another
> application, why not implement a full parser in that application? The
whole point
> of an analyzer is to make sense of the data passed into it for the user.
And what
> about protocols that could be send with different encodings (BER, XER,
etc)
> there a field type should still be the same - eg UINT16 - but the
representation
> on the wire is totally different.

The beauty of this is specifically that the other app does not need to do
any parsing at all. It fully relies on Wireshark's parsing engine to get
the tree of fields, the offset/length/type/name/encoding/... of each field
and instead of having to write custom code for each protocol (you know, a
parser), the app simply has to implement generic handling for FT_STRING,
FT_UINT16, etc... and the various encodings. So for your BER encoded
integer, having an FT_UINT16 with an ENC_BER encoding flag for example and
a size of 4 (type+length+2 bytes for most short ints) would convey all the
info needed to handle the field.


That would require having an encoder able to understand Wireshark output
syntax and the structure of the protocol you are using to perform the
encoding (text based, TLV, CSN1, ASN1, etc): all the encoding rules related
to the protocol you are playing with. Having a decoder / parser is far from
giving you an encoder.

As Eric explained, having the HTTP response code as a number simply gives
you much more advantages (filtering, comparisons, inequalities...) than
having it in text. So this is clearly not a bug as it was done on purpose.


Will such a thing work for absolutely every single protocol out there ?
Maybe not, but it works for the vast majority of them.


I have some doubts regarding this statement, for the reason I gave above.
But as I'm eager to learn, if you can come up with a PoC I would be very
interested to look at it :)

I did a very similar thing on top of Network Monitor in a previous life at
Microsoft (I had the code available being internal to the company) and it
was a huge success, helping many people in various ways.


So what kind of encoding did it support?


>       How about having an #ifdef for such things so that Wireshark's
use-case
> today does not have to suffer the addition of generated fields, and such
> automation cases could still get full fidelity parsing by building
Wireshark's
> engine with the specific #define ? Today they have to build directly
against
> Wireshark's code anyway so that shouldn't be a problem.
>
>
>       __________________________________________________________
> _________________
>       Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org
> <mailto:wireshark-dev () wireshark org> >
>       Archives:    https://www.wireshark.org/lists/wireshark-dev
> <https://www.wireshark.org/lists/wireshark-dev>
>       Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-
> dev <https://www.wireshark.org/mailman/options/wireshark-dev>
>                    mailto:wireshark-dev-request () wireshark org <mailto:
wireshark-
> dev-request () wireshark org> ?subject=unsubscribe

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org>
Archives:    https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request () wireshark org?subject=unsubscribe

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev () wireshark org>
Archives:    https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request () wireshark org?subject=unsubscribe