Re: libpcap : An entry in the manual about multithreading

[Guy Harris <gharris () sonic net>]

On May 7, 2023, at 9:27 AM, Michael Richardson <mcr () sandelman ca> wrote:

> Frederick Virchanza Gotham <cauldwell.thomas () gmail com> wrote:
> > I think that there should be a page in the libpcap manual that
> > explicitly explains the multithreading capabilities and limitations.
>
> okay, that sounds reasonable.
> git clone  https://github.com/the-tcpdump-group/tcpdump-htdocs

Either a separate page or something in the main pcap(3PCAP) page.

> > Some libraries have an entry in the manual stating that the library is
> > not threadsafe at all. Nine times out of ten, you're safe to use these
> > libraries from multiple threads so long as you use an exclusive lock.
>
> I don't think that libpcap has been tested in this way.
> I think it would work, and I don't think we use any thread local storage.

We do now, as of

        https://github.com/the-tcpdump-group/libpcap/commit/b10eefd47f979a339aaeb247bf47cc333aa7ba91

which was done in order to fix a case where some libpcap routines weren't thread-safe.

> > Some other libraries are thread-safe but the manual states that the
> > handle returned from the 'open' function can only be used by one
> > thread.
>
> This is where libpcap is.

Yes.  We do not support using a pcap_t handle from multiple threads without the caller using a mutex, except for 
calling pcap_breakloop() from a thread other than the thread using the pcap_t.

> > I remember seeing a page somewhere on the internet that read something
> > along the lines of: "The libpcap library is thread-safe, i.e. multiple
> > threads can call libpcap functions concurrently. However once you've
> > obtained a handle from pcap_open_live, that handle can only be used
> > exclusively by one thread -- with the exception of the pcap_breakloop
> > function".
>
> I don't think we ever said this.

We have never stated that in the documentation, but I may have answered a question on a Q&A site with such an answer.

That's certainly our *policy*, at least as of making the filter compiler reentrant; prior to doing so, pcap_compile() 
was known to be very much not thread-safe.  To fix that, I changed it to use a reentrant parser and scanner (which is 
why it now requires Bison or Berkeley YACC, and a sufficiently recent version of Flex), so different threads should be 
able to compile pcap filters in parallel.

However, it was later discovered that we'd missed pcap_datalink_val_to_description_or_dlt(), pcap_statustostr(), 
bpf_image(), and pcap_next_etherent(), all of which work around C's tragic low level of support for strings by 
formatting into a static buffer and returning a pointer to that, so that the callee doesn't have to free the string 
when it's no longer needed:

        https://github.com/the-tcpdump-group/libpcap/issues/1174

which was fixed by the aforementioned commit.

> > I think the debug build of libpcap should have runtime asserts to
> > ensure that the same thread is always operating on any given pcap_t*
> > handle. For example there could be a global map of pcap_t* handles to
> > thread ID's, something like:
>
> >    struct Mapping { pcap_t *handle; pthread_t thread_id; };
>
> >    Mapping mappings[32u];
>
> I could tolerate this.

Except, of course, that pcap_breakloop() shouldn't do that check, as noted above.

That would certainly test the thread safety of code *using* libpcap, as long as they're testing with the debug build of 
libpcap, and as long as this is a platform for which "release" and "debug" builds are provided.  Unfortunately, the 
only platform I know of where that's a common notion if Windows, unless I've missed something in the lands of Apple, 
Linux, *BSD, Solaris, etc. developers, and I don't know whether Npcap provides separate release and debug builds of 
Npcap.

It wouldn't test the thread safety of libpcap *itself*, however.  That would require tests of the sort done by the 
people who submitted the bug mentioned above.
_______________________________________________
tcpdump-workers mailing list -- tcpdump-workers () lists tcpdump org
To unsubscribe send an email to tcpdump-workers-leave () lists tcpdump org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s