Re: Need precompiled source files in libpcap

[Guy Harris <guy () alum mit edu>]

On Jun 25, 2016, at 7:25 AM, Yang Luo <hsluoyb () gmail com> wrote:

> On Sat, Jun 25, 2016 at 3:09 PM, Guy Harris <guy () alum mit edu> wrote:
>
> > On Jun 24, 2016, at 8:58 PM, Yang Luo <hsluoyb () gmail com> wrote:
> >
> > > I found that there are a lot of “precompiled source code files" like grammar.c, scanner.c, grammar.h, ,scanner.h, 
> > > etc. (which are also listed in .gitignore). They are NOT contained in the libpcap repo.
> > >
> > > I personally want to ship them in the "Win32-Extensions" folder of the Npcap repo, so the Npcap developers can just 
> > > build the "libpcap.sln" project with one click, and will get the wpcap.dll.
> > >
> > > Do you think this is a good idea? Or Windows developers should also build everything from the bottom?
> >
> > You can either
> >
> >         1) install Flex and Bison yourself, use them to build those files, and ship them with the Npcap source 
> > (along with the .y and .l files!)
>
> I have successfully built libpcap on my Ubuntu, and got the generated source files. They can be used in the Windows 
> build process.

I wouldn't rely on a UN*X version of Flex or Bison generating a file that can be built on Windows; for example, some 
versions of either Flex or Bison would generate source files that included <unistd.h>, which is a UN*X-specific header 
file.

This also means that if you have a source directory on a file server, or that's otherwise shared between multiple 
operating systems, and the software uses Flex or Bison, and you've just done a UN*X build in that directory, if you're 
going to do a Windows build, you may need to do not just a clean (to get rid of object files), you may need to do a 
"distclean" or whatever clean operation removes generated files (to get rid of files generated by Flex or Bison) - 
people have had problems with Wireshark builds due to not getting rid of the generated files first.

> > or
> >
> >         2) require that developers install Flex and Bison.
>
> In future, I will make sure that the Windows developers compile those generated source files.

That's the best solution.
 
> > You will need Flex 2.5.31 or later.  See, for example
> >
> >         https://sourceforge.net/projects/winflexbison/
> >
> > for Flex and Bison for Windows.
>
> I will try the Windows version Flex and Bison later.

The section in the Wireshark Developer's Guide on setting up a Windows environment:

        https://www.wireshark.org/docs/wsdg_html_chunked/ChSetupWin32.html

includes instructions on setting up Flex and Bison, as Wireshark also requires them.

> Besides these two things, are there other files that need to be prebuilt? I know I need to read the "configure" file 
> in the root dir to see what it does,

No, the "configure" file is *itself* a generated file - the source is in configure.ac and aclocal.m4 - so you don't 
want to read it (unless you're trying to debug a problem in the configure script itself).

The best thing to read would be the Makefile.in file, which is the template from which the configure script generates 
the Makefile.  What you should look for are lines of the form

        XXXXX.c: ...

and

        XXXXX.h: ...

which are Makefile rules to generate .c and .h files.

> but this file is too long..
> If you can tell me what needs to be pre-built besides those two, it would be great..

The other generated files are:

        version.c, version.h - unfortunately, generating those from the VERSION file requires several UN*X tools, so 
you'd have to either:

            require developers to install Windows versions of those tools from Cygwin or some other place;

            come up with an equivalent procedure that uses tools that come with Windows;

            manually generate them.

        bpf_filter.c - currently, the BPF interpreter is in a subdirectory of the source directory (so that it can be 
picked up and dropped into a UN*X kernel; that's probably not that necessary these days, as UN*Xes have their own 
versions in their source trees, and they need to pick up any changes we make to the libpcap BPF interpreter by hand), 
and the build procedure makes a symbolic link to it.  For Windows, you probably should make a copy for now; I'll ask 
whether it's time to just put the BPF interpreter in the top-level libpcap source directory.

> I noticed this docs: https://github.com/the-tcpdump-group/libpcap/blob/master/INSTALL.txt, and it seems to require to 
> run "./configure" (a shell script), which is for Linux. So I think there's no Windows alternative config command for 
> now?

No, and there are no plans to have the autoconf script work for Windows.

> > We also support CMake:
> >
> >         https://cmake.org
> >
> > on both UN*X *and* Windows.
>
> Can CMake do what the UN*X "./configure" command does?

It's supposed to.

On UN*Xes, it doesn't yet do *all* the things that the configure command does - there are some XXX comments in 
CMakeLists.txt about that.  I eventually plan to make it do all the checks in question.

On Windows, it should do all the necessary checks.

> On Windows, the build is very easy, just click "build" in Visual Studio. But the ".configure" process seems to have 
> no replacement on Windows. Is that CMake?

CMake is an alternative to autotools; the vendor's description of it is:

        CMake is an open-source, cross-platform family of tools designed to build, test and package software. CMake is 
used to control the software compilation process using simple platform and compiler independent configuration files, 
and generate native makefiles and workspaces that can be used in the compiler environment of your choice. The suite of 
CMake tools were created by Kitware in response to the need for a powerful, cross-platform build environment for 
open-source projects such as ITK and VTK.

A number of open-source projects use it - including Wireshark.  Wireshark currently supports both autotools (automake, 
autoconf) and CMake for building, several of the UN*X buildbots do CMake builds, and I think Debian now uses CMake 
rather than autotools when building Wireshark.

CMake can generate build files for a number of build tools:

        https://cmake.org/cmake/help/v3.6/manual/cmake-generators.7.html

including makefiles for UN*X "make", makefiles for Windows Nmake, projects for several different versions of Visual 
Studio, and projects for Xcode.

It has a GUI tool for Windows; I've never used it, so I don't know how convenient it is to use, or whether, once you've 
generated Visual Studios project files, it'll start Visual Studio for you or if you then have to start it yourself.  (I 
do all my building from the command line on both UN*X and Windows.)

> > > I think just ship them in Npcap repo will be a good idea?
> >
> > We won't be putting the generated versions into the libpcap repository, and I would *not* recommend putting them 
> > into the Npcap repository.  *If* you want to provide the generated source files in the Npcap source release, as 
> > they're provided in libpcap source releases, I would recommend that you generate them as part of the process of 
> > building an Npcap source release.
>
> Have you put the generated source files into the source release?

I thought we were doing that, but we're not.
_______________________________________________
tcpdump-workers mailing list
tcpdump-workers () lists tcpdump org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers