Re: Error building nmap on amd64

[David Fifield <david () bamsoftware com>]

On Fri, Aug 22, 2008 at 01:47:33PM -0600, David Fifield wrote:
> On Tue, Aug 19, 2008 at 09:01:08PM +0200, Sven Klemm wrote:
> > when trying to build nmap on amd64 I am getting the following error.
> >
> > make[2]: Entering directory `/tmp/nmap/nselib-bin'
> > ./libtool --tag=CC --silent --mode=compile gcc -I../liblua  -g -O2 -c
> > bit.c
> > ./libtool --tag=CC --silent --mode=link gcc -avoid-version -module
> > - -rpath /usr/local/libexec/nmap/nselib-bin -L../liblua  -o bit.la
> > bit.lo -llua -lm
> > /usr/bin/ld: /tmp/nmap/liblua/liblua.a(lapi.o): relocation R_X86_64_32
> > against `luaO_nilobject_' can not be used when making a shared object;
> > recompile with -fPIC
> > /tmp/nmap/liblua/liblua.a: could not read symbols: Bad value
> > collect2: ld returned 1 exit status
> > make[2]: *** [bit.so] Error 1
> > make[2]: Leaving directory `/tmp/nmap/nselib-bin'
> > make[1]: *** [nsestdlib] Error 2
> > make[1]: Leaving directory `/tmp/nmap'
> > make: *** [all] Error 2
> >
> > The reason for the error is liblua is not compiled with -fPIC which is
> > needed for shared libraries on amd64. When I manually build liblua
> > with -fPIC I don't get any error. configure should probably check if
> > - -fPIC is needed for shared libraries and enable it for liblua.
>
> I can reproduce this on amd64. It used to work, but it appears to have
> stopped working when I checked in r9353. That added -llua to the link
> line used to create shared modules; it was intended to make loading of
> shared modules work when Nmap was compiled statically. Note that it used
> to work without -fPIC, so I think the -fPIC warning is just a symptom of
> a different problem.
>
> The recommended way to build shared modules is to link the main
> executable with liblua, and use the linker flag -Wl,-E which causes
> symbols to be dynamically exported. When bit.so looks for lua_pushnumber
> it finds it within the nmap executable.
>
>       "Static linking is incompatible with the use of dlopen()"
>       http://lua-users.org/lists/lua-l/2007-11/msg00110.html
>
> When linking statically -Wl,-E has no effect and symbols like
> lua_pushnumber are not exported for the use of shared modules. That was
> a bug reported here:
>
>       http://seclists.org/nmap-dev/2008/q1/0405.html
>
> That fixed the problem on x86 but (as I now know) it doesn't work on
> platforms like amd64 where you can't just link static and dynamic
> objects. Linking each module with liblua also makes them bigger: bit.so
> increases in size from 16000 to 390000 bytes.
>
> The ability to link Nmap statically is important because that's how the
> RPMs are built. Some solutions I see are
> * Compile liblua with -fPIC unconditionally,
> * Modify the liblua makefile to use -fPIC when necessary,
> * Modify the module loading system so that modules can be compiled in
>   statically or loaded dynamically.

After thinking about this some more and discussing it with Fyodor, I
have this proposal: remove support for shared modules from NSE.

Currently there is only one shared module, bit.so, and it's tiny. We
used to have another one, pcre.so, but it was changed to be linked
statically into the nmap executable when we ran into a similar problem
last year:

        Libtool, static libraries, and NSE modules
        http://seclists.org/nmap-dev/2007/q4/0085.html

The base problem is that statically linking the nmap executable and
having dyamically loaded NSE modules are incompatible. We could work
around it by using -fPIC and linking each module separately with liblua,
but the Lua wiki insists that this approach "is *not* a good idea."

        "Do Not Link Modules to the Lua Core Libraries"
        http://lua-users.org/wiki/BuildingModules

This change change would remove support for shared NSE modules and
change bit into a statically linked module like bin, hash, and pcre.
This is a moderately substantial change so please speak up if you have a
reason why shared modules should be retained.

For more on NSE shared modules see
http://nmap.org/book/nse-implementation.html#nse-implementation-c-modules

David Fifield

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org