Static nmap builds for infosec via Docker

Posted on 08.06.2016

I am considering doing an Infosec training course where I will need a static copy of many useful utilities, such as (the exercise here) nmap. However, having tried to do this on my plain host, I found that nmap's make static was frustrating at best. So this post documents the process I underwent to do this inside of a docker container.

Firstly, let's get docker out of the way. Docker is a container system that Rory M absolutely loves and has been encouraging me to use for a long time. David F similarly recommended this approach. OK, so, time to stop mocking docker and actually learn how to use it. Here is a quick redux:

  1. Images are, well, file system images present inside the container and contain all of the userspace goodness we need.
  2. Tags are just names for containers/images, a little bit like git tags identify specific sha1 commits in git.
  3. Containers are specific instances of images. You can have multiple containers per image, obviously.
  4. Committing to an image updates that image with any local resident changes.
  5. Host volumes can be mounted.

There's the basics. Without further ado, I went and set up my image:

docker run -i -t fedora bash

a crazily short amount of time later and I found myself inside a bash prompt on a fedora container. My next step was to decide how exactly to go about building, since Fedora's glibc-static package didn't work with my nmap tests previously. Googling for static nmap binaries will get you this page and I have reason to believe the author cross compiled with musl. As it turns out there is a handy set of scripts for building a musl gcc cross compiler already in a repository! If you like you can learn how to do this yourself but personally life is way too short, so I will be using the existing option.

Now we have our container the first thing to do is to install our needed cross compilation dependencies. These are fedora-specific names for the packages; adjust to taste:

dnf remove -y vim-minimal
dnf install -y vim gcc gcc-c++ tar wget bzip2 lzma gzip make autoconf
dnf install -y libtool git hg patch gmp-devel libmpc-devel mpfr-devel byacc bison file
dnf install -y texinfo
dnf install -y flex

The first line removes vim-minimal, which is installed in the container by default. Next I set up my working directories:

mkdir /opt/cross
mkdir /work

and then cloned the build scripts repository:

cd /work
hg clone https://bitbucket.org/GregorR/musl-cross
cd musl-cross

at this stage you need to set some options in config.sh in this folder, specifically the target you intend to build for (or architecture at least). The actual config.sh is more verbose than this, but you essentially Needs

ARCH=x86_64
CC_BASE_PREFIX=/opt/cross
MAKEFLAGS=-j8
GCC_STAGE1_NOOPT=1

That's it. Now just type:

./build.sh

and go and get a coffee or six.

This will, eventually, succeed. If it doesn't, you are missing something from the dependencies list for gcc (in preparing this post, my build failed repeatedly as I forgot odd tools and others were not in the container, like file). Now you are ready to begin cross-compiling stuff!

I fancied beginning with openssl, for which I lost precious hours of my life. I went through the usual process of setting up an openssl source tree in my work folder:

cd /work
mkdir openssl
cd openssl
wget https://www.openssl.org/source/openssl-1.0.2h.tar.gz
tar xvf openssl-1.0.2h.tar.gz
cd openssl-1.0.2h

that done, there were several misfires in attempting to compile it (actually, I started with openssl-1.10-pre5, but found out it used an old, deprecated API to do async handling, which has been causing issues). I couldn't quite get a static binary and then I stumbled upon this mailing list entry. Essentially you need to edit Configure and add this single line:

"linux-x86_64-musl", "gcc:-m64 -DL_ENDIAN -DTERMIO -O3
-Wall::-D_REENTRANT::-static:SIXTY_FOUR_BIT_LONG RC4_CHUNK DES_INT
DES_UNROLL:${x86_64_asm}:elf:dlfcn:linux-shared:-fPIC:-m64:.so.\$(SHLIB_MAJOR).\$(SHLIB_MINOR):::64",

You should add this just below a similar line beginning linux-x86_64, we will use our -musl target instead of this. Next we need to fix an issue with termios/termio:

sed -i 's/-DTERMIO/-DTERMIOS/g' Configure

and now, now we can configure! Firstly, a warning. If you want to use this for production these options are NOT safe. I am deliberately thinking ahead of testing against old, vulnerable systems using say testssl.sh. Ok, so now you've read that warning, do this:

./Configure no-shared enable-ssl3 enable-ssl3-method enable-weak-ssl-ciphers
enable-egd enable-heartbeats enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5
--prefix=/opt/cross/x86_64-linux-musl/ linux-x86_64-musl

Now we give the make scripts a bit of a hint as to what we want them to do, going forward:

export CC=/opt/cross/x86_64-linux-musl/bin/x86_64-musl-linux-gcc
export CXX=/opt/cross/x86_64-linux-musl/bin/x86_64-musl-linux-g++
export LD=$CC

and then we run, as advised by configure:

make depend

Openssl's make depend will complain a lot about a missing stddef.h. Unfortunately it is basically broken and you simply need to ignore it. You can now run:

make
make install

and off it goes. Again this is coffee time. When it is done, there is something we need to fix for later: openssl assumes $prefix/lib64 as the lib directory, but our compiler assumes $prefix/lib. No worries, we make some symlinks:

cd /opt/cross/x86_64-linux-musl/lib
ln -s ../lib64/libcrypto.a
ln -s ../lib64/libssl.a

Next we need libpcap. Thankfully this is fairly straightforward, so I will quote the steps I took in one:

cd /work
mkdir libpcap
cd libpcap
wget http://www.tcpdump.org/release/libpcap-1.7.4.tar.gz
tar xvf libpcap-1.7.4.tar.gz
cd libpcap-1.7.4
./configure --disable-shared --prefix=/opt/cross/x86_64-linux-musl/
make
make install

This is the usual wget, configure, make, make install routine that you should be able to use. Now the fun starts. Another dependency of nmap is liblinear. To get started, I did this:

cd /work
mkdir liblinear && cd liblinear
wget http://www.csie.ntu.edu.tw/~cjlin/liblinear/liblinear-2.1.tar.gz
tar xvf liblinear-2.1.tar.gz
cd liblinear-2.1

and promptly found no configure file! Ok, that's actually alright, since we already set CC and CXX above. However, if you look inside the Makefile you will see it attempts to build a shared object, which we absolutely do not want. Patch the makefile so it reads:

AR=ar rcv
RANLIB=ranlib
CXX ?= g++
CC ?= gcc
CFLAGS = -Wall -Wconversion -O3 -fPIC
LIBS = blas/blas.a
SHVER = 3
OS = $(shell uname)
#LIBS = -lblas

all: train predict

lib: linear.o tron.o blas/blas.a
        $(AR) liblinear.a $<
        $(RANLIB) liblinear.a

At the top and now type:

make
make lib

Unfortunately there is no make install so we have to do this part manually. No worries:

cp liblinear.a /opt/cross/x86_64-linux-musl/lib
cp linear.h /opt/cross/x86_64-linux-musl/include

and now, we get to the monster itself: nmap. As usual, we set up:

cd /work
mkdir nmap && cd nmap
wget https://nmap.org/dist/nmap-7.12.tar.bz2
tar xvf nmap-7.12.tar.bz2 && cd nmap-7.12

Ok, so, now, how to configure? Well, I used:

./configure --prefix=/opt/cross/x86_64-linux-musl/ --without-zenmap
--without-ndiff --without-nping --with-libua=included --with-pcap=linux
--with-libpcap=/opt/cross/x86_64-linux-musl/ --with-openssl=/opt/cross/x86_64-linux-musl/

Specifically, I disabled the zenmap gui, got rid of ndiff/nping and configured my pcap and openssl from our cross compiled tree (definitely static). Wait for this to be done and then type:

make static

and boom, you should be done! To test we have achieved our mission:

$ file ./nmap
./nmap: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped

Perfect. Incidentally, if at this stage you type make install the nmap build scripts default to the shared option. I couldn't be bothered to fix them at this stage, so I just did:

cp ./nmap /opt/cross/x86_64-linux-musl/bin

for the moment.

Now, having done all this work I very much wanted to keep this environment. As previously mentioned, we're currently in a container with an id like root@aabbccddeeff - and what we want is to commit a new image. To do this, type exit and go back to your host. Then:

docker commit -m "x64 cross compile working" -a "Your Name" aabbccddeeff fedora-musl-static

replacing aabbccddeeff with the identifier in your container, and substituting an appropriate message, name and tag to suit your needs. Expect this to take quite a while, as the difference between the images is at this stage quite large.

Now, if we want to make a new container running against this new image, do:

docker run --i -t fedora-musl-static /bin/bash

and you'll find yourself back in the same system! Pretty neat, right? There is one final step - I wanted my binaries outside this environment, so I ran:

docker run -v /tmp/crosscompileout:/output -i -t fedora-musl-static /bin/bash

instead. This mounts /tmp/crosscompileout inside the VM as /output and anything copied there appears on the host system for usage. So, just:

cp /opt/cross/x86_64-linux-musl/bin/nmap /output
cp /opt/cross/x86_64-linux-musl/bin/openssl /output

and you now have some static binaries to be working with on Linux.