BAM Weblog

Running Unpatched Binaries on NixOS

Brian McKenna — 2017-03-22

I have been using NixOS since the end of 2014, both at home and for work. One thing which has been a bit annoying over these years is when I have to execute a downloaded binary. Atlassian has some internal tools which are written in things like Go. I package these binaries and install them into my user profile but sometimes it can be annoying to spend the 5 minutes packaging something when you want to run something just once or right now.

The first problem is the dynamic linker doesn't exist at the usual place:

/lib64/ld-linux.so.2

Because there's not even a /lib64 directory. When packaging, we use patchelf to make the binary refer to a dynamic linker in the Nix store.

The second problem is then getting the dynamic linker to find linked libraries. When packaging we use patchelf to change the binary's rpath.

I have come up with 2 ways to work around these problems. I'm not yet sure which method is best, so I'll describe both.

The first method is a giant FHS chroot environment. The chroot environment will put the linker and libraries in the standard places. I have a command installed into my user profile so that I can quickly enter the environment:

with import <nixpkgs> { };

buildFHSUserEnv {
  name = "enter-fhs";
  targetPkgs = pkgs: with pkgs; [
    alsaLib atk cairo cups dbus expat file fontconfig freetype gdb git glib 
    gnome.GConf gnome.gdk_pixbuf gnome.gtk gnome.pango libnotify libxml2 libxslt
    netcat nspr nss oraclejdk8 strace udev watch wget which xorg.libX11
    xorg.libXScrnSaver xorg.libXcomposite xorg.libXcursor xorg.libXdamage
    xorg.libXext xorg.libXfixes xorg.libXi xorg.libXrandr xorg.libXrender
    xorg.libXtst xorg.libxcb xorg.xcbutilkeysyms zlib zsh
  ];
  runScript = "$SHELL";
}

I can then run enter-fhs to get into a chrooted shell. Commands like enter-fhs -c ./my-binary also work. If the binary needs an extra library that's not listed, I just add it. Not ideal.

The second method is by putting a hook into nix-shell and calling the dynamic linker on the binary. The hook allows specifying the libraries I want via the nix-shell command:

$ nix-shell -p gcc.cc alsaLib dbus gtk 'callPackage <nix-files/ld_library_path.nix> { }'

[nix-shell]$ runBinary ./my-binary my-arg-1 my-arg-2

Where the ld_library_path.nix expression is:

{ makeSetupHook }:

makeSetupHook { } ./ld_library_path.sh

And ld_library_path.sh is:

addLdLibraryPath () {
    if [ -d $1/lib ]; then
        export NIX_LD_LIBRARY_PATH="$1/lib${NIX_LD_LIBRARY_PATH:+:}$NIX_LD_LIBRARY_PATH"
    fi
}
runBinary() {
    LD_LIBRARY_PATH=$NIX_LD_LIBRARY_PATH:$LD_LIBRARY_PATH $(< $NIX_CC/nix-support/dynamic-linker) $@
}
envHooks+=(addLdLibraryPath)

Docker is a third method which I used to use, but it can also be annoying to configure a Docker container with persistence to run a binary.

Hopefully the above tools can help you figure out how to quickly run unpatched binaries on NixOS. If you don't need to do things quickly, it's probably worth learning how to package existing binaries for real.

Please enable JavaScript to view the comments powered by Disqus.