Building Maven packages with Nix#

Wed, 10 May 2017 01:06:07 +0000

[ Note to Clojurists using Leiningen: I found, towards the end of this maven2nix process, that the pom.xml files that lein pom creates are not actually sufficient to the task of building jars. If that's why you're reading this, this may not be the right thing to be reading. ]

Nixpkgs has some tools for building Maven projects but (as far as I can determine) absolutely no examples of their use. It took me some time to figure things out from trawling through mailing lists, not least because my Nix-fu is not (yet?) up to reading the buildMaven source and shell script and saying "hey, that's obvious". I write these notes in the hope that eddies in the space-time continuum will cause them to pop up a week ago when I needed them, but failing that, maybe they'll be useful to someone else who struggles with it.

If you have a Java project whose dependencies are managed by Maven, you will find that building a Nix derivation for it is non-trivial, because maven expects to be able to figure out the dependencies and download them as it runs. Nix doesn't appreciate this, because derivations should be pure functions of their inputs and not depend on what happens to be hosted on random web sites at any given time. Indeed, proper "pure" builds are run in a context that doesn't even allow them outbound network access.

Other languages with similar package dependency managers have a similar problem: for example, Ruby projects that use Bundler, or Node projects that use npm/yarn. The solution is also similar: we break it into two parts. First, generate a list of all the dependencies - including the transitive dependencies - so that Nix can download them into the store; second, run the build in "offline" mode and tell it to find the dependencies in the store instead of on the Internet.

Get the package list

My overpowering impression of Maven based on having used it for three days is that it is entirely composed of plugins. Great plugins have little plugins upon their backs to bite 'em, and little plugins have lesser plugins - and so, ad infinitum. In this case, the plugin is called org.nixos.mvn2nix:mvn2nix-maven-plugin and it provides a command called mvn2nix. In principle using this is a simple matter of writing (or somehow obtaining) a pom.xml file, then doing

$ mvn org.nixos.mvn2nix:mvn2nix-maven-plugin:mvn2nix # don't do this

to generate a file called project-info.json. However, in practice there are a couple of issues:

as of May 2017 (and since Aug 2015) this generates a file which causes buildMaven to complain that the flag `authenticated` is not present.

The issue is that mvn2mix generates different output for dependencies that already exist in your local repository than it does for packages it fetches, and the workaround (as described in the issue) is to run it against an empty directory:

$ mvn  -Dmaven.repo.local=$(mktemp -d -t maven)  org.nixos.mvn2nix:mvn2nix-maven-plugin:mvn2nix # mmm

a little bit of post-processing is also required: sometimes the URLs generated by mvn2nix have repeated / characters in their paths.

"https://repo1.maven.org/maven2//logkit/logkit/1.0.1/logkit-1.0.1.pom"
                                ^ non-ideal

This is the kind of thing for which sed would be the unix-philosophy-solution, if you can handle the resulting leaning toothpick syndrome. Or you could just use Perl

$ cat project-info.json | jq . | \
 perl -pe 's,https://repo1.maven.org/maven2//,https://repo.maven.org/maven2/,g' > $$ \
     && mv $$ project-info.json # or something

This probably also constitutes a bug but I've not yet taken the time to find out if it's with mvn2nix or with my generated POM, or whether it's been reported previously. Anyway, achievement unlocked.

Writing the Nix expression.

I've done this as a standalone expression that you can drop into your project as default.nix and run nix-build on: if you were writing something to add to the packages set in the nixpkgs repo then I think the preamble would look different (caveat: don't understand, haven't tried it).

with import  {};
let jar = buildMaven ./app/project-info.json;
in stdenv.mkDerivation rec {
  buildInputs = [ makeWrapper ];
  name = "my-app";
  version = "0.1.0";
  builder = writeText "install.sh" ''
    source $stdenv/setup
    mkdir -p $out/share/java $out/bin
    cp ${jar.build} $out/share/java/
    makeWrapper ${jre_headless}/bin/java $out/bin/${name} \
      --add-flags "-jar $out/share/java/${jar.build}"
  '';
}

Some points of note

buildMaven does the necessary invocations of mvn, then returns a set with the attribute build which is a jar file installed in the store. But that's probably not much use on its own, so ...
we put it in $out/share/java, because nix packaging guidelines say that we should, and
on the assumption that we are packing an application not just a library, we have also added a script in bin/ to start it up.
this is for a headless app: if your app does GUI, you will want to use the full jre not jre_headless

I'm not a Nix expert (did I say that already?). Where this is non-idiomatic it's more likely because I don't understand the idiom than through any kind of conscious decision process.

Coming soon

As I said at the start, this doesn't actually work for Clojure/Leiningen projects. Look out for another blog entry soon that describes a whole other completely different approach you can use there.