Java 11 Modules, Building

I'm basically done modularising the code at work - at least the active code. I rather indulgently took the opportunity to do a massive cleanup - pretty well all the FIXMEs, should be FIXMEs, TODOs, dead-code and de-duplication that's collected over the last few years. Even quite a few 'would be a bit nicers'. It's not perfect but it was nice to be able to afford the time to do it.

I'm still trying to decide if I break the projects up into related super-projects or just put everything in the single one as modules. I'm aiming toward the latter because basically i'm sick of typing "cd ../blah" so often, and Netbeans doesn't recompile the dependencies properly.

I'm going to reset the repository and try using git. I don't like it but I don't much like mercurial either.

Building

At the moment I have a build system which uses make and compiles at the module level - i.e. any source changes and the whole module is recompiled, and one can add explicit module-module dependencies to control the build order and ensure a consistent build.

One reason I do this is because there is no 1:1 correspondance between build sources and build classes. If you add or remove nested or anonymous (or co-located) classes from a source file that adds or removes .class files which are generated. So to ensure there are no stale classes I just reset it on every build.

This isn't too bad and absolutely guarantees a consistent build (assuming one configures the inter-module dependencies properly) but the compiler is still invoked multiple times which has overheads.

Building Faster

Really the speed isn't a problem for these projects but out of interest i'm having a look at a couple of other things.

One is relatively simple - basically use JSR-199 to create a compiler server something like the jdk uses to build itself.

The more complicated task is incremental builds using GNU Make. I think I should be able to hook into JavacTask and with a small bit of extra code create something akin to the "gcc -MD" option for auto-generating dependencies. It has the added complication of having to detect and remove stale .class files, and doing it all in a way that make understands. I've already done a few little experiments today while I was procrastinating over some weeding.

Using JavacTask it is possible to find out all the .class files that are generated for a given source file. This is one big part of the puzzle and covers the first-level dependencies (i.e. %.java: %.class plus all the co-resident classes). One can also get the AST and other information but that isn't necessary here.

To find the other dependencies I wrote a simple class file decoder which finds all the classes referenced by the binary. Some relatively simple pattern matching and name resolution should be able to turn this into a dependency list.

Actually it may not be necessary to use JavacTask for this because the .class files contain enough information. There is extra overhead because they must be parsed, but they are simple to parse.