Free Software JavaFX

I've been keeping a (rather loose) eye out on the availability of a fully free-software JavaFX for a while. Whilst all distributions now ship OpenJDK by default and that works fine for anything not JavaFX, the OpenJFX stuff has still been work in progress and hasn't generally been available.

Anyway I thought i'd try to compile it myself. Of course the first thing I noticed was that a preview build IS now available but I decided to keep going anyway so I could build a complete integrated JDK/JRE rather than having to manually link it in at runtime.

For the first part of the problem, following the build instructions was quite straightforward, at least on Ubuntu 16.04. I manually installed a couple of the specific build tools required and a couple of extra -dev packages. I didn't bother with building the bundled webkit for now. Even on this gutless under-RAM'd machine it didn't even take terribly long. I used the openjdk 10 for the compiler.

Then I looked into integration witht the jdk (later section in the build instructions). I was too lazy to read most of the build instructions so I just went with configure && make, although after a few iterations I settled on:

$ mkdir build
$ cd build
$ ../configure  --with-import-modules=../../rt/build/modular-sdk \
  --with-jtreg=../../jtreg \
  --prefix=/opt \
  --with-native-debug-symbols=none
$ make product-images
  

Some time later it's all done and I ran some simple tests and Bob's your uncle. Stripping the debug symbols just reduces the size of the install (significantly) although it's probably worth keeping them at this early stage.

The slowest part was checking out the mecurial repositories.

Oh, I couldn't get the jdk tests to run :-/ jtreg just complained that it couldn't determine the JVM version. Trying to search the interwebs has so far been fruitless - it's not terribly important for now and I successfully ran the bootcycle-build which self-compiles after bootstrapping which is at least some bit of testing.

Update: Not sure where I got it from but the jtreg I had was out of date so it couldn't handle the new version string. Starting on the jtreg page got me the latest (4.2b12) and now the testing is running.

Torrents

Although this box doesn't have much disk it does have enough for a few big blobs so I thought I'd look into distributing a build (because, well!). To that end I sussed out setting up a private tracker and seeding torrents directly from this machine.

Anyway i've mostly worked it out but it isn't quite ready but i'll get to it eventually.

Java 11

With the short lifetime of Java 9 and 10 i've basically just ignored them completely - i'm still using 8.x everywhere. But Java 11 is going to be LTS release so it's probably time to start into it.

Having a fully free build of the whole platform is an important part of that for my hobby code. Now I just need the motivation ...

Update 2: And NetBeans, which is still not supporting a Java that was EOL'd 3 months ago. I'm sure they've nearly go the licensing sorted out though!

Update 3

Ok so I guess i'm a bit out of the loop (and search engines failed me again, it took me hours to come across this stuff, mostly by accident).

JavaFX is going to removed from the JDK. And additionally the JRE will also go away(?) and basically replaced with standalone platform-specific builds via jlink. Actually i'm not sure how the latter will work, maybe that's just wrt the openjdk; and in any event Oracle are committing to another few years of Java 8 releases.

Given that all the 'action' is server-side I guess it makes some sense. Just not for me!

Huh, I wonder why the current jdk build process allows linking in the JavaFX module then. Unclear. Just doing so breaks the tests so it's probably just the build system lagging slightly. Then again the make_runargs.sh script used to run against the javafx module doesn't seem to work against openjdk11 anyway. Well unless there's some other meachanism but neither javafxpackager nor javapackager get built - does jlink do all that now? So far my searching has been pretty futile and a lot of the documentation is either out of date or hard to find. I suppose it's still basically work in progress and/or i'm just not that motivated to dig deep enough.

Well I did waste the day anyway so whatever. At least AMD announced some nice hardware and Intel made an arse of themselves with their 'first' 5Ghz cpu demo which needed to be plugged into a fridge to run for 10 seconds.

Oh, there is also openjfx in ubuntu repos, but it only goes to 8. I can't remember if i looked at it in the past, probably did but I think I was looking at java9 at the time not realising how short lived it would be. I should've just stayed with 8.

Update 4

Well I did find something about running against the standalone javafx on this blog post. I guess all that javapackager stuff is gone or something since webstart and applets are defunct? Not that I ever used it anyway. Shrug.

Also apparently using jlink and making a partial copy of the jre for every application will be `easier' than just sending out a jar and having a central single installation of a jre. Wot?

Everyone seems to be a bit upset that Oracle are only going to provide commercial support for the oracle jdk11+ too - but given how much they've invested in making the openjdk feature parity I think they did ok for such an `evil' company. I mean, it's not like they forked some public commons like Linux or something as Google did.

github and m$

I only had a couple of long abandoned projects on github but now i've deleted my account. I don't see the immediate reason why m$ would want to buy it but it can't be for a good one for anyone else.

I wonder if they'd have bought if it git had the same meaning in american as it does in english - i.e. bastard, fuckwit, etc.

But anyway I guess it's just as well I didn't move anything there when google code shut down, saves me the hassle of doing it again.

Backend stuff

Winter has hit here and along with insomnia i'm not really feeling like doing much of an evening but i've dabbled a few times and basically ported the Java version of a tree-revision database to C.

At this point i've just got the core done - schema/bindings and most of the client api. I'm pretty sure it's solid but I need to write a lot of testing and validation code to make sure it will be reliable and performant enough, and then write a bunch more to turn it into something interesting.

But i've been at a desk for 10 hours straight and my feet are icy cold so it's not happening tonight.

Evolution and S/MIME

So I noticed there was a S/MIME security fault in a bunch of email software - including Evolution.

Now my memory is a bit faded because it was 15+ years ago but I'm pretty sure we wrote the code to handle this case (mostly Larry and Jeff). For this each decoded segment was displayed separately with a special gtkhtml tag to reset the html parser between blocks. Although it might have only been on the signature level so I could be wrong but in general it didn't just dump the whole email to HTML for all sorts of reasons. The MIME parser could handle all sorts of broken streams so truncated HTML was expected to come up once in a while.

Of course that must've all been thrown away when the renderer was replaced by the 'better' renderer from apple going by some of the reports of the 'vulnerability'.

Not that i've ever used S/MIME or gpg - it's pretty much useless to me since nobody I know knows how to use it and hardly anyone uses email these days anyway.

I was also horrified to see that evolution now uses cmake. Well just as well I completely ignored the project after I took a voluntary redundancy ... I would've gone absolutely ballistic! Not that compiling with libtool didn't suck complete arse but at least it worked.

But GNOME was already going to shit back before I quit, both due to redhat throwing their weight around and Miguel being such an obnoxiously microsoft fanboi. Haven't touched it in any meaningful way (or Evolution) in over a decade and all I see of it is going backwards by continously copying the next shitty GUI-trend-of-the-month and/or being bullied into shitty designs by a bunch of fuckwits.

Oops

Had a bug in my fastcgi code, that broke the blog for some web clients depending on their ID string. It just happened to break on mobile phones more often. Oops.

King PUSS

Some photos of the cat.

He's a bit of a pretty-boy but he's smarter than he looks.

Ostensibly his name is Cooper (as in Cooper's Original Pale Ale).

But I just call him cat.

c dez port

I had a couple of hours to burn Sunday morning so I ported over the rest of the dez code to C, although I didn't feel like testing it till I had some hours to burn today.

Anyway, I fixed some bugs and ran some tests. It's only about 30-50% faster than the Java version on the bible test for practical "limit" values. The patches generated aren't necessarily identical because of some minor changes in the hash table design but the differences are minor. The C code also requires some more bounds and error checking for robustness.

I also added CRC32 checksums to the file format as a quick check that the input and output aren't corrupted.

cdez + other stuff

I started porting dez to C to look at using it here somewhere. Along the way I found a bug in the matcher implementation but otherwise got very distracted trying to gain a few neglible percent out of the delta sizes by manipulating the address encoding mechanism.

I tried modifying the matcher in various ways - experimenting with the hash table details. These involved including the hash value (i.e. to reduce spurious string matching - it just slows it down) or using a separate index table (no real difference). Probably the most surprising was that the performance was already somewhat better than covered in the dez benchmarks. Both considerably faster processing and smaller generated deltas. I guess that must have been an earlier implementation and I need to update them. For example the bible compression test only takes 11 seconds and creates a 1 566 019 byte delta - or 65% of the runtime at 90% of the output size.

This insprired me to play with the chain limit tunable - which sets how deep the hashtable chain gets before it starts to throw away older values. Using a setting of 5 (32 depth) it just beats the previous published results but in only 0.7s - still somewhat slower than 0.1 for gzip but at least it's not out of the range of practicality. This is where I found the bug in the entry discard indexing which was an easy fix.

This does mean that the other timings I did are pretty much pointless though - using a larger block search size than 1 just produces so much worse results and it's still slower. I haven't tried with a large source input string however, where a chain limit will truncate the search space prematurely.

Then I spent way too much time and effort trying various address encoding mechanisms to try to squeeze a little bit more out of the algorithm. In the end although I managed to get about 2.5% best case improvement in some cases I doubt it's really worth worrying about. However some of the alternative address encoding schemes are conceptually and mechanically simpler so I might use one of them (and break the file format).

Because of all that faffing about I never really got very far with the cdez conversion although I have the substring matcher basically done which is the more complex part. The encoding/decoding code is quite involved but otherwise straightforward bit bashing.

Update I tried a different test - one where i simulated the total delta size of encoding 180 revisions of jjmpeg development - not a particularly active project but still a real one. The original encoding is easily the best in this case.

bloggone

For some reason the blog went offline for a few hours. It kept getting segfaults in libc somewhere. All I did to fix it was run make install (which simply copied the binary into the cgi directory and didn't rebuild anything) and it started working again. Unfortunately I didn't think to preserve the binary that was there to find out why it stopped working.

Something to keep an eye on anyway.