Java Modules

So I might not be giving a shit and doing it for fun but I'm still looking into it at work.

After a couple of days of experiments and quite a bit of hacking i've taken most of the libraries I have and re-combined them into a set of modules. Ostensibly the modules are grouped by functionality but I also moved a few bits and pieces around for dependency reasons.

One handy thing is the module-info (along with netbeans) lets you quickly determine dependencies between modules, so for example when I wanted to remove java.desktop and javafx from a library I could easily find the usages. It has made the library slightly more difficult to use because i've moved some methods to static functions (and these functions are used a lot in my prototype code so there's a lot of no-benefit fixing to be done to port it) but it seems like a reasonable compromise for the first cut. There may be other approaches using interfaces or subclasses too, although I tend to think that falls into over-engineering.

Spi

One of the biggest benefits is the service provider mechanism that enables pluggability by just including modules the path. It's something I should've looked into earlier rather than the messy ad-hoc stuff i've been doing but I guess things get done eventually.

I've probably not done a good job with it yet either but it's a start and easy to modify. There should be a couple of other places I can take advantage of it as well.

Redesign

I'm also mid-way through cleaning out a lot of stuff - cut and paste, newer-better implementations, or just experiments that take too much code and are rarely used.

I had a lot of stream processing experiements which just ended up being over-engineered. For example I tried experimenting with using streams and a Collector to calculate more than just min/sum/max, instead calculating multi-dimensional statistics (i.e. all at once) on multi-dimensional data (e.g. image channels). So I came up with a set of classes (1 to 4 dimensions), collector factories, and so on - it's hundreds of lines of code (and a lot of bytecode) and I think I only use it in one or two places in non-performance critical code. So it's going in the bin and if i do decide to replace it I think I can get by with at most a single class and a few factory methods.

The NotAnywhereZone

Whilst looking for some info on netbeans+cvs I tried finding my own posts, and it seems this whole site has effectively vanished from the internet. Well with a specific search you can still find blog posts on google, but not using the date-ranges (maybe the date headers are wrong here). All you can find on duckduckgo is the site root page.

So if you're reading this, congratulations on not being a spider bot!

Not Netbeans 9.0, Java 11

Well that effort was short-lived, no CVS plugin anymore.

It's not that hard to live without, just use the command line and/or emacs, but today i've already wasted enough time trying to find out if it will ever return (on which question I found no answer, clear or otherwise).

It was also going to be a bit of a pain translating an existing project into a 'modular' one, even though from the perspective of a makefile it's only a couple of small changes.

Netbeans 9, Java 11

So after months of rewriting license headers netbeans 9 is finally out. So I finally had a bit more of a serious look at migrating to openjdk + openjfx 11 for work.

Eh, it's going to be a bit of a pain but I think overall it should be an improvement worth the effort. When i'm less hungover and better-slept i'll start looking into jmodularising my projects too.

One unfortunate bit is that netbeans doesn't seem to support native libraries in modules so i'll need to use makefiles for those. This is one of the more interesting features of jmods so i'm also looking into utilising that a bit more.

At the moment as i'm looking into some deep learning stuff so i've got a lot of time between drinks - pretty much every stage of it is an obnoxiously slow process.

Lots of other little things to look into as well, and now the next yearly contract has finally been done I don't have an easy excuse for putting in fuck-all hours!

Dead Cats Dont Bounce

My cat died sometime in the last few weeks.

He was acting a bit strange and sore for a while. I later found out my nephew had stepped on him - looking at his fucking phone no doubt. But following that he did seem to recover - he wasn't quite himself but he didn't seem to be bothered by anything as such. Then one day he stopped turning up for food and I haven't seen him since. However I started to smell dead-animal from a spot he used to sleep in. Unfortunately (or perhaps fortunately) I can't get to where it is - deep under a low deck - so I just have to wait for the flies and bugs to do their work. It's also right next to the loungeroom so it's a bit unpleasant depending on the weather.

Rest In Peace gentle killer.

Life goes on ...

In other news.

Work has been slow for a few months - partly my mood, partly the work, and significantly the countract hasn't been renewed. There's still some money remaining but it's running out. The org is just being funny about contractors, they'd rather deal with multinationals who fleece them and the country than with local businesses.

Not that I really care, barely been doing 2 days/week of work and i'm still earning enough to blow way way too much at the pub. At least those 2 days have been solid lately, customers are super-happy with everything. The weather is slowly improving but still has a ways to go to be beer drinking weather, not that it's stopped me so far. The the weird bloke in a kilt that hangs around a certain pub. Hmmm.

Nephew is pissing me off by just being here (i'm still not sure how annoyed i should be with him over the whole stepping on the cat thing). He was only supposed to be living with me for a couple of months while he prepared the paperwork to go into the army after emigrating from The Philippines but he decided to do an apprentiship instead and he's been here over a year now. Mostly nothing major but it's the little things like having to clean the stove up to use it every time he's used it before, stack the dishwasher properly, remind him to do things, and well just having someone else banging around the house. His old man is visiting this week so hopefully I can find out when he's moving out.

I've been lazy lazy for months but I finally did a few things on the garden and around the house - pruning, mulching, weeding, clearing up piles of wood. Build a short staircase from the deck to the ground (one day to be paving). A few days here and there of warmer weather helped.

My mood has continued to be pretty flat for the most part, and worse than that every now and then. Often but not always sleep related - i'm always tired but sometimes more tired than others. I have to stop reading the news and forums - the world is just so fucked up and so are too many of the people in it. Every time I read about big corps or politicians I just want to go drink (or just sleep).

Odds n Sods

Not playing games much but when I do it's usually No Man's Sky. I think the game design is getting a little unfocused but it's still an ok way to blow a few hours now and then. I really dislike the new sentinal timeout mechanics though.

Looking at building a mini-ITX Ryzen machine but just can't decide on bits and pieces. APU or 2700x+small GPU? I started building a small case by hacking away at a bigger one (approx 200x400x400mm -> 200x200x200mm) but that's still work in progress.

As an ongoing thing i've been poking around a start-up that's looking at doing some machine vision/learning/artistic stuff, but i'm just not certain I can commit to it and I just haven't been coding much outside of work. It has the potential to be a whole heap of fun but it just hasn't grabbed me so far.

Another shitty `technology' company

Oh FFS.

I have my previous workstation for work sitting idle so I thought i'd drop in an xubuntu install and try building openjdk & openjfx on it. It's got a 6x core I7-980 and plenty of RAM so it should be ok right?

Well all went well until I tried to build webkit, just for completeness. Result - consistent ICE inside g++. Blast. Well I thought it was consistent until I tried it with a fresh build of gcc 7.3, this also crashed but in a different place and when I went back to the system gcc I noticed the crash whilst repeatable wasn't in a consistent place. Actually it started crashing everywhere, even inside various jvm based tasks.

This is typically a symptom of system problems, specifically RAM. I looked in the BIOS incase it's been overclocked but it is so ancient there's no settings for RAM, I ran a few memory testers, I tried various numbers of threads for the build.

Then I remembered Intel and their notorious bugs this year causing system stability problems in some cases. I tried to find the options to turn off the bug mitigations but (in part due to isp maintenance at just that moment) I gave up and just booted with the 4.10.x kernel.

Oh look, works fine now (well, it compiles cleanly, webkit tests still fail!)

Perhaps this is a failure of Canonical, or the Linux developers? No, ultimately it's because Intel cut too many corners and have shit hardware. Then again any company that could design something as poor as HPET in this day and age is obviously fucking incompetent.

On a related note i've been eyeing off a Ryzen system every few months. I price one up and think about it but ultimately leave it for the time being. I'm just not doing enough computing beyond 'read internet' to justify it. Another thing I can't decide on is between some 'low-end' APU system or a beastly 2700X machine. The RAM is still so $$$ here and you need good ram for either. At least the last time I specced one up I noticed from some benchmarks than a 2700X would pretty much cream that old I7-980 at 1/4 of the price (or less, not that I paid for it).

No nice things

So my experiment with bittorrent here has ended before it even started. I was running transmission-daemon as the seeder - but apart from a couple of short tests only had it running with no torrents being seeded.

Despite that i'm getting 500MB+ traffic PER DAY! just from peer requests (I guess, as there seems to be no way to find out what's going on inside transmission).

And the thing is I never publically released the torrent files it was seeding so some port scraper has done it and added it to some peer exchange. Despite blocking the port i'm still getting dozens of incoming packets per minute but I guess they'll quieten down over time.

Blah.

Fortunately I don't pay for traffic.

Ahh google, the `great' advertising company!

Oh nice one, they've decided to block all of Blender's videos on youtube because they don't want to turn on advertising.

Essentially blackmailing them into becoming part of their slurping adversiting empire where they get to make most of the money from peoples labour whilst paying a pittance. Despite not being a real user of it, as one of the original supporters who helped fund the freeing of Blender way back when with a few hundred dollars I'm pretty apalled they would be treated this way (although not surprised).

Fortunately the technology is coming together so that alternatives exist to a monolithic/expensive server such as PeerTube which uses WebTorrent, or InterPlanetary File System (IPFS), and others. blender.org is experimenting with a peertube instance at video.blender.org already.

Of course that only works so long as bittorrent isn't blocked, or WebRTC isn't blocked, or backbone operators start to throttle protocols competing with those that are part of their corporate conglomerate or have not been paid for. Or regulatory capture though some pro-establishment law effectively bans it (like the EU Copyright `Article 13' crap happening now).

Even if some backlash makes them change their mind it's just another example of the problem of corporate centralisation/ownership of culture.

Free Software JavaFX

I've been keeping a (rather loose) eye out on the availability of a fully free-software JavaFX for a while. Whilst all distributions now ship OpenJDK by default and that works fine for anything not JavaFX, the OpenJFX stuff has still been work in progress and hasn't generally been available.

Anyway I thought i'd try to compile it myself. Of course the first thing I noticed was that a preview build IS now available but I decided to keep going anyway so I could build a complete integrated JDK/JRE rather than having to manually link it in at runtime.

For the first part of the problem, following the build instructions was quite straightforward, at least on Ubuntu 16.04. I manually installed a couple of the specific build tools required and a couple of extra -dev packages. I didn't bother with building the bundled webkit for now. Even on this gutless under-RAM'd machine it didn't even take terribly long. I used the openjdk 10 for the compiler.

Then I looked into integration witht the jdk (later section in the build instructions). I was too lazy to read most of the build instructions so I just went with configure && make, although after a few iterations I settled on:

$ mkdir build
$ cd build
$ ../configure  --with-import-modules=../../rt/build/modular-sdk \
  --with-jtreg=../../jtreg \
  --prefix=/opt \
  --with-native-debug-symbols=none
$ make product-images
  

Some time later it's all done and I ran some simple tests and Bob's your uncle. Stripping the debug symbols just reduces the size of the install (significantly) although it's probably worth keeping them at this early stage.

The slowest part was checking out the mecurial repositories.

Oh, I couldn't get the jdk tests to run :-/ jtreg just complained that it couldn't determine the JVM version. Trying to search the interwebs has so far been fruitless - it's not terribly important for now and I successfully ran the bootcycle-build which self-compiles after bootstrapping which is at least some bit of testing.

Update: Not sure where I got it from but the jtreg I had was out of date so it couldn't handle the new version string. Starting on the jtreg page got me the latest (4.2b12) and now the testing is running.

Torrents

Although this box doesn't have much disk it does have enough for a few big blobs so I thought I'd look into distributing a build (because, well!). To that end I sussed out setting up a private tracker and seeding torrents directly from this machine.

Anyway i've mostly worked it out but it isn't quite ready but i'll get to it eventually.

Java 11

With the short lifetime of Java 9 and 10 i've basically just ignored them completely - i'm still using 8.x everywhere. But Java 11 is going to be LTS release so it's probably time to start into it.

Having a fully free build of the whole platform is an important part of that for my hobby code. Now I just need the motivation ...

Update 2: And NetBeans, which is still not supporting a Java that was EOL'd 3 months ago. I'm sure they've nearly go the licensing sorted out though!

Update 3

Ok so I guess i'm a bit out of the loop (and search engines failed me again, it took me hours to come across this stuff, mostly by accident).

JavaFX is going to removed from the JDK. And additionally the JRE will also go away(?) and basically replaced with standalone platform-specific builds via jlink. Actually i'm not sure how the latter will work, maybe that's just wrt the openjdk; and in any event Oracle are committing to another few years of Java 8 releases.

Given that all the 'action' is server-side I guess it makes some sense. Just not for me!

Huh, I wonder why the current jdk build process allows linking in the JavaFX module then. Unclear. Just doing so breaks the tests so it's probably just the build system lagging slightly. Then again the make_runargs.sh script used to run against the javafx module doesn't seem to work against openjdk11 anyway. Well unless there's some other meachanism but neither javafxpackager nor javapackager get built - does jlink do all that now? So far my searching has been pretty futile and a lot of the documentation is either out of date or hard to find. I suppose it's still basically work in progress and/or i'm just not that motivated to dig deep enough.

Well I did waste the day anyway so whatever. At least AMD announced some nice hardware and Intel made an arse of themselves with their 'first' 5Ghz cpu demo which needed to be plugged into a fridge to run for 10 seconds.

Oh, there is also openjfx in ubuntu repos, but it only goes to 8. I can't remember if i looked at it in the past, probably did but I think I was looking at java9 at the time not realising how short lived it would be. I should've just stayed with 8.

Update 4

Well I did find something about running against the standalone javafx on this blog post. I guess all that javapackager stuff is gone or something since webstart and applets are defunct? Not that I ever used it anyway. Shrug.

Also apparently using jlink and making a partial copy of the jre for every application will be `easier' than just sending out a jar and having a central single installation of a jre. Wot?

Everyone seems to be a bit upset that Oracle are only going to provide commercial support for the oracle jdk11+ too - but given how much they've invested in making the openjdk feature parity I think they did ok for such an `evil' company. I mean, it's not like they forked some public commons like Linux or something as Google did.