unusually long arms?

Since getting skinny again i've been forced (against my will!) to at least look into finding some clothes that fit.

On the weekend I went to look at some locally made woolen stuff (expensive!) and the smallest size was still a bit baggy for the thin tshirts (but they are intentionally a rather loose cut). But the jumpers of the same size were just too short in the arms and the shop assistant kindly suggested my arms were unusually long! Hah, maybe, maybe not. I went with a medium jumper and some small tshirts and i'm probably all shopped out for a while after that effort so I wont find out if she was right till I try other clothes. God knows when i'll wear them.

In high school photos I was always at the side in the first standing row and with rows being ordered by height that put me squarely in short-arse territory. When I did work experience at the end of year 11 they kept joking how I was going to "grow up" to be a jocky (funny men!), yet somehow by the time I started uni 2 and a bit years later i'd reached 6'; so maybe i am an odd shape (or it's not me that's odd!).

I had resigned myself to being about 90kg for the foreseeable future at least and I even expected that to be a little optimistic. Losing nigh on 4 1/2 bags of spuds in 9 months seems ... excessive; that was just under 25% of my total mass. I seem to recall measuring myself rather inaccurately at over 1m around the waist at some point, ... and now it's easily under 800mm (those spuds had to fit somewhere I guess). An old mate I worked with before Ximian reckons i'm thinner than when I left to start at Ximian nearly 16 years ago. Yay I guess? FWIW I asked the dr if i was underweight last week and he said no way and even if he just based it on BMI == 20 that's good enough for me.

It certainly makes riding up hills a lot easier. I took the roadie out for a roll last weekend for a 65km round-trip to Port Noarlunga which has a few rises and on the day sported a stiff (and cold!) southerly to head into on the way up. Given its been so long since i've been that far or that hilly (such as it is) it was about the right distance from a cold-start - i was a bit tired when I got home but barely sore the next day (well apart from the arse, it will take a few more trips to get saddle hardened to that seat again). I've also been going to the beach any day it's hot enough and splashing around like a drowning cat-in-a-sack which makes for some solid exercise together with the 1 hour riding required. It's pretty dull though and the 1/2 hour ride home is enough to heat up again but I had to get out of the house.

So apart from the gout i'm probably on my way to being physically in better shape than ever. And on the gout, after a bit of a hiccup of misunderstanding that should be working towards becoming a mostly-non-issue too (albeit with a daily pill).

Pity about the head though, "utterly miserable" pretty much sums that up. But that summation has rarely been very wrong at any time for as far back as I care to remember.

OpenCL 2.1 + java = zcl 0.x?

I noticed Khronos released the OpenCL 2.1 spec recently so I spent this morning updating zcl to include all the functions.

Since I don't have a suitable implementation nothing is tested and there's probably some typos and so on. I found a few small bugs in the enum tables while I was there.

But what took most of the time was the property queries. Each OpenCL object type has one or more query functions but rather than implement them all I use a tagged query function which branches to the correct function entry point at the lowest level but shares all the rest of the code. But then I had to add some specialist variants, and specialisations for return types and overloaded parameters - it started to get unwieldy and a new query type on CLKernel meant it wasn't going to be enough anyway.

So I said fuck that for a joke and just redid the whole mechanism.

For the basic 5-parameter queries I still share most of the code but I now add any type-specific queries separately. To cope with the api and code bloat i distilled the java side interface down to only two entry points for each query:

    native <T> T getInfoAny(int type, int ctype, int param_name);
    native <T> T getInfoAnyV(int type, int ctype, int param_name);

The first is a scalar query and the second an array one. It just means it now has to box primitive return types for scalar queries which is unlikely to have any measurable performance impact but the Java helpers which wrap the above interfaces in type-friendly calls could always be replaced with native equivalents if it was an issue.

This let me merge some internal jni code and delete a lot of snot and I moved the re-usability to a different layer so that the more specific queries can share most of the code. For example this was the previous set of native interfaces on CLObject, and although this covered the kernel and program specific 6-argument queries like GetProgramBuildInfo() it was getting a bit messy.

    native long getInfoLong(int type, long subtarget, int param);
    native long[] getInfoLongA(int type, long subtarget, int param);
    native int getInfoInt(int type, long subtarget, int param);
    native byte[] getInfoByteA(int type, long subtarget, int param);
    native <T> T getInfoP(int type, long subtarget, int param, int ctype);
    native <T extends CLObject> T[] getInfoPA(int type, long subtarget, int param, int ctype);
    native long getInfoSizeT(int type, long subtarget, int param);
    native long[] getInfoSizeTA(int type, long subtarget, int param);

... It seemed like a good idea at the time.

The exposed interfaces remain the same (like getInfoString(param), getInfoInt(param), etc).

Given the complete lack of interest and because it needs some testing anyway I wont be releasing a zcl-0.6 just yet.

bloody peecee

I've been having a few issues with my machine lately so as i had nothing better to do yesterday I had a look into fixing it.

First issue I looked at was that I broke OpenGL when i upgraded to the latest catalyst driver. I tried various kernels, even a long-overdue update of slackware (which lead to a another few hour's diversion earlier in the month trying to un-fuckup firefox as much as i could), using the slackware kernel, etc. But finally I used strace and slackpkg to determine it was using a stale .so left over from AMD's previous abomination of an install script so once that was deleted all was well. SVM still wont work in any of my code - which is still baffling as it works from the AMD SDK samples. I should really post something on the AMD forums but i haven't used it for a while.

The other problem has been only half my RAM started showing up a few months ago; the bios shows 8G but linux only sees 4G. This is ok for what i normally use the machine for but netbeans brings it to it's knees if you have a couple of projects open. I thought it was down to a faulty DIMM and If i hadn't forgotten and needing to come home earlier yesterday I might have bought some. But I tried each individually and they seemed to work - passing the linux make test at least. I tried lower speeds, voltages, all sorts of things. It appears a fairly common problem but nobody had much of a solution.

I even upgraded the bios - which was another couple of hour diversion as that cleared the EFI boot record and I had to re-figure out how to re-install it. I used a USB-bootable tool-linux distribution called grml - very impressed - I just dd'd it onto the usb stick, and it even provides an EFI boot record so i could just use efibootmgr to re-add it. I know i've tried puppylinux multiple times before but it never worked so I might get a usb stick just to keep this handy.

The last thing I tried was dusting the sockets and then trying to jiggle the cpu in it's socket - I don't have any thermal paste so I couldn't get at it properly but i wiggled a bit without freeing the heatsink.

Now ... that appeared to work. I did some tests and so on and it seemed to be ok, ... I even wrote a post about it.

But then I had to tinker, so i was playing with the memory speeds - to see if "faster" ram really made any real difference - but I broke the whole machine so i had to power cycle it a few times till the BIOS reset everything, ... and yeah back to 4G of ram. Blast. I swore at it, shut it off, and went to bed early.

On a cold boot I did get 8G out of it but then the system crashed while I was using it, so something is amiss. It could still be the RAM I suppose but at this point i'm more inclined to believe it is the motherboard or PSU. I don't feel like investigating further for the moment and will just see how it goes (i just cold booted and it's 8G again *shrug*).

However, some good did come of all this.

When I was playing with kernels I went to the trouble to actually go through all the config options and fuck off everything I didn't need. It's down to 3.2M packed including my system filesystem with no module so i don't need an initrd, and it boots a bit faster (not that it was any slouch). I'm also trying the fully-preemptive kernel and i'm liking it so far, even under very heavy load the system remains interactive such that you barely notice at all. Hmm, I just noticed the sound mixer is missing, NM.

But the really good bit is now the CPU runs almost cold - i don't know if it's the BIOS upgrade, the kernel customisation, or some BIOS setting that changed (i already had it on 45W TDP because it got too hot), but the difference is marked. Previously any high-load task such as a 'make -j4' would cause the fan to kick in almost immediately and if i didn't also up the case fan speed (manually) fairly promptly the whole desk would take off. I've currently got a kernel build running for a few minutes triggering a load-average of over 8, with the case fan on it's slowest setting; and the CPU fan hasn't throttled up at all.

Maybe the machine is just running a lot slower now - but I can't really tell so what does it matter if it is.

Update: it got unstable so i took one dimm out. Seems ok so far. Maybe it's just the PSU? It seems like the most fragile component and it's almost certainly underpowered anyway (sigh, a poor decision that one). I dunno. I started looking at new cases & psus but if it remains stable albeit in a reduced capacity I'll be in no rush ...

time is an illusion

But so is reality so it doesn't make any difference.

I've got too much of it to fill with boring tedium either way.

OpenCL 2.0 + Java = zcl 0.5, or ~= 1.0 beta

I spent a wet morning doing some clean up and packaging of another zcl build and just finished updating the home page and uploading the source.

Although I just bumped the revision, this is getting pretty close to a 1.0 release. It's still got a few missing bits but it's mostly because the documentation is a bit broken beyond the README. It is only compatible with Java 8.

The home page has more details but the big points are that it now garbage collects everything (with explicit override), the lambda interfaces (trivial though they are), dynamically links to libOpenCL, fills out the extension framework and implements some extensions, and supports cross-platform building of native code.

I had to add a small code-generator to make the dynamic linking practical but it relies on the strict formatting of cl.h and does nothing fancy

Now i've got cross platform sorted out i'll probably do all my work to this interface rather than jogamp/jocl because it's just nicer to use and easier to work with. This might not mean any more frequent updates but at least it should get tested more. But apart from not being able to get SVM working at all on my machine (sdk demo works, cut and pasted bits from demo, or any other thing i write - crash) i've encountered very few bugs anyway.

I've probably covered enough of the new stuff in the blog previously so probably wont have much to add, but the curious are welcome to ask.

yay summer

Just got back from the beach. Should've had a camera - water was literally like glass. It was overcast so the horizon vanished in places which is always a nifty effect. Water was crystal clear. It is also still pretty cold though and once you go out past the sand-bar things get a bit nippy. Had the cold water all to myself though there were a good few people out and about but almost nobody went beyond the shoreline unless they were on some floating craft. The closest beach is a comfortable 30m ride and between jetties so it's quiet, with no posers or jet-skies (so far?) and the sand is really nice (best sand on any beach i've been on). Should be a nice night too if the stillness keeps up, it's absolutely dead-calm again (@6pm) which we've had a fair bit so far this spring.

Nice to be 'skinny' again too. You don't notice when you're putting it on but its like i just stopped carrying 2x10KG sacks of flour (and a carton of milk!) around with me everywhere, which is definitely noticeable now it's gone. Not that it ever stopped me getting my gear off at the beach it feels better losing the dugong. Its like winding the block back about 15 years, well apart from the mental and physical scars, aches and pains, and now all-grey hair. Although i'm still getting used to looking down at my skinny knees when i'm cycling. I got some new shorts a couple of weeks ago and they already feel loose - but i think they were a pretty generous "Size 32" to start with.

Of course I should be having a beer or nice limey g&t right about now but the gout is already niggling (probably from a couple of beers i had monday with a visitor, first time in weeks) - my toe was even a bit sore after the first dip. Sigh. More fucking tea I guess. For me it seems the main cause is just not drinking enough water (or actually not weeing enough) - but by enough that means at least 3L per day just to start with. Which gets to be a bit of a drag and if i'm doing physical things i'm not sure I can even drink enough to offset the sweating.

It makes going out a pretty abysmal experience because i just don't enjoy myself. Actually I haven't been enjoying myself much going out the last few years but a couple of beers at least made it passable for the occasional highlight. I can have a few but might suffer later so usually don't risk it. Not that i've had much opportunity anyway but i've been almost dreading when they do turn up - i'd rather just garden or find some way to pass time by myself lately. Maybe it'll pass but I don't care if it doesn't. Speaking of which someone just called and i turned him down, but i might change my mind in the next hour. I was all settled in after the trip to the beach and don't feel like dressing up to hit the city, if i can even find clothes that still fit. I never like going out on Saturday anyway as it's too obnoxious and busy and the crowd a bit too keen, and i'm a little underslept and too cranky to put up with that shit anymore.

I've been eating a lot of chillies at home - i'm getting pretty steadily through my stash of frozen habaneros from a few years ago so hopefully I don't run out before something else comes online. I might have to buy an advanced seedling as growing from seed here without a greenhouse takes a very long time due to the cold nights. I finally finished a bottle of Blairs "Possible Side Effects" sauce this week - i can't remember when i opened it but it's years past it's use-by date although I didn't use it for some time. It's really expensive here in AU so i kept it anyway and after i topped it up with some lime juice it tasted better. It's not the nicest flavour but it's ok, and it's hot. Now onto the "Ultimate Insanity" sauce which has been in the fridge too long ... But due to just not eating much anymore and the palatte destruction of lots of chillies often with limes or lemons ... eating out has become super-bland and quite unappealing.

Work has been quite taxing (in a good way, although i'm a little blah on the whole need to work right now) so i haven't been coding much outside of that. I'm not watching much TV. I watch a small amount of twitch/youtube stuff but not that often. I've been playing PS4 a bit more - although it's mostly DRIVECLUB and Res0Gun. Res0Gun is just a brilliant game and although I'm still pretty mediocre at it the more you play the better you get - it's very pinballish. I turn up my new amp enough to feel the explosions. Yum. The other regular is DC which I do some single-events, community challenges, or occasionally tour events. None of my friends have it so I just see what's there (one just got a ps4 but he's not into car games). I've started doing many-lap races to learn some tracks and cars and with sped-up time it makes them pretty interesting once you get into the swing of things. With the new PS communities I played a bit with the photomode and uploaded some for something to do. I got a couple of messages so I guess somebody saw them but it's a little inflexble at the moment - something like a web forum with each 'community' a single topic. I even tried streaming on twitch today which at least I confirmed my router didn't crap-out, but i'll have to be in the right mood to try that again. And check with a GNU box how it looks.

I finally finished the last book i was reading - it really dragged on, and/or i kept falling asleep on the last bit. I dunno some dreary battle against evil by characters i didn't give a shit about (with a weird and entirely unnecessary prologue which explained what happened afterwards). And then straight onto the next of the 12 part(?) epic it is part of. This could take a while ...

I guess it's time to decide on what to do this fine and warm evening.

I can grab my tea and go wander out in the garden in a pair of shorts until the grass gets too cool and then watch some shit on TV or play a game or just go to bed ...

Or get tarted up enough to go to a pub (not much tarting admittedly) and head to the currently trendy part of town to drink water and hang out with my leery mate and his wanker friends (actually only one is obnoxious, if he's even there) while they get drunk and perve on the chicks half their age. And then ride home feeling afterwards probably feeling miserable (it's just happened the last few times).

Yeah I already decided when he called. Time to go see what the bugs have been eating if it hasn't cooled down too much while i've been here. Still dead calm at least.

Catching up

Not much going on so today here's a diary entry ...

I've added a few little things to ZCL - it's coming along quite nicely, I should probably work toward another release. I'm slowly adding the functional-like stuff into it, I decided to go with 'q.offer()' as the enqueue function, and 'ofXX()' as the factory methods. Together with the garbage collector support it does offer some interesting possibilities for code-reuse but i'm still experimenting with it in practice.

I needed to access a webcam so i added another api to jjmpeg (but i might move it) which just wraps v4l2 devices directly from the file descriptor (no library). Actually I did that a while ago but have slowly been filling it out as I needed more functionality. OTOH I started looking into the total snot-show that webcam access is on microsoft platforms and decided to give up - you can't even build the media-framework libraries with mingw-w64 as far as I can tell and you need vs of some form just to get the "system" headers. Ugh.

However I did find webcam-capture library which has already solved these problems. It's probably what i will look at as a fallback, but on linux efficiency is a bit low. The simplest webcam dump to JavaFX with my library generates almost no garbage (it provides static access directly to the driver buffers) and uses 50% less cpu time compared to using the low-level interfaces it provides despite a pretty expensive YUYV conversion step. The high level swing one is 4x the cpu overhead.

Along the way I found that was actually using videoInput "library" but I couldn't get that to link (cross compile at any rate, it should work with the ms sdk) - and in any event that just uses the directshow stuff which I had working ... i dunno, years ago. But the driver no longer works for the webcam i'm using and i'd have to buy it, ... so yeah that can wait.

And that ultimately led me to openimaj which probably would've saved me doing almost all that myself, although i would've needed to understand it anyway for OpenCL translation. And maven, ffs. But I guess I should at least have a look.

Discovery of useful software isn't that easy these days with so much noise - even if you go looking which I can't say I did ...

I've also been using a small amount of OpenGL and interoperating with OpenCL. It's become all a bit ... naff, and JOGL has been necessarily messed up to support all this nafficity. Pity I had to do this now rather than in a couple of months otherwise I would be looking at vulkan instead but with any luck that will be out soon enough to move to it before I need to get too far into GL (with the steammachines in november?). I'm sure microsoft will find some way to totally fuck-up it's cross-platform parts again though. My current thinking is that i will write a java binding for it once I have my hands on it (if only just-because) but I haven't looked into it all so far. But removing the static per-thread state should make it a lot saner.

For now I have some simple classes to do some off-screen rendering, and some OpenCL interop which is enough for what I want. Access to an output texture in JavaFX would be nice but it sounds like this is just not going to happen. Although one would expect a vulkan backend to be done at some point it will probably suffer the same hiding issues (well, with good reason I guess). If I really needed more performance i'd just use another toolkit - which is sad as that doesn't appear to be the intended vision of the javafx designers.

On that interop I had to fill out the extension mechanism in ZCL. I followed the prototype I'd created earlier. Currently each extension is provided by a different CLExtension class. It holds a pointer which in C-land is a function table resolved on the platform, and each platform object manages these. At first I was just going to use this as the mechanism for accessing extensions but it quickly becomes messy - you have to find the platform the object belongs to an in some cases this requires multiple queries (e.g. q.getDevice().getPlatform()). One approach I tried was to hide that by providing the extension methods directly on the object they extend - e.g. new CLContext or CLCommandQueue methods. These then manage looking up the extension and invoking the correct method for the given object. The details still need to be resolved but only once per object and it's all handled java-side.

There's a bit more behind the original mechanism than just code tidyiness for the extensions - they could potentially be loaded at runtime, or written separately from the core library. But on reflection how useless is that? The problem with this approach is each extension has it's own object - this is good and bad in that eventually you end up with a table required per context, queue, device, or whatever.

I think putting the extension methods on the target object is correct and after that the details don't really matter so much since it's an internal detail. But (on the fly design) I guess i should just maintain a CLPlatform reference on each object which can be extended and handle it that way. The extension objects will still be per-extension which keeps a cleaner namespace but they only need to be set per-platform which doesn't happen often. I'm pretty sure all objects have a 1:1 platform relationship, that would be the only thing to throw a spanner in the works; but the whole extension mechanism wouldn't work if that were the case.

Somewhere along this journey i came across some C++ code for something, I can't remember what it was. It was how I find most C++ code - it's been so over-engineered almost all of the actual lines of code is just boilerplate. The workings are so hidden I gave up trying to find it. It's just as bad in Java land where everyone wants to write a fucking framework before they even get started. Cut the bullshit and get to the point. C++ shows its heritage as being born from a time when "Software Engineering" was going to solve the worlds software problems by taking the programmer out of programming; using UML and CASE tools and auto-generating everything. It really shows, it is not a good language. This craziness was at it's peak just as I was going through uni and its done it's fair share of damage to the world and clearly continues to if abominations like C++ still exists.

That'll do for now.

OpenVX

Here's something I missed being out of the loop for the last good while: OpenVX.

Although it appears so did the rest of the industry? Another standard sent to rot?

I'll have to have a closer look though; if it is well thought out it will be worth borrowing a few ideas at the least as it's a problem i've come up with multiple solutions for myself and it's always useful to get a fresh perspective.

Or maybe it'll give me something to play with along with ZCL either as a binding and/or a pure Java (re)implementation for prototyping.

I wonder how far away vulkan is for the rest of us.

Update: Some movement has occurred on the HSA front recently. I suppose just some more announcements or partnerships or something. There's a lot of technology and software that needs to come together to get anywhere and it's taking it's time. I wonder if it'll ever get traction though. I guess it solves one of the problems with OpenCL - you actually have to program parallel software which is a skill few have the time to learn and fewer companies have the budget to fund. At least it has the potential to stay in the compiler layer, hidden from your typical code-monkey.

OpenCL notes

As tangentially related observation I've been working on some OpenCL stuff of late and I came across a problem I just couldn't get to run very fast on the dev GPU. I ended up resorting to using an OpenCL/CPU kernel and another queue but along the way i tried a couple of other interesting things.

One was using a native kernel written in Java - but this still requires a CPU driver / queue since the GPU driver i'm using doesn't support native kernels. ZCL has fairly simple interface to this:

public interface CLNativeKernel {

    /**
     * Native kernel entry point.
     *
     * @param args arguments, with any CLMemory objects replaced with
     * ByteBuffer.
     */
    public void invoke(Object[] args);
}

public class CLCommandQueue {
...
public native void enqueueNativeKernel(
    CLNativeKernel kernel,
    CLEventList waiters,
    CLEventList events,
    Object... args) throws CLException;
}

The JNI hides the details but it behaves the same way as the C code whereby any memory objects in the argument list are replaced by pointers (ByteBuffer here). Not sure if i'll keep the varargs prototype because it is inconsistent with every other enqueue function and only saves a little bit of typing. I'll review it when i look at the functional stuff i mentioned in the last post.

Which can be used efficiently and conveniently in Java 8:

  CLBuffer points = cl.createBuffer(0, 1024 * 4);
  q.enqueueNativeKernel((Object[] args)-> {
    // object[0] = ByteBuffer = points contents
    // object[1] = Integer = 12
    // object[1] = Integer = 24
  }, null, null,
  points, 12, 24);

Since my prototype harness didn't have a CPU queue until I subsequently added it my first approach was to emulate a native kernel using event callbacks and user events. It actually worked pretty well and was about the same runnning time, although it took a bit more (fairly straightforward) code to set up.

One approach I took was to have two queues - the primary 'gpu' queue where the work is done, and another one used for the memory transfer and rendezvous point.

  // setup CLEventLists and the user event

  // gpu part of work
  gpu.enqueueXXKernel(..., gpudone);
  // prepare for cpu component
  work.enqueueReadBuffer(..., gpudone, readdone);
  // prepare for gpu again
  work.enqueueWriteBuffer(..., cpudone, writedone);
  // make sure q is ready
  gpu.enqueueMarkerWithWaitList(writedone, null);

  memdone.setEventCallback(CL_COMPLETE, (CLEvent e, int status) -> {
    // do the work
    cpudoneevent.setUserEventStatus(CL_COMPLETE);
  });

In this case all the enqueue operations are performed at once and events are used to synchronise. This simplifies the callback code a little bit. Now i'm looking it it there's probably no need for the separate queue if the gpu queue is synchronised with it anyway. (like with most of these examples it is a summary of what i came up with, but not the full sequence of how i got there which explains some of the decisions).

This is a trivial approach to ensuring the 'gpu' queue behaves as the caller expects: that is, as if the work was performed in sequence on the queue and without having to pass explicit events. I'm using the read/write interfaces rather than map/unmap or otherwise mostly out of habit, but the data in question is quite small so it shouldn't make much difference either way.

And FWIW for this problem ... this approach or the java NativeKernel one actually runs a tiny bit quicker than using the OpenCL/CPU device let alone the GPU (all wall-clock time on the opencl q).

I had to make some small tweaks to the CLEventList code to make this all work and to tie properly into the garbage collection system. Mostly this was adding a CLEvent array rather than just using the pointer array and fixing the implementation. I kept the pointer array to simplify the jni lookup. I also had to have construction go through the same mechanism as the other CLObjects so they retain global reference uniqueness. This greatly simplifies (i.e. completely removes) reference tracking which is always nice with async callbacks. I think it should all "just work"; it does from the Java side - but i need to check from the OpenCL side of things whether actions like setUserEvent() adds an explicit reference bump.

This is a prime example of what HSA should solve well, but for now this is what i've got to work with.

I've been so busy with details i haven't had a chance to even look at any OpenCL 2.0, let alone OpenVX, HSA, or much else. And frankly the spirit is just not that willing of late. Spring is just the latest of a litany of excuses for that.