More on project panama

So i've continues to work on code utilising project panama/jdk.foreign to bind directly to C apis. Most of the work has gone into one of the vulkan bindings in panamaz - i started a 3rd one that creates the api directly from registry but utilises templates heavily to be a bit less hairy than the first attempt. It's still grown to be pretty hairy but at least it runs fast and generates a good quality api. I might split it out into it's own project soon as netbeans is getting a bit unwildy opening panamaz with so source files to work with.

Now i'm going through the vulkan tutorial and using tha to refine the api a bit further. I'm working towards using signed distance fields for digital art of some sort. What? Who knows, i'm lacking inspiration.

Also my heart isn't really in it at the moment so it's a pretty slow burn. I've stopped tracking the development version and i'm just using openjdk 18 - the primary reason is that netbeans wont open projects properly when i use my development builds so i've kinda given up. The api has moved on quite a bit from jdk 18 so it will be quite a bit of change when it comes to update unfortunately. I'd been using emacs to get around this but without code completion Java isn't a lot of fun.

notzed.nativez, java.make

I've been working on an update java.make which removes the JNI bits and replaces whem with jdk.foriegn bits via an updated notzed.nativez (jdk-foreign branch). I had made a couple of mistakes: using .PHONY targets as dependencies just doesn't work, and trying to hook everyting to the 'classes remade' sentinal wasn't sufficient granularity. I've added a few more sentinal markers which can be used as dependency targets and this has greatly simplified the dependency setup and allows builds to properly parallelise.

Long term i'll probably update jjmpeg and zcl to use these new mechanisms but for now they are not a priority.

I have another background project going to try to optimsie the build process. I have almost all the code done in both c and java to handle accurate incremental builds, e.g. it can find out what needs to be remade, what needs to be deleted and so on, run a compiler server, and generate makefile dependency rules, but for now i'm using javac with '-m'.

netbeans

Along the way I ran into some performance issues with Netbeans and java code completion. As part of the code completion it runs through a routine which filters out the visible symbols based on Java's scoping rules, but the routine uses an N^2 algorithm, and to make matters worse the inner loop does a string comparision based on output of CharSequence:toString() - which is very expensive, genereates a ton of garbage, and by far the majority of the time expense. Even the somewhat modest 3000 constants from vulkan will slow the routine down to around 500ms on a ryzen cpu. Oddly there was alreayd a solution to this coded up but for some reason it was never hooked up to all uses of the visibility check. I reported it with some potential fixes and then found out jira is no longer the bugtracker for netbeans. Sigh, it sucked but at least it wasn't github.

Netbeans code completion seems more broken than it used to be as well since they switched away from the gpl licensed nb-javac. It wont really complete properly until you've added the import and syntax errors (you know, the ones that exist as you're writing new code because it's decided to repase it mid-phrase) will break all sorts of things. Now i've got a codebase I can edit i might see if i can at least build a local copy without all that tooltip bullshit that keeps getting in the way of what you're doing. Be nice to fix the slow performance of the output window too - there's no reason it should be so miserably slow.

jdk.vector

A few weeks ago I also played a bit with the vector api which allows one to access SIMD instruction sets natively in java. It's a bit better than I thought it was when I first looked at it (it seemed very clumsy), but it's also less abstract than I had hoped. It's more or less a mapping to AVX intrinsics - you need to specify the vector width and although you have abstracted intrinsics you must code to the platform the same way you need to with intrinsics otherwise the performans tanks. Also suffers from the general problem of compilers over-usign registers so it's easy to get spills if you try to unroll anything.

I was mostly poking around with graphics related maths functions like matrices, one routine was a 4x4 matrix inversion using gauss-jordan elimination. I tried to unroll the inner loops entirely using SIMD selection logic to implement the row sorting but it didn't create much speed-up. Oddly I tried a branchy bit of code that generated some very odd hotspot output that executed microbenchmarks quite a bit faster - it seems to branch off to a sort of exception point, re-arrange the registers via memory swaps and then re-enter the main loop through some hotspot function. Kinda hard to follow what was going on with the assembler output. I might post about this later if i revisit it.

Panama Experiments

After a pretty long break I started playing with the OpenJDK panama branch again - this is the in-development mechanism for Java to be able to call C functions using metadata rather than requiring a C binding library via JNI.

Whilst I get the impression the developers just aren't very experienced with the way C works and keep trying to shoe-horn C libraries into a fixed and awkward model of how they think they should work, overall it's much improved than where it was last time I looked. Of course it's a moving target still so even the updates i've made will break shortly.

Apart from updating to changes in the panama api the main changes have been working on a more flexible api compiler utilising programmable templates, and at least attempting to move away from write-once perl. It's surprising how much more work this took, about 5x the effort and you don't seem to get a lot of benefit out of it initially.

I also have a vulkan wrapper that uses the vulkan registry xml document as the sole source. This allows it to better group various features and automagically generate a nice object-oriented api. One feature it has is auto-generating 'nice' constructors so using the api is pretty much equivalent to the C api, it can also auto-imply some parameters like array lengths. This was an earlier iteration of the code so is definitely write-once code, i'm considering trying to covnert this to use the templating framework from the other generator but it's a huge amount of work.

I'm still working on how ResourceScopes can fit with the various api's generated. In some cases it's an ok fit, in others it's has a very large impedence mismatch which is difficult to resolve in a satisfactory manner. It would be nice if there was something with a bit more granular control available as once you start using scopes you're forced to use it everywhere, or just not use it. They map to problems where you have a bunch of resources you allocate incrementally and then free together, but there are many scenarios in C apis that just don't work like this at all. I've had some discussions on the mailing lists but so far i'm not terribly satisfied with the offered solutions, we'll see I suppose.

The REAMDE has a bit more info.

gcc export plugin

One tool that might be useful for others is the gcc plugin to export all the available type information. It's a pretty messy bit of code because it was mostly written by trial and error and viewing the output of debug_tree(). I admit I didn't read the gcc-internals documentation enough but it's pretty dense and hard to follow, and gcc plugins are few and far between.

Getting the named parameters for function declarations was particularly problematic - in some cases it's simple as you get the parameters, then the function type itself. But an untyped function pointer declared as a field, or as a parameter, comes to the plugin in a different way, sometimes only being able to be resolved after the plugin is finished. I still have some 'dangling' function parameters left over but they don't seem to be important and even on a very complex set of header files like ffmpeg all the fields and named parameters are being resolved properly.

The output of the plugin is a perl hash which completely describes all the structs, unions, function pointers, and enums found in the header files.

Because #defines aren't available to gcc plugins a separate programme is used to generate these from the output of cpp. Because #defines are just C it isn't trivial to convert these to real values so the programme generates a C file which is then compiled and executed to generate the perl output. It uses some gcc features to create a pseudo-function overloading to imply the type of #defines, and has flexible filtering mechanisms to only grab those of interest. It's a bit fragile but can be controlled to avoid some broken cases. Also due to the need to execute the extraction step cross compiliation will be difficult or impossible but that's a hurdle for later.

Have at it

I spent most of January working pretty solidly on this new generator but i've not felt like working on it much for a little while. The ResourceScope stuff is just about to be renamed (the justification seems pretty weak to me but whatever) so it'll all break but i'll patch it up when those changes settle down.

Browse the source code, the git checkout url is https://code.zedzone.au/git/panamaz.

I'm using the foreign-jextract branch of https://github.com/openjdk/panama-foreign.git.

A year down

It's been a year since I decided to quit my job, well yesterday was to the day. Also a Friday. At the time I put two weeks notice in but they completely stopped communicating with me after one week so I just gave up.

It's been a weird year. In some ways it seemed to drag on, in other ways it seems to have gone fast - I guess in general i've just lost track of time since i've got nothing in particular to mark the days. I've spent way way too much time (and money) at the pub, and it's getting to the point that I have to try to move on. It's making me feel like a total loser just hanging around mostly drinking by myself - or worse drinking with some random who turns out to be a total fuckwit (last one started telling me Hitler was only trying to protect his country ... sigh) or someone who gets funny on booze. And somehow my circle of friends keeps shrinking, I can probably count on one hand 'friends' i've seen in the last 6 months, and I might not even need another hand for good aquaintances. I suppose the thing which most upsets me is I just don't have any other way to socialise, and although I don't mind a good booze-up, it's just too bloody much doing it almost every day. I try to think of things but even without COVID making a mess of group activities I really just don't like doing organised shit anyway. Maybe it's my lot in life to live alone until I die - it's certainly been a consistent experience so far.

I cut back on the pub a lot this week anyway and spent a lot of time hacking on zcl/foriegn-abi and playing games. Going through God of War, although I don't really how it turned into an open-world RPG - it's just not the same sort of game anymore and i'm a bit lost and well, basically grinding to get enough gear to progress. Sigh. I also got Gran Turismo Sport because it was on sale - bloody 120GB download (what for? shitloads of crap videos i don't give a fuck about?), I dunno it's 'ok' I guess and the Mt Panorama circuit has a tigher (narrower, more realistic) feel to it, but it just seems like i'm playing GT3 again. At least the license tests are easier.

I've continued to work on fitness and general health - been on some long rides (many 60km, one 80km) and even started going for a swim at the beach but it's been cold and windy all week so that's dropped off for now. Down to about 76Kg which is about 'ideal weight' according to probably-questionable sites I can find on the internet (my old scales were bunk but I think I was around 88Kg 2 years ago when i broke my hip). Of course I look pretty skinny but people are just oversized these days. I started having insomnia troubles so I'm trying to fix that using sleep restriction (CBT-I) with ... mixed results so far. I'm mostly struggling to stay up till midnight every night if i'm just at home alone playing games on the couch or using the pc. And so if I nod off for a bit around 10 or 11 it makes it harder to get to sleep at 12. Oh well it's an ongoing effort and maybe it's getting a bit better, it's one of those things that's hard to tell because you can't remember much from lack of sleep.

Fish as anti-deppressant?

I had to go the GP for something else and mentioned the insomnia and a lot of anxiety i've been having lately ... and it just boiled down to psychologist or anti-depressants. Neither of those worked for me in the past and the 6(?) pills I tried either did nothing (with shitty side-effects) or made things worse, sometimes much worse. I took the script because I couldn't remember if i'd had that one before and lo and behond - yeah, and it was another failure. So I decided to try something else - eat fish every day. Dunno if it was just a placebo but it had an immediate effect, literally the next day my mood felt better. Well not the mood exactly but it's like a mild 'warm glow' that settled into my body below my stomach which makes it easier to just feel 'good'. Sort of what one would hope of taking an anti-depressant although none I ever took had such a significant, lasting and well, completely side-effect-free result as simply having a couple of sardines or mackeral once a day.

It's been 5 or 6 weeks since I started and it's remained - and already helped me weather some shitty situations. Improved my skin a bit too, although I was hoping it would fix my cold hands and feet too but it hasn't. It was such a foreign feeling to start with, you know, not feeling miserable all the time.

Semi-Retirement

So I dunno if i'll work again - if i live cheaply and nothing major changes in my life I can probably afford to not work.

But fuck it's boring. And lonely.

So many hours to fill every day and not much to fill it with. If I walk to the pub I'll spend 2 hours walking but there's still so much day to go. A decent ride might be 3 hours or 4 if i'm really keen. Can only read or playstation so much. Gardning. And yeah people get sick of hearing how boring it is when they're too busy to get any time to themselves (maybe that's why my circle of friends keeps shrinking!). I can do some coding and so on but I dunno what's the point?

The loneliness is the real killer at the moment. Sigh. Well for now i'm trying to fix things one at a time - physical health, check, insomia, work in progress, then?

Guess i'll go for a ride to the beach today, although it's set to be warm the water will probably be too cold for a dip but I might get keen. YOLO and all that.

Life update

I'm still suffering pretty severe burnout from my last job - nearly 9 months after quitting although I barely had any work leading up to it either (due to covid/shceduling issues with the job) so it feels somewhat longer. A few weeks ago a mate got me to do a little bit of consulting work - fucking python of all things - and while I didn't hate it and the pay was ok i'm not sure I can be bothered even putting an invoice in because that just means more paperwork and tax crap to deal with. I live a rather modest life and can cruise on savings/investments basically indefinitely at this point assuming no big life changes.

Life is otherwise mostly ok. Still rather lonely and isolated on the whole, I suppose it's just in my nature. The only "friends" I see regularly are bar staff and other hospitality people i've gotten to known over the years by simply spending so much time at pubs. It has it's moments I suppose but every now and then the pathetic-hollowness of it all eats away at me and I bottom out for a week or two before I get over myself and get back to it.

My leg still hurts from the broken hip, mostly it seems to be tendons in the knee which wilted during the recovery and are difficult to strenghen up. I've been walking quite a bit the whole time but it just wasn't enough it seems. As the weather is slowly improving I've been working harder around the house and going for half-decent rides on the bike (2-3 hours) and that seems to finally be making a dent.

Could do to lose a couple of kilos, mostly just to drop an inch or two of belly fat so I can wear my utility kilts again! I've been working on it anyway and generally getting pretty fit.

The COVID-19 vaccine roll-out in Australia has been a total shit-show. Every time I've tried to use the shitty web-site to book an appointment it's been at least 2 months away - if there were even any slots available. Plus you can't do it without a mobile phone number which is something i'm loathe to give out. My GP ran out of shots when I tried calling them last month. Another GP wanted a health record from my other GP first (not meant to be required for a vaccine) - i.e. two bloody appointments. Just got sick of fucking around and noticed a chemist I go to was doing shots so just walked in yesterday and got an AstraZeneca shot. See why couldn't it have been that simple from the start? It's not the 'preferred' one for someone my age but it's much less anxiety on that than all the time i've wasted looking up shots over the last few months. So far no real side-effects apart from a sore arm.

I suppose the main problem is I just feel like i'm waiting to die and have been for decades. Nothing has really changed for me lifestyle wise since about uni - except now I don't have to work and I can afford to drink out. I don't like sleeping in so get up no problem but most things seem to be just going through the motions filling time. Be it reading or playing games, house or garden work, even just extending walks because I have time to blow. It's not all bad I suppose but it wears you down eventually. I'm spending much less time alone in an attempt to address that part, but it has mixed results I suppose. One side-effect of this is hobbies don't hold much appeal since they generally require alone time. I had been reading reddit/news obsessively for months and decided to finally bin that a week ago. This just means even more time to fill ... i've been going to bed early a lot! I can't say it's particularly liberating or anything but it's one less pointless thing I was doing I guess.

Oh well, life goes on.

Summer Cruising aka Long Service Leave

Overcrowding at Grange at midday.

The warm weather is winding down here but I did get a chance to get to the beach a couple of times. The water was a little brisk but on the refreshing end rather than numb appendages end of the scale.

I didn't get there as much as one would like but then our summer was a bit of a fizzer, I think we only had a couple of days over 40 and not many more over 35. After a dry winter we've had a drier summer so the garden needed constant watering, the veggies were a bit shit but i should be getting a decent crop of chillies - the only problem is they're about 2 months too late and will shortly stop growing but maybe some of the ones in the pots will keep going for a bit longer if i can find a sunnier spot in the garden.

Apart from that I'm just trying to enjoy my `long service leave' of indeterminable length. I've had some jobs suggested to me but it just turns my mind off, I feel really burnt out still. I've been trying to get into various hobbies from photography to video games to electronics, cooking, wood/metal work, sewing, reading ... but I just can't seem to get past a littler tinkering before losing interest in most things. I guess I need some specific projects, I should finish my workbench for one but it's a bit tricky getting shit without a car.

Trying to keep active socially as well, spending too much time at pubs and bars (which is getting a bit bloody expensive these days) but also trying to visit friends more often. Cycling a lot more than I have done in a long time although far short of where I've been in the past. Walking quite a lot. I lost a little weight and gained a lot of fitness at least. My broken hip hurts pretty much constantly but it isn't enough to bother me most of the time and whilst being active brings different pains it's better than sitting for lengthy periods so that's another factor driving me off my arse and potentially out of the house.

My "new" computer isn't doing much - it's just a web box most of the time. I keep the kernel and Slackware-current updated and have been keeping a loose eye on technology like Java but it's more out of habit than anything. I'm not doing any hacking - i'm still using playerz every day to play music but I was half-way through some work on playlists when I stopped. I guess the weather's been too good a lot of the time to waste it inside anyway.

Quit my job!

Decided to finally pull the pin on work. It's been coming for years but some issues regarding long service leave and some minor conflicts with the new manager finally tipped me over.

It's uncanny just how much my mood has lifted since the decision to leave - and is still up there a week later after a bit of a hard slog working on handover material.

I have absolutely no plans! I've got savings enough to tide me over for an extended period so I don't even have to rush to look for work.

Not sure what else i'll get up to but i'm sure along the way i'll have time to dabble in more Free Software than I have for a while and maybe start interacting with the wider "commnity" more. The music player continues to come along too and more on that later.

playerz: a music player

I got sick of mocp crashing every couple of days so i've finally got the music player I was working on last year into a state good enough to replace what I was using it for. It's still early alpha software and I don't have a project page for it or any documentation but the code is at https://code.zedzone.au/. It requires an old version of libeze, possibly the earliest that was checked in but I haven't checked.

Originally I was coding this for a beagleboard or the like where the player would be used without a graphical interface. It would automatically detect and scan new media and remember old media so re-inserting media would be fast. I also experimented with a 'talking' menu driven by a remote (Mele airmouse). Since i've now moved my old pc to under the TV i'm just targeting that now.

It consists of a bunch of services which comminucate using posix message queues.

music-player

Plays the music and allows navigation forward/reverse through the (shuffled) playlist. A rudimentary 'talking menu' has been implemented using espeak-ng but seems broken at the moment.

ffmpeg is used to parse and decode the media and to perform automatic volume leveling, and ALSA is used for audio output.

disk-indexer

Recursively scans mounted disks (or directories) looking for music files and gathers metadata from them and writes them to an index. It handles re-scans efficiently by only scanning for changes. It also generates the shuffle table across all files. lmdb is used for the database backend.

input-monitor

Monitors a raw input device for user input and sends commands off to the player. It is hard-coded to the Mele air-mouse but due to some issues with the USB stack device naming in Linux it can't always find the mouse device which is used for some of the controls.

disk-monitor

Checks for removable media to become available or be removed via raw kernel UEVENTs. New media is mounted read-only and passed to the disk indexer for a (re)scan. Removed media is marked as unavailable.

audio-cmd

A command line tool to send various commands to the player such as next or previous and so on.

mocp was having trouble with a collection of about 160K music files I have, and the TUI was pretty slow every time I started it. Indexing same said files takes about 20 minutes with playerz (I dunno roughly on par with mocp?) but restarting the player is basically instant.

I'm doing some web stuff for work at the moment so will probably look at some web control stuff, although for now the Mele air-mouse does most of what I need (skip to next song).

Even though i've done fuck all this year i'm getting a bit sick of work. Nothing interesting to work on and the company I work for has been trying to bilk me out of long service leave (13 weeks paid time off) - something I should have had 5 years ago. Really just want to retire but i'm a bit too young yet for that even if I have really nothing to work for anymore.

Going nowhere

Still going though.

At some point a few months ago I almost completely lost interest in coding - my new computer has just been collecting dust. This is the first time i've turned it on in a couple of months, and all I did at that time was upgrade the kernel. Same with this time actually but I also upgraded netbeans to 12.0 because I wasn't doing anything else.

Work is super-slow, partly due to COVID, partly due to timing with respect to breaking my leg, partly the project i'm on and a good dose of I don't give a fuck.

I've been spending a lot of time alone at home, again partly due to COVID but also plenty of I don't give a fuck. Even on a cold day if there's sun about sitting in the backyard with a book and some (nettle) tea is preferrable to drinking beer alone in a cold pub. I seem to feel the cold terribly these days. I'm getting out when I feel the need to, the feeling comes and goes.

I've spent a bit of time playing games, mostly No Man's Sky and a lot of Dreams. Once I got to >$U 4billion in NMS there doesn't seem much point since I avoid the multiplayer but I still go poke around every now and then to see what's cooking.

Dreams is a lot more interesting so i've spent a lot of time in that. I worked on a few starts of games, but I find desiging levels isn boring as fuck so I usually don't get past working out the base gameplay and a demo level. Dreams logic is very limited so it makes doing much with it a bit difficult, but just the challenge of those restrictions is entertaining enough, at least in measured doses. There's some really impressive stuff in there though if you can find it. Should be interesting to see what the PS5 can do with it, and if there's a new PSVR coming.

Till next time ...