termz

Every now and then I think about the sad state of affairs regarding terminal emulators for X11. It's been a bit of a thing for a while - it's how i ended up working at Ximian.

I stopped using gnome-terminal when i stopped working on it and went back to xterm. I never liked rxvt or their ilk and all of the 'desktop environment' terminal emulators are pretty naff for whatever reason.

xterm works and is reliable but with recent (being last 10 years) X Windows System servers the text rendering performance plummeted and even installing the only usable typefaces (misc-fixed 6x13, and 10x20, and sometimes xterm itself) became a manual job. Whilst performance isn't bad on this kaveri box I also use an uber-intel machine with a HD7970 where both emacs and xterm runs like an absolute pig whenever any GL applications are running, and it isn't even very fast otherwise (i'm talking whole desktop grinding to a halt as it redraws exposes at about 1 character column per SECOND). It's an "older" distribution so that may have something to do with it but there is no direct indication why it's so horrible (well apart from the AMD driver but i have no choice for that since it's used for OpenCL dev). I might upgrade it to slackware next year.

Ho hum.

Anyway I started poking last night at a basic xterm knockoff and got to the point of less sort of running inside it and now i'm thinking about ways I might be able to implement something a bit more complete. I'm working in Java and have a tiny bit of JNI to get the process going and handle some ioctl stuff (which seems somewhat easier now than it was in zvt, but portability is not on the agenda here).

zvt

When I wrote ZVT the primary goal was performance and to that end considerable effort was expended on making a terminal state machine which implemented zero-copy and zero-garbage algorithms. zero-copy is always a good thing but the zero-garbage was driven by the very slow malloc on Solaris at the time and my experience with Amiga memory management.

Another part of the puzzle was display and the main mechanism was inspired by some Amiga terminal emulators that used COPPER lists to re-render rows to the screen in arbitrary order without requiring them to be re-ordered in memory (memory bandwidth was a massive bottleneck when using pre 1985-spec hardware in 199x). I used a cyclic double-linked (exec) list of rows and to implement a scroll I just moved a row from the start to the end of the list which takes 8 pointer updates and a memset to clear it (and it also works for partial screen scrolls). By tracking the last row a given one was actually displayed at I could -at-any-point-later- attempt to create an optimal screen-update sequence including using blits for scrolling and minimising glyph redraws to only those that had changed. The algorithm for this was cheap and reliable if a little fiddly to get correct.

This last point is important as it allows the state machine to outpace the screen refresh rate which always becomes the largest bottleneck for terminal emulators in 'toolkit' environments. This is where it got all it's performance from.

new hardware, new approach

Thinking about the problem with current hardware my initial ideas are a little bit different.

I still quite like the linked list storage for the state machine and may go back to it but my current idea is instead to store a full cell-grid for the displayable area. I can still make full-screen scrolling just as cheap using a simple cyclic row trick (infact, even cheaper) but sub-region scrolling would require memory copies - but at the resolution of 4-bytes-per-glyph these are insanely cheap nowadays.

This is the most complex part of the emulator since it needs to implement all the control codes and whatnot - but for the most part thats just a mechanical process of implementing enough of them to have something functional.

I would also approach rendering from an entirely different angle. Rather than go smart i'm going wide and brute-forcing a simpler problem. At any given time - which can be throttled based on arbitrary metrics - I can take a snapshot of the current emulator screen and then asynchronously convert that to a screen display while the emulator continues to run on it's own thread.

For a basic CPU renderer it may still require some update optimisation but given it will just be trivial cell fonts to copy it probably wont be appreciably cheaper to scroll compared to just pasting new characters every time. And obviously this is utterly trivial code to implement.

The ultimate goal (and why the fixed-array grid backing is desirable) would be to use OpenCL or OpenGL (or more likely Vulkan if it ever gets here) to implement the rendering as a single pass operation which touches each output pixel only once. This would just take the raw cell-sized rectangle of the terminal state machine as it's only variable input and produce a fully rendered and styled framebuffer as the result. Basically just render the cells as a low-res nearest-neighbour texture lookup into a texture holding the glyphs. The former is a tiny tiny texture in GPU terms and rendering a single full-screen NN textured quad is absolutely nothing for any GPU. And compared to the gunk that is required to render a full-screen of arbitrary text through any gui toolkit ever it's many orders of magnitude less effort.

Ideally this would only ever exist at full-resolution in the on-screen framebuffer memory which would also make it extremely cheap memory wise.

But at least initially I would be going through JavaFX so it will instead have to have multiple copies and so on. The reason to use JavaFX is for all the auxiliary but absolutely necessary fluff like clipboard and dnd operations. I don't really like tabbed terminals (I mean I want to use windows as windows to multitask, not as a stack to task switch) but that is one way to ameliorate the memory use multiplication this would otherwise create.

So to begin with it would be extremely fat but that's just an implementation detail and not a limitation of the design.

worth bothering?

Still mulling that over. It's still a lot of work even if conceptually it's almost trivial.

Yep, I was bored.

I made a Workbench2.0 window theme for xfce tonight.

(yes i realise the depth and zoom buttons are swapped but i'm not re-taking these shots. Oh blast, the bottom-left pixel should be black too.).

Actually why I did this came about in a rather round-a-bout way.

I had to spend all day in Microsoft Windows today debugging some code and as white backgrounds strain my eyes too much I spent some time trying to customise netbeans and the system itself such that it was usable. Netbeans was a copy of the config on another machine plus a theme change and installation of the dejavu fonts. But after a bit of poking around with the 'classic windows' theme I found I could change more than I expected and set it up somewhat amiga-like, as far as that goes (wider borders, flat colours, etc). The theme editor is a pretty dreadful bit of work-experience-kid effort like most of the config windows in that shitstain of an OS shell.

So that got me thinking - it always bothered me that ALL the usable XFCE4 themes have "microsoft windows 95 blue" borders, so i poked around and found the theme I was using (Microcurve) and started randomly editing the XPMs until I ended up with this.

The zoom button is maximise, but the since the depth button doesn't function that way I mapped that to minimise - which will probably take a while to get used to since its no longer familiar and doesn't exactly match it's function. I had to create a new pin button which is definite programmer-art but fits the simple flat design well enough. I already had the close button in the correct spot but decided to drop the menu and shade buttons since I never use them anyway.

I did try to have the close and depth buttons right to the edge as they should be but they get cut-off when the window is maximised, messed up if not all buttons are included in the decoration, and it meant I couldn't animate depressing them properly. So I extended the side borders to the top. I made the bottom thicker too - I cant fucking stand trying to hit a stupid 1-pixel high button to resize the windows anyway (particularly when the mouse normally lags as focus changes turning it into a detestable mini-game every time) and it adds a pleasant weight to the windows. The bottom corners are wider as well which also affects the resize handles in a positive way.

Yeah it's still blue ... but at least it's a different blue, of an older and more refined heritage.

PS gimp 2.8 is a fucking pain in the arse to use.

Update: I decided to publish what I have on a Workbench2.0 theme home page. This contains updated screenshots.

Update: Looks like CDE?

Yeah, no.

unusually long arms?

Since getting skinny again i've been forced (against my will!) to at least look into finding some clothes that fit.

On the weekend I went to look at some locally made woolen stuff (expensive!) and the smallest size was still a bit baggy for the thin tshirts (but they are intentionally a rather loose cut). But the jumpers of the same size were just too short in the arms and the shop assistant kindly suggested my arms were unusually long! Hah, maybe, maybe not. I went with a medium jumper and some small tshirts and i'm probably all shopped out for a while after that effort so I wont find out if she was right till I try other clothes. God knows when i'll wear them.

In high school photos I was always at the side in the first standing row and with rows being ordered by height that put me squarely in short-arse territory. When I did work experience at the end of year 11 they kept joking how I was going to "grow up" to be a jocky (funny men!), yet somehow by the time I started uni 2 and a bit years later i'd reached 6'; so maybe i am an odd shape (or it's not me that's odd!).

I had resigned myself to being about 90kg for the foreseeable future at least and I even expected that to be a little optimistic. Losing nigh on 4 1/2 bags of spuds in 9 months seems ... excessive; that was just under 25% of my total mass. I seem to recall measuring myself rather inaccurately at over 1m around the waist at some point, ... and now it's easily under 800mm (those spuds had to fit somewhere I guess). An old mate I worked with before Ximian reckons i'm thinner than when I left to start at Ximian nearly 16 years ago. Yay I guess? FWIW I asked the dr if i was underweight last week and he said no way and even if he just based it on BMI == 20 that's good enough for me.

It certainly makes riding up hills a lot easier. I took the roadie out for a roll last weekend for a 65km round-trip to Port Noarlunga which has a few rises and on the day sported a stiff (and cold!) southerly to head into on the way up. Given its been so long since i've been that far or that hilly (such as it is) it was about the right distance from a cold-start - i was a bit tired when I got home but barely sore the next day (well apart from the arse, it will take a few more trips to get saddle hardened to that seat again). I've also been going to the beach any day it's hot enough and splashing around like a drowning cat-in-a-sack which makes for some solid exercise together with the 1 hour riding required. It's pretty dull though and the 1/2 hour ride home is enough to heat up again but I had to get out of the house.

So apart from the gout i'm probably on my way to being physically in better shape than ever. And on the gout, after a bit of a hiccup of misunderstanding that should be working towards becoming a mostly-non-issue too (albeit with a daily pill).

Pity about the head though, "utterly miserable" pretty much sums that up. But that summation has rarely been very wrong at any time for as far back as I care to remember.

OpenCL 2.1 + java = zcl 0.x?

I noticed Khronos released the OpenCL 2.1 spec recently so I spent this morning updating zcl to include all the functions.

Since I don't have a suitable implementation nothing is tested and there's probably some typos and so on. I found a few small bugs in the enum tables while I was there.

But what took most of the time was the property queries. Each OpenCL object type has one or more query functions but rather than implement them all I use a tagged query function which branches to the correct function entry point at the lowest level but shares all the rest of the code. But then I had to add some specialist variants, and specialisations for return types and overloaded parameters - it started to get unwieldy and a new query type on CLKernel meant it wasn't going to be enough anyway.

So I said fuck that for a joke and just redid the whole mechanism.

For the basic 5-parameter queries I still share most of the code but I now add any type-specific queries separately. To cope with the api and code bloat i distilled the java side interface down to only two entry points for each query:

    native <T> T getInfoAny(int type, int ctype, int param_name);
    native <T> T getInfoAnyV(int type, int ctype, int param_name);

The first is a scalar query and the second an array one. It just means it now has to box primitive return types for scalar queries which is unlikely to have any measurable performance impact but the Java helpers which wrap the above interfaces in type-friendly calls could always be replaced with native equivalents if it was an issue.

This let me merge some internal jni code and delete a lot of snot and I moved the re-usability to a different layer so that the more specific queries can share most of the code. For example this was the previous set of native interfaces on CLObject, and although this covered the kernel and program specific 6-argument queries like GetProgramBuildInfo() it was getting a bit messy.

    native long getInfoLong(int type, long subtarget, int param);
    native long[] getInfoLongA(int type, long subtarget, int param);
    native int getInfoInt(int type, long subtarget, int param);
    native byte[] getInfoByteA(int type, long subtarget, int param);
    native <T> T getInfoP(int type, long subtarget, int param, int ctype);
    native <T extends CLObject> T[] getInfoPA(int type, long subtarget, int param, int ctype);
    native long getInfoSizeT(int type, long subtarget, int param);
    native long[] getInfoSizeTA(int type, long subtarget, int param);

... It seemed like a good idea at the time.

The exposed interfaces remain the same (like getInfoString(param), getInfoInt(param), etc).

Given the complete lack of interest and because it needs some testing anyway I wont be releasing a zcl-0.6 just yet.

bloody peecee

I've been having a few issues with my machine lately so as i had nothing better to do yesterday I had a look into fixing it.

First issue I looked at was that I broke OpenGL when i upgraded to the latest catalyst driver. I tried various kernels, even a long-overdue update of slackware (which lead to a another few hour's diversion earlier in the month trying to un-fuckup firefox as much as i could), using the slackware kernel, etc. But finally I used strace and slackpkg to determine it was using a stale .so left over from AMD's previous abomination of an install script so once that was deleted all was well. SVM still wont work in any of my code - which is still baffling as it works from the AMD SDK samples. I should really post something on the AMD forums but i haven't used it for a while.

The other problem has been only half my RAM started showing up a few months ago; the bios shows 8G but linux only sees 4G. This is ok for what i normally use the machine for but netbeans brings it to it's knees if you have a couple of projects open. I thought it was down to a faulty DIMM and If i hadn't forgotten and needing to come home earlier yesterday I might have bought some. But I tried each individually and they seemed to work - passing the linux make test at least. I tried lower speeds, voltages, all sorts of things. It appears a fairly common problem but nobody had much of a solution.

I even upgraded the bios - which was another couple of hour diversion as that cleared the EFI boot record and I had to re-figure out how to re-install it. I used a USB-bootable tool-linux distribution called grml - very impressed - I just dd'd it onto the usb stick, and it even provides an EFI boot record so i could just use efibootmgr to re-add it. I know i've tried puppylinux multiple times before but it never worked so I might get a usb stick just to keep this handy.

The last thing I tried was dusting the sockets and then trying to jiggle the cpu in it's socket - I don't have any thermal paste so I couldn't get at it properly but i wiggled a bit without freeing the heatsink.

Now ... that appeared to work. I did some tests and so on and it seemed to be ok, ... I even wrote a post about it.

But then I had to tinker, so i was playing with the memory speeds - to see if "faster" ram really made any real difference - but I broke the whole machine so i had to power cycle it a few times till the BIOS reset everything, ... and yeah back to 4G of ram. Blast. I swore at it, shut it off, and went to bed early.

On a cold boot I did get 8G out of it but then the system crashed while I was using it, so something is amiss. It could still be the RAM I suppose but at this point i'm more inclined to believe it is the motherboard or PSU. I don't feel like investigating further for the moment and will just see how it goes (i just cold booted and it's 8G again *shrug*).

However, some good did come of all this.

When I was playing with kernels I went to the trouble to actually go through all the config options and fuck off everything I didn't need. It's down to 3.2M packed including my system filesystem with no module so i don't need an initrd, and it boots a bit faster (not that it was any slouch). I'm also trying the fully-preemptive kernel and i'm liking it so far, even under very heavy load the system remains interactive such that you barely notice at all. Hmm, I just noticed the sound mixer is missing, NM.

But the really good bit is now the CPU runs almost cold - i don't know if it's the BIOS upgrade, the kernel customisation, or some BIOS setting that changed (i already had it on 45W TDP because it got too hot), but the difference is marked. Previously any high-load task such as a 'make -j4' would cause the fan to kick in almost immediately and if i didn't also up the case fan speed (manually) fairly promptly the whole desk would take off. I've currently got a kernel build running for a few minutes triggering a load-average of over 8, with the case fan on it's slowest setting; and the CPU fan hasn't throttled up at all.

Maybe the machine is just running a lot slower now - but I can't really tell so what does it matter if it is.

Update: it got unstable so i took one dimm out. Seems ok so far. Maybe it's just the PSU? It seems like the most fragile component and it's almost certainly underpowered anyway (sigh, a poor decision that one). I dunno. I started looking at new cases & psus but if it remains stable albeit in a reduced capacity I'll be in no rush ...

time is an illusion

But so is reality so it doesn't make any difference.

I've got too much of it to fill with boring tedium either way.

OpenCL 2.0 + Java = zcl 0.5, or ~= 1.0 beta

I spent a wet morning doing some clean up and packaging of another zcl build and just finished updating the home page and uploading the source.

Although I just bumped the revision, this is getting pretty close to a 1.0 release. It's still got a few missing bits but it's mostly because the documentation is a bit broken beyond the README. It is only compatible with Java 8.

The home page has more details but the big points are that it now garbage collects everything (with explicit override), the lambda interfaces (trivial though they are), dynamically links to libOpenCL, fills out the extension framework and implements some extensions, and supports cross-platform building of native code.

I had to add a small code-generator to make the dynamic linking practical but it relies on the strict formatting of cl.h and does nothing fancy

Now i've got cross platform sorted out i'll probably do all my work to this interface rather than jogamp/jocl because it's just nicer to use and easier to work with. This might not mean any more frequent updates but at least it should get tested more. But apart from not being able to get SVM working at all on my machine (sdk demo works, cut and pasted bits from demo, or any other thing i write - crash) i've encountered very few bugs anyway.

I've probably covered enough of the new stuff in the blog previously so probably wont have much to add, but the curious are welcome to ask.

yay summer

Just got back from the beach. Should've had a camera - water was literally like glass. It was overcast so the horizon vanished in places which is always a nifty effect. Water was crystal clear. It is also still pretty cold though and once you go out past the sand-bar things get a bit nippy. Had the cold water all to myself though there were a good few people out and about but almost nobody went beyond the shoreline unless they were on some floating craft. The closest beach is a comfortable 30m ride and between jetties so it's quiet, with no posers or jet-skies (so far?) and the sand is really nice (best sand on any beach i've been on). Should be a nice night too if the stillness keeps up, it's absolutely dead-calm again (@6pm) which we've had a fair bit so far this spring.

Nice to be 'skinny' again too. You don't notice when you're putting it on but its like i just stopped carrying 2x10KG sacks of flour (and a carton of milk!) around with me everywhere, which is definitely noticeable now it's gone. Not that it ever stopped me getting my gear off at the beach it feels better losing the dugong. Its like winding the block back about 15 years, well apart from the mental and physical scars, aches and pains, and now all-grey hair. Although i'm still getting used to looking down at my skinny knees when i'm cycling. I got some new shorts a couple of weeks ago and they already feel loose - but i think they were a pretty generous "Size 32" to start with.

Of course I should be having a beer or nice limey g&t right about now but the gout is already niggling (probably from a couple of beers i had monday with a visitor, first time in weeks) - my toe was even a bit sore after the first dip. Sigh. More fucking tea I guess. For me it seems the main cause is just not drinking enough water (or actually not weeing enough) - but by enough that means at least 3L per day just to start with. Which gets to be a bit of a drag and if i'm doing physical things i'm not sure I can even drink enough to offset the sweating.

It makes going out a pretty abysmal experience because i just don't enjoy myself. Actually I haven't been enjoying myself much going out the last few years but a couple of beers at least made it passable for the occasional highlight. I can have a few but might suffer later so usually don't risk it. Not that i've had much opportunity anyway but i've been almost dreading when they do turn up - i'd rather just garden or find some way to pass time by myself lately. Maybe it'll pass but I don't care if it doesn't. Speaking of which someone just called and i turned him down, but i might change my mind in the next hour. I was all settled in after the trip to the beach and don't feel like dressing up to hit the city, if i can even find clothes that still fit. I never like going out on Saturday anyway as it's too obnoxious and busy and the crowd a bit too keen, and i'm a little underslept and too cranky to put up with that shit anymore.

I've been eating a lot of chillies at home - i'm getting pretty steadily through my stash of frozen habaneros from a few years ago so hopefully I don't run out before something else comes online. I might have to buy an advanced seedling as growing from seed here without a greenhouse takes a very long time due to the cold nights. I finally finished a bottle of Blairs "Possible Side Effects" sauce this week - i can't remember when i opened it but it's years past it's use-by date although I didn't use it for some time. It's really expensive here in AU so i kept it anyway and after i topped it up with some lime juice it tasted better. It's not the nicest flavour but it's ok, and it's hot. Now onto the "Ultimate Insanity" sauce which has been in the fridge too long ... But due to just not eating much anymore and the palatte destruction of lots of chillies often with limes or lemons ... eating out has become super-bland and quite unappealing.

Work has been quite taxing (in a good way, although i'm a little blah on the whole need to work right now) so i haven't been coding much outside of that. I'm not watching much TV. I watch a small amount of twitch/youtube stuff but not that often. I've been playing PS4 a bit more - although it's mostly DRIVECLUB and Res0Gun. Res0Gun is just a brilliant game and although I'm still pretty mediocre at it the more you play the better you get - it's very pinballish. I turn up my new amp enough to feel the explosions. Yum. The other regular is DC which I do some single-events, community challenges, or occasionally tour events. None of my friends have it so I just see what's there (one just got a ps4 but he's not into car games). I've started doing many-lap races to learn some tracks and cars and with sped-up time it makes them pretty interesting once you get into the swing of things. With the new PS communities I played a bit with the photomode and uploaded some for something to do. I got a couple of messages so I guess somebody saw them but it's a little inflexble at the moment - something like a web forum with each 'community' a single topic. I even tried streaming on twitch today which at least I confirmed my router didn't crap-out, but i'll have to be in the right mood to try that again. And check with a GNU box how it looks.

I finally finished the last book i was reading - it really dragged on, and/or i kept falling asleep on the last bit. I dunno some dreary battle against evil by characters i didn't give a shit about (with a weird and entirely unnecessary prologue which explained what happened afterwards). And then straight onto the next of the 12 part(?) epic it is part of. This could take a while ...

I guess it's time to decide on what to do this fine and warm evening.

I can grab my tea and go wander out in the garden in a pair of shorts until the grass gets too cool and then watch some shit on TV or play a game or just go to bed ...

Or get tarted up enough to go to a pub (not much tarting admittedly) and head to the currently trendy part of town to drink water and hang out with my leery mate and his wanker friends (actually only one is obnoxious, if he's even there) while they get drunk and perve on the chicks half their age. And then ride home feeling afterwards probably feeling miserable (it's just happened the last few times).

Yeah I already decided when he called. Time to go see what the bugs have been eating if it hasn't cooled down too much while i've been here. Still dead calm at least.