About Me

Michael Zucchi

 B.E. (Comp. Sys. Eng.)

  also known as Zed
  to his mates & enemies!

notzed at gmail >
fosstodon.org/@notzed >

Tags

android (44)
beagle (63)
biographical (104)
blogz (9)
business (1)
code (77)
compilerz (1)
cooking (31)
dez (7)
dusk (31)
esp32 (4)
extensionz (1)
ffts (3)
forth (3)
free software (4)
games (32)
gloat (2)
globalisation (1)
gnu (4)
graphics (16)
gsoc (4)
hacking (459)
haiku (2)
horticulture (10)
house (23)
hsa (6)
humour (7)
imagez (28)
java (231)
java ee (3)
javafx (49)
jjmpeg (81)
junk (3)
kobo (15)
libeze (7)
linux (5)
mediaz (27)
ml (15)
nativez (10)
opencl (120)
os (17)
panamaz (5)
parallella (97)
pdfz (8)
philosophy (26)
picfx (2)
players (1)
playerz (2)
politics (7)
ps3 (12)
puppybits (17)
rants (137)
readerz (8)
rez (1)
socles (36)
termz (3)
videoz (6)
vulkan (3)
wanki (3)
workshop (3)
zcl (4)
zedzone (26)
Friday, 04 December 2015, 08:04

Clothes are expensive.

I overdid this week a bit and had to get out of the house today. It was also hot for the first time in a couple of weeks so I had to get out into it. I had plans to buy a few things but they partially fell through and all I did was end up pissed off.

I did find some (pretty crappy) sort-of-actual-shorts (as opposed to no-they're-actually-longs, you know, they actually end above the fucking knee) because the typical length shorts of the day are too hot, too uncomfortable, and rub the top of my legs in ways that cause soreness. Now I don't have such a fat-arse at least it's possible to have them fit the leg when they fit the waist. One of the pair I got is basically white - they look like 60s sports day shorts, I just need the white singlet and dunlops and the no 1 haircut to go with an old family portrait, but they're just for for around the house. Either the sizes are a little "generous" or i'm down to a small now and that size definitely wasn't tight.

Actually I haven't really stopped losing weight despite eating pretty normally and doing enough basic exercises to gain muscle. This week I measured a minimum "under 69.0" for the first time in decades (but that was just the extreme of the trough). I somewhat expected it to level off by now.

I went looking for a battery for my bike headlight - the charger for the chinese junk I bought broke but the light is fine and i've been charging the mystery battery with a PSU I made in uni - but i got really pissed off at the dickhead at jaycar who couldn't do more than point at the shelves I'd already looked through so i gave up and walked out. If i just want to get their cheap junk i can get it myself, i only went to the shop to see if they had knew anything I didn't and to support a local business. dx.com it is I guess. Last time I tried the battery bar in the city but the shop assistant was utterly clueless and the 'tech guy' was a condescending fuckwit and it seems more like a "junk bar" than a battery bar anyway.

There's two local shops I will NEVER return to ever again. I made the effort to go to a local shop physically and with a literal pocket full of cash, and they weren't interested in even attempting to close any sort of sale. Fortunately at the expensive wool-clothes shop I went to after the battery bar I had a long chat to the bored shop assistant and she tried to upsell everything 'male' they sold (which isn't much but i hadn't intended to get a jumper when i went in).

I bought some dried fruit from The Market and tried to find a silicone gasket for a (stove-top) coffee maker for one of my brothers - but nobody sells them. At least one guy said it was because they never wear out - which is the whole bloody point! I got one for my machine at Gaganis Bros but they're still in the bad books for their bicycle parking arrangements they didn't have many back then anyway.

At this point i was ready to fall apart or have an accident (i've just been in a foul mood all week and this made it worse) so I said fuck it and had a beer at the Exeter. I quite like the pub itself and the staff are decent but this isn't the first time i've looked around and realised I was surrounded by other middle-aged men so I decided if I have any more I should try somewhere else. I decided at this point to go buy some bike shit after seeing it last time - I grabbed some new shy shorts (30 was the right fit, so yeah I guess I am small now) - the ones i've been using lately are 36 and they were too tight last year but now they feel like nappies with their built-in chamois and the draw-string. Also too long and too hot. Ugh I spent like 170 on another pair of cargoish things at the same time back then which I had been using but they feel like clown pants now, they were even bigger (and both have faded poorly, why are they all black anyway??). And a couple of jersey's which are possibly a bit on the tight side but we'll see - most they had were black and that's too hot so I got a white one and a red one to match my roadie (hey why not). One specific jersey I used most recently still has serviceable cloth but got stretched too much to be passable so at some point I will see if i can take it in somehow. Those expensive bike faded shorts are too big for that but I might also try to take in some other stuff I've never even worn if i can get my sewing machine to cooperate (stuff i was too fat to fit, and now falls off).

So that bike shit (incl, shoes I bought last week, new pedals and cleats), the wool tshirts and jumper, the "POW" PE shorts, jerseys and shorts, a new tyre, well over a grand. It is what it is.

I ended the afternoon with a few beers at the Austral - fuckit, if i get gout so be it (but so far so good?). I had 4 total for the day but I think 2 was probably what I should've stopped as after those two I just wasn't' really feeling anything good and all it left me with was a headache and a churned guts which still persists a couple of hours later.

I contemplated life and it came up very very short. As ever.

I even thought about tattoos. Why? To belong to "something" I guess? I didn't think about them too much.

Tagged biographical, rants.
Wednesday, 02 December 2015, 10:19

dumb lockless 'queue' idea

I was playing with linux signals the need arose for a lockless queue, although given the mess of poll vs signal and so on i might just use a pipe for signalling as i've done before. But such a queue is useful for that case as well.

Unless you're dealing with exotic hardware or performance I imagine that a basic bounded lockless queue is fairly easy to implement using a two-index compare and swap but I was thinking about the unbounded case. Not that unbounded is necessarily a good idea in many cases for other reasons.

After a couple of mistakes I think I arrived at a very simple solution which should work. It's actually half a stack, and half a something else I don't even know the name of, but it can be made to appear as a queue cheaply.

enqueue

This is implemented as a push onto the head of a single-linked list. This can be implemented very simply using a compare and swap against the single pointer of the stack head. The next pointer of the node can be freely updated until it is owned by the queue once the compare and swap succeeds.

void enqueue(queue q, msg m) {
  queue old, new;

  do {
    old = q;
    new.head = m;
    m.next = old.head;
  } while (!compare_and_swap(q, old, new));
}
dequeue-all-reverse-order

This is no dequeue operation, instead there is a dequeue-all which removes every element currently on the queue but returns them in reverse order. It just clears the stack head and returns it's old value. This can be implemented trivially using an atomic exchange operation with NULL.

queue dequeue_all(queue q) {
  q = exchange(q, null);
  return q ? q.head : null;
}
dequeue

This has to be performed as a separate step but it is performed local to the reader. It should be efficient enough on typical processors. In the (very) unlikely event that order isn't important then it isn't even necessary.

void dequeue_foreach_rec(msg stack, func cb) {
   if (q) {
     dequeue_foreach_rec(stack.next, cb);
     cb.apply(stack);
   }
}

void dequeue_foreach(queue a, func cb) {
  dequeue_foreach_rec(dequeue_all(q), cb);
}

It supports multiple writers. Whilst it still functions "correctly" for multi-reader queues it probably isn't terribly useful for that case as it's first-come-first-gets-all with no attempt or possibility of balanced delivery.

Tagged hacking.
Tuesday, 01 December 2015, 05:56

more termz

I tried a few more variants on the OpenCL rendering code but none are all that fast - the overheads kill it and whilst it does free the CPU up a little bit it isn't much. Probably not worth more time unless I look at OpenGL instead.

I added resizing. It meant I had to add some locking to the snapshot routine but it only needs to lock around the resize operation so it adds almost no overhead to normal operation (merely detecting a resize has occurred). I'm not yet sure what's supposed to happen with saved cursors and alternate screens when a resize occurs, probably just clip to size. Unfortunately there seems to be no way to set the WM_NORMAL_HINTS on the javafx stage, so there's no way to make it size to cells properly.

I did a bunch of benchmarking and profiling. One thing I tried was another test to compare to xterm - now at full-screen. Running "find . -name '*.c' | xargs cat" from the root of the linux 3.19.8 source tree. After a couple of runs this is about 25 seconds on termz, and 16 minutes in xterm. Well. Yeah it's a silly test but all those ls -l's during the day add up.

Looking at the memory profiler it doesn't really use too much heap during operation, just a few MB and most of that is the image and javfx. I mean nor should it, there isn't data structures to maintain. But having multiple compilers in ram (jvm, OpenCL), their generated output, and all the added overhead of the runtime support needed for those really adds up so it's very very fat in practice. I guess if multiple terms ran on the same jvm it would be ok.

enough yet?

So at this point i'm not really sure what i'll do with it. I'll probably poke at the edges when i'm bored and eventually when I get around to it I will dump what I have to my software site as the result of a weekend-and-a-bit-hack.

After that I'm not sure. It's actually quite functional and robust already (well, compared to the effort in) and wouldn't take much more work to turn it into a usable terminal for me - add some scrollback (pretty simple), mouse selection stuff (not that hard), and sort out some of the keyboard details (reading obtuse documents and testing). So maybe that will happen.

If I got that far adding the "10x20" typeface would probably be on the cards. Fixed-size outline fonts would be possible by just pre-rendering them but to me they just aren't terminal fonts.

Anything further such as a re-usable term component (which might actually be of use to the world) would require substantially more work on the i18n side of things and I don't feel like learning enough to do that properly.

Make

I did a bit poking at the java makefile stuff and it's to the point where i'm using it for out-of-ide builds of termz and zcl and will look into using it on other projects. That's the best way to find bugs/what works and what doesn't and previous attempts never got that far. For all it's gnumakefile obtuseness it's really rather compact at under 200 lines excluding comments and I didn't put any effort into making it particularly small. And that includes targets for javadocs, source jars, and binary builds.

Tagged hacking, java, javafx, opencl, termz.
Sunday, 29 November 2015, 12:32

GNU make and java

Today I had another go at looking at using meta-rules with gnu make to create a 'automatic makefile' system for building java applications. Again I just "had a look" and now the day is all gone.

This is the current input for termz.

java_PROGRAMS=termz

termz_VERSION=0.0
termz_DISTADD=Makefile java.make \
 jni/Makefile \
 jni/termz-jni.c

# compiling
termz_SOURCES_DIRS=src
termz_RESOURCES_DIRS=src
termz_LIBADD=../zcl/dist/zcl.jar

# packaging (runtime)
termz_RT_MAIN=au.notzed.termz.TermZ
termz_RT_LIBADD=../zcl/dist/zcl.jar

# native targets
termz_PLATFORMS=gnu-amd64
# native libs, internal or external.  path:libname
termz_RT_JNIADD=../zcl/jni/bin:zcl jni/bin:termz

# Manually hook in the jni build
jni/bin/gnu-amd64/libtermz.so: build/termz_built
 make -C jni TARGET=gnu-amd64

clean::
 make -C jni clean

include java.make

make (jar)

This builds the class files in build/termz, then the jni libraries via the manual hook. In another staging area (build/termz_dist) it merges the classes and the resource files (stripping the leading paths properly). It uses javapackager to create an executable jar file from this staged tree which includes references to the RT_LIBADD libs (moved to lib/). Finally it copies the RT_LIBADD jar files into bin/lib/ and the native libraries into bin/lib/{platform} so that the jar can be executed directly - only java.library.path must be set.

Not shown but an alternative target type is to use java_JARS instead of java_PROGRAMS. In this case jar is used to package up the class and resource files and in this case no staging is needed.

make tar

This tars up all the sources, resources, and DISTADD into a tar file. No staging is required and all files are taken 'in-place'. The tar file extracts to "termz-0.0/".

make clean

Blows away build and bin and (indirectly) jni/bin. All 'noise' is stored in these directories.

I'm using metaprogramming so that each base makefile can define multiple targets of different types. This is all using straight gnu make - there is no preprocessing or other tools required.

java.make does all the "magic" of course. While it's fairly straightforward ... it can also be a little obtuse and hairy at times. But the biggest difficulty is just deciding what to implement and the conventions to use for each of them. Even the variable names themselves.

There were a couple of messier problems that needed solving although i'd solved the former one last time I looked at this some time ago.

class-relative names

For makefile operation the filenames need to be specified absolutely or relative to the Makefile itself. But for other operations such as building source jars one needs the class-relative name. The easiest approach is just to hard-code this to "src/" but I decided I wanted to add more flexibility than this allows.

The way I solved this problem was to have a separate variable which defines the possible roots of any sources or resources. Depending on what sort of representation I need I can then match and manipulate on these roots to form the various outputs. The only names specified by the user are the filenames themselves.

For example when forming a jar file in-place I need to be able to convert a resource name such as "src/au/notzed/terms/fonts/misc-fixed-semicondensed-6x13.png" into the sequence for calling jar as "-C" "src" au/notzed/terms/fonts/..." so that it appears in the correct location in the jar file.

I use this macro:

# Call with $1=root list, $2=file list
define JAR_deroot=
    $$(foreach root,$1,\
 $$(patsubst $$(root)/%,-C $$(root) %,\
  $$(filter $$(root)/%,$2))) \
   $$(filter-out $$(addsuffix /%,$1),$2)
endef

I'll list the relevant bits of the template which lead up to this being used.

# Default if not set
$1_RESOURCES_ROOTS ?= $$($1_RESOURCES_DIRS)

# Searches for any files which aren't .java
$1_RESOURCES_SCAN := $$(if $$($1_RESOURCES_DIRS),$$(shell find $$($1_RESOURCES_DIRS) \
  -type d -name CVS -o -name '.*' -prune \
   -o -type f -a \! -name '*java' -a \! -name '*~' -a \! -name '.*' -print))

# Merge with any supplied explicitly
$1_RES:=$$($1_RESOURCES_SCAN) $$($1_RESOURCES)

# Build the jar
$$($1_JAR): $(stage)/$1_built
        ...
        jar cf ... \
          $(call JAR_deroot,$$($1_RESOURCES_ROOTS),$$($1_RES)) \
        ...

At this point I don't care about portability with the use of things like find. Perhaps I will look into guilified make in the future (i'm a firm believer in using make as the portability layer as the current maintainer is).

And some example output. The last 2 lines are the result of this JAR_deroot macro.

jar cf bin/termz-0.0.jar   -C build/termz . \
 -C src au/notzed/termz/fonts/misc-fixed-6x13-iso-8859-1.png \
 -C src au/notzed/termz/cl/render-terminal.cl 

A lot of this stuff is there to make using it easier. For example if you just want to find all the non-java files in a given directory root and that contains the package names already you can just specify name_RESOURCES_DIRS and that's it. But you could also list each file individually (there are good reasons to do this for "real" projects), put the resources in other locations or scatter them about and it all "just works".

How?

Just looking at the JAR_deroot macro ... what is it doing? It's more or less doing the following pseudo-java, but in an implicit/functional/macro sort of way and using make's functions. It's not my favourite type of programming it has to be said, so i'm sure experts may scoff.

  StringBuilder sb = new StringBuilder();
  // convert to relative paths
  for (String root: resourcerootslist) {
    for (String path: resourcelist) {
      if (path.startsWith(root+"/")) {
        String relative = path.replace("^" + root + "/", "");

        sb.append("-C").append(root)
          .append(relative);
      }
    }
  }

  // include any with no specified roots
resource:
  for (String path: resourcelist) {
    for (String root: resourcerootslist) {
      if (path.startsWith(root+"/")) {
         continue resource;
      }
    }
    // path is already relative to pwd
    sb.append(path);
  }

  return sb.toString();

This is obviously just a literal translation for illustrative purposes. Although one might notice the brevity of the make solution despite the apparent verbosity of each part.

Phew.

going native

Native libraries posed a similar but slightly different problem. I still need to know the physical file location but I also need to know the architecture - so I can properly form a multi-architecture runtime tree. I ummed and aahd over where to create some basic system for storing native libraries inside jar files and resolve them at runtime but i decided that it just isn't a good idea for a lot of significant reasons so instead I will always store the libraries on disk and let loadLibrary() resolve the names via java.library.path. As it is still convenient to support multi-architecture installs or at least distribution and testing I decided on a simple naming scheme that places the architecture under lib/ and places any architecture specific files there. This then only requires a simple java.library.path setup and prevents name clashes.

Ok, so the problem is then how to define both the architecture set and the library set in a way that make can synthesise all the file-names, extensions (dll, vs so), relative paths, manifest entries in a practical yet relatively flexible manner?

Lookup tables of course ... and some really messy and hard to read macro use. Oh well can't have everything.

Libraries are specified by a pair of values, the location of the directory containing the architecture name(s), and the base name of the library - in terms of System.loadLibrary(). These are encoded in the strings by joining them with a colon so that they can be specified in a single variable. The final piece is the list of platforms supported by the build, and each library must be present for all platforms - which is probably an unnecessary and inconvenient restriction in hindsight.

This is the bit of code which converts the list of libraries + platform names into platform-specific names in the correct locations. I'm not going to bother to explain this one in detail. It's pretty simple just yuck to read.

#
# lookup tables for platform native extensions
#
# - is remapped to _, so usage is:
#
#  $(_$(subst -,_,$(platform))_prefix) = library prefix
#  $(_$(subst -,_,$(platform))_suffix) = library suffix
#
_gnu_amd64_prefix=lib
_gnu_amd64_suffix=.so
_gnu_amd32_prefix=lib
_gnu_amd32_suffix=.so
_mingw32_amd64_prefix=
_mingw32_amd64_suffix=.dll
_mingw32_amd32_prefix=
_mingw32_amd32_suffix=.dll

# Actual jni libraries for dependencies
$1_JAR_JNI=$$(foreach p,$$($1_PLATFORMS), \
  $$(foreach l,$$($1_RT_JNIADD), \
   $$(firstword $$(subst :, ,$$l))/$$(p)/$$(_$$(subst -,_,$$p)_prefix)$$(lastword $$(subst :, ,$$l))$$(_$$(subst -,_,$$p)_suffix)))

Thinking about it now as i'm typing it in a simpler solution is possibly in order even if might means slightly more typing in the calling Makefile. But such is the way of the metamake neophyte and why it takes so long to get anywhere. This is already the 2nd approach I tried, you can get lost in this stuff all too easily. I was thinking I would need some of this extra information to automatically invoke the jni makefile as required but I probably don't or can synthesise it from path-names if they are restricted in a similar fashion and I can just get away with listing the physical library paths themselves.

Simple but restricted and messy to implement:

termz_PLATFORMS=gnu-amd64
termz_RT_JNIADD=../zcl/jni/bin:zcl jni/bin:termz

vs more typing, more flexibility, more consistency with other file paths, and a simpler implementation:

termz_RT_JNIADD=../zcl/jni/bin/gnu-amd64/libzcl.so \
  ../zcl/jni/bin/mingw32-amd64/zcl.dll \
  jni/bin/gnu-amd64/libtermz.so

Ahh, what's better? Does it matter? But nothing matters. Nothing matters.

Ok the second one is objectively better here isn't it?

After another look I came up with this to extract the platform directory name component:

$(lastword $(subst /, ,$(dir $(path))))

make'n it work

One other fairly large drawback of programming make this way is the abysmal error reporting. If you're lucky you get a reference to the line which expands the macro. So it's a lot of hit and miss debugging but that's something i've been doing since my commodore days as a kid and how I usually work if i can get away with it (i.e. building must be fast, i.e. why I find it so important in the first place).

And if you think all of that looks pretty shit try looking at any of the dumb tools created in the java world in the last 20 years. Jesus.

damn work

It seems i hadn't had enough of the terminal after the post last night - i was up till 3am poking at it - basically another whole full-time day. I created a custom terminfo/termcap - basically just started with xterm and deleted shit I didn't think I cared for (like anything mouse or old bandwidth savers that looked like too much effort). But I looked up each obtuse entry and did some basic testing to make sure each function I left in worked as well as I could tell. Despite the documentation again there are a lot of details missing and I had to repeatedly cross checked with xterm behaviour. Things like the way limiting the scroll region works. And I just had a lot of bugs anyway from shoddy maths I needed to fix. I then went to look at some changes and netbeans had been too fat so I went down the rabbit-hole of writing meta makfiles so I could do some small tests in emacs ... and never got that far.

I really needed a proper break from work-like-activities this weekend too, and a lot more sleep. At least I did water the garden, mow the lawn, wash my undies, and run the dishwasher.

Tagged hacking, java.
Saturday, 28 November 2015, 14:52

opencl termz

I was just going to "try something out" while I waited for the washing to finish ...

... so after a long and full day of hacking ...

The screenshot is from an OpenCL renderer. Each work item processes one output pixel and adds any attributes on the fly, somewhat similar in effect to how hardcoded hardware might have done it. I implemented a 'fancy' underline that leaves a space around descenders. The font is a 16x16 glyph texture of iso-8859-1 characters. I haven't implemented colour but there's room for 16.

On this kaveri machine with only one DIMM (== miserable memory bandwidth) the OpenCL routine renders this buffer in about 35-40uS. This doesn't sound too bad but it takes 3uS to "upload" the cell table input, and 60uS to "download" the raster output (and this is an indexed-mode 8-bit rather than RGBA which is ~2x slower again), but somehow by the time it's all gone through OpenCL that's grown to 300-500uS from first enqueue to final dequeue. Then add on writing to JavaFX (which converts it to BGRA) and it ends up ~1200uS.

I'm using some synchronous transfers and just using buffer read/write so there could be some improvements but the vast majority of the overheads are due to the toolkit.

So I guess that's "a bit crap" but it would still be "fast enough". For comparison a basic java renderer that only implements inverse is about 1.5x slower overall.

But for whatever reason the app still uses ~8% cpu even when not doing anything; and that definitely isn't ok at all. I couldn't identify the cause. Another toolkit looks like a necessity if it ever went beyond play-thing-toy.

I got bored doing the escape codes around "^[ [ ? Ps p" so it's broken aplenty beyond the bits I simply implemented incorrectly. But it's only a couple days' poking and just 1K3LOC. While there is ample documentation on the codes some of the important detail is lacking and since i'm not looking at any other implementation (not even zvt) i have to try/test/compare bits to xterm and/or remember the fiddly bits from 15 years ago (like the way the cursor wrapping is handled). I also have most of the slave process setup sorted beyond just the pty i/o - session leaders, controlling terminals, signal masks and signal actions, the environment. It might not be correct but I think all the scaffolding is now in place (albeit only for Linux).

FWIW a test i've been using is "time find ~/src" to measure elapsed time on my system - after a couple of runs to load everything into the buffer cache this is a consistent test with a lot of spew. If I run it in an xterm of the same size this takes ~25s to execute and grinds big parts of the desktop to a near halt while it's active. It really is abysmal behaviour given the modern hardware it's on (however "underpowered" it's supposed to be; and it's considerably worse on a much much faster machine). The same test in 'termz' takes about 4.5s and you'd barely know it was running. Adding a scrollback buffer would increase this (well probably, and not by much) however this goes through a fairly complete UTF-8 code-path otherwise.

The renderer has no effect on the runtime as it is polled on another thread (in this instance via the animation pulse on javafx). I don't use locks but rely on 'eventual consistency'. Some details must be taken atomically as part of the snapshot and these are handled appropriately.

Right now I feel like i've had my fill for now with this. I'm kinda interested, but i'm not sure if i'm interested enough to finish it sufficiently to use it - like pretty much all my hacking hacked up hacks. Time will be the teller.

Tagged hacking, java, javafx, opencl, termz.
Friday, 27 November 2015, 09:05

termz

Every now and then I think about the sad state of affairs regarding terminal emulators for X11. It's been a bit of a thing for a while - it's how i ended up working at Ximian.

I stopped using gnome-terminal when i stopped working on it and went back to xterm. I never liked rxvt or their ilk and all of the 'desktop environment' terminal emulators are pretty naff for whatever reason.

xterm works and is reliable but with recent (being last 10 years) X Windows System servers the text rendering performance plummeted and even installing the only usable typefaces (misc-fixed 6x13, and 10x20, and sometimes xterm itself) became a manual job. Whilst performance isn't bad on this kaveri box I also use an uber-intel machine with a HD7970 where both emacs and xterm runs like an absolute pig whenever any GL applications are running, and it isn't even very fast otherwise (i'm talking whole desktop grinding to a halt as it redraws exposes at about 1 character column per SECOND). It's an "older" distribution so that may have something to do with it but there is no direct indication why it's so horrible (well apart from the AMD driver but i have no choice for that since it's used for OpenCL dev). I might upgrade it to slackware next year.

Ho hum.

Anyway I started poking last night at a basic xterm knockoff and got to the point of less sort of running inside it and now i'm thinking about ways I might be able to implement something a bit more complete. I'm working in Java and have a tiny bit of JNI to get the process going and handle some ioctl stuff (which seems somewhat easier now than it was in zvt, but portability is not on the agenda here).

TermZ? Glyphs are greymaps extracted directly from the PCF font.

zvt

When I wrote ZVT the primary goal was performance and to that end considerable effort was expended on making a terminal state machine which implemented zero-copy and zero-garbage algorithms. zero-copy is always a good thing but the zero-garbage was driven by the very slow malloc on Solaris at the time and my experience with Amiga memory management.

Another part of the puzzle was display and the main mechanism was inspired by some Amiga terminal emulators that used COPPER lists to re-render rows to the screen in arbitrary order without requiring them to be re-ordered in memory (memory bandwidth was a massive bottleneck when using pre 1985-spec hardware in 199x). I used a cyclic double-linked (exec) list of rows and to implement a scroll I just moved a row from the start to the end of the list which takes 8 pointer updates and a memset to clear it (and it also works for partial screen scrolls). By tracking the last row a given one was actually displayed at I could -at-any-point-later- attempt to create an optimal screen-update sequence including using blits for scrolling and minimising glyph redraws to only those that had changed. The algorithm for this was cheap and reliable if a little fiddly to get correct.

This last point is important as it allows the state machine to outpace the screen refresh rate which always becomes the largest bottleneck for terminal emulators in 'toolkit' environments. This is where it got all it's performance from.

new hardware, new approach

Thinking about the problem with current hardware my initial ideas are a little bit different.

I still quite like the linked list storage for the state machine and may go back to it but my current idea is instead to store a full cell-grid for the displayable area. I can still make full-screen scrolling just as cheap using a simple cyclic row trick (infact, even cheaper) but sub-region scrolling would require memory copies - but at the resolution of 4-bytes-per-glyph these are insanely cheap nowadays.

This is the most complex part of the emulator since it needs to implement all the control codes and whatnot - but for the most part thats just a mechanical process of implementing enough of them to have something functional.

I would also approach rendering from an entirely different angle. Rather than go smart i'm going wide and brute-forcing a simpler problem. At any given time - which can be throttled based on arbitrary metrics - I can take a snapshot of the current emulator screen and then asynchronously convert that to a screen display while the emulator continues to run on it's own thread.

For a basic CPU renderer it may still require some update optimisation but given it will just be trivial cell fonts to copy it probably wont be appreciably cheaper to scroll compared to just pasting new characters every time. And obviously this is utterly trivial code to implement.

The ultimate goal (and why the fixed-array grid backing is desirable) would be to use OpenCL or OpenGL (or more likely Vulkan if it ever gets here) to implement the rendering as a single pass operation which touches each output pixel only once. This would just take the raw cell-sized rectangle of the terminal state machine as it's only variable input and produce a fully rendered and styled framebuffer as the result. Basically just render the cells as a low-res nearest-neighbour texture lookup into a texture holding the glyphs. The former is a tiny tiny texture in GPU terms and rendering a single full-screen NN textured quad is absolutely nothing for any GPU. And compared to the gunk that is required to render a full-screen of arbitrary text through any gui toolkit ever it's many orders of magnitude less effort.

Ideally this would only ever exist at full-resolution in the on-screen framebuffer memory which would also make it extremely cheap memory wise.

But at least initially I would be going through JavaFX so it will instead have to have multiple copies and so on. The reason to use JavaFX is for all the auxiliary but absolutely necessary fluff like clipboard and dnd operations. I don't really like tabbed terminals (I mean I want to use windows as windows to multitask, not as a stack to task switch) but that is one way to ameliorate the memory use multiplication this would otherwise create.

So to begin with it would be extremely fat but that's just an implementation detail and not a limitation of the design.

worth bothering?

Still mulling that over. It's still a lot of work even if conceptually it's almost trivial.

Tagged hacking, java, javafx, opencl, termz.
Monday, 23 November 2015, 14:32

Yep, I was bored.

I made a Workbench2.0 window theme for xfce tonight.

Focused

(yes i realise the depth and zoom buttons are swapped but i'm not re-taking these shots. Oh blast, the bottom-left pixel should be black too.).

Actually why I did this came about in a rather round-a-bout way.

I had to spend all day in Microsoft Windows today debugging some code and as white backgrounds strain my eyes too much I spent some time trying to customise netbeans and the system itself such that it was usable. Netbeans was a copy of the config on another machine plus a theme change and installation of the dejavu fonts. But after a bit of poking around with the 'classic windows' theme I found I could change more than I expected and set it up somewhat amiga-like, as far as that goes (wider borders, flat colours, etc). The theme editor is a pretty dreadful bit of work-experience-kid effort like most of the config windows in that shitstain of an OS shell.

So that got me thinking - it always bothered me that ALL the usable XFCE4 themes have "microsoft windows 95 blue" borders, so i poked around and found the theme I was using (Microcurve) and started randomly editing the XPMs until I ended up with this.

Unfocused

The zoom button is maximise, but the since the depth button doesn't function that way I mapped that to minimise - which will probably take a while to get used to since its no longer familiar and doesn't exactly match it's function. I had to create a new pin button which is definite programmer-art but fits the simple flat design well enough. I already had the close button in the correct spot but decided to drop the menu and shade buttons since I never use them anyway.

Pinned

I did try to have the close and depth buttons right to the edge as they should be but they get cut-off when the window is maximised, messed up if not all buttons are included in the decoration, and it meant I couldn't animate depressing them properly. So I extended the side borders to the top. I made the bottom thicker too - I cant fucking stand trying to hit a stupid 1-pixel high button to resize the windows anyway (particularly when the mouse normally lags as focus changes turning it into a detestable mini-game every time) and it adds a pleasant weight to the windows. The bottom corners are wider as well which also affects the resize handles in a positive way.

Yeah it's still blue ... but at least it's a different blue, of an older and more refined heritage.

PS gimp 2.8 is a fucking pain in the arse to use.

Update: I decided to publish what I have on a Workbench2.0 theme home page. This contains updated screenshots.

Update: Looks like CDE?

Yeah, no.

Sunday, 22 November 2015, 12:21

unusually long arms?

Since getting skinny again i've been forced (against my will!) to at least look into finding some clothes that fit.

On the weekend I went to look at some locally made woolen stuff (expensive!) and the smallest size was still a bit baggy for the thin tshirts (but they are intentionally a rather loose cut). But the jumpers of the same size were just too short in the arms and the shop assistant kindly suggested my arms were unusually long! Hah, maybe, maybe not. I went with a medium jumper and some small tshirts and i'm probably all shopped out for a while after that effort so I wont find out if she was right till I try other clothes. God knows when i'll wear them.

In high school photos I was always at the side in the first standing row and with rows being ordered by height that put me squarely in short-arse territory. When I did work experience at the end of year 11 they kept joking how I was going to "grow up" to be a jocky (funny men!), yet somehow by the time I started uni 2 and a bit years later i'd reached 6'; so maybe i am an odd shape (or it's not me that's odd!).

I had resigned myself to being about 90kg for the foreseeable future at least and I even expected that to be a little optimistic. Losing nigh on 4 1/2 bags of spuds in 9 months seems ... excessive; that was just under 25% of my total mass. I seem to recall measuring myself rather inaccurately at over 1m around the waist at some point, ... and now it's easily under 800mm (those spuds had to fit somewhere I guess). An old mate I worked with before Ximian reckons i'm thinner than when I left to start at Ximian nearly 16 years ago. Yay I guess? FWIW I asked the dr if i was underweight last week and he said no way and even if he just based it on BMI == 20 that's good enough for me.

It certainly makes riding up hills a lot easier. I took the roadie out for a roll last weekend for a 65km round-trip to Port Noarlunga which has a few rises and on the day sported a stiff (and cold!) southerly to head into on the way up. Given its been so long since i've been that far or that hilly (such as it is) it was about the right distance from a cold-start - i was a bit tired when I got home but barely sore the next day (well apart from the arse, it will take a few more trips to get saddle hardened to that seat again). I've also been going to the beach any day it's hot enough and splashing around like a drowning cat-in-a-sack which makes for some solid exercise together with the 1 hour riding required. It's pretty dull though and the 1/2 hour ride home is enough to heat up again but I had to get out of the house.

So apart from the gout i'm probably on my way to being physically in better shape than ever. And on the gout, after a bit of a hiccup of misunderstanding that should be working towards becoming a mostly-non-issue too (albeit with a daily pill).

Pity about the head though, "utterly miserable" pretty much sums that up. But that summation has rarely been very wrong at any time for as far back as I care to remember.

Tagged biographical.
Newer Posts | Older Posts
Copyright (C) 2019 Michael Zucchi, All Rights Reserved. Powered by gcc & me!