The brain dump

Last night I was pretty exhausted - the neighbours had some bricks delivered around 7am which woke me too early after what was a pretty lousy night of sleep. I got up with the idea of turning off the watering system (starts at 7am) after it had done the pot plants and kind of forgot about that and ended up hacking for nearly 15 hours straight.

Ok so it was probably a bit obsessively intensive, but i'm pretty surprised I still get such a level of buzz out of hacking code after 25 years of doing it for fun. In terms of effort and intensity, it could have been any other xmas holiday from any of those previous years. On the other hand i'm writing more and better code than ever before because the platform is fuller than ever before. The only downside is that I can't keep up the intensity for as many consecutive weeks as I used to and end up with aches and pains from sitting still for so long.

But as I intended to take a bit of a break for a few days I thought i'd jot down some feature ideas, todo's and wishlists in my engineering notebook (a paper one) while the ideas were still hot.

As I wrote things down my memory started to vanish very rapidly - as if waking from a dream. I got the main feature points down fairly completely but I was pretty stumped at the todos. Although only minutes before I had a mind full of all those little refactoring, renaming, and tweaking ideas for improving the codebase, by the time I went to write them down I drew a near blank. I've experienced something similar before when I left the Evolution project; as I wrote all my knowledge down all the anxieties from keeping track of the 'known bugs' and 'future plans' in my head just melted away. But it wasn't quite so abrupt, although both instances were very cathartic. To be honest i'm still surprised I can keep so much in my head for weeks at a time - when I often forget where I put a screwdriver only a minute prior.

So it was a very good idea to write it down and let me get a good nights sleep. And although my brain is now bereft of most of the details I managed to write most of them down, and i'm sure the others are in the source as FIXMEs or will come back when I get into it again - if they were important.

Scrollbars!

So apparently one of the side-effects of the touch-interface 'revolution' is the abolishment of scrollbars.

Well fuck that for a poke up the arse with a glowing poker.

Even before the latest firefox came along with a serious bug on my *ainol tablet (I love it, even the the logo includes a diagrammatic anus) which makes it almost unusable - flicking the screen normally-but-not-always makes it flick in the wrong direction - this no-scrollbar idea makes the device itself very clumsy for reading any web pages more than about 3 screens high. The android browser is just the same. And given some of the pages I read are dozens of pages long, i'm faced with a usability nightmare.

Just bring the friggan scrollbars back ...

It's probably related to another annoying "web 3" feature - infinite scrolling screens that load on demand. They might solve a technical problem and have a bit of 'wow' factor, but they are not a usability improvement. And they kill some pretty important features we've come to expect like searching within a page. But I suppose we have to get used to this sort of dumbing down of the internet - such a big chunk it is all about advertising wrapped in entertainment; not learning, education, or information.

While i'm ranting about firefox mobile i find it pretty much a pain to work with even without that killer bug - there are too many small buttons too close together, the tab system is really annoying, as is the click-to-title-bar-to-go-to-another-screen thing, with the popup keyboard always getting in the way. It has a habit of zooming far too often when I go to move around - a mis-fingered 'scroll' turning into a 'press'. If we already have a two-finger zoom, why not bloody stick to it? I just don't see the point of overloading gestures to the point of confusion. Well here's two fingers for the designers: Y, and while i'm at it: !). Mouse/keyboard users have other mechanisms for this.

The whole point of "touch" is it is supposed to be natural and "intuitive", but if you have to concentrate hard every time you try to press on something it has lost any edge it had.

I don't really know the answer for the tiny-button-problems as it would need some serious thought but for all the serious thought the UI designers have put into the current offerings they still have a distance to go. The only practical thing I can think of from the top of my head is an icon/menu bar which POPS onto the screen (doesn't slide, doesn't fade, doesn't push the content over to one side, doesn't fucking animate) and has nice big buttons for the most useful functions. You know, like the microsoft 'charms' thing, although it sounds like they overloaded 'gestures' to the point of absurdity (i'd really like to know what that dickhead in the 'clean up the party' ad is supposed to be doing when he draws a circle and a line on the photo near the start). Gestures suck because they're completely non-discoverable and require a lot of practice to get consistent with.

But pressing inside a page should be reserved for links or focus. Not for zooming-sometimes-in-sometimes-out. It's just too easy to do it by mistake.

Inflexion points

Hmm, and on a completely unrelated note, I think i've hit one of those inflexion points where I suddenly change a food forever. It's getting close to the point where my coffee goes black, strong, sugarless. I use a Baletti stove-top coffee maker which makes great coffee easily and cheaply and with the right coffee, amount of milk and temperature it's pretty near to a decent 'bought one', but it's a bit hit and miss. Yesterday I ran out of milk and although I usually can't stomach the black coffee I managed ok. About the time I started using one of these machines my coffee went sugarless - I can't stand even a small amount of sugar now. Although a little sugar and milk is still needed in tea, but I don't drink that as often.

It reminds me that one day after having had cornflakes every morning for breakfast for years I suddenly decided it was too sweet and decided I vaguely remembered vita-brits being pretty nice. Can't stand cornflakes now, although I could probably go some sugarless vita-brits once in a while. Although these days breakfast is usually left-overs, coffee, toast, or often just reading my morning round of blogs and organising my on-screen workbench.

Hmm, my nice neighbours are getting some work done in their yard next to this room, I should go mow the lawn or something before I get rooted to the chair for another day.

DuskZ GMUD, & Source

I just got confirmation of the original source license so I decided to get it all out of the way tonight. I kept thinking it was Friday today, but since that's tomorrow I wanted to clean up then then try harder not to get sucked into hacking on it for the rest of my long-weekend.

I've kept most of the code GPL 2 or later but decided to exercise the 'or later' on the client side and used GPLv3 for the new files and for the whole package.

The source has been checked in to the DuskZ project. Well the DuskZ keeps with my naming convention theme, and it will soon diverge enough to need another name.

As I started without a version control system there is no detailed changelogs, but lets just say the changes have been 'considerable'.

Test Run

The code is pre-alpha but it should run the current test game from Zabin (with some bugs), although only the initially imported version will work. I already have pending changes to break it.

First check out all 3 modules from the DuskZ googlecode and build it. I use NetBeans 7.3 beta but presumably ant will work.

Then checkout the old test game from DuskRPG github. The compatible game files are in "DuskFiles/Dusk2.7.3"

Start the server:

$ cd {path-to-DuskRPG}/DuskFiles/Dusk7.7.3
$ java -jar {path-to-DuskServer}/dist/DuskServer.jar

Start the client:

$ cd {path-toDuskRPG}/DuskFiles/Dusk7.7.3/www
$ java -jar {path-to-DuskZ}dist/DuskZ.jar

A recent Oracle version of java is currently required as the JavaFX stuff hasn't been fully freed yet for the openjdk. Fingers crossed it will be a matter of months.

Connect to localhost, port 7474 and enter a username/password to create a new account. If all's well you'll get dumped into the game, or not - no promises at this point.

Update: I forgot to add www to the client starting directory.

Java Scripting?

Well to "relax" I did some reading up on JavaScript from Java ... and to be honest it's "bloody awesome". I've done plugin languages and extensions before in C and Java so i'm no stranger to the binding issue, but this makes it almost "zero work" if you already have well defined objects - which should be a goal anyway.

Anyway, it's tidy and 'included', which is nice - no wonder Oracle is trying to modularise Java, it's getting big.

As one might know from reading this blog i'm not a huge fan of writing applications in JavaScript ... but it does have it's uses and this one seems like a perfect fit.

One of the last major components of the original Dusk I haven't already gutted is the scripting language - but it's now on the chopping block too. One reason is the code is pretty opaque and difficult to maintain, another is that the 'scripts' are the whole game, the finally it is a reasonable enough implementation (compact pre-compiled code) and still works. But the thought of extending it to support planned new features is a bit unpleasant.

I think breaking the whole existing part-game is worth it in the long run. And efficiency isn't much worth if it isn't usable - and computers are somewhat faster than they were 15 years ago.

It's also what I would call a "marketing" or "management" feature: JavaScript has a huge following and a ton of resources, whereas trying to learn the arcane and obtuse syntax of Dusk's script engine is not going to be high on anyone's list of things to do before they die.

I played with it a bit more and realised you can basically do everything you could do in Java from any JavaScript script, including file and network i/o ... so it looks like i'll have to delve into SecurityManager. I've always avoided it as the documentation is obtuse and bereft of any useful examples - and we all know how complexity is such a winner when security is at issue. I'll also have to work out using it in a multi-threaded context, if that's possible.

Well I suppose life wasn't meant to be easy.

Time passes.

Ahh on threading it looks like everything has to be from the same doPrivileged() call. I chose to create a threadpool and submit jobs to that, and use Future to get the results if required. Seems to be a reasonable compromise. I've implemented a watchdog timer too, but because the ScriptEngine has no way of interrupting or stopping execution (should be something on the Context?) I'm forced to use the dangerous "Thread.stop()". Well what can you do eh? I guess it's more than a "few lines of code", but on the other hand I have a nice easy to use - and probably safe - script mechanism.

This deserves more of a write-up but I was supposed to have an early minute today so it will have to wait.

Update: I had more of a look at scripting including the SecurityManager aspects and documenting the current script system and how it's used.

Unfortunately the sandboxing turns out to be much more involved than I wanted. I can fairly easily do a basic sandboxing such as preventing disk and network i/o outside of prescribed locations (no thanks to the documentation; it's unreadable shit), but beyond that things get quite tricky. One reason is the rhino engine itself needs to do things like reflection in order to execute any script.

I experimented with some nasty stuff like scanning the stack-trace for 'legal' accesses, but I dunno, it seems complex enough that I wouldn't trust it. So for the moment my initial solution will stick with the simple high-level solution of restricting file and network access and put some trust that the players that reach a high enough level to create scrips that will behave themselves - as one did in the past.

I've still yet to come up with the solution i want for how the scripts should be structured. I've thought about everything from a simple global-context scripts to having the entire mob persistent state exist within local variables on objects within the JavaScript engine and all scripts merged into a single pre-compiled programme. Something in-between will probably be the result - as usual - but I need to implement at least a couple of ideas and see how they work in practice. One desires as much flexibility and simplicity as possible, but you can't maximise both at the same time. I initially thought of simply having global-state inline code, but that doesn't allow the use of 'return'. So now i'm thinking more of defining a function within the code that matches the filename. This also allows pre-compilation. And certain objects such as things or items require a number of related scripts, so it probably makes sense to wrap them in an object and just call methods on that.

The existing script language is pretty weird. Prefix notation for operators and effectively everything is just a function (which parses itself - the syntax is not parse-able without semantic information). Might take a while to grok and convert the scripts - a compiler which converts the scripts probably isn't worth the effort. One detail I hadn't previously been aware of is the number of general player commands that are implemented as scripts - and the complexity thereof. Most of the other scripts are fairly simple or at least straightforward. It's not an absolute necessity to convert all of them or even any of them - but having an existing game working is a good test-bed.

There are also some scripts which don't really do much and simply define global "game-theme" behaviour - yet need to run very often. I will look into whether these can be instead be implemented by Java code and supplied at run-time via a plugin/extension mechanism. This still allows for customised behaviour without recompiling the whole application, together with better space and time performance.

Tall tiles

Had a little hacking session this morning and implemented "tall tiles" into the client. The server doesn't need to know about them although I had to write a converter for Tiled format for the test map I generated.

Tall tiles allow for a great deal more flexibility in design for oblique or isometric projected maps without complicating any of the game logic. Well, ... it seems a pretty obvious feature for anything but a top-down game.

If one looks closely one can see the partially obscured player sprite. Even things like the tree in-front of the path/building/or other trees is completely impossible with the original Dusk engine - without creating separate tiles for every possible background.

As I need to store multiple images and so on I decided to store them in a jar file, with a manifest describing some meta-data about the files therein. In tiled I create a separate set of tiles for each tile size, and the converter creates one packed image for each tile set and stores it in the jar. I intend to use the same mechanism (and jar) for player and mob sprites too, allowing a lot more flexibility there as well, and possible even use tiled to manage those too (allowing for example symbolic names for sprites rather than using numbers which are hard to keep synchronised).

Rendering is straight forward from top to bottom and I render the map then the player. I put all the player labels and highlights above everything else because well it just seemed like the right thing to do.

Server

I decided to change the way tile actions are defined. Rather than have a separate script for each tile by number, I instead have a property file (why write my own parser, again) which defines which script is executed for each tile that needs one. By default tiles cannot be walked or seen through so values only need to be set for open tiles. This makes it a lot easier to manage, and I will probably hard-code some constants like "true" as well. I'm using Tiled to set the properties on the tiles which define which scripts are run, and the converter dumps these to a file the server understands.

Actually last night I also started working on a binary protocol for the server. I did the map update and a couple of others. But then I kind of got hung up on trying to encode some of the more detailed messages and just wasn't feeling the love. Probably a more re-usable approach using simpler "property lists" might be more loveable.

Source?

Some licensing queries I have should be sorted soon, and then i'll put the code somewhere once I've fixed up the files.

Anyway i'm pretty pleased with my progress this week (considering i've never worked on any game code ever before), and this little 'win' is probably a good opportunity to take a break, before I get overly keen and try to tackle multiple maps or layers and sink my last weekend of leave down the toilet!

Tooling and stuff

Did a bit more refactoring and bug fixing this morning in the server. Removed a bunch of Vector types, and improved the code a bit. Found and fixed a concurrency bug.

Now i'm starting to get the codebase under control - and learning enough about it - i'm starting to get more wild ideas and think about other issues.

One is tooling - the bundled editor isn't too good, and there is already a java-based editor for tile maps called Tiled (although it's moved to C++ now - ugh - the original still works ok and doesn't need a pile of dependencies installed). So I had a bit of a play with jaxb and jxc and created an exporter for the current map format to Tiled's native format.

It works ok although bogs down (on this laptop) at this sort of scale - the map is pretty big.

Writing an exporter isn't much more work, although since i've gone this far i'm starting to think of other ideas to include at same time ...

Taller tiles

Taller tile images which sit on the same base square allow for things like trees and forests and posts you can walk behind. i.e. pretty useful for a designer.

I think this should be pretty easy to implement as a first cut, the rendering code only needs to align based on the height of the tiles, and the image storage needs fiddling with. A slight further complication is to avoid pop-in, but the server can send more of the map than is shown.

Taller sprites

This again is fairly simple and solving the problems for the tiles should help solving it for the sprites.

Multiple maps

I think that huge map is pretty unwieldy, 'rooms' are just defined as teleports to other closed sections of the map, and obviously that doesn't scale particularly well. Obvious solution is to have a separate map just big enough for each 'scene', and add the map id as a move/teleport argument. I think this loosely connected graph of separate regions has a more 'muddish' feel to it too.

I think that most of the game logic doesn't really need to change, as most entities will be tied to a given map so it doesn't need to turn into a 3d problem.

Layers

Another idea is to have multiple layers of rendering as a mostly cosmetic feature. I initially thought this would be pretty easy but to do it well will require some extra meta-data.

To keep the problem in two dimensions the entities will either have to be locked to one visual layer which might be a bit limiting, or some extra meta-data provides that information. e.g. a non-visible plane which indicates which layer objects are rendered on. Currently there can be behaviour attached to a type of tile that you're standing on and although this layer-indicator system would also work with that it might be useful to have a separate plane which defines the behaviour - although that is more design work.

But it could certainly be interesting, and allow for some interesting effects like parallax.

Binary protocol

Currently the protocol is a mix of binary (char) and strings. Actually it's a bit of a mis-mash of mis-match - each end is writing to a DataOutputStream, and each end is reading from a StreamReader. Although it works it isn't really supposed to, and having a DataInputStream on the other end would allow binary interactions.

Initially the layer stuff sounded good but the more I thought about it I wasn't so sure. I played with some taller tree tiles in Tiled and I thought it looked pretty nice. For an angled-down view just using a taller image gets you much of what you might want layers for without the extra overhead.

de-awt-ed

``I'll just fix up a few little things'' turned into ``Another full-day of hacking'' on my fork of Dusk.

I finally managed to de-awt everything. Well there are a couple of missing features (text viewer/editor), but the last major window to convert is in:

To make this a bit nicer I think I need to modify the protocol so the server tells me more about the items such as item counts as it does with the inventory command. And there are other possibilities such as graphics, and "detailed description". It should probably also split up equipment and general inventory in some way or another ...

Well at least the layout didn't take any time today even if it looks it.

What took all the time was rewriting another big chunk of the client code map tracking and abstracting away the user interface from the client logic.

For the map code I again abstracted it into a separate structure and provided accessors and indices of various things, although now i'm reflecting on it I don't think some of the stuff I coded is even used since the algorithms that used them are gone. For example there was code to find entities by name - but internally i'm just using unique ids now so having an index by name it isn't required anymore. One major change I did was to just keep track of entities (players, mobs etc) using absolute coordinates rather than trying to keep them forever up-to-date with the current viewing window. Apart from being a great deal easier to use it is obviously more efficient. There were a lot of "x - offset1 + offset2" type things everywhere anything was accessed, and instead I just made the map to be relative to the top-left corner rather than the centre, and track the player location separately.

I also changed entity based actions to use id numbers instead of indexed names. Basically if there is more than one object of a given type on the screen then some messy code gives each one a unique number. The problem is the client and server do this independently, and I couldn't be stuffed finding out how it works - one of the benefits of the map abstraction was killing a pile of ugly code, and it was buggy anyway. Since the server and client already uses unique id's for everything I just added support in the server for doing targeted actions based on id instead. The client can always do the lookup if the command line needs to support it, and I guess I can now clean up that mess in the server too.

And finally the entire user interface and system interaction is hidden behind a re-usable interface, for the obvious sub-java/sub-gnu operating system option.

So far the de-awting seems to have made is more stable too. Although I also fixed some thread issues in the client as the game runs on a separate thread to JavaFX.

Shopfront

Given the day started raining I spent most of it hacking on Dusk again. Coded up a shop-front.

Yeah the styling is naff, but it isn't meant to be better. The main window blurs when it's shown.

I gotta say the layout was a lot messier than I would have thought. VBox and HBox only allow consistent alignment for all items in the stack - which seems pretty pointless to me. I'd have thought a common use-case for a VBox would be a stack of things with different alignments. And I couldn't get GridPane or BorderPane to work very well either, although perhaps I could now with the following knowledge.

Basically to perform arbitrary alignment and expansion of cells within a vbox or hbox, one has to embed a hbox or vbox for each containee, set it to fill, and then set the containee alingment on that. It's just rather clumsy. As for the table view I had to hard-code everything to make it behave at all.

And learning a new toolkit is always a pain - probably took 5 hours to do this one simple window even though i've been working with layout based toolkits since gtk 1. And even then it seems to have some nasty javafx bug in which the table view in the Sell tab doesn't render properly. I really must try this stuff on another box to see if it is just something wrong locally like the shitty Intel IGP or it being 32-bit.