Sore throat, sore fingers, sore head, sore heart.

I had a party for a friend visiting from Chicago last week. Drank way too much of course. Not bad for a school night, although I planned ahead and took the day off. Then I caught a bad cold or a mild flu off a house-mate, and got stuck with that over the whole bloody weekend.

Over that same weekend I worked on an introduction to Cell B.E. programming. Quite a lot. A series of worked examples starting from a plain C Mandelbrot generator, through various phases of SPUification and SIMDisation up to a final high performance(?) implementation. And all the trials and tribulations, discoveries and tricks along the way. I'm looking to publish a chapter a week or so to a new free software Cell hacking site that will hopefully come together soon. I wrote it using texiweb, using literate programming. Eight postable chapters plus introduction and appendices so far, and there are still plenty of ideas to explore.

And to top it off, yesterday I was cooking some meat on the BBQ, and I wanted to check the flame setting -- which tends to involve looking under the front. I didn't notice the folding chair right where my head was heading to, so I got a rather heavy bump on my forehead that seems to hurt more now than it did at the time.

After the party and flu I finally went to work today but I think I could've done with another day off. Apart from not being fully past the virus, it feels as though I'm suffering withdrawal from all the hyper-active hacking I've been doing on the couch over the last few days. Empty heart, irritable disposition, and feelings of exhaustion and despair.

Layout via Optimisation

Since I mentioned linear programming in the last entry I've managed to wrap my head around it enough to come up with a possible solution for my layout issues.

Example layout problem:

If I just consider the horizontal coordinates, the following known values and relationships describe this layout:

B centre = A centre

B width = 40

C width = B width / 2

C left = B right + 5

D width = 100

D right = B right

Want to know:

Left and right of every box.

Now a recursive layout algorithm has trouble with this. It can't determine the offset of B without knowing the size of A, but the size of A depends on the offset of B. You need to do at least a couple of size and layout passes, and even then I'm having trouble getting it working (and I don't have the perserverance to keep trying).

It seems at first that solving the simultaneous equations would work. But in general this isn't useful because there is nothing to stop negative widths being created -- which isn't terribly useful. And determining enough additional terms to constrain the solution (e.g. that D left = A left) requires calculation itself.

Changing this to an optimisation problem allows it to be defined without the need for it to be fully specified. And constraints keep the values pointing the right way.

After a bit of fiddling about, I think I've come up with a basic systematic procedure which should allow the determination of the results using the simplex algorithm. It may not be the most optimal setup, but it appears to be relatively robust so far.

1. Trivially define the objective function in terms of the sum of every left and right X coordinate.
2. Define width relationships - the known knowns, or the known unknowns.
3. Define container constraints - every child is fully inside its parent. Unless the parent size is defined explicitly, in which case do not add any implicit relationship.
4. Define layout constraints.

Applied to the problem above, first, trivially define the object function in terms of the left and right bounds of every box:

p = ar + al + bl + br + cl + cr + dl + dr

This is a minimisation problem in p.

Next define the widths:

br - bl = 40

dr - dl = 100

ar - al >= 0

cr - cl >= 0 (since we know this is B width / 2, we could just use 20 too)

Define the container relationship -- everything inside A must be inside A:

br - ar <= 0 cr - ar <= 0 dr - ar <= 0 al - cl <= 0 al - bl <= 0 al - dl <= 0 Everything is relative to A, so set A's left edge to 0, so it doesn't float up to meet the highest other boundary. al = 0 Now define the layout relationships. B centre = A centre bl + (br - bl)/2 = (ar - al) / 2 or bl/2 + br/2 - ar/2 + al/2 = 0 C width = B width / 2 cr - cl = (br - bl)/2 or br/2 - bl/2 - cr + cl = 0 C left = B right + 5 cl = br + 5 or cl - br = 5 D right = B right dr = br dr - br = 0 And running the simplex method over this system generates everything I need directly: Optimal Solution: p = 650; ar = 160, al = 0, bl = 60, br = 100, cl = 105, cr = 125, dl = 0, dr = 100 (calculated using http://www.zweigmedia.com/RealWorld/simplex.html) Nice. And just as a test, lets shrink D to be smaller than B, say 30.

Optimal Solution: p = 440; ar = 90, al = 0, bl = 25, br = 65, cl = 70, cr = 90, dl = 35, dr = 65

Still works.

It should be able to calculate both dimensions and deeper structures simultaneously. This will allow relationships across dimensions, e.g. A width = B height * factor + offset.

One thing the recursive layout routine handled ok was affine transforms (after a lot of mucking about mind you). I guess this should handle them too if I get the coordinate space correct. Relative position calculations need to be in parent space, but the layout calculations need to be in global space.

Now I just have to work out how to calculate those parameters from expression trees, and find out if a solver will run fast enough. Actually I'm pretty confident of the performance question, even a trivial implementation I wrote for work did thousands of rows and columns in a few hundred ms.

Z**D

Well I've been hacking on, of all things, a 2D structured diagram drawing application.

It came out of a lack of satisfaction with pretty well any GUI application I've tried. They look good on paper but when you try to do things with them you find all sorts of nasty limitations. One of the biggest problems they have is creating lots of objects of the same kind with different settings (think, uml classes). On the other hand, there are purely text or language based formats, which are good for editing lots of items quickly, and you can control layout very well, but they have steep learning curves, and making small tweaks to attain visual perfection is very time consuming. I haven't had the chance to use many CLI applications -- perhaps GNUPlot is the closest. They provide graphical output but no chance for graphical input. Like GNUPlot, many are focused on scientific users too.

So I want to experiment with a combined CLI, GUI, and ``active source'' user interface. I guess it might end up being a bit ambitious, but it'll give me something to keep me off the streets for a while.

Some of the ideas so far:

• CLI, GUI, and programming language are all fully first-class interactive data input options.
• CLI is an old-school CAD-like interface, where data can be input numerically.
• A ``live'' text editor where edits are reflected in the display at or near real-time.
• A typical pick'n'place GUI as well, at least for creating objects and adjusting layout.
• Item references in CLI or Editor or GUI are highlighted in the alternative input views.
• Declarative language allows flexibility, yet still allows direct GUI relationships.
• Language more human oriented than parser oriented.
• Custom graphical objects defined in declarative language, including object-like features.
• Custom graphical objects also defined in Java classes.
• Flexible layout mechanisms based on anchor points and vector arithmetic. Somehow available from GUI too.
• Heirarchical objects and grouping.
• Cascading styles using same syntax as language.

I'm more interested in the HCI factors than producing a real product, but I guess I'll see how it goes. And how long I keep interested.

I've done some work on the language parser, and the language itself is an interesting problem, and there's plenty of room to experiment. Do I go for minimalistic syntax, and put more of the semantics into the execution of the AST, or do I wrap up more of the language in reserved words? Do I use special symbols (e.g. ~ for a curve, "foo" for a label), or stick to English keywords (curve x ..., label "blah" ...)? GUI-modifications-to-source operations --- essentially graphical refactoring tools --- will probably be quite some work too.

And I have some layout code that mostly works. Although I've already hit some issues with containers which must automatically size to their contents. Because of the arithmetic and relationship based layout system there is the potential for complex relationships that are difficult, expensive, or impossible to calculate using recursive layout procedures and delayed arithmetic. Maybe linear programming will come to the rescue, if I can get my head around it - should be able to just calculate all the layout at once. But ... It will need some symbolic expression refactoring, which doesn't sound too fun.

Things afoot

I keep writing (long) posts and then decide against posting them. Social anxiety or something. How strange for an internet jot pad.

Lately I've started to get a bit busier and more serious with my hacking, so once I get something to a non-embarassing state, I might actually move forward with putting bits out there. Though, the reaching of a state of non-embarassment might mean a long wait. And i'm a little recitent about baring all to the world like that -- people might point and laugh, or even worse, start using it, leading to the descending spiral of the un-fun of on-going maintenance.

To that end I've been looking into hosting -- perhaps a bit prematurely, but it needs to be considered at some point. I had a quick look at google code, but it's license limitations rub me up the wrong way (one long-term project is server based, so AGPL is almost certainly the way i'm headed which rules it out anyway) -- it is also full of lots of rubbish, ugly, and doesn't appear to be maintained by humans. I never liked Sourceforge, it's got a sort of crappy host-of-last-resort feel to it. A mate mentioned github, but that seems a bit too light on features - mailing lists and issue trackers and so on, and I have no overt love for git, or proprietary services either.

Savannah will probably be the one -- it ticks all the boxes on features, and philosophy, and although svn has it's issues, I know how to use it. It even has some quite nice themes -- why on earth they use that god-awful super-ugly default is beyond me, but if you log on you can select some nice ones which turn it into a completely different modern-feeling site (yes I realise this thing has a shit theme too -- but this is just a single crappy `blog roll'). Their information is a little out of date on Java, to the point they suggest not using it, which I find a little odd -- but so it is in general with the FSF and GNU sites, someone needs to refresh their information. As far as I can tell (which is not far) it's all nice and happily free now. If it doesn't end up being suitable, I'll keep looking.

Apart from a couple of Java projects, a few people (well, me included) on the psubuntu list are slowly discussing setting up some sort of site for free-software Cell hacking. Since my interests have moved on a bit, and right now i'm quite busy with `other shit', I'm not sure I want to commit to such an endeavour, but I have some material which could probably help get the ball rolling.

The project i'm on at work will most probably be winding up around the end of the FY -- so I may be forced to have a LOT more spare time in a short few months. Actually i'm quite looking forward to the possibility, as the current project-winding-up phase isn't terribly exciting, and I've had more than a few gut-fulls of .NET and Windows XP to boot. Something might turn up by then, but if it doesn't, I wouldn't mind a bit of `time to myself' to re-discover some of the `knack' that went missing of late.

Right now, I just wish there were more hours in the day, and I was capable of being more alert for more of the ones that are here.

Summer

Hmm, nearly been 3 months since last I poked a few bytes onto this blog. Sure has been a hot summer around here - nearly 2 straight weeks over 40 - topped 46 one day. After that, even 35 feels pretty comfortable. Fortunately no fires around here, but that might be more luck than anything - we've had no rain to speak of since December - which is reasonably uncommon even for this, the 'driest state in the driest continent', as the beer ad said. I had a good few weeks off over Christmas - and spent most of the time playing ps games or outside wrecking my back yard - only just starting to get back into hacking.

Work has been pretty dull - ongoing maintenance, documentation, bug fixing and testing - you know, all the great stuff you put off till you can't put it off any longer. Unfortunately, the bell tolls for me. At least i'm doing the doco using Lyx -- that's come along a bit since I last looked at it. Pretty comfortable environment for pumping out documentation, much better than any crappy word processor -- faster, easier, and better quality output. Although their 'WYSIWYM' mantra is bullshit - none of the character formatting is semantic based (c.f. texinfo for instance), and defining your own styles is overly complex.

On the home hacking front, after playing around with C for a little, i've been poking a bit around Java again. I like the simplicity, things just make sense. It's still a fresh of breath air after all that c-hash rubbish.

I thought i'd look at re-implementing the CMS i'd been poking around with in C for the last few months -- but not really going anywhere -- in Java. I was going to do all UML stuff just to give it a go, but the tools around all suck, and emacs and a bit of pseudo-code solved my problems more easily. Oh well, maybe later. The Berkeley DB Java Edition is really nice - maps very nicely to Java, so removes all the boiler-plate and marshalling I needed to do in C. Also simplifies the index setup by automating some of the fiddly details. So it was pretty trivial to re-implement the database part in Java. Although it certainly helped that i'd done it before and knew the schema I wanted. I also worked out a way to simplify the schema significantly, whilst keeping all of the major features of the design, from 5 tables to 3, and fewer tables to manually maintain. I looked through and old document I wrote about it previously and it didn't read particularly well, and is probably before I got the previous design working, so I might write the new one up one day.

Now i'm working on the texinfo parser bit. This needed doing from scratch since the C version used flex and I didn't want to copy it - it was needlessly complex and fragile anyway. And it turns out a hand-rolled parser and lexical analyser is really quite simple to write and should be easier to maintain and extend. I guess I may go back and convert it to C at some point as well, which should be trivial (a C version would be handy for me for various reasons). I've got most of the basic parser framework done, although I still need to implement output translation, and meta-data capture. Oh well, no rush.

Hmm, I woke up from a stupid nightmare 19 hours ago, how come I'm not sleepier?

Oh the economic side-show has been interesting so far, I wonder when we're getting to the main event? Idiot pollies are still going around here either making out everything's hunky-dory, or illogically blaming the other side for any problems we're facing or going to face. Japan's massive drop in export income might just be the action point that drives the fact home that we're helplessly at the mercy of the rest of the world's use of our expensive dirt for our own prosperity, and local labour laws or arguing about how much money is being splashed around will have little to do with any long-term outcome.

Swings and rounabouts.

Ok, back to a bit of hacking. There's something about spending all day chained to a windows xp desktop and a visual studio session that simply sucks the life out of you. Leaving little time or energy to pursue other hobbies or interests. But with the lengthening days and a lack of TV to watch, I've found time to look back at Java again.

The last time I did any Java was 1998-1999, so it's been a long time between drinks, so to speak. Although I have tried to like eclipse, I never really did get to like it. And the last time I went looking for plugins all the interesting looking ones (for what I was looking at) were out of date, or had nothing but dead links in them - so that put me off even bothering. So this time I tried netbeans - and by chance 6.5 had just been released.

Well, I was mostly impressed actually. Swing actually isn't that bad. And it's a breath of fresh air to have some decent documentation for a change after dredging through the crap for WPF. Or even to look at the source-code! Wow! Almost like the 'good old' GNOME days where I built everything from glib up locally so had a much better time of debugging. At first the GUI editor seemed as crappy as the WPF one - defaulting to using hard-coded layouts. But then I couldn't find any table or v/hbox/stackpanel's and thought the whole system was really stuffed. But then I discovered that the layout container can automatically snap to theme-specified offsets, and can align columns and rows, and actually aligns baselines rather than bounding boxes. Wow. I don't know if it'd be easy to use from code, but with the netbeans editor it is quite nice. Far better than gtk+'s (and WPF's) fixed-sized crap and having to worry about whether it's an inner or outer margin, it just does that automagically, and even scales for different themes. Nice. Anyway, after struggling with vbox/hbox/table (ugh), and then grid/stackpanel (sigh, even worse) for years, I instantly fell in love (I think it's a GroupLayout, or maybe it's a netbeans-specific AbsoluteLayout).

Oh, and it all runs quite snappy too. Not that it isn't a bit of a pig (but machines are so much bigger now - so it isn't really an issue), but UI apps seem just fine - infact netbeans is a lot faster than visual studio, in every way. A lot faster. I do wish you could set bitmap fonts though - I'd love to just use 'fixed' as my font.

I thought i'd miss 'delegates' - as a C coder I littered my code with function pointers all the time, I really couldn't get enough of them, and c-hash delegates are much the same (although I don't use them the same way). But using anonymous classes actually looks neater for some reason - it's basically what c-hash is doing 'under the bonnet', and often the delgate callbacks do so little they don't warrant their own function anyway. Properties are sort of nice - but they are only syntactic sugar, they don't actually do anything at all differently to what a getter and setter does - they provide no additional facilities automatically like property monitoring. c# events are quite nice though, again they are really only syntactic sugar, but they do save a lot of boilerplate code.

Apart from that, it struck me just how much like Java that c# really is. In true ms fashion all they did was copy and complicate. The simplicity of the Java language is nice.

Some of the libaries are also nice too. Simple. But others - you just wonder what people are thinking. The JSR for binding properties is awful. It looks complicated and messy to use - much of it in the name of 'type safety' - but then they go and use some scripting language for binding expressions that isn't checked till run-time anyway. Sigh. Everyone seems obsessed with standardising and formalising every little thing too. Sometimes the smaller simpler stuff can just be left to the implementor. Some of the 'design pattern' guys seem to have gone off the deep end - adding over-doses of complexity upon complexity.

The application at work is a single-user destkop application that uses a SQL RDBMS as a backend. One thing we've contemplated is moving it to a multi-user, server-based system. Now i would simply not consider using .NET to do this. So that was another reason to re-visit Java. Ahh j2ee 5. Well one thing that can be said for Java. You've got a lot to choose from, a lot of it is free, some of it is extremely good quality. It's really like another world completley from the closed money-grabbing greedy eco-system in the stagnant cesspit surrounding the outflow from microsoft. There just isn't any comparison - they're not even in the same league, perhaps not even the same sport.

I'm still trying to get my head around the persistence framework. I kinda liked the older Entity model, because things were coded directly, at least initially it seems easier to understand (or at least, I figured out how it worked at one point, although I never used it). The newer model does a few things unfamiliar to me, but i'm sure I could get used to it. Our .NET code uses a custom entity-like system and a lot of custom SQL (lots of messy joins), which would be quite difficult to move to the persistence query language, but most of it could be moved to views or stored procedures as well (and probably should be at that). I'd considered nhibernate, but it was a bit immature at the time, and quite frankly I didn't see the worth in investing all that time to learn another meta-language to re-describe what was already in the database (and I still dont - i'm glad as hell that j2ee 5 uses xml meta-descriptors sparingly).

The new EJB stuff is quite nice in some areas. Last time I worked on Java this stuff didn't even exist. We were using CORBA directly. The automatic CORBA servant generation from interfaces is ... well it's nice. But there are some `strange' limitations. Well they're not really strange - it forces a particular architecture which will scale transparently. But if you don't really need that it does limit things needlessly - like passing around client-hosted interfaces. Although facilities like JMS can often be used to implement the same sort of things anyway. JMS is nice.

One problem is that although all of these really nice facilities exist - it can be a real pain getting them to actually work. I was playing with JMS and even though I was using the 'correct' mechanism for sending messages, I was running out of connections. Bug in the app server perhaps? I'm not sure. Not knowing is a bit of a problem. And with a distributed application I was hoping to re-use the persistence objects remotely, but that doesn't really work. Ho hum, back to the (manual) grind of load--copy to mirror object--return, etc. In another case I tried changing a persistence object from being LAZY to EAGER loaded - it crashed with what seemed to me an interface mismatch between server and client. Couldn't work that one out. Actually in general netbeans+glassfish seems terribly unreliable at consistently rebuilding dependencies. Maybe i'm doing something wrong, but even with less than a dozen files I often have to run it twice, or shutdown everything and clean+build to get new code loaded (this is something that affects visual studio too).

I shall continue to tinker.

Sackboy, alias The DRM Kid

Ahh well, so my `worst fears' about the LittleBigPlanet (LBP) moderation system seem to have come to fruition. I knew when Sony decided to recall all copies because of a possible religious offence only days before a world-wide launch, at no doubt very high cost, we were in for a messy ride if not a complete cluster-fuck.

http://forums.littlebigworkshop.com/lbp/board/message?board.id=pod&thread.id=214 http://boardsus.playstation.com/playstation/board/message?board.id=littlebigplanet&thread.id=197353 http://www.psxextreme.com/ps3-news/4085.html

It seems LBP's moderation system is as harsh as it is limited. From what I can tell, levels get deleted from the server with no explanation for even the slightest infringement - you cannot even play your local copy if you are online, and they retain persistent identifiers so you cannot copy/edit and resubmit them either. There is no ratings system, so everything has to be child-friendly (why does the PS3 have a parental control setting then?), but also it means it has to be inoffensive to everyone - everywhere. Unfortunately, if you look hard enough you can always find someone who will find offence in whatever it is you are doing, no matter how commonplace or inoffensive it may be to you and your associates.

Then there are those nefarious so-called intellectual `property' laws. Trademarked, copyright, or even patented(!) items cannot be represented in the game. So I guess no one-click shopping levels! And these are only compounded by the difficulty in understanding what they actually mean and how they are applied - which might not even be possible if they are trying to comply with some sort of lowest-common-denominator of laws from all countries that have the PSN. For a family-oriented game, almost certainly to be used without supervision of legal counsel, the likely-hood of running afoul of these laws is quite high. And when their home-spun creation vanishes without explanation, I'm sure they'll let all of their friends know of their frustration and anger. Hey maybe they can get Fry in to do some more tutorials - teaching the general populace about the evils of copyright violation!? `Here's the DRM Kid, he'll protect us from those nasty pirates!' (rather ironic if it happened, given his recent piece extolling Free Software for The GNU Project's 25th anniversary).

Ahh, what a mess these laws have created for all of us - even those who wanted them.

But what about the game itself?

I actually bought LBP on Friday and spent a lot of the weekend playing it. I had considered the censorship issue and thought it would probably work itself out. Reading the current threads on the forums does make me feel less of the game, although of course it doesn't change the gameplay of the built-in levels.

It looks gorgeous, with the materials, textures and lighting, sound and effects all spot on (apart from the jump 'whoosh' sound, which I don't particularly care for). The level designs are varied and mostly interesting, and there are constant puzzle elements when trying to advance, or get to bonus items on a level. The controls are over-all pretty good, although since everything is physics based it can take a little getting used to compared to programmed behaviour. The 3 levels of depth is a bit fiddly at times, but it isn't a deal-breaker. It has some frustrations - checkpoints are limited so death can mean the repeat of an entire level. The `emote' functionality (happy/angry/sad/scared) is mostly pointless - usually you're too far away to even see it properly, and it sort of wobbles and flickers a bit when you can - which makes it look a bit silly. It also definitely isn't a young kids game, it is too difficult for that, requiring exact timing and jumping in places - although some of the mini-games are for everyone. Multiplayer can make level traversal more difficult - but that is part of the fun too. e.g. interfering with someone so you get more of the goodies. And each level has at least one 2-player puzzle for extra items. But on many scenes the camera doesn't pan enough, so you can get left behind very quickly - after a short timeout you die until whatever player the camera is on gets to the next checkpoint. I haven't played online.

The user-created levels are a mixed bag - there are so many that it is difficult to find the really good ones. There are too many (utterly pointless and annoying) `get trophies quick' levels high in the list, and people are already asking on forums for levels to be `hearted' for the purpose of gaming the system. So it's mostly just a random guessing game, although with time the tagging system will probably become more accurate and thus useful. Levels load pretty quickly - although i've had some failed loadings, but a retry normally has them work. The polish and difficulty varies greatly, but there is a fairly wild mix, from simple platform games, to races, to puzzle games and so forth - even a side-scrolling shooter with a sort of string-puppet feel to it(!).

The level editor is necessarily quite complicated. But building things out of real materials with motors, switches, connectors and so forth is fairly intuitive - and a heap of fun. It will take a lot of time to create a good level - and with the broken moderation system in place, it reduces the enthusiasm somewhat - it could vanish without explanation if you make a mistake and offend some culture you're not even aware of, for example.

I don't know if the moderation thing will get sorted out. Hopefully it can be toned down a bit - at least it must tell people what they did wrong and give them the opportunity to fix it. There are plenty of people who want to make levels in good faith, but if just one level gets deleted permanently with no explanation or recourse, I can imagine they'll give up on the game forever - and probably be angry enough to let everyone else know too. A tiered system with some more grown-up `may-offend but not-illegal' content would be bloody nice too. e.g. no possibility of art or satire, just `commercial-mainstream that is so inoffensive it manages to offend' type levels - which will end up being boring. A multi-tier system is probably too much to ask, but they clearly need a better system for accidental problems through ignorance - otherwise this game will lose sales.

I suspect it may be a scalability issue - rather than check or edit the levels to fix problems they just blocked the level (or a grief-report blocks the level immediately, until it is reviewed, and they have a back-log). I'm still willing to give them the benefit of the doubt, but I guess time will tell on this one, as it always does.