Sleep n Whinge

ugh, what a crappy day. I hit the grog a bit hard last night (sister dropped by for a couple of hours on her way to the airport), and subsequently had very little sleep; and the neighbours decided today was a good day to re-start the work on the extensions next door. Had a nap about 5, at least until some dodgey scam out of India rang up about 7:30. Blah.

But I played a bit with some code during the day. I poked around with my slideshow creator, working on some more transition wipes - worked out a 'clock' transition which seemed to take much longer than it should have (for lack of inspiration I'm looking at the SMIL stuff for ideas). I was going to write a very simple front-end gui for it, but just didn't have the motivation for that today.

Then I got totally side-tracked with some other stuff: I noticed javafx builds are finally available for gnu/linux, looking at the swingx demo (there's a couple of things that look interesting), the image filters it uses. Mr Huxtable also has an interesting article about BufferedImage stuff (which i'm sure i've read before but must have forgotten about): and that got me thinking about changing the way jjmpeg's helpers work with images as it uses 3BYTE_BGR types and direct DataBuffer access.. And that got me thinking about JNIEnv.GetPrimitiveArrayCritical (to avoid 2 copies), and well by this time I was too hung-over and tired to do anything useful.

I also noticed the neighbours were building a really big verandah which will block most of the direct light into my bathroom, and they over-cut a bit of a tree that hangs over the boundary. And I got a letter from my insurance company whining about an over-charge they shouldn't have been making in the first place. All all that together with the severe lack of sleep, put me in a terrible mood and made me feel really rather miserable. And now it's 3am and they'll be at it at 7am again next to my bedroom window so tomorrow probably wont be much better ...

Update: Oh fun, 7:25am, shit radio station was bad enough, now it's with the jack-hammer.

VideoZ

I had a go at writing a simple 'media mixer' today. So far it's only video, but i'm already thinking about how to do the sound (hence some work on JOAL yesterday, I'm planning on using OpenAL-Soft to do the mixing, which gives me '3d sound' for free as well). Sound is a bit more difficult than video ...

As output it generates an encoded video file; using jjmpeg of course.

With a small amount of code i've got a slideshow generator, together with affine transforms, opacity, and video or still pictures. I'm just using Java2D for all the rendering: so the compositor is fairly slow, but it's workable.

But, the biggest part of any real application such as this is the user interface for setting up the animation parameters ...

Paged layout, busy dot.

After poking around at jjmpeg a bit this morning, I played a bit more with ReaderZ. First I added an animated 'busy' icon for when the reader is busy, and moved the epub html loader to another thread so it animates. It's ugly, but it works. I simplified the use of the event manager as well.

Then I redesigned the BlockLayout code in CSZ so that I could sub-class it to create a paged media layout. It isn't 'conformant' by any stretch, and has a bug with tall images, but at least it forces lines to align to a new page once they've overflowed the viewport.

During this I realised I probably wont be able to get away with a single-pass for the layout. e.g. if you have an auto-sized box, it's size depends on things like the size of floats and the lineboxes inside of it. But you have to lay these all out before you can determine what it is, and then must lay them out again afterwards once you've determined the real size you're working with (also required for things like text-align). I might have to lay out individual words instead so then the second layout can be fast as well as letting the layout be handled separately from the text object.

jjmpeg transcoding

Well I had a go at transcoding using jjmpeg. I added the binding required to get it to work and added a new JJMediaWriter class to handle some of the details.

It doesn't work very well - many formats just crash. But at least avi with a few formats works. I presume i have some problems with the buffer sizes or some-such.

Update: A misunderstanding of the JNI api means I was getting a ByteBuffer pointing to 0, rather than a null ByteBuffer. I've fixed that up and now the transcode demo works a bit better. I'm still not flushing the decoders on close, so it isn't complete yet.

Sources:

e-reader, epub

Had a mini hack-fest today, and whipped up an e-pub backend for ReaderZ based on CSZ.

I added very basic img tag support as well, as is obvious.

Apart from the code to parse the content.opf file from the .epub archive which was fairly small, I spent the most time trying to work out a URL handler for a made-up 'epub:' protocol. I copied the way the jar: protocol handler distinguises between the base archive and the filename using "!/" - this is so that the normal url resolution mechanism work. But I also wanted to resolve by the manifest ID and I use the url fragment for that (although in hindsight I probably don't need it). But anyway in the end it wasn't much code, and having it there made everything 'just work', which was nice.

I also had to deal with all the crap XML brings along: i.e. dtd resolution.

The actual viewer is a bit unwieldy as it works as a set of html pages. So you need to pan around to read each 'page' (i.e. chapter, or whole book), and changing pages flips between the items in the spine (i.e. chapters or whole book). To do better than that I really need a paginating layout engine: which is something for later.

I have no svg support not surprisingly, so title pages which are pure svg come up a re-assuring blank.

Still a bit slow opening new chapters, but what can you do eh?

It's all been checked in to ReaderZ and CSZ.

floats n stuff

I made some more progress on CSZ. The latest thing I have sort-of working are floats.

I think i'm interpreting the bits i've implemented correctly: floats are quite limited so the layout logic isn't terribly complex. I still have no borders or padding (and I removed the fudge factor I had in before) so it looks a bit cramped.

It's still sad just how much crap you need to get to even this point ...

I just got a call from work and they want me back in a couple of weeks, so I might turn down the effort a bit so I can psych myself up for that. Maybe i'll finally use that kobo as a reader of books too ...

CSZ

Well I kept poking away at the XHTML/CSS stuff, for want of something better to do. I had a couple of wins along the way.

The cascading and inheritance is working somewhat better now, and I added a few more properties. Given that I'm not very familiar with all the various rules, I think I have a fairly efficient resolution mechanism by indexing various bits and pieces. The layout system is still crap, and very very incomplete, but at least I have baselines aligned now.

This is a totally contrived example, everything apart from the text layout and typefaces are hacked in one way or another.

I tried it on the kobo ... it's still fairly slow, but it's better (I think: TBH i can't remember what I tried the other stuff on). Still hampered by the text layout though.

For a 200k file (which is mostly just <p> elements), scanning the file, resolving the properties and generating the box list takes only a fraction of a second. I'm not trying to resolve or use very many properties though. About all i'm using `in anger' are some of the font-* properties.

But then performing the layout (as a single page) takes about 7s (once the jvm is warmed up), which is mostly due to TextLayout. I will have to try it with a simpler font than the one the JVM comes with.

Rendering is fairly ok (relative to the e-ink anyway) and all i'm doing is painting every textlayout in the whole tree ...

I uploaded it to MediaZ anyway, in the new CSZ module.

Hmmm, I should really take a break from hacking for a bit. But i'll believe that when I see it.

xhtml, css, boxes n shit

For some stupid reason I delved into CSS and XHTML and rendering thereof.

CSS is so deceptively simple: a few boxes, layouts in lines, and a few properties to set. Saying the devil is in the details here isn't doing the term justice. It's all in the details. And they're ugly.

From the fairly complex cascading rules, to the number of properties. The layout merging. The badly written documentation: filled with "x inline y box" "a block b box c" to such specificity, and no with definitions it is quite difficult to decipher what it's even talking about. It's also quite hard to debug, since it needs a fair bit of data structure to represent it.

CSZ

Anyway, after some mucking about, I have a relatively complete CSS lexer and parser, a fairly incomplete cascade resolver, a fairly incomplete layout engine, and a very incomplete style system. It's just enough to show paragraphs of text with some basic formatting. For a book reader I don't want the document to control the text too much anyway.

I'm attempting to do it while streaming the input, and (obviously thus) in a single pass using the pull parser from XMLStreamReader. Therefore initial parsing is quite quick, but it's still taking a relatively long time to lay out the boxes ...

TextLayout

And the problem here is TextLayout. It's just quite slow. I tried my own version of layout using FontMetrics.getCharsWidth(), but inside that just creates a TextLayout anyway, so it's even slower (or maybe not, now it's a bit faster?).

I know why it's so complicated; for laying out complex scripts and handling all the special cases. Anyway, that is the primary factor of constraint on performance at the moment, although as the implementation is so far from finished, i'm sure it wont be the last one.

(I played some more, and the font used plays a big part in the speed taken, so there's hope yet).

I suppose I should try it on the kobo to see how it goes there.

Dead end?

It's taken a lot of effort to get this far, and i'm not really happy with the result. So i'm not sure if i'll keep plugging away at it or throw it away (and if i need such a functionality, use cssbox). There's a lot left to get it to be useful for anything.

About Me

Tags