ePub3 Test Cases Are On the Anvil

The [UNDER CONSTRUCTION] ePub3 Test Suite is getting stronger by the day. It is refreshing to see this well-defined core test suite take shape; especially after the disaster and near uselessness of the IDPF ePub3 so-called development books.

It is unclear whether this is an IDPF "initiative" but the contributors all seem to be IDPF members. I haven't seen any specific announcements about the work from the IDPF; perhaps I wasn't paying attention.

If you are interested in viewing and downloading the ePub Test Suite you can access it from GitHub here. But be warned it is still very much [UNDER CONSTRUCTION] so may be of limited value at this time. It is just exciting to know this core tool is being made available only two whole years after the release of the specification.

AZARDI

AZARDI was of course the first ePub3 reader in the entire known universe, being released just one month after the specification was released. We were a bit slower with the Fixed-Layout version because we wanted it to be truely specification compliant AND useful immediately for textbooks with rich interactivity. That was released mid-2012. You can see the power of AZARDI with ePub3 sample reflowing and fixed-layout books Here, Here and especially Here.

One rich difference between AZARDI Desktop and the other ePub3 reading system (iBooks) is that it is based on Mozilla (IE Firefox). Unfortunately the Test Suite books are Webkit oriented rather than universal. The universe weeps! There is also going to be an interesting rendering system orientation in the near future as Google's Blink emerges from its little "O Solo Mio" closet.

We selected Mozilla because we needed to deliver a desktop reading system that worked on Windows, Linux and MacIntosh and natively supported MathML. The options were precisely one!

To create AZARDI we had to create dozens and dozens and dozens of test cases. We have presentation test-books, technical test-books, load testing test-books with 1000 chapters and hundreds of embedded fonts (as examples). This is how serious software is developed.

A toe in the test book waters...?

So it is good to see this tentative toe into the test book water or more correctly Test Suite water. However from the construction of the books the motivation appears to be targeted at specification compliance fear policing rather than performance and development.

The final objective seems to be the ability to allow a reading system to  be ranked for compliance against the specification as if that is even relevant in 2013 with the raft of meaningless and crazy features that comprises much of the ePub3 specification.

The Test Suite mixes required and optional features somewhat randomly. Ideally these would have been split into separate test books. The root Document 0100 is a bit of a feature hot-pot. It should have probably been two or three documents. Also the most important document of all —Document 0150: Navigation— has been strangely relegated low on the list. If I was testing a reading system I would want this one first.

Whatever!

The important thing is an effort is being made whatever the motivation and document structure. This will certainly be a better assessment of ePub3 specification compliance than the rather weird BISG ePub3 reader support grid which doesn't even list a lot of major features and is sadly full of "marketing truths"!

You never know, next the IDPF may announce an ePub3 Reading System Support Medal system. That would be so arbitrarily cool.

Understanding the Test Books

At this stage there is no documentation for the test books so by just accessing the file it can be difficult to understand what each of the magical 30 test books is doing and why they exist. Infogrid Pacific to the rescue!

Over on our APEX tutorial site we have prepared a full deconstruction of all the Test Books as they are in their current avatars (remember they are a work very much [UNDER CONSTRUCTION] and more likely to change than not).

Because we are deeply involved in implementing production systems, delivery systems and reading systems for ePub3 we like to get our feet in the water early (that's how we got the first AZARDI released so quickly after the completion of the specification).

It was fun racking AZARDI up against the (at this stage [UNDER CONSTRUCTION]) IDPF Test Suite.

This is a very exciting step-forward for ePub3. Far more than the Readium project this brings ePub3 pluses and minuses, goods and bads into empirical focus in a way no waffle dialogue does or can.

The ePub3 Test Suite has rough edges and shows its desktop cut and past construction (just ignore the typos and spelling errors. Don't be such a publisher!). The authors obviously don't have the luxury of a serious authoring environment like IGP:Digital Publisher to do the test book production. (We have indirectly offered an IGP:Digital Publisher Portal at no charge... but no interest.)

The Test Book Construction

Having mentioned the construction of the test cases, this just highlights the classic problem of testers who have to create AND maintain a combination of subjective and objective tests.

It takes a lot of thinking and a lot of preparation work to create digital content test cases which can be evaluate in just seconds. The ePub3 Test Suite team is doing quite well at their first attempt. Kudos for that.

Fixed Layout...?

The fixed-layout test cases (0200-0220) are a little behind the game and are reasonably and obviously influenced by the iBooks very partial ePub3 specification implementation; and the messy Readium implementation that gets pretty much nothing right in fixed layout.

Specific criticisms are:

1. The books don't have covers (that applies to all the books except Document 0130).
2. The test-case statement grammar is ambiguous
3. No test-cases for left-center-right page positioning (yet)
4. No mixed spine-property test-cases of significance.
5. The fact that the test cases seem to imply the cover/opening section is right aligned on a spread just like the Apple thing. We addressed this issue in a blog post nearly a year ago.

The test cases should put a priority on the spread layout left-center-right positioning as a significant designer tool in book presentation. We use the spread positioning feature to put covers in the center of a viewport, title pages to the right, copyright pages to the left, etc.. In AZARDI we gave the option of associating EPUB:TYPEs with the left-center-right position options on the odd occassion we are using Fixed Layout to mimic a print book.

As the Fixed Layout Test Suite develops we suggest you compare them against the IGP demonstration book FLO (Fixed Lay Out) Grows Up. which clearly demonstrates so many of the current missing basic components.

And in ePub3 Conclusion

The backward looking to a world of eInk and dedicated reader features is still front and center in the IDPF ePub3 specification as can clearly be seen encapsulated in the ePub3 Test Suite.

We only have to look at features like the crazy CFI weirdness to understand that any reading system developer has to have the tools to sort the chaff from the wheat in this large and very sporadic specification.

A strong and useful value of the ePub3 Test Suite books is they make it very easy for a reading system developer, and eBook production system developers to understand, decide and define which features to support for books, platforms and user 2013 expectations without being specification cludgelled.

The ePub3 Test Suite Document feature lists in tabular form helps cut through the clutter.

No Reading System does fully support the ePub3 specification. More importantly no Reading System should support the full ePub3 specification.

Posted by Richard Pipe