Mobile web and HTML inside JavaME with LWUIT

In the video: A use case of using HTMLComponent. Used both to render offline content within the JAR (The events detailed descriptions) and also to access external web-flows (Facebook, Google forms). Fonts look better on actual devices (video compression takes its toll...)

One of the things I always missed in J2ME was the ability to perform some web-based flows or display HTML in general as part of the application (and not by sending the user to the native browser with platfromRequest, not knowing if he'll ever come back...).

Most of the other mobile platforms do have solutions that allow embedding web seamlessly into the application, but in J2ME it never existed. So this is why I am glad to unveil my latest project which was done as part of my consulting work at Sun Microsystems: HTMLComponent.

HTMLComponent is a LWUIT component that allows J2ME developers to render HTML documents (local or remote) that conform to the XHTML Mobile Profile 1.0 standard. It truly revolutionizes the things you can do in J2ME and brings the platform to new heights.

Use Cases

There are many use cases for HTMLComponent, here are a few I can think of:

Rendering rich-text locally - The simplest one is when you need to render some rich text locally - for example a text segment that part of it should be bold/italic and an image next to it, or maybe even a small table - to do that with current tools is not an easy task. But with HTMLComponent, you can simply compose an HTML document describing that exact scenario and you're done! This is classic for Help/About pages and can also scale up to rendering parts of your UI with HTML (Since it supports forms).

Dynamic Content and UI - Another common use case is dynamic content and UI. When you release your app, sometimes you need parts of it to be dynamic - for example a catalog section for your app that changes daily or a news page or an elaborate details page. Up until now developers had to make their own frameworks to support such cases and most of the time it wasn't truly 100% dynamic. With HTMLComponent you can simply have the app refer to a URL, and serve HTML there - and since HTML is not only the content but also the layout and even UI, your catalog can look totally different without you changing anything in the application and having to release a new version of it.

Using your existing web flows - A third use case is when you already have a web server-side, and instead of writing the whole thing again for mobile, and having to handle 2 separate processes you want to utilize the already existing process for mobile. A good example for that is registration and login - why recreate this process if you can simply redirect users from your app to the existing registration URL?

Embedding third parties web flows - the fourth use case is when you want your users to be able to access third parties existing mobile web content. For example you have an LBS app and want to let your users check the weather in an affiliated site - no problem - just provide the URL. Other examples can be referring users to online dictionaries, translators, search engines and other services.

Anyway - The real strength here is the fusion between the client application and the web. It is ideal for use cases in which a simple mobile web site is not adequate enough - and an application needs to rely also on web-based flow for dynamic content/UI (either existing or specially designed for the app)

Getting HTMLComponent

And now you probably ask yourself: When can I get it and for how much? Well the answers are the best you could hope for: Now and for free!

HTMLComponent is a part of Sun's LWUIT framework, a UI toolkit that allows you to create great looking and portable UI for J2ME applications. I worked with LWUIT since it came out back in mid 2008 as a consultant for various companies. I did several project including adding RTL support to LWUIT, and in the past few months I have joined the LWUIT team at Sun as a consultant.

One of the best things about LWUIT is that it's open-sourced and can be used also for commercial purposes for free, and HTMLComponent is no different here, so basically this solution is available now for all J2ME developers out there and can be checked out from the LWUIT SVN (Note that it is not included in the LWUIT 1.3 drop, so downloading the latest drop wouldn't help, you have to check out the sources from the SVN).

If you check it out, don't forget to check out also the LWUITBrowser project which I wrote to demonstrate what you can do with HTMLComponent. I'd like to emphasize that a browser is not the use case we're aiming for, since all mobile phones (at least those with J2ME...) have native browsers that nowadays support even more than XHTML-MP1. But still this demo is the best way to show what types of documents HTMLComponent can render without us having to build a whole server side.

About XHTML-MP 1.0

And now a few words about the XHTML Mobile Profile 1.0 standard: XHTML-MP 1.0 is a subset of XHTML adapted to mobile. The standard supports most of the basic elements such as Images, Fonts, Lists, Tables, Forms and even CSS. It does not support however Javascript and frames and also other tags or attributes. The full details of which tags it supports can be found in this document, and also a more detailed list of which attributes in each tag are supported can be found here, though I am not sure of its accuracy.

The XHTML-MP1 standard should be sufficient for most use cases above, and as for the fourth use case above (using existing third party content) - While there are not many sites that conform 100% to the XHTML-MP1 standard, a lot of mobile sites are very close, and since the parser in the package is not very strict they can be rendered. Just make sure you're using the mobile versions, since using regular web will surely be unsuitable.

Detailed support list

And now for those of you who stayed this long, here's a more detailed list of what is supported and what's not (in general and in the current version):

Text - Obviously text is supported... But that also includes most of the important ways to format it in HTML including alignment, paragraphs, div, new lines using the BR tag etc. Text justification is not supported yet (align="justify") and will hopefully be added later on.

Fonts - Fonts can be changed anytime for example by using B for bold, I for italic BIG for a bigger font, SMALL for a smaller etc. Also header definitions (The H1-6 tags) and other less known font directives (such as KBD, CITE etc.) are supported, but all subject to availability of such font in the device. If the needed fonts are not available in your device or don't look as expected, you can always add LWUIT bitmap fonts and assign them to the various tags.

Images - Images are supported and basically any image format that your device's JVM support will work here. Image download is done asynchronously by using a number of threads, so the document loads up even before all images are available. The number of threads can be tuned and image downloading can also be turned off. Note that HTMLs containing large images may cause memory problems and/or cause the page to look weird.

Lists - Lists are supported including nesting. This includes ordered lists (the OL tag) and unordered lists (the UL tag) with their items (the LI tag) and also definitions list (the DL, DT and DD tags).

Forms - Forms are also supported. Both the GET and POST methods are supported as a form action. The HTML package takes care of URL encoding and most input field types are supported including: text fields and areas, check boxes, radio buttons, combo boxes, hidden fields and of course the reset and submit buttons. Regular buttons (both input type=button and the button tag) are not supported due to the fact that they can only function with Javascript. Also not supported is the File upload control (input type="file") as it is not part of XHTML-MP1.

Tables - Tables which are still the best way to layout HTML documents, are supported including nested tables. There are still some open issues here and there but these will be solved over time.

Char entities - In order to be able to write HTML reserved characters such as the tag brackets or non-displayable/type-able characters, char entities are used and they are supported. All char entities up until ASCII code 255 are supported and beyond that you can add your own.

HTTP - HTMLComponent is mainly a UI component and its job is to render HTML documents. As such it does not deal with network/IO and the complexities of the HTTP protocol. It does use an interface by the name of DocumentRequestHandler that should be implemented by developers to fetch documents from wherever they need and in whatever protocol they want to use. Since the most common uses are fetching HTML via HTTP, there is an implementation of this interface in the LWUITBrowser project that handles the HTTP communication and knows to take care of request/response headers, cookies, redirects, network errors etc.

Parsing - HTMLComponent uses an internal parser to parse the given HTMLs. The parser is not 100% strict and will also accept some errors in the document. The developer can control however which errors to ignore and continue parsing and which errors to stop on by implementing the HTMLCallback interface. Also note that some errors may be too fatal for the parser, so try to stick to the XHTML-MP1 standard and especially always close tags that have been opened and in the correct hierarchical order.

Refresh/Redirect - HTMLComponent performs refresh/redirect directive that are defined in the META tag and HTTP_EQUIV attribute of the HTML. As for redirect response codes, these should be implemented in the request handler.

CSS - While CSS (or more accurately WCSS) is a part of the XHTML-MP1 standard, it is still not supported in the current version. The meaning is there is no way to change individual style properties of elements. So for example while you can determine the colors of regular text, links and other elements (with LWUIT styles) as a whole per component type (i.e. all text will be black, all links will be blue), there's no way yet to make one word blue and the other red. But this will be added as CSS will be introduced later on.

Embedded objects - While the OBJECT and PARAM tags are a part of XHTML-MP1, embedded objects are not supported, and there are currently no plans to support those (as they would require writing mechanisms that actually run these objects which is a whole different issue...)

Anyway, that's about it - hope you'll find this new piece of software helpful for your innovative mobile project, any comments and feedback will be appreciated so we can further develop this technology.