During the JavaOne 2007 keynote session, Jasper Potts, Ken Russell, and I demonstrated a photo editing web application called IRIS. The Interactive Rich Imaging Software demo makes good use of Java for a rich, compelling web client.

We wanted our web app to be able to integrate more closely with the user's machine than is possible using traditional web technologies. Some of our features included: true 3D hardware accelerated graphics; drag 'n drop photo uploading; fullscreen slideshows; and dynamic asynchronous upload and download of images. To implement these features we decided to mix Applets with our HTML code. This article will detail how we did it, and how you can too.

To better understand this article, and the accompanying code samples, I suggest you either watch this screencast or visit the IRIS site and experiment with IRIS yourself.

Why Applets?

Why did we choose to use Applets? (Er... besides the fact that we work on the desktop team!). We chose to use Applets because they provide very powerful features that are hard, if not impossible, to find in other web technologies.

First, the Java platform is much more powerful than the browser when it comes to multithreading. The browser essentially uses a single threaded rendering loop / Javascript engine. AJAX helps get around this limitation by introducing a single API for executing asynchronous requests (the famed XMLHttpRequest), but this is a far cry from true multithreading.

Threading was designed into Java from the start. It provides such useful APIs as Thread, Runnable, Future, ExecutorService, ConcurrentHashMap, AtomicBoolean, and many, many more. Massively threaded web tier applications have relied on these APIs for years. For thread happy apps, Java (and by extension Applets) make a whole lot of sense. IRIS uses multithreading for everything from uploading photos to searching for users to running the slideshow.

Second, Applets provide a robust security model which allow web apps to integrate closely with the user's machine. When you digitally sign an Applet, the Java platform will allow your Applet to access the user's hard drive, network, or other resources (if the user grants such access, of course). This was an absolute requirement for IRIS. One of the features we wanted to support was the ability to allow the user to drag a bunch of photos from their desktop or from another application onto our web page. We wanted fun effects to occur during the drag over, and on the drop. This kind of desktop integration requires more power and flexibility than the traditional browser model supports.

Third, we wanted to support full screen mode and true hardware accelerated 3D graphics with perspective transforms. We also wanted to leverage our knowledge and experience as desktop developers when designing and creating this web application.

Finally, at JavaOne we made a lot of noise about the JavaFX platform. As many have rightly pointed out, in order for JavaFX to succeed, we needed a more compelling Applet story. While there are many improvements underway for Applets (we hear you!), we also knew that there were a lot of cool things that could be done today with Applets that nobody had really taken advantage of. We wanted IRIS to showcase these features.

Using Applets

Before diving in, it would be good to briefly review how we get Applets into our web pages to begin with. Applets have been around for a long time. So long in fact that HTML has a special tag just for them: <APPLET>. While the <APPLET> tag has been officially deprecated in favor of the <OBJECT> tag, I've found that <APPLET> is often better supported. Or at least, easier to use. I'll include an example of how to use the <APPLET> tag.

I know I'm committing major transgressions for using deprecated API, but I always use the <APPLET> tag. I just like it. Give me a break, I'm a desktop guy!

Non trivial applets are usually packaged into Jar files. At its simplest, you can define such Applets by specifying the Applet width and height, codebase, archive, and code attributes:

    <applet applet width="100%" height="45px"
                 codebase="http://swinglabs.org/iris/applets/editor"
                 archive="Editor.jar,Common.jar,Applet.jar"
                 code="org.jdesktop.iris.ToolBox" />


In this example snippet, I've defined that the width should occupy 100% of the available space, while the height is fixed at 45 pixels. The codebase attribute defines the base URL to use when resolving project artifacts, such as jar files or image files. The archive attribute contains your comma separated list of Jar files. Finally, the code attribute lists the Applet itself.

In addition to such simple parameters, you can also specify your own Applet specific parameters. These are specified in HTML by using <param> subtags. Each <param> tag has two attributes: name and value. In the Applet, you can retrieve these parameters by simply calling "getParameter". Here is an example:

    <applet applet width="100%" height="45px"
                 codebase="http://swinglabs.org/iris/applets/editor"
                 archive="Editor.jar,Common.jar,Applet.jar"
                 code="org.jdesktop.iris.ToolBox">
        <param name="color" value="#2342FF"></param>
        <param name="foo", value="bar"></param>
    </applet>

In Applet class:

    String foo = getParameter("foo");


Using Native Code with Applets

One of the key benefits of using Java in a web application is the ability to use native code. For example, in IRIS we use the Java OpenGL bindings (JOGL) to get true 3D effects. For safety, only signed Applets can use native code, and must get permission from the user. Unfortunately, paying for a certificate from a public Certificate Authority (such as Verisign) can be prohibitively costly. Does everyone who wants to use JOGL have to buy their own certificate? And how do we include native code in our Applet?

The Java Native Launch Protocol (JNLP) provides a powerful and convenient solution. The JNLP protocol defines an XML file that describes all of the resources necessary to run a Java desktop application. Using work pioneered in the JOGL project, we can reuse JNLP files for describing the resources of an Applet, including the use of native resources.

Instead of putting your Applet directly in the web page, put the JNLPAppletLauncher Applet in its stead. The JNLPAppletLauncher uses a JNLP file to delegate to your Applet. To try out the JNLPAppletLauncher. You can get it here _______________

Enough theory, here's the actual HTML snippet we use in IRIS for the Editor Applet.

    <applet id="editorApplet" name="editorApplet" width="100%" height="100%" MAYSCRIPT
                 codebase="http://swinglabs.org/iris/applets/editor"
                 archive="Editor.jar,Common.jar,swinglabs-dev.jar,jogl.jar,gluegen-rt.jar,msg.jar"
                 code="com.sun.opengl.util.JOGLAppletLauncher">
         <param name="subapplet.classname" value="org.jdesktop.iris.editor.EditorApplet"></param>
         <param name="subapplet.displayname" value="Editor"></param>
         <param name="progressbar" value="true"></param>
         <param name="codebase_lookup" value="false"></param>
    </applet>


As you may with any HTML tag, we have given our <APPLET> tag an id and a name. These will be used later in conjunction with CSS and Javascript to manipulate our Applet within the page. Next, we've specified the width and height of the Applet.

You'll notice that we've also specified an attribute that has no value, but is simply declared: MAYSCRIPT. This attribute instructs the web browser that the Applet should be allowed to execute script commands against the page. This will become very important in a few minutes.

TODO Migrate this example to JNLPAppletLauncher

There were a couple interesting lessons I learned while writing IRIS. The first is that cross browser HTML/CSS issues are an absolute nightmare. The vast majority of our time writing the UI was spent tinkering with page layout in order to get the visual design we wanted. The other thing I learned was that working with Javascript was actually a pretty decent experience!

Java/Javascript Integration

To enable the sort of rich interaction between Applets and HTML that IRIS needed required tight integration between our Applets and the DOM of the page. This was both easier, and harder, than I expected. It was easier in that making arbitrary Javascript calls and executing arbitrary scripts against the page is pretty well supported on all major browsers. What made it more difficult than I expected was the singled threaded nature of the Javascript execution environment in the browser. It is way too easy to end up in deadlocks.

Calling Java from Javascript

We needed to be able to call into the Applet from our web page. In IRIS, this happens whenever you search for a user name. When you type a Flickr user name into the search field and hit enter, we call a method in the "Toolbox" Applet (the bar on the bottom of the page) called loadAlbums. This method then uses Java multithreading libraries to communicate with Flickr, search for photos, and insert any results into the web page.

It turns out that calling Applet methods from Javascript is really simple. Just call the method! The first code snippet here is from the toolbox.jsp file. The Applet has a name and id of "toolbox".

    document.toolbox.loadAlbums(searchString)


This ends up calling the following method in the ToolboxApplet class:

    public void loadAlbums(final String search) {/*...*/}


As you can see, calling Applet methods is really straightforward.

Calling Javascript from Java

After we retrieve search results from Flickr, our Applet needs to be able to call Javascript methods in the web page to manipulate the DOM and insert the search results. Fortunately, on pretty much all the major browsers (including FireFox, Mozilla, Internet Explorer, Safari, and Opera) there is a Java-to-Javascript integration API called LiveConnect. LiveConnect is an API originally included in Netscape Navigator for allowing just this kind of web page to Java integration. In the years since it was introduced, every major browser has implemented the LiveConnect APIs.

The basic technique that we employed was to execute arbitrary Javascript calls against the page using LiveConnect. We found this technique to work very well.

Another option we explored that I'd like to see supported in a future version of the Java plugin was the ability to manipulate the DOM directly via API from the Applet. There is basic support for the W3C HTMLDocument API (which is the same DOM API that all browsers support and that Javascript works with), but I'd like to see it fully and properly supported in the Java platform.

The LiveConnect netscape.javascript.JSObject class provides a very lightweight API which enables you to make javascript calls to the browser. We used two methods on JSObject: getWindow and eval. The getWindow method takes an Applet as an argument and returns the ____Document?____ that owns the Applet as a JSObject instance. With this ____Document?______ you can then access the DOM of the entire page.

For example, to change the background color of a div with id "test", we could execute the following calls from within our Applet:

    JSObject page = JSObject.getWindow(myApplet);
    page.eval("test.background=#AABBCC");


Rather than manipulating the DOM directly, as in this example, in IRIS we created a handful of Javascript methods in the web page itself and simply called these methods from within the Applet. We found this to be a rather clean solution since it kept all the code for creating and manipulating the web page in one place: within the page itself.

Beware of Deadlocks

When integrating web pages with Applets, there is one major issue you must be aware of. Since the javascript engine of the browser is single threaded, you must be careful not to cause deadlocks when calling between Java and Javascript. Whenever you make a call from Java->Javascript or from Javascript->Java, it must be an asynchronous call. On the Java side, be sure not to block the calling thread.

An easy way to avoid deadlocks is to use the following recipe:

1. On all calls into Java, on the Java end, wrap the logic of the method in a SwingUtilities.invokeLater() call.
2. The method signatures of all methods in Java called from Javascript should return void. This helps to keep you honest.

If you are careful and know what you are doing, you could have some methods in Java that return a value immediately. But be careful!

Tips 'n Tricks

Now that we've covered the fundamentals of communicating between Java and Javascript, let's get into the meat and cover some of the tips 'n tricks we used to integrate our Applets with the web page.

When you run IRIS and select a photo album to view, you will notice in the toolbox an icon that allows you to edit a photo. If you click this button, you'll notice a bit of an animation effect and a seemingly non-rectangular Applet will appear, allowing you to edit the photo while the rest of the page appears dimmed. There are a couple tricks we use to get these effects.

Trick #1: How to hide/show applets

This is actually a pretty simple trick for those familiar with HTML and CSS. We put the Applet in its own <div> within the page and simply hide/show the div from the Toolbox Applet by invoking a javascript method written for this purpose.

Here is the actual Javascript method in photos.jsp which shows the Editor Applet and the semi-translucent dark "veil".

In the ToolboxApplet class:

    public void showEditor() {
        page.eval("showEditor();");
    }


In the photos.jsp page:

    function showApplet() {
        if (BrowserDetect.browser == "Firefox") {
            // fix for scrollbars painting over applet in firefox
            MM_findObj('ALBUMS').style.overflow = 'hidden';
            MM_findObj('MAINPAGE').style.overflow = 'hidden';
        }

        var dim = xDocSize();
        var w = dim.w;
        var h = dim.h;
        xHeight('editorApplet', h - 110);
        xShow('VEIL');
        xShow('EDITOR');
    }


You'll notice that we have a work-around in this code for Firefox (did I mention how much I hate cross browser HTML/CSS issues?). We also use a javascript library to hide and show named divs. This libary helped us to work around various cross browser issues. We also calculate the size of the Applet based on the available screen real estate. Finally, the Applet is shown by toggling the visibility of the div that contains it.

Trick #2: Veiling the Page

In addition to using div visibility, we also leverage z-ordering in the CSS spec in order to get the "veil" and the editor Applet to appear over top of everything else in the page. It should be as simple as that. Unfortunately, the java plugin (at least for FireFox and Internet Explorer) doesn't properly support z-ordering. No matter what layer you put the Applet in, it will "blow through" all the layers above it. For example, the black half-transparent "veil" that we put up will only paint over the toolbox Applet until the toolbox Applet causes a repaint. I'm not sure if this is a browser bug or a plugin bug, but the net effect is the same: we need to alter our design. What we decided to do was simply not to veil the Toolbox.

We hope to resolve Applet z-ordering in an update release of Java 6. Respecting the browser z-order would let us do some nifty effects much more easily.

Trick #3: Non-rectangular Applets

Unfortunately, at this time you can't have a non-rectangular Applet, just as you cannot have a non-rectangular window or dialog in Swing. This issue is being actively worked on, and targeted for release in the Java 6 Update N release. In the meantime, we need to be clever. The image below looks like a non-rectangular Applet. How did we get this effect? The editor window is actually a rectangular Applet bordered by a set of semi-transparent PNG images. The images provide the non rectangular "shape" while the Applet provides all the content.

TODO Insert image with red dashed lines indicating where the edges of the Applet are.

The Close, Previous, Next, and Perspective buttons in the top-right corner of the Applet are actually just HTML images! By using our Java-to-Javascript bridge, we were able to send commands back and forth between the page and the Applet, effectively allowing the HTML images and a smattering of Javascript to drive the editor. How cool is that?

Trick #4: D&D With Applets

Perhaps one of the more exciting features demonstrated in IRIS is the ability to drag and drop from the desktop to the web page. This actually didn't require many lines of code, but it was rather tricky to get right. There were two issues I ran into.

First, you must be careful not to cause deadlocks. The thread that handles the D&D operation appears to be the same thread used by the browser for rendering the page. Thus, when we got a drop event, I was calling into the Javascript engine and asking it to perform some task, but this immediately caused a deadlock because the Swing EDT wouldn't exit until it returned from the browser and the browser wouldn't execute until the EDT finished executing! By following the guidelines set forth earlier with regards to using SwingUtilities.invokeLater, you can avoid both these problems.

Below is a screenshot of an actual drag and drop operation. As you can see, the iris opens to accept the images.

insert screenshot

Conclusion

I've shown in this article how to include Applets in your web pages, and how to communicate between the page and your Applet. I've also shown some of the tips and tricks we used to get Applets that appear shaped. I hope this article, and the IRIS demo, will help fire your imagination and show what kinds of cool things you can accomplish with Applets. I also strongly encourage you to keep your eye out for Java 6 Update N, since it will help greatly with the deployment problems commonly associated with using Applets.

In summary, Applets: Think Again!