The Java 2 user interface

How has the Java UI evolved and what's on the horizon?

Graphical and user interface capabilities have progressed in leaps and bounds since the early days of the Java language. The Java 2 platform contains a sophisticated cross-platform user interface architecture that consists of numerous high-level components, an advanced feature-rich device-independent graphics system, and a host of multimedia extensions. In this article, we'll explore this progression, examine the capabilities of the current version 1.3 in detail, and finish by looking to the future to see what release 1.4 will offer.

Matt Chapman (mchapman@uk.ibm.com), Software Engineer, IBM

Matt Chapman graduated with a BSc. in Computer Science from the University of Warwick in 1996. He now works as a software engineer at the IBM Centre for Java Technology in Hursley, UK, and has worked on Java virtual machine implementations and user interface technologies on various platforms, in particular Linux and AIX. He can be contacted at mchapman@uk.ibm.com.



24 July 2001

Prior to the release of the Java 2 platform, the Abstract Window Toolkit (AWT) was the extent of the Java platform's graphical capabilities. Various technologies, such as Swing, were introduced as optional extensions. With the Java 2 platform, most of these extensions have found their way into the core as part of the Java Foundation Classes (JFC). JFC refers to the entire set of graphical and user interface technologies included in the Java 2 platform, including AWT and Swing. In this article, we'll explore each of the major components of the JFC and then discuss some of the optional extensions.

The heart of the JFC: Swing

Swing, a GUI toolkit with a rich set of components, forms the heart of the JFC's user interface capabilities. It is both a replacement for the components the AWT provides and also a big step forward.

When integration was the priority

In the first releases of the JDK, integration with the native platform was considered a priority and so the AWT provided components that were implemented using the native components of each platform (in the Java programming vernacular, these are now known as heavyweight components). For example, on UNIX platforms the java.awt.Button class was implemented with a Motif PushButton widget.

The same Java application had a different appearance on each platform, but the intention was that the different implementations were functionally equivalent. Of course, this is where the problems start. For simple interfaces, the equivalence is true, but the model breaks down as complexity increases simply because the components are different, and they will always behave slightly differently in some situations no matter how many bugs are fixed and how many times parts of the AWT are rewritten.

The other problem that cropped up by placing a priority on integration was functionality. The AWT provided only a limited set of components because of the "lowest common denominator" approach -- a particular component or function can only be provided if it is available on every platform. A classic example is mouse buttons. Back in JDK 1.0.2, there was no way to distinguish between mouse button presses because the Macintosh had only one mouse button, and so every other platform had to behave as if it too supported only one mouse button.

As the language became more of a platform in its own right, the approach to GUIs moved toward identical appearance and behavior across all platforms. To achieve this goal, the native components have to be abandoned as much as possible. But, clearly, some native code is still required. You can't make a window appear on UNIX without X System Window calls being involved.

Enter Swing, which achieved this goal by making use of a subset of the AWT, including the basic drawing operations and the certain classes in the java.awt package: Container, Window, Panel, Dialog, and Frame.

Best of all possible approaches

Swing does not completely follow the "Java language as a platform" route. Instead, it combines the best of both approaches by offering a bridge back to the native platforms.

The mechanism for establishing this bridge is referred to as Pluggable Look-and-Feels (which is pretty close to the concept of themes, popular in the Linux community). Each Swing component has a model of its functionality and a separate appearance (the look-and-feel), which can be set in advance or changed on the fly.

Swing provides a Java look-and-feel (previously known as Metal), separate ones for the Windows and Motif platforms, and one for the Macintosh platform (as an extra option). The platform look-and-feels don't use the native components of the platform like the AWT does. Instead, they use lightweight components that are drawn to have the same appearance as the native components. This is good for functionality, but there are always some differences in look or behavior, so complex interfaces will never be identical to ones that use native components.

Furthermore, you can roll your own look-and-feel, which is a great ability to have when crafting one for highly specialized applications or when providing a corporate look-and-feel across a range of applications.

Figures 1, 2, and 3 present a screen from each of the main look-and-feels.

Figure 1. The Java look-and-feel
Figure 1. The Java look-and-feel
Figure 2. The Windows look-and-feel
Figure 2. The Windows-style look-and-feel
Figure 3. The Motif look-and-feel
Figure 3. The Motif look-and-feel

To see what's new for Swing in release 1.4, you can skip to the last section of this article, UI changes in store in 1.4.


Platform-independent drag and drop

JDK 1.1 added a general mechanism, found in the java.awt.datatransfer package, that enabled the transferring of data between and within applications, as well as the ability to manipulate the system clipboard.

The java.awt.dnd package was introduced in the Java 2 version. This package builds on the data-transfer mechanism by providing drag-and-drop facilities that can operate in a platform-independent manner within a single Java application or between two Java applications. It can also behave in a platform-dependent manner in order to integrate with the drag-and-drop facilities of the native platform.

The Drag and Drop (DND) API is quite challenging to use because it operates at a high level of abstraction to support the different ways in which it can work and because it is designed to operate on arbitrary datatypes, as specified by the java.awt.datatransfer.Transferable interface. Let's take a look at an example.

How the DND API works

First, a DragSource object is required to initiate a drag-and-drop operation. The object is used to create a DragGestureRecognizer, which specifies the events that designate the start of a drag operation, for example, holding down the left mouse button over a particular component and then moving the mouse.

Upon recognizing the gesture, the DragSource object can then be used to define the start of the operation with the information that is to be transfered.

Next, a valid destination for the drag operation is defined by passing a component to a DropTarget object or by calling the new setDropTarget() method of Component.

A class implementing the DropTargetListener interface is used to respond to the various events that can occur during a drag-and-drop operation. The main event is the drop operation, which is passed a DropTargetDropEvent object. This object can be queried as to the nature of the data being dragged.

The listener can then either reject or accept the drag and transfer the data, completing the drag-and-drop operation.


Enabling the disabled: Accessibility

The JFC Accessibility API equips Java applications so they can be accessed by users of all abilities, including people with sight-, hearing-, or dexterity-related difficulties. These might include the inability to discern visible or auditory cues or to operate a pointing device.

Two of the most important features of accessibility support are screen readers and magnifiers. Screen readers allow users to interact with a GUI by creating an off-screen representation of the interface and passing this to a speech synthesizer or a Braille terminal. Screen magnifiers provide an enlarged window of the screen, typically from 2 to 16 times the normal size. They generally keep track of pointer movements and changes in input focus and adjust the enlarged view accordingly. In addition, techniques such as font smoothing may be used to create a clearer picture.

The Java Accessibility Bridge

Some host systems, such as Microsoft Windows, provide their own accessibility features. By default, Java applications do not fully support them. For example, with native applications the screen magnifier detects when the input focus is switched to a different user interface component, such as by using the Tab key, and it adjusts the portion of the screen that is being magnified to show the component that now has the input focus.

However, Swing applications use lightweight components, which are treated as images by the operating system, instead of discrete components. This means the screen magnifier cannot track changes in input focus in the same way as with native applications.

This is exactly the problem the Java Accessibility Bridge for Windows solves. It creates a map between events relating to lightweight components and native system events. By using the Bridge, Java applications that support the Accessibility API are then fully integrated with the Windows accessibility support.

Resources at the end of this article offers links to more accessibility-related resources.


From primitive to advanced: Java 2D

Before the Java 2 platform, graphical capabilities in the language were rather primitive, limited to solid lines of single-pixel thickness; a few geometric shapes such as ovals, arcs, and polygons; and basic image-drawing functionality. All that changed with the introduction of the Java 2D API, which contains a substantial feature set.

The core of this API is provided by the java.awt.Graphics2D class, which is a subclass of java.awt.Graphics. The remainder of the API is provided by other packages within the java.awt hierarchy, including java.awt.color, java.awt.font, and java.awt.geom.

The java.awt.Graphics2D class

This class is a subclass of java.awt.Graphics, the class that provided graphical capabilities prior to the Java 2 release. The reason for this arrangement: backwards compatibility. Components are still rendered by calling their paint() method, which takes a Graphics object.

In the current version of the language, though, the object is really a Graphics2D object. This means that a paint() method can either use the Graphics object as a Graphics object (using the old drawing methods) or cast it to a Graphics2D object. If it uses the second option, then any of the additional capabilities of the 2D API can be used.

The java.awt.geom package

The java.awt.geom package provides a number of classes relating to two-dimensional geometry, such as Arc2D, Line2D, Rectangle2D, Ellipse2D, and CubicCurve2D. Each of these is an abstract class, complete with two non-abstract inner classes called Double and Float (which are subclasses of the abstract outer class).

These classes allow the various geometric shapes to be constructed with coordinates of either double or float precision. For example, newEllipse2D.Float(x,y,w,h) will construct an ellipse bounded by a rectangle of width w and height h, at position (x,y), in which x, y, w, and h are all floating-point values.

Also in this package is the AffineTransform class, which forms a core element of the 2D API. An affine transformation is one in which parallel lines remain parallel after the transformation. Examples of this type of transformation include such actions as translation, rotation, scaling, shearing, or any combination of these. Each transformation can be represented by a 3x3 matrix that specifies the mapping between source and destination points for the transformation.

Instances of the AffineTransform class can be created directly from a matrix of floating-point values, although they are more usually created by specifying one or more translation, rotation, scaling, or shearing operations. Mostly double-precision values are used, and angles are measured in radians (not degrees as used by the Arc2D class).

Text rendering

The text capabilities of the Java 2D API are impressive. They include:

  • Anti-aliasing and hinting for improved output quality
  • The ability to use all the system-installed fonts
  • The ability to apply the same operations (rotation, scaling, painting, clipping, and so on) to text as to graphic objects
  • Support for adding embedded attributes to strings (such as font, size, weight, and even images)
  • Support for bi-directional text (to enable right-to-left character runs like you would encounter in Arabic and Hebrew)
  • Primary and secondary cursors that can navigate through text containing both right-to-left and left-to-right character runs
  • Advanced font-measurement capabilities, surpassing those of the old java.awt.FontMetrics class
  • Layout capabilities to word-wrap and justify multi-line text

An example of Java 2D at work

Listing 1 contains the code for a basic example that uses some of the Java 2D methods, plus a couple of other methods. It is a complete program, but it only scratches the surface of the capabilities offered by the 2D API. (The output is shown in Figure 4.)

Listing 1. A simple Java 2D example
import java.awt.*;
import java.awt.geom.*;

public class j2d extends Frame {
    public static void main(String args[]) { new j2d(); }
    public j2d() {
        setSize(220,180);
        setVisible(true);
    }
    public void paint(Graphics g) {
        // Obtain a Graphics2D object
        Graphics2D g2 = (Graphics2D)g;

        // Set the rendering quality.
        g2.setRenderingHint(RenderingHints.KEY_ANTIALIASING,
            RenderingHints.VALUE_ANTIALIAS_ON);

        // define a linear colour gradient
        GradientPaint gp = new GradientPaint(0, 60, Color.red,
                                             0, 120, Color.yellow);
        Ellipse2D r = new Ellipse2D.Float(30, 60, 160, 60);
        g2.setPaint(gp);
        g2.fill(r);

        // set rotation
        g2.transform(AffineTransform.getRotateInstance(Math.PI/8));
        g2.setFont(new Font("Serif", Font.BOLD, 85));
        g2.setPaint(Color.blue);
        // set compositing rule with transparency
        g2.setComposite(AlphaComposite.getInstance(
                                       AlphaComposite.SRC_OVER, 0.5f));
        g2.drawString("J2D",50,70);
    }
}
Figure 4. Output of Listing 1
Output of Listing 1

For further resources on the Java 2D API, see Resources.


Sound finally catches up: Audio APIs

The audio capabilities have been steadily improving with each major release of the language. In JDK releases up to and including 1.1, there was only basic support for 8-bit mono sound, found in the java.applet package. An audio clip in AU format could be loaded using a URL and then only three operations were available: play, stop, and loop. The version 1.2 release improved that situation.

From 8 to 16 bits

The first Java 2 release, version 1.2, incorporated a new sound engine, licensed from a company called Beatnik, to provide support for 16-bit stereo sound in most common formats, including AU, WAV, AIFF, and several MIDI formats.

This version also saw the addition of a new static method, java.applet.Applet.newAudioClip(), which made it easier to use sound in applications, not just in applets. Then version 1.3 made a giant leap forward.

The Java Sound API

Version 1.3 of the platform saw the biggest step forward. Using the same Beatnik sound engine with all the previous capabilities, this version added a whole new API -- the Java Sound API.

The Java Sound API provides an extensive set of audio capabilities and is accessed through the new javax.media.sound.sampled and javax.media.sound.midi packages.

The javax.media.sound.sampled package contains classes to play back sound from a variety of sources, as well as classes to mix up to 64 channels together. Each audio signal can be controlled and processed in various ways, such as converting between file formats and applying effects like gain and reverb. Audio capture is also supported, from input sources such as microphones.

The javax.media.sound.midi package provides MIDI capabilities, which include loading instruments and generating sound from MIDI data and playing back MIDI sequences.

What's left? Hardware acceleration

There's still one significant area in which Java audio capabilities are lacking -- hardware acceleration. This lack of support means that whatever the wonderful hardware capabilities of your sound card, those abilities will not be used. This structure does provide good cross-platform portability, but at the cost of high CPU overhead and lower performance.

The JavaSound demo, included in the standard demo package, shows off many of the features of the Java Sound API. See Resources for more related information.


Automated events: AWT Robot

The java.awt.Robot class, new in version 1.3, is designed to generate input events on the native platform. Operations provided include:

  • Moving the mouse pointer to a given position on the screen
  • Generating mouse button press and release events
  • Generating keyboard press and release events

This functionality opens up whole new possibilities for test automation. For example, a sequence of events could be recorded once and then automatically played back any number of times in a fully automated way. Additional capabilities provide the means of testing the pixel color of a screen coordinate or even capturing a whole area of the screen as an image buffer.

To see what's new for AWT in release 1.4, you can skip to the last section of this article, UI changes in store in 1.4.


Multimedia options: Java Media APIs

The Java Media APIs are a set of resources covering an extensive range of multimedia technologies. Some of them, such as the 2D and sound APIs, are part of the core J2SE platform; the rest are currently optional extensions, but some of them will no doubt find their way into the core in the future. The other APIs in this area are Java 3D, Advanced Imaging, Image I/O, the Java Media Framework (JMF), and Speech.

Java 3D

The Java 3D API provides a set of object-oriented interfaces that support a simple, high-level programming model, enabling developers to build, render, and control the behavior of 3D objects and visual environments.

The API includes a detailed specification document and implementation for packages javax.media.j3d and javax.vecmath.

Advanced Imaging

Operations covered by this specification enhance a user's ability to manipulate images. It includes such operations as contrast enhancement, cropping, scaling, geometric warping, and frequency domain processing.

This type of functionality is applicable to various fields, such as astronomy, medical imaging, scientific visualization, meteorology, and photography.

Image I/O

This API defines a pluggable framework for reading and writing images of various formats. This new API is being designed through the Java Community Process.

Java Media Framework (JMF)

The JMF is an API for incorporating audio, video, and other time-based media into Java applications and applets. This optional package extends the multimedia capabilities of the J2SE platform.

Speech

The Java Speech API allows developers to incorporate speech technology into user interfaces for Java applets and applications. The API specifies a cross-platform interface to support command-and-control recognizers, dictation systems, and speech synthesizers.

This blanket API is divided into several specifications:

  • Java Speech API Specification (JSAPI)
  • Java Speech API Programmer's Guide
  • Java Speech API Grammar Format Specification (JSGF)
  • Java Speech API Markup Language Specification (JSML)

There is no Sun reference implementation for this API, but there are numerous third-party implementations, including Speech for Java (available from IBM alphaWorks), which uses ViaVoice to support voice-command recognition, dictation, and text-to-speech synthesis.

Listing 2 illustrates the synthesis aspect of the Java Speech API. It also uses the JSML to control the way in which the synthesizer speaks the text (defining the pitch, speed, emphasis, and so on).

Listing 2. Synthesis and JSML
import javax.speech.synthesis.*;\

public class TrySpeech {
    public static void main(String args[]) {
        Synthesizer synthesizer = Central.createSynthesizer(null);
        synthesizer.allocate();
        try {
            synthesizer.speak("Hello I am a computer", null);
            synthesizer.speak(
                "Hello I am a <EMP>computer</EMP>", null);
            synthesizer.speak(
                "Hello I am a <PROS PITCH="+10%">computer</PROS>",
                null);
        } catch (Exception e) { e.printStackTrace(); }
    }
}

See Resources for additional information on the Speech APIs.


UI changes in store in 1.4

The upcoming release, version 1.4, code-named Merlin, is going to present many new features along with the obligatory bug fixes. We'll highlight some of those changes.

What's new with Swing

The Merlin release will bring extensive improvements to the Swing toolkit. The new functionality includes:

  • An indeterminate progress bar that will indicate action to the user without quantifying the degree of completeness. These components will be useful when you don't know in advance how long something is going to take (establishing a network connection, for example).
  • A JSpinner component that lets the user select from an ordered list using arrow buttons to scroll through the list.
  • A JFormattedTextField component that lets developers restrict the set of characters allowed in a text component. This function is great for inputting dates and numbers.
  • New JTabbedPanes, which are enhanced to allow for scrollable tabs. Currently the tabs spill over onto multiple rows if they cannot all fit onto one row, which can be very awkward for the user because the pane has to rearrange the tabs when one on another row is selected. The alternative used here is to keep all the tabs on one row, but provide arrow buttons to scroll along the row.
  • Enhancements to the Windows look-and-feel to make it match the native platform more closely. These enhancements include cosmetic changes to match Windows 2000, such as gradient fills in title bars, and better integration with the Windows desktop to respond to changes the user makes to the desktop settings.
  • A large number of bug fixes and minor enhancements, including the capability of attaching sounds to Swing events.

AWT gets a new focus model

In release 1.4 the AWT is being given a completely new focus model, which has long been a problem area. The existing focus model has been plagued by countless bugs and greatly lacks functionality, such as not being able to query which component currently has the focus. The problems are generally caused by the differences in the way different platforms handle input focus, as well as by the complication of new lightweight components that require separate focus handling.

As much of the new model as possible has been implemented with shared code; the platform-dependent code has been minimized. The heart of the new model is the class java.awt.KeyboardFocusManager. This class will provide the public API calls to query and set numerous aspects of the focus state, such as which component currently has the focus and the order of components in a focus traversal cycle.

One final note: The new focus model is not completely backwards-compatible with current versions, so existing applications will need to be thoroughly tested.

Other AWT features

Mice with a scroll wheel as a middle mouse button are fairly popular, so support is being added. It takes the form of a new MouseWheelEvent class, a MouseWheelListener interface, and a new addMouseWheelListener() method to the java.awt.Component class (and the corresponding removeMouseWheelListener() method).

Support is being added for Headless Java. This addition means that Java applications running on a server without a GUI environment will still be able to use the AWT, for example, to create images in memory for sending out over the network.

Finally, other changes include enhancements to the drag-and-drop functionality, the ability to create frames without decorations, and numerous bug fixes.

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Java technology on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Java technology
ArticleID=10567
ArticleTitle=The Java 2 user interface
publish-date=07242001