3D graphics for Java mobile devices, Part 1: M3G's immediate mode

Playing games on mobile devices is a fun pastime. Up until now, hardware performance has favored classic game concepts that use addictive game play, but simple graphics. Today, Tetris and Pac-Man are increasingly complemented by two-dimensional action games with extensive graphics. Consequently, the next step is to move toward 3D graphics. Sony's PlayStation Portable shows the graphics power you can put into a mobile device. Although the average mobile phone is technologically behind this specialized game machine, you can see where the market is heading. The Mobile 3D Graphics API (M3G for short), defined in Java Specification Request (JSR) 184, is an industry effort to create a standard 3D API for mobile devices that support Java programming.

M3G's API can be divided roughly into two parts: immediate and retained mode. In immediate mode, you render individual 3D objects. In retained mode, you define and display an entire world of 3D objects, including information on their appearance. You can imagine immediate mode as the low-level access to 3D functions, and retained mode as a more abstract, but also more comfortable, way of displaying 3D graphics. In this article, I'll explain the immediate mode APIs. The second part of this series shows how to use retained mode.

Alternatives to M3G

M3G is not alone. HI Corporation's Mascot Capsule API is popular in Japan, where all three major operators use it in different incarnations, and abroad. Sony Ericsson, for example, ships phones with both M3G and HI Corporation's proprietary API. Application developers report on Sony Ericsson's Web site that Mascot Capsule is a stable and fast 3D environment.

JSR 239, Java Bindings for OpenGL ES, targets similar devices to M3G. OpenGL ES is a subset of the widely known OpenGL 3D library and is becoming the de facto standard for native 3D implementations on constraint devices. JSR 239 defines a Java API that resembles OpenGL ES's C interface as much as possible, making it easy to port existing OpenGL content. As of September 2005, JSR 239 is still in early draft status. I can only speculate whether it will make any impact on mobile phones. However, while not compatible with its API, OpenGL ES did influence M3G's definition: JSR 184's expert group stipulated that it be possible to efficiently implement M3G on top of OpenGL ES. If you know OpenGL, you will recognize many M3G features.

Despite the alternatives, M3G has the support of all the major phone manufacturers and operators. Although I've mentioned games as the main attraction, M3G is a general-purpose API that you can use to create all kinds of 3D content. It will be the 3D API to use for mobile phones for years to come.

Back to top

Your first 3D object

As a first example, you'll create a cube as displayed in Figure 1.

The cube lives in what M3G defines as a right-handed coordinate system. If you take your right hand and stretch out your thumb, index finger, and middle finger so that each finger is in a right angle to the other two fingers, then the thumb is your x axis, the index finger the y axis, and the middle finger the z axis. Try to align your thumb and index finger with the x and y axis in Figure 1a; your middle finger should then point toward you. I've used eight vertices (the cube's corner points) and placed the cube's center at the coordinate system's origin.

As you can see in Figure 1, the camera that shoots the 3D scene looks toward the negative z axis, facing the cube. The camera's position and properties define what the screen later displays. Figure 1b shows a side view of the same scene so you can easily see what part of the 3D world is visible to the camera. One limiting factor is the viewing angle, which is comparable to using a camera lens: a telephoto lens has a narrower view than a wide-angle lens. The viewing angle thus determines what you can see to the sides. Unlike the real world, 3D computing gives you two more view boundaries: near and far clipping planes. Together, the viewing angle and the clipping planes define what is called the view frustum. Everything inside the view frustum is visible, everything outside is not.

This is all implemented in the VerticesSample class, whose members you can see in Listing 1.

package m3gsamples1;

import javax.microedition.lcdui.*;
import javax.microedition.m3g.*;


/**
 * Sample displaying a cube defined by eight vertices, which are connected
 * by triangles.
 *
 * @author Claus Hoefele
 */
public class VerticesSample extends Canvas implements Sample
{
 /** The cube's vertex positions (x, y, z). */
 private static final byte[] VERTEX_POSITIONS = {
   -1, -1,  1,    1, -1,  1,   -1,  1,  1,    1,  1,  1,
   -1, -1, -1,    1, -1, -1,   -1,  1, -1,    1,  1, -1
 };

 /** Indices that define how to connect the vertices to build
  * triangles. */
 private static int[] TRIANGLE_INDICES = {
   0, 1, 2, 3, 7, 1, 5, 4, 7, 6, 2, 4, 0, 1
 };

 /** The cube's vertex data. */
 private VertexBuffer _cubeVertexData;

 /** The cube's triangles defined as triangle strips. */
 private TriangleStripArray _cubeTriangles;

 /** Graphics singleton used for rendering. */
 private Graphics3D _graphics3d;

VerticesSample inherits from Canvas to be able to draw directly to the display. It also implements Sample, which I defined to help organize this article's different source code samples. VERTEX_POSITIONS defines eight vertices in the same order, as shown in Figure 1a. For example, vertex 0 is defined as position (-1, -1, 1). Remember that I placed the cube's center at the coordinate system's origin; therefore, the cube's edges will be two units long. The camera's position and viewing angle will later define the amount of pixels a unit will occupy on the screen.

The vertex positions are not enough, however; you also have to say what geometry you want to build. Like in a dot-to-dot painting, you have to connect vertices with lines until you see the resulting drawing. M3G poses one restriction though: you must build the geometry from triangles. Triangles are popular with 3D implementations because you can define any polygon as a set of triangles. A triangle is a basic drawing operation on which you can build more abstract operations.

Unfortunately, if you had to describe the cube with triangles alone, you would need 6 sides * 2 triangles * 3 vertices = 36 vertices; this would be a waste of memory as many vertices are duplicated. To reduce memory, you first separate the vertices from their triangle definitions. TRIANGLE_INDICES defines the geometry using the indices of the VERTEX_POSITIONS array and can thus reuse vertices. Next, you reduce the number of indices by using triangle strips instead of triangles. With a triangle strip, new triangles reuse the preceding triangle's last two indices. For example, the triangle strip (0, 1, 2, 3) will translate to two triangles (0, 1, 2) and (1, 2, 3). When you follow the triangle definitions in TRIANGLE_INDICES with Figure 1a, where I marked each corner with the respective index, you'll see that the triangles jump wildly between sides. This is just a pattern to avoid the definition of several strips. I managed with eight vertices and one single triangle strip with 14 indices for the cube.

You use the rest of the class members to draw the cube. Listing 2 shows their initialization.

 /**
  * Called when this sample is displayed.
  */
 public void showNotify()
 {
   init();
 }


 /**
  * Initializes the sample.
  */
 protected void init()
 {
   // Get the singleton for 3D rendering.
   _graphics3d = Graphics3D.getInstance();

   // Create vertex data.
   _cubeVertexData = new VertexBuffer();

   VertexArray vertexPositions =
       new VertexArray(VERTEX_POSITIONS.length/3, 3, 1);
   vertexPositions.set(0, VERTEX_POSITIONS.length/3, VERTEX_POSITIONS);
   _cubeVertexData.setPositions(vertexPositions, 1.0f, null);

   // Create the triangles that define the cube; the indices point to
   // vertices in VERTEX_POSITIONS.
   _cubeTriangles = new TriangleStripArray(TRIANGLE_INDICES,
       new int[] {TRIANGLE_INDICES.length});

   // Create a camera with perspective projection.
   Camera camera = new Camera();
   float aspect = (float) getWidth() / (float) getHeight();
   camera.setPerspective(30.0f, aspect, 1.0f, 1000.0f);
   Transform cameraTransform = new Transform();
   cameraTransform.postTranslate(0.0f, 0.0f, 10.0f);
   _graphics3d.setCamera(camera, cameraTransform);
 }

As a first step in init(), you obtain the graphics context for 3D drawing. Graphics3D is a singleton and _graphics3d holds a reference for later use. Next, you create a VertexBuffer to hold the vertex data. As you'll see later, you can assign several kinds of information to a vertex, and VertexBuffer contains all of them. Currently, the only information you need is vertex positions in a VertexArray that is set using _cubeVertexData.setPositions(). The constructor of VertexArray takes the number of vertices (eight), the number of components per vertex (x, y, z), and the size of each component (one byte). Because the cube is so small, one byte is enough to hold one coordinate. You can also create a VertexArray using short values (two bytes) if you want to create larger objects. However, you can't use real numbers, only integers. Next, you initialize a TriangleStripArray with the indices from TRIANGLE_INDICES.

The final part of the initialization code is the camera setup. In setPersective(), you set the viewing angle, the aspect ratio, and the clipping planes. Notice that the aspect ratio and the values for the clipping planes are float values. M3G requires floating point support, which has been available since CLDC 1.1, from the Java Virtual Machine (JVM). After setting the perspective, you move the camera away from the cube so you have a full view of the object. You do this with a translation, a topic I'll explain in more detail in the section about Transformations. For now, trust me that postTranslate() with a positive third parameter moves the camera along the z axis.

After the initialization, you render the scene to the screen. Listing 3 shows this.

 /**
  * Renders the sample on the screen.
  *
  * @param graphics the graphics object to draw on.
  */
 protected void paint(Graphics graphics)
 {
   _graphics3d.bindTarget(graphics);
   _graphics3d.clear(null);
   _graphics3d.render(_cubeVertexData, _cubeTriangles,
       new Appearance(), null);
   _graphics3d.releaseTarget();
 }

In paint(), bindTarget() assigns the Canvas's graphics context to Graphics3D. This enables rendering 3D objects until releaseTarget() is called. After a call to clear() erases the background, you draw the cube by using the vertex data and triangles created in init(). Many of Graphics3D's methods throw unchecked exceptions, but I decided to do without a try/catch block because most of the errors are unrecoverable. You can find this sample's complete source code in VerticesSample.java.

To display the sample, I wrote a simple MIDlet. You can get it from the download section together with the article's entire source code. You can see the result of running the sample in Figure 2.

It's difficult to tell that the rectangle on the screen is a cube because I placed the camera directly in front of it, which is like standing too close to a white wall. Why is it white? Because I haven't yet assigned any colors and white is the default. The next section will fix that.

Back to top

Vertex colors

When I created the VertexBuffer, I mentioned that you can assign several kinds of information to a vertex -- colors for example. Graphics hardware processes vertices in a pipeline like a factory builds cars in an assembly line. It chases vertex after vertex through the different processing steps until each one reaches the screen. In this architecture, all data for each vertex must be available at the same time. Imagine how everything would slow down if the worker that assembles the hood had to get the screws from a different place each time.

Figure 3 shows the cube's first five vertices, laid out flat and including color information in (R, G, B) format. The numbers at the corners are again the indices used in the triangle strip.

If you assign the colors to the vertices, what happens to the pixels inside the triangles? One possibility is to use one vertex color for the whole triangle; another is to interpolate the colors between two vertices and create a color ramp. M3G lets you choose between both options. In flat-shading mode, you use the color of a triangle's third vertex across the whole triangle. If you define the first triangle in Figure 3 as (0, 1, 2), the color will be red (255, 0, 0). In smooth-shading mode, each pixel inside the triangle gets its own color by interpolation. The pixels between index 0 and 2 would start with a green color (0, 255, 0) that changes slowly to red. Several triangles share the vertices at index 2 and 3, which means that they also share the color because one vertex can have exactly one color.

In Figure 3, I also indicated the order in which the indices are defined. For example, (0, 1, 2) defines the first triangle's vertices counter-clockwise; the second triangle (1, 2, 3) is defined clockwise. This is called the winding of a polygon. You use it to determine which side is the front or the back. Looking at the cube from the front, you might assume that you always see the outside, but what if you could open the box? You would want to see the inside too. Each side of the cube has two sides: a front and a back. Per default, counter-clockwise order indicates the front.

There is one little problem though: As Figure 3 shows, the winding in a triangle strip changes for each successive triangle. The convention is that a strip's first triangle defines its winding. When I wrapped one triangle strip around the whole cube in Listing 1, I started with a triangle that is wound counter-clockwise (0, 1, 2). This way, I implicitly defined the outside of the cube as the front face and the inside as the back face. Depending on your needs, you can tell M3G to render only the front faces, only the back faces, or both. The latter is useful if the cube has a half-open lid and you can see both inside and outside at the same time. If possible, you should disable faces that you can't see because it speeds up rendering. The process of excluding triangles from rendering is called culling.

Listing 4 shows how to use vertex colors.

 /** The cube's vertex colors (R, G, B). */
 private static final byte[] VERTEX_COLORS = {
   0, (byte) 255, 0,             0, (byte) 255, (byte) 255,
   (byte) 255, 0, 0,             (byte) 255, 0, (byte) 255,
   (byte) 255, (byte) 255, 0,    (byte) 255, (byte) 255, (byte) 255,
   0, 0, (byte) 128,             0, 0, (byte) 255,
 };


 /**
  * Initializes the sample.
  */
 protected void init()
 {
   // Get the singleton for 3D rendering.
   _graphics3d = Graphics3D.getInstance();

   // Create vertex data.
   _cubeVertexData = new VertexBuffer();

   VertexArray vertexPositions =
       new VertexArray(VERTEX_POSITIONS.length/3, 3, 1);
   vertexPositions.set(0, VERTEX_POSITIONS.length/3, VERTEX_POSITIONS);
   _cubeVertexData.setPositions(vertexPositions, 1.0f, null);

   VertexArray vertexColors =
       new VertexArray(VERTEX_COLORS.length/3, 3, 1);
   vertexColors.set(0, VERTEX_COLORS.length/3, VERTEX_COLORS);
   _cubeVertexData.setColors(vertexColors);

   // Create the triangles that define the cube; the indices point to
   // vertices in VERTEX_POSITIONS.
   _cubeTriangles = new TriangleStripArray(TRIANGLE_INDICES,
       new int[] {TRIANGLE_INDICES.length});

   // Define an appearance object and set the polygon mode. The
   // default values are: SHADE_SMOOTH, CULL_BACK, and WINDING_CCW.
   _cubeAppearance = new Appearance();
   _polygonMode = new PolygonMode();
   _cubeAppearance.setPolygonMode(_polygonMode);

   // Create a camera with perspective projection.
   Camera camera = new Camera();
   float aspect = (float) getWidth() / (float) getHeight();
   camera.setPerspective(30.0f, aspect, 1.0f, 1000.0f);
   Transform cameraTransform = new Transform();
   cameraTransform.postTranslate(0.0f, 0.0f, 10.0f);
   _graphics3d.setCamera(camera, cameraTransform);
 }

 /**
  * Renders the sample on the screen.
  *
  * @param graphics the graphics object to draw on.
  */
 protected void paint(Graphics graphics)
 {
   _graphics3d.bindTarget(graphics);
   _graphics3d.clear(null);
   _graphics3d.render(_cubeVertexData, _cubeTriangles,
       _cubeAppearance, null);
   _graphics3d.releaseTarget();

   drawMenu(graphics);
 }

In the class member section, I define each vertex color in VERTEX_COLORS. I put the colors into a new VertexArray in init() and assign them to the VertexBuffer with a call to setColors(). I also initialize an Appearance object called _cubeAppearance that _graphics3d.render() uses to change the cube's look. Part of _cubeAppearance is PolygonMode, which contains methods to change polygon-level attributes, including which faces are displayed. To interactively change these attributes, I added a keyPressed() method, which Listing 5 shows.

 /**
  * Handles key presses.
  *
  * @param keyCode key code.
  */
 protected void keyPressed(int keyCode)
 {
   switch (getGameAction(keyCode))
   {
     case FIRE:
       init();
       break;

     case GAME_A:
       if (_polygonMode.getShading() == PolygonMode.SHADE_FLAT)
       {
         _polygonMode.setShading(PolygonMode.SHADE_SMOOTH);
       }
       else
       {
         _polygonMode.setShading(PolygonMode.SHADE_FLAT);
       }
       break;

     case GAME_B:
       if (_polygonMode.getCulling() == PolygonMode.CULL_BACK)
       {
         _polygonMode.setCulling(PolygonMode.CULL_FRONT);
       }
       else
       {
         _polygonMode.setCulling(PolygonMode.CULL_BACK);
       }
       break;

     case GAME_C:
       if (_polygonMode.getWinding() == PolygonMode.WINDING_CCW)
       {
         _polygonMode.setWinding(PolygonMode.WINDING_CW);
       }
       else
       {
         _polygonMode.setWinding(PolygonMode.WINDING_CCW);
       }

       break;

     // no default
   }

   repaint();
 }

Pressing the respective key changes one of three properties: the shading mode (flat or smooth), the culling (whether you see the cube's outside or inside), and the winding (whether counter-clockwise triangles indicate front faces or back faces). Figure 4 shows these different options. VertexColorsSample.java is this sample's source code in its entirety.

Back to top

Transformations

At the beginning, I used a Transform object to move the camera backward so I could see the whole cube. In the same way, you can transform any 3D object.

You can mathematically express transformations as matrix operations. A vector -- for example, the camera position -- multiplied by the correct matrix for translation results in a vector that moves accordingly. A Transform object represents such a matrix. For the most common transformations, M3G provides three easy-to-use interfaces that hide the underlying mathematics:

• Transform.postScale(float sx, float sy, float sz): Scales the 3D object in the x, y, and z direction. Values greater than 1 enlarge the object by the given factor; values between 0 and 1 reduce it. Negative values will scale and mirror at the same time.
• Transform.postTranslate(float tx, float ty, float tz): Moves the 3D object by adding the given values to the x, y, and z coordinates. Negative values move the object in the direction of the negative axis.
• Transform.postRotate(float angle, float ax, float ay, float az): Rotates the object with the given angle around the axis that goes through (0, 0, 0) and (ax, ay, az). Positive values for the angle rotate the object clockwise if you look along the positive rotation axis. For example, postRotate(30, 1, 0, 0) rotates an object by 30 degrees around the x axis.

All operation names start with "post," which means that the current Transform object is multiplied from the right with the given transformation matrix -- the order of matrix operations matters. If you turn 90 degrees to the right and walk two steps, you end up in a different position than if you first walked two steps and then turned. You can achieve the above walking instructions by calling two post methods behind each other: postRotate() and postTranslate(). The call order determines which walking instruction you achieved. Because of the post-multiplication, the transformation you use last is applied first.

M3G has a Transform class and a Transformable interface. All immediate mode APIs take a Transform object as a parameter that modifies its associated 3D object. On the other hand, you use the interface Transformable to transform nodes that are part of a 3D world in retained mode. I discuss this in Part 2 of this series.

You can see the sample demonstrating transformations in Listing 6.

 /**
  * Renders the sample on the screen.
  *
  * @param graphics the graphics object to draw on.
  */
 protected void paint(Graphics graphics)
 {
   _graphics3d.bindTarget(graphics);
   _graphics3d.clear(null);
   _graphics3d.render(_cubeVertexData, _cubeTriangles,
       new Appearance(), _cubeTransform);
   _graphics3d.releaseTarget();

   drawMenu(graphics);
 }

 /**
  * Handles key presses.
  *
  * @param keyCode key code.
  */
 protected void keyPressed(int keyCode)
 {
   switch (getGameAction(keyCode))
   {
     case UP:
       transform(_transformation, TRANSFORMATION_X_AXIS, false);
       break;

     case DOWN:
       transform(_transformation, TRANSFORMATION_X_AXIS, true);
       break;

     case LEFT:
       transform(_transformation, TRANSFORMATION_Y_AXIS, false);
       break;

     case RIGHT:
       transform(_transformation, TRANSFORMATION_Y_AXIS, true);
       break;

     case GAME_A:
       transform(_transformation, TRANSFORMATION_Z_AXIS, false);
       break;

     case GAME_B:
       transform(_transformation, TRANSFORMATION_Z_AXIS, true);
       break;

     case FIRE:
       init();
       break;

     case GAME_C:
       _transformation++;
       _transformation %= 3;
       break;

     // no default
   }

   repaint();
 }

 /**
  * Transforms the cube with the given parameters.
  *
  * @param transformation transformation (rotate, translate, scale)
  * @param axis axis of translation (x, y, z)
  * @param positiveDirection true for increase, false for decreasing
  *                          value.
  */
 protected void transform(int transformation, int axis,
     boolean positiveDirection)
 {
   if (transformation == TRANSFORMATION_ROTATE)
   {
     float amount = 10.0f * (positiveDirection ? 1 : -1);

     switch (axis)
     {
       case TRANSFORMATION_X_AXIS:
         _cubeTransform.postRotate(amount, 1.0f, 0.0f, 0.0f);
         break;

       case TRANSFORMATION_Y_AXIS:
         _cubeTransform.postRotate(amount, 0.0f, 1.0f, 0.0f);
         break;

       case TRANSFORMATION_Z_AXIS:
         _cubeTransform.postRotate(amount, 0.0f, 0.0f, 1.0f);
         break;

       // no default
     }
   }
   else if (transformation == TRANSFORMATION_SCALE)
   {
     float amount = positiveDirection ? 1.2f : 0.8f;

     switch (axis)
     {
       case TRANSFORMATION_X_AXIS:
         _cubeTransform.postScale(amount, 1.0f, 1.0f);
         break;

       case TRANSFORMATION_Y_AXIS:
         _cubeTransform.postScale(1.0f, amount, 1.0f);
         break;

       case TRANSFORMATION_Z_AXIS:
         _cubeTransform.postScale(1.0f, 1.0f, amount);
         break;

       // no default
     }
   }
   else if (transformation == TRANSFORMATION_TRANSLATE)
   {
     float amount = 0.2f * (positiveDirection ? 1 : -1);

     switch (axis)
     {
       case TRANSFORMATION_X_AXIS:
         _cubeTransform.postTranslate(amount, 0.0f, 0.0f);
         break;

       case TRANSFORMATION_Y_AXIS:
         _cubeTransform.postTranslate(0.0f, amount, 0.0f);
         break;

       case TRANSFORMATION_Z_AXIS:
         _cubeTransform.postTranslate(0.0f, 0.0f, amount);
         break;

       // no default
     }
   }
 }

The paint() method now has a Transform object, _cubeTransform, as the fourth parameter in the call to _graphics3d.render(). The modified keyPressed() method contains code to change the transformation interactively using transform(). The GAME_C key switches between rotating, translating, and scaling the cube. The UP/DOWN key changes the x axis of the current transformation, LEFT/RIGHT the y axis, and GAME_A/GAME_B the z axis. Press FIRE to reset the cube to the original position. You can find the complete source code in TransformationsSample.java.

Back to top

Depth buffer and projection

Using transformations, I want to demonstrate two concepts that I've used already but haven't yet explained: projection, which defines how 3D objects map to a 2D screen, and the depth buffer, which is a way to render objects correctly according to their distance from the camera.

To see the rendered image from the camera's point of view, you have to convert the 3D world to camera space by taking the camera position and orientation into account. In the previous sample, I called Graphics3D.setCamera() with a Camera and a Transform object. You can think of the latter as the camera transformation or as the instruction to M3G on how to convert from world coordinates to camera coordinates -- both definitions are correct. Finally, three-dimensional objects are displayed on a two-dimensional screen. So far, Camera.setPerspective() told M3G to do a perspective projection when converting from a 3D to 2D space.

Perspective projection works like the real world: when you look down a long, straight road, it seems that the road's boundaries meet at a point at the horizon. Objects along the road become smaller the further the distance to the camera. You can also ignore perspective and draw all objects the same size, no matter how far away they are. This can make sense for applications such as CAD programs, where it is easier to work on drawings without perspective. To disable perspective projection, Camera.setParallel() replaces Camera.setPerspective().

In camera space, an object's z coordinate specifies its distance to the camera. If you render several 3D objects with different z coordinates, you would of course expect objects closer to the camera to obscure objects further away. By using the depth buffer, objects are rendered correctly. The depth buffer has the same width and height as the screen but holds z coordinates instead of color values. It stores the distances to the camera of all the pixels drawn on the screen. However, M3G draws a pixel only if the pixel is closer to the camera than the existing one at the same position. You can test this by comparing the incoming pixel's z coordinate with the value in the depth buffer. Thus, enabling the depth buffer renders objects according to their 3D position, independent of the order of Graphics3D.render() commands. Conversely, if you disable the depth buffer, you have to pay attention to the order in which you draw your 3D objects. You enable or disable the depth buffer test when you bind the target graphics to Graphics3D. When you use the overloaded version of bindTarget() that takes one parameter, you enable the depth buffer by default. When using bindTarget() with three parameters, you can explicitly switch the depth buffer on and off by using a boolean value as the second parameter.

You can change the two properties, depth buffer and projection, as Listing 7 shows:

 /**
  * Initializes the sample.
  */
 protected void init()
 {
   // Get the singleton for 3D rendering.
   _graphics3d = Graphics3D.getInstance();

   // Create vertex data.
   _cubeVertexData = new VertexBuffer();

   VertexArray vertexPositions =
       new VertexArray(VERTEX_POSITIONS.length/3, 3, 1);
   vertexPositions.set(0, VERTEX_POSITIONS.length/3, VERTEX_POSITIONS);
   _cubeVertexData.setPositions(vertexPositions, 1.0f, null);

   // Create the triangles that define the cube; the indices point to
   // vertices in VERTEX_POSITIONS.
   _cubeTriangles = new TriangleStripArray(TRIANGLE_INDICES,
       new int[] {TRIANGLE_INDICES.length});

   // Create parallel and perspective cameras.
   _cameraPerspective = new Camera();

   float aspect = (float) getWidth() / (float) getHeight();
   _cameraPerspective.setPerspective(30.0f, aspect, 1.0f, 1000.0f);
   _cameraTransform = new Transform();
   _cameraTransform.postTranslate(0.0f, 0.0f, 10.0f);

   _cameraParallel = new Camera();
   _cameraParallel.setParallel(5.0f, aspect, 1.0f, 1000.0f);

   _graphics3d.setCamera(_cameraPerspective, _cameraTransform);
   _isPerspective = true;

   // Enable depth buffer.
   _isDepthBufferEnabled = true;
 }


 /**
  * Renders the sample on the screen.
  *
  * @param graphics the graphics object to draw on.
  */
 protected void paint(Graphics graphics)
 {
   // Create transformation objects for the cubes.
   Transform origin = new Transform();
   Transform behindOrigin = new Transform(origin);
   behindOrigin.postTranslate(-1.0f, 0.0f, -1.0f);
   Transform inFrontOfOrigin = new Transform(origin);
   inFrontOfOrigin.postTranslate(1.0f, 0.0f, 1.0f);

   // Disable or enable depth buffering when target is bound.
   _graphics3d.bindTarget(graphics, _isDepthBufferEnabled, 0);
   _graphics3d.clear(null);

   // Draw cubes front to back. If the depth buffer is enabled,
   // they will be drawn according to their z coordinate. Otherwise,
   // according to the order of rendering.
   _cubeVertexData.setDefaultColor(0x00FF0000);
   _graphics3d.render(_cubeVertexData, _cubeTriangles,
       new Appearance(), inFrontOfOrigin);
   _cubeVertexData.setDefaultColor(0x0000FF00);
   _graphics3d.render(_cubeVertexData, _cubeTriangles,
       new Appearance(), origin);
   _cubeVertexData.setDefaultColor(0x000000FF);
   _graphics3d.render(_cubeVertexData, _cubeTriangles,
       new Appearance(), behindOrigin);

   _graphics3d.releaseTarget();

   drawMenu(graphics);
 }

 /**
  * Handles key presses.
  *
  * @param keyCode key code.
  */
 protected void keyPressed(int keyCode)
 {
   switch (getGameAction(keyCode))
   {
     case GAME_A:
       _isPerspective = !_isPerspective;
       if (_isPerspective)
       {

         _graphics3d.setCamera(_cameraPerspective, _cameraTransform);
       }
       else
       {
         _graphics3d.setCamera(_cameraParallel, _cameraTransform);
       }
       break;

     case GAME_B:
       _isDepthBufferEnabled = !_isDepthBufferEnabled;
       break;

     case FIRE:
       init();
       break;

     // no default
   }

   repaint();
 }

You use the GAME_A key to switch between perspective and parallel projection. GAME_B enables or disables the depth buffer. You can find the full source code in DepthBufferProjectionSample.java. In Figure 6, you can see the effects of the different settings.

Back to top

Lighting

In a room without light, everything appears black. It's surprising then that you could see anything in the previous samples that didn't contain light. Vertex colors and, as you'll see later, textures don't need light; they will always display in the defined color. However, light can modify them; light adds depth to a scene.

The direction of light reflected from an object depends on its alignment. If you point a flashlight to a mirror that is orthogonal in front of you, the light will reflect back to you. If the mirror is tilted, the light's entry and exit angle will be exactly the same. The general idea is that you need a direction vector that is orthogonal to the lit surface. This vector is called a normal vector or normal for short. M3G calculates the shading based on the normal, the light's location, and the camera's location.

Again, a normal is a per-vertex attribute, and the shading of pixels between vertices is either interpolated (PolygonMode.SHADE_SMOOTH) or taken from a triangle's third vertex (PolygonMode.SHADE_FLAT). Because the cube has eight vertices, one way to supply normals would be to specify vectors that point from the cube's center toward its corners, as Figure 7a shows. However, this would result in an incorrectly shaded cube. The shading color would be shared among three sides, making the edges invisible and giving the cube a round look. What would be fine for a sphere doesn't work for a cube. Figure 7b shows how you can create hard edges by using 24 normals, 4 per side. Because one vertex can only have one normal, you must duplicate the vertices as well.

After you can calculate lighting using the normals, you need to tell M3G what kind of light you want. Light comes in various forms: light bulbs, the sun, and a flashlight, among others. Their counterparts in M3G are omnidirectional, directional, and spotlight.

• Omnidirectional light originates in one point and shines equally in all directions. A light bulb without a lampshade produces such a light.
• A directional light emits parallel rays in one direction. The sun is so far away that you can consider its rays parallel. A directional light doesn't have a position, only a direction.
• A spotlight is comparable to a flashlight or a spot used in theater. Its light casts a cone shape and illuminates objects where the cone meets with the surface.

In the real world, light also bounces off of objects and illuminates others nearby. If you switch on your bedroom light, some light will reach under the bed even if the light doesn't have a direct path to it. Raytracers render phenomenally realistic images by following the path from the camera to a light source -- and they need a phenomenally long time to do so. For interactive frame rates, a simpler model must suffice: ambient light. Ambient light illuminates objects from every direction at a constant rate. You could model the previous bed scene with an ambient light to lighten up all objects to some degree and create an additional omnidirectional light.

Listing 8 demonstrates how to set different lights.

  // Create light.
   _light = new Light();
   _lightMode = LIGHT_OMNI;
   setLightMode(_light, _lightMode);
   Transform lightTransform = new Transform();
   lightTransform.postTranslate(0.0f, 0.0f, 3.0f);
   _graphics3d.resetLights();
   _graphics3d.addLight(_light, lightTransform);

 /**
  * Sets the light mode.
  *
  * @param light light to be modified.
  * @param mode light mode.
  */
 protected void setLightMode(Light light, int mode)
 {
   switch (mode)
   {
     case LIGHT_AMBIENT:
       light.setMode(Light.AMBIENT);
       light.setIntensity(2.0f);
       break;

     case LIGHT_DIRECTIONAL:
       light.setMode(Light.DIRECTIONAL);
       light.setIntensity(1.0f);
       break;

     case LIGHT_OMNI:
       light.setMode(Light.OMNI);
       light.setIntensity(2.0f);
       break;

     case LIGHT_SPOT:
       light.setMode(Light.SPOT);
       light.setSpotAngle(20.0f);
       light.setIntensity(2.0f);
       break;

     // no default
   }
 }

You can see each mode's result in Figure 8. The sample cube is lit with all four types of light. In each case, the light is white, directly in front of the camera, and faces three sides of the cube.

The omnidirectional light is strongest at the vertex facing the light and slowly fades out. The spotlight, on the other hand, produces a sharp change between light and dark, where the cone of the spot ends. If I had defined the spot with a larger cone, the result would have been similar to the omnidirectional light. The ambient light illuminates the cube from every direction. The cube looks flat because the image lacks shade. Finally, the directional light gives each side a different color. Within a side, the color stays the same because of the light's parallel rays.

The lighting is not exact; otherwise, the cone cast by the spotlight would be round. This is because of the complexity of lighting calculations, which the mobile phone's implementation simplifies. We could help the quality by adding more triangles to each side of the cube. Although the triangles wouldn't define a visible geometry, they would give M3G more control points (and more numbers to calculate).

Back to top

Materials

A light can cause different effects. A shiny silver ball reflects the light in a different way than a sheet of paper does. M3G models these material characteristics with the following attributes:

• Ambient reflection: The light that is reflected by an ambient light source.
• Diffuse reflection: The reflected light is scattered equally in all directions.
• Emissive light: An object can send out light to imitate glowing objects.
• Specular reflection: The light that reflects off objects with a shiny surface.

You can set a color for each material attribute. The shiny silver ball's diffused color would be silver and its specular component white. To get the final object color, the material's color mixes with the light's color. If you point a blue light toward the silver ball, it would turn bluish.

Listing 9 shows how to use materials:

   
  // Create appearance and the material.
   _cubeAppearance = new Appearance();
   _colorTarget = COLOR_DEFAULT;
   setMaterial(_cubeAppearance, _colorTarget);

 /**
  * Sets the material according to the given target.
  *
  * @param appearance appearance to be modified.
  * @param colorTarget target color.
  */
 protected void setMaterial(Appearance appearance, int colorTarget)
 {
   Material material = new Material();

   switch (colorTarget)
   {
     case COLOR_DEFAULT:
       break;

     case COLOR_AMBIENT:
       material.setColor(Material.AMBIENT, 0x00FF0000);
       break;


     case COLOR_DIFFUSE:
       material.setColor(Material.DIFFUSE, 0x00FF0000);
       break;

     case COLOR_EMISSIVE:
       material.setColor(Material.EMISSIVE, 0x00FF0000);
       break;

     case COLOR_SPECULAR:

       material.setColor(Material.SPECULAR, 0x00FF0000);
       material.setShininess(2);
       break;

     // no default
   }

   appearance.setMaterial(material);
 }

setMaterial() creates a new Material object and sets the colors with setColor() using the respective color component identifier. The Material object is then assigned to an Appearance object, which is used in the call to Graphics3D.render(). Although not shown here, you can use Material.setVertexColorTrackingEnable() in order to use the vertex colors for ambient and diffuse reflection instead of using Material.setColor(). Both lights and materials are implemented in LightingMaterialsSample.java. Using keys, you can combine different lights with materials to experiment with the effects.

In Figure 9, the different material characteristics are displayed using an omnidirectional light. Each screenshot sets a single color component to red to demonstrate its effect in isolation.

The ambient reflection only reacts to ambient light; thus, using an omnidirectional light has no effect. The diffuse material component creates a matte surface, while the emissive component creates a glowing effect. The specular color component emphasizes the shininess. Again, you could improve the shading quality by using more triangles.

Back to top

Textures

So far, I've shown you two ways of changing your cube's appearance: vertex colors and materials. Still, the result looks artificial. In the real world, you'd have too many details. This is where textures help. Textures are images wrapped around 3D objects like paper wrapped around a gift. You have to choose the right paper for the occasion and decide how to align it. You must make the same decisions for 3D programming.

By now, you have probably guessed that I'll introduce yet another per-vertex attribute. For each vertex, a texture coordinate defines which texture position will be used. M3G then maps the texture to fit your object. Imagine a stretchy wrapping paper that you pin to your gift at the vertices. The texture's pixels to which these coordinates refer are called texels. Figure 10 shows how you can map a square texture with 128 x 128 texels to the cube's front face.

Texture coordinates are named (s, t) in order to distinguish them from the (x, y) used for vertex positions (in literature, (u, v) is commonly used instead). Coordinates (s, t) are defined so (0, 0) is the texture's upper left corner and (1, 1) is its lower right corner. Accordingly, if you want to map the lower left corner (-1, -1) of the cube's front face to the texture's lower left corner, you must assign the texture coordinates (0, 1) to vertex 0.

Because you define texture coordinates relative to a texture's corners, images of any size have the same coordinates. M3G interpolates values between 0 and 1 to the closest texel; for example, 0.5 would refer to the middle of the texture. If a texture coordinate is outside the range of 0 to 1, M3G lets you decide what happens. The coordinates are either wrapped around (for example, a value of 1.5 is the same as 0.5) or the values are clamped, which means that any value less than 0 would be 0 and any value greater than 1 would be 1. A texture's width and height can be different, but must be powers of 2, such as 128 in Figure 1. Implementations must support texture sizes at least up to 256, which is one of the optional properties in M3G.

Graphics3D.getProperties() returns a Hashtable filled with implementation-specific properties, such as the maximum texture dimension or the maximum supported number of lights. The documentation to getProperties() contains a list of properties and their minimum requirements. Before using a feature that exceeds that value, you should first check whether your device's implementation supports it.

You can see the use of textures in Listing 10.

 /** The cube's vertex positions (x, y, z). */
 private static final byte[] VERTEX_POSITIONS = {
   -1, -1,  1,    1, -1,  1,   -1,  1,  1,    1,  1,  1, // front
    1, -1, -1,   -1, -1, -1,    1,  1, -1,   -1,  1, -1, // back
    1, -1,  1,    1, -1, -1,    1,  1,  1,    1,  1, -1, // right
   -1, -1, -1,   -1, -1,  1,   -1,  1, -1,   -1,  1,  1, // left

   -1,  1,  1,    1,  1,  1,   -1,  1, -1,    1,  1, -1, // top
   -1, -1, -1,    1, -1, -1,   -1, -1,  1,    1, -1,  1  // bottom
 };

 /** Indices that define how to connect the vertices to build
  * triangles. */
 private static final int[] TRIANGLE_INDICES = {
    0,  1,  2,  3,   // front
    4,  5,  6,  7,   // back
    8,  9, 10, 11,   // right
   12, 13, 14, 15,   // left
   16, 17, 18, 19,   // top
   20, 21, 22, 23,   // bottom
 };

 /** Lengths of triangle strips in TRIANGLE_INDICES. */
 private static int[] TRIANGLE_LENGTHS = {
   4, 4, 4, 4, 4, 4
 };

 /** File name of the texture. */
 private static final String TEXTURE_FILE = "/texture.png";

 /** The texture coordinates (s, t) that define how to map the
  * texture to the cube. */
 private static final byte[] VERTEX_TEXTURE_COORDINATES = {
   0, 1,   1, 1,   0, 0,   1, 0,   // front
   0, 1,   1, 1,   0, 0,   1, 0,   // back
   0, 1,   1, 1,   0, 0,   1, 0,   // right
   0, 1,   1, 1,   0, 0,   1, 0,   // left
   0, 1,   1, 1,   0, 0,   1, 0,   // top
   0, 1,   1, 1,   0, 0,   1, 0,   // bottom
 };

 /** First color for blending. */
 private static final int COLOR_0 = 0x000000FF;

 /** Second color for blending. */
 private static final int COLOR_1 = 0x0000FF00;

 /**
  * Initializes the sample.
  */
 protected void init()
 {
   // Get the singleton for 3D rendering.
   _graphics3d = Graphics3D.getInstance();

   // Create vertex data.
   _cubeVertexData = new VertexBuffer();

   VertexArray vertexPositions =
       new VertexArray(VERTEX_POSITIONS.length/3, 3, 1);
   vertexPositions.set(0, VERTEX_POSITIONS.length/3, VERTEX_POSITIONS);
   _cubeVertexData.setPositions(vertexPositions, 1.0f, null);

   VertexArray vertexTextureCoordinates =

       new VertexArray(VERTEX_TEXTURE_COORDINATES.length/2, 2, 1);
   vertexTextureCoordinates.set(0,
       VERTEX_TEXTURE_COORDINATES.length/2, VERTEX_TEXTURE_COORDINATES);
   _cubeVertexData.setTexCoords(0, vertexTextureCoordinates, 2.0f, null);

   // Set default color for cube.
   _cubeVertexData.setDefaultColor(COLOR_0);

   // Create the triangles that define the cube; the indices point to
   // vertices in VERTEX_POSITIONS.
   _cubeTriangles = new TriangleStripArray(TRIANGLE_INDICES,
       TRIANGLE_LENGTHS);

   // Create a camera with perspective projection.
   Camera camera = new Camera();
   float aspect = (float) getWidth() / (float) getHeight();
   camera.setPerspective(30.0f, aspect, 1.0f, 1000.0f);
   Transform cameraTransform = new Transform();
   cameraTransform.postTranslate(0.0f, 0.0f, 10.0f);
   _graphics3d.setCamera(camera, cameraTransform);

   // Rotate the cube so we can see three sides.
   _cubeTransform = new Transform();
   _cubeTransform.postRotate(20.0f, 1.0f, 0.0f, 0.0f);
   _cubeTransform.postRotate(45.0f, 0.0f, 1.0f, 0.0f);

   // Define an appearance object and set the polygon mode.
   _cubeAppearance = new Appearance();
   _polygonMode = new PolygonMode();
   _isPerspectiveCorrectionEnabled = false;
   _cubeAppearance.setPolygonMode(_polygonMode);

   try
   {
     // Load image for texture and assign it to the appearance. The
     // default values are: WRAP_REPEAT, FILTER_BASE_LEVEL/
     // FILTER_NEAREST, and FUNC_MODULATE.
     Image2D image2D = (Image2D) Loader.load(TEXTURE_FILE)[0];
     _cubeTexture = new Texture2D(image2D);
     _cubeTexture.setBlending(Texture2D.FUNC_DECAL);

     // Index 0 is used because we have only one texture.
     _cubeAppearance.setTexture(0, _cubeTexture);
   }
   catch (Exception e)
   {
     System.out.println("Error loading image " + TEXTURE_FILE);
     e.printStackTrace();
   }
 }

In init(), I add the texture coordinates that are defined as the class's static members to the VertexBuffer. As in the lighting example, I use four vertices per side of the cube and map each vertex to one corner of the texture. Note that I used a scale of 2.0 as the third parameter to _cubeVertexData.setTexCoords(). This tells M3G to multiply all texture coordinates by this value. In effect, the texture is using only a quarter of the cube's side. I do this to show you M3G's clamping and wrapping feature. If the texture is clamped, it'll only be drawn in the upper left corner. If the texture is wrapped, it'll be tiled over the whole side.

The texture itself is loaded with Loader.load() and assigned to a Texture2D object. You could also use MIDP's Image.createImage(), but the Loader class is the fastest way if you want to read a texture from a Java Archive (JAR) file. The resulting Texture2D object is then set as the texture for the cube appearance.

When using texturing, you might still want to use colors obtained by lighting or directly assigned to vertices. For this purpose, M3G supports different blending functions that are set with a call to _cubeTexture.setBlending(). In init(), I use Texture2D.FUNC_DECAL, which blends the texture with the underlying vertex color depending on an alpha value. The gray bits in the texture image in Figure 10 are 60 percent transparent. Instead of setting the vertex colors or using lighting, I set a default color for the cube with _cubeVertexData.setDefaultColor(), which means that all triangles of the cube will have the same color. With blending, you can also use multiple textures on top of each other to achieve additional effects.

I built in another optional feature of M3G. As you saw in the lighting section, the rendering quality depends on the number of triangles you use -- the smaller the distance between vertices, the better the interpolation. The same is true for textures. High quality in context to textures means that the texture maps without distortion. Textures like the one in Figure 10 are vulnerable because they contain straight lines, which make a distortion obvious. M3G provides a cheap way, in terms of processing power, to solve the problem. An optional perspective correction flag can be set with PolygonMode.setPerspectiveCorrectionEnable(), as you can see in Listing 11.

 /**
  * Checks whether perspective correction is supported.
  *
  * @return true if perspective correction is supported, false otherwise.
  */
 protected boolean isPerspectiveCorrectionSupported()
 {
   Hashtable properties = Graphics3D.getProperties();
   Boolean supportPerspectiveCorrection =
       (Boolean) properties.get("supportPerspectiveCorrection");

   return supportPerspectiveCorrection.booleanValue();
 }

 /**
  * Handles key presses.
  *
  * @param keyCode key code.
  */
 protected void keyPressed(int keyCode)
 {
   switch (getGameAction(keyCode))
   {
     case LEFT:
       _cubeTransform.postRotate(-10.0f, 0.0f, 1.0f, 0.0f);
       break;

     case RIGHT:
       _cubeTransform.postRotate(10.0f, 0.0f, 1.0f, 0.0f);
       break;

     case FIRE:
       init();
       break;

     case GAME_A:
       if (isPerspectiveCorrectionSupported())
       {
         _isPerspectiveCorrectionEnabled = !_isPerspectiveCorrectionEnabled;
         _polygonMode.setPerspectiveCorrectionEnable(
             _isPerspectiveCorrectionEnabled);
       }
       break;

     case GAME_B:
       if (_cubeTexture.getWrappingS() == Texture2D.WRAP_CLAMP)
       {
         _cubeTexture.setWrapping(Texture2D.WRAP_REPEAT,
             Texture2D.WRAP_REPEAT);
       }
       else
       {
         _cubeTexture.setWrapping(Texture2D.WRAP_CLAMP,
             Texture2D.WRAP_CLAMP);
       }
       break;

     case GAME_C:
       if (_cubeVertexData.getDefaultColor() == COLOR_0)
       {
         _cubeVertexData.setDefaultColor(COLOR_1);
       }
       else
       {
         _cubeVertexData.setDefaultColor(COLOR_0);
       }
       break;

     // no default
   }

   repaint();
 }

In the sample, isPerspectiveCorrectionSupported() checks whether the implementation supports perspective correction. If it does, you can interactively switch the flag on or off in keyPressed(). I also added options to change the way the texture maps on the cube (clamped or repeated) and an option to change the blending color. Changing the blending color demonstrates how you can easily mix colors with textures to create additional effects. Find the complete sample in TexturesSample.java.

Figure 11 shows the texture mapping with different options.

Back to top

What's next

I've covered a lot of ground in this article, including the details of creating cubes by using vertex data, how to take pictures of cubes with a camera, using lights and materials on cubes, and how to create realistic-looking cubes with textures. Many cubes.

The cubes served well to demonstrate concepts, but they aren't milestones of complex 3D design. In the introduction, I emphasized the gaming aspect of 3D graphics. Assembling vertex data manually, as done in the samples, makes detailed game worlds a daunting task. You need a way to design a 3D scene in a modeling tool and import that data into your program.

After the model is imported, you have to find a way to organize the data. With the VertexBuffer approach, you have to remember all transformations as well as the relationship between objects. An upper arm connects to a lower arm, which has a hand. You must place arm and hand relative to each other. M3G eases this task by having a scene graph API, retained mode, which you can use to model a complete world of objects and its properties. This is exactly what I'll discuss in the second part of this series.

Back to top

Download

Information about download methods

Resources

Learn

• Get to know details about the devices that support M3G on the manufacturer Web sites from Sony Ericsson, Nokia, and Motorola.
  
  
• The four-part series "J2ME 101" (developerWorks, November 2003) provides an introduction to the Java 2 Platform, Micro Edition and the Mobile Information Device Profile (MIDP).
  
  
• Learn about Java 3D in "Java 3D Joy Ride" (developerWorks, November 2001). Java 3D is a technology for the Java 2 Platform, Standard Edition.
  
  

Get products and technologies

• Download the latest specification of the Mobile 3D Graphics API for J2ME (JSR 184).
  
  
• Use Sun's Java Wireless Toolkit version 2.2 or higher to develop your own M3G applications.
  
  
• The Java Wireless Toolkit can also be embedded into the Eclipse IDE by using the EclipseME plug-in.
  
  
• NetBeans with the Mobility Pack 4.1 is another open source IDE that supports developing applications with the Java Wireless Toolkit.
  
  
• Try the Mobile 3D Viewer on alphaWorks to display 3D models on a Pocket PC device.
  
  

Discuss

• Participate in the discussion forum.
  
  
• Discuss games and 3D-related questions around Java in Sun's Java Games Forums.
  
  
• developerWorks blogs: Get involved in the developerWorks community.
  
  

About the author

Comments

Back to top

Trademarks

Table of contents

• Alternatives to M3G
• Your first 3D object
• Vertex colors
• Transformations
• Depth buffer and projection
• Lighting
• Materials
• Textures
• What's next
• Download
• Resources
• About the author
• Comments