3D development with WebGL, Part 3: Add user interaction

Creating 3D apps and data visualizations

The WebGL API gives JavaScript developers the ability to tap directly into the powerful built-in 3D graphics acceleration capabilities of today's PC and mobile-device hardware. Supported transparently in modern browsers, WebGL makes it possible to create high-performance 3D games, applications, and 3D-enhanced UIs for mainstream web users. This article concludes a three-part series for JavaScript developers who are new to WebGL. Follow along as series author Sing Li guides you through the development of a complete 3D game application and a prototype for a data-visualization UI.

Share:

Sing Li (westmakaha@yahoo.com), Consultant, Makawave

Sing LiSing Li has been a developerWorks author since the site's inception, writing articles and tutorials that cover a variety of web and Java topics. He has more than two decades of system engineering experience, starting with embedded systems, crossing over to scalable enterprise systems, and now back full circle with web-scale mobile-enabled services and "Internet of things" ecosystems.



04 February 2014

Also available in Chinese Japanese

Hardware-accelerated 3D performance is now at the fingertips of every JavaScript developer, thanks to ubiquitous support for WebGL in desktop and mobile browsers. In Part 2 of this WebGL series, you experimented with two high-level WebGL libraries: Three.js and SceneJS. You observed how these libraries encapsulate the complexity of raw WebGL development and enable rapid 3D development through simple, conceptually pure APIs. You learned how to:

  • Rotate a pyramid in 10 lines of Three.js code, instead of 100 lines of raw WebGL
  • Leverage object-oriented design via Three.js classes, rapidly clone 3D objects, and create scenes that display multiple objects in motion
  • Use Three.js to learn fundamental 3D concepts, including scene graphs, meshes, materials, lighting, and camera placement
  • Animate wireframe objects in a scene
  • Program a scene that combines multiple motions — motion intrinsic to an object and motion of an entire group of objects
  • Create a dramatic scene through the use of lighting and shadow effects
  • Increase the realism of rendered objects via texture mapping using photo-realistic textures
  • Generate arbitrary irregularly shaped 3D geometries using the Three.js shape geometry API
  • Work with off-center rotation of 3D objects
  • Obtain complex ready-to-render prefabricated 3D meshes and textures from online 3D warehouses/repositories and include them in your 3D scenes
  • Plan, stage, and code animated fly-through visualization of multi-room 3D spaces using tweening, enhanced with easing
  • Create complex 3D scene graphs easily using the SceneJS library and contrast SceneJS's design with the Three.js library

You're now ready to put what you've learned so far into practice by creating 3D applications and data visualizations. This final article of the series introduces 3D user-interaction concepts and techniques, then guides you through the development of two 3D applications: a complete 3D tic-tac-toe game and an interactive data-visualization UI. See Download to get the sample code.

By the end of this article, you'll have more than enough knowledge to apply WebGL and tackle 3D requirements in your upcoming web development projects.

User interaction with 3D scenes

Thus far in the series, you've built scenes with animated 3D objects and moving cameras (fly-through or walk-through), but all with preprogrammed motions that the user doesn't influence. The user experience has been similar to watching a video. In practice, many 3D applications — games and data visualizations among many others — require interaction with users.

With a high-level WebGL library such as Three.js, adding user interaction can be straightforward. Load up imatlight.html in a browser. This is the lighting-and-shadow-effect example from Part 2 (matlight.html) with 3D user control of camera position added. Figure 1 shows the scene rendered by imatlight.html.

Figure 1. 3D scene that users can interact with (in Safari on OS X)
Image shows 3D scene that users can interact with (in Safari on OS X)

Start interacting with the scene. Using the mouse, you can:

  • Pan the scene horizontally or vertically: Click and hold the right mouse button while moving the mouse.
  • Move into or away from the scene to show more or less detail: Click and hold the mouse wheel (or center) button while moving the mouse. You can also rotate the mouse wheel for the same effect.
  • Orbit around the scene while maintaining the current distance from it: Click and hold the left mouse button while moving the mouse.

With these motions and some practice, you should be able to navigate around a 3D-rendered scene effectively and quickly.

Figure 2 shows imatlight.html after some interactions have occurred. You see the Figure 1 scene from an entirely different angle.

Figure 2. Another perspective on imatlight.html after user interaction (in Chrome on OS X)
Image shows another perspective on imatlight.html after user interaction (in Chrome on OS X)

Listing 1 shows the code for imatlight.html. The highlighted lines are the code added to matlight.html (from Part 2) to incorporate user interaction into the scene.

Listing 1. Interaction with a 3D scene (imatlight.html)
<!doctype html>
<html>
<head>
<title>developerWorks WebGL Three.js Interactive Lights and Shadows Effect Example</title>
  <script src="Three.js" ></script>
  <script src="js/controls/OrbitControls.js"></script>

  <script type="text/javascript">

  function draw3D()  {
    var controls;

    function animate() {
      requestAnimationFrame(animate);

      pyramid1.rotateY(Math.PI/180);
      sphere.rotateY(Math.PI/180);
      cube.rotateY(Math.PI/180);
      multi.rotateY(Math.PI/480);
      renderer.render(scene, camera);
    }
    function updateControls() {
      controls.update();
    }

    var geo = new THREE.CylinderGeometry(0,2,2,4,1, true);
    var pyramid1 = new THREE.Mesh(geo, new THREE.MeshPhongMaterial({color: 0xff0000}));
    pyramid1.position.set(-2.5, -1, 0);

    geo = new THREE.SphereGeometry(1, 25, 25);
    var sphere = new THREE.Mesh(geo, new THREE.MeshPhongMaterial({color: 0x00ff00}));
    sphere.position.set(2.5, -1, 0);

    geo = new THREE.CubeGeometry(2,2,2);
    var cube = new THREE.Mesh(geo,new THREE.MeshPhongMaterial({color: 0x0000ff })   );
    cube.position.set(0, 1, 0);

    var camera = new THREE.PerspectiveCamera(  45, 1024/500,0.1, 100);
    camera.position.z = 10;
    camera.position.y = 1;

    controls = new THREE.OrbitControls( camera );
    controls.addEventListener( 'change', updateControls );

    var multi = new THREE.Object3D();
    pyramid1.castShadow = true; sphere.castShadow = true;
    multi.add(cube);
    multi.add(pyramid1);
    multi.add(sphere);
    multi.position.z = 0;

    geo = new THREE.PlaneGeometry(20, 25);
    var floor = new THREE.Mesh(geo, new THREE.MeshBasicMaterial({color : 0xcfcfcf}));
    floor.material.side = THREE.DoubleSide;
    floor.rotation.x = Math.PI/2;
    floor.position.y = -2;
    floor.receiveShadow = true;

    var light = new THREE.DirectionalLight(0xe0e0e0);
    light.position.set(5,2,5).normalize();
    light.castShadow = true;
    light.shadowDarkness = 0.5;
    light.shadowCameraRight = 5;
    light.shadowCameraLeft = -5;
    light.shadowCameraTop = 5;
    light.shadowCameraBottom = -5;
    light.shadowCameraNear = 2;
    light.shadowCameraFar = 100;

    var scene = new THREE.Scene();
    scene.add(floor);
    scene.add(multi);
    scene.add(light);
    scene.add(new THREE.AmbientLight(0x101010));

    var div = document.getElementById("shapecanvas2");

    var renderer = new THREE.WebGLRenderer();
    renderer.setSize(1024,500);
    renderer.setClearColor(0x000000, 1);
    renderer.shadowMapEnabled = true;
    div.appendChild( renderer.domElement );
    animate();

  }

</script>

</head>
<body onload="draw3D();">

  <span id="shapecanvas2" style="border: none;" width="1024" height="500"></span>

  <br/>
  </body>

</html>

OrbitControls API for Three.js

Optional Three.js control APIs

The Three.js code distribution includes myriad alternative user input/output support APIs for 3D hardware and use cases, all in the examples/js/controls source directory. TrackballControls.js supports user interaction via a trackball. FirstPersonControl.js supports a user-interaction pattern familiar to many first-person view (FPV) game players. FlyControl.js supports a flight-simulator-style camera control supporting roll and pitch. OculusControls.js supports user interaction via an in-development (and highly anticipated) precision consumer head-tracking immersive virtual-reality device, the Oculus Rift.

In Three.js, mouse interaction is supported via the OrbitControls.js API, which is located in the examples/js/controls directory of the Three.js source tree. Because not all 3D applications require user interaction, OrbitControls and other APIs for interacting with several other hardware devices are optional libraries (see the Optional Three.js control APIs sidebar).

OrbitControls works by moving the location of the camera within the 3D scene in unison with mouse input. The following two lines of code instantiate the control and parameterize it with the camera from the 3D scene:

controls = new THREE.OrbitControls( camera );
controls.addEventListener( 'change', updateControls );

The OrbitControls change listener makes a callback on the updateControls() function, which is defined as:

function updateControls() {
  controls.update();
    }

In the imatlight.html 3D scene, the animate() callback function is already updating the object rotations upon screen refreshes via an rAF hook; this is why updateControls() calls nothing other than controls.update(). If the scene being rendered is static, rAF is not hooked, and rendering occurs only when control changes are detected. In such cases, the updateControls() function should also call the renderer's render function to update the scene.


Designing a 3D game

The next project you'll tackle is a fully playable 3D game that requires user interaction: 3D tic-tac-toe. Two opposing players are represented by different-colored playing pieces. One player's pieces are red, and the other player's are green. The players take turns placing their pieces inside a 3x3x3 cubic "cage." The first player with three pieces lined up in a row, along any dimension, wins. The code presented here implements a computer player (who controls the red pieces) that you can play against. The core "engine" code is independent and can be adapted for a human-vs.-human game, perhaps implemented over a network.

Figure 3 shows the 3D cube in which the game is played. You can see 3x3x3=27 positions where playing pieces can be placed, all of which are occupied by white spheres.

Figure 3. The 3D tic-tac-toe playing arena (in Firefox on OS X)
Image shows the 3D tic-tac-toe playing arena (in Firefox on OS X)

As the players orbit the "cage" and ponder the next move, each available position that the mouse passes over turns to yellow. The player can then click the selected sphere to commit the move, and the sphere changes to that player's color.

Text displayed on the screen indicates whose move it is and when a player wins. Figure 4 shows a winning combination for the computer-based red player and the resulting on-screen display. After a win, the next click anywhere on the screen resets the game.

Figure 4. Red player's victory (in Chrome on OS X)
Image shows red player's victory (in Chrome on OS X)

Load up tictacthreed.html and try a game for yourself. You can orbit around the playing cage using your mouse as you did in imatlight.html. (Both pages use the Three.js OrbitControls API.) The computer player is not particularly smart, and you can readily win a game. When the game is over, click anywhere to start a new one. The loser of the previous game gets the first move in the new game.

After you have a chance to try a few games, check out the source code for tictacthreed.html (see Download). The code in Listing 2 is the portion of tictacthreed.html that creates the 3D playing cage for the pieces.

Listing 2. Creating the 3D playing cage
var base = new THREE.Geometry();
for (var z=-1; z<1; z++ ) {
  base.vertices.push(
    new THREE.Vector3(0, 0 ,z), new THREE.Vector3( 3, 0, z ),
    new THREE.Vector3(0, 1 ,z), new THREE.Vector3( 3, 1, z ),
    new THREE.Vector3(1, 2 ,z), new THREE.Vector3( 1, -1, z ),
    new THREE.Vector3(2, 2 ,z), new THREE.Vector3( 2, -1, z )
  );
}
for (var x=1; x<3; x++ ) {
  base.vertices.push(
    new THREE.Vector3(x, 1 ,1), new THREE.Vector3( x, 1, -2 ),
    new THREE.Vector3(x, 0, 1), new THREE.Vector3( x, 0, -2 )
     );
}
var  cage = new THREE.Line(base, new THREE.LineBasicMaterial(), THREE.LinePieces );
cage.position.set(-1.5,-0.5, 0.5);

A total of 12 intersecting lines form the cage. The first four are on the x-y plane. The end coordinates for the second set of four are the same as those for the first set, except that the z component is -1 instead of 0. The last four lines consist of two sets that have the same end coordinates except for the x value. The THREE.Line constructor is used to create the cage, consisting of line "pieces" (line segments that are not connected to one another). After construction, the completed cage is translated so its center is at origin (0,0,0).

Generating the spheres marking the playing positions

To generate the white spheres that mark the possible playing positions, tictacthreed.html uses the iterative code shown in Listing 3.

Listing 3. Generating the white spheres
var geo = new THREE.SphereGeometry(0.3, 25, 25);
var range = [-1, 0, 1];
var idx = 0;
    range.forEach(function(x) {
    range.forEach(function(y) {
    range.forEach(function(z) {
var tempS = new THREE.Mesh(geo, new THREE.MeshPhongMaterial({color: 0xffffff}));
    tempS.ID = idx++;
    tempS.claim = UNCLAIMED;
    pos.push(tempS);
    tempS.position.set(x, y, z);
    scene.add(tempS);
    })
  )

});

Implementation of the computer player

The computer player is implemented through the redComputerMove() function. Whenever it's the red player's turn, redComputerMove() is called to make the move. This function first scans all the winning combinations in the wins array to determine if it can win with its next move. If it can't, it rescans the combinations to determine if it must block a green player's impending win (because green has claimed two positions in a winning combination and the remaining one is unclaimed). The scan of winning combinations is assisted by the countClaim() helper function. If redComputerMove() can't win and doesn't need to block, it determines the next move by traversing either a preferred positions array or taking the first available unclaimed position. Under this strategy, the computer plays "reasonably" but doesn't win every game. You can certainly improve on the playing strategy.

The spheres all have a diameter of 0.6 (radius of 0.3) so users can see them through the cage. The code in Listing 3 creates and places white spheres into all 27 playable positions. Note that although all 27 spheres use the same geometry object (named geo) for mesh construction, each has a separate instance of THREE.Material. This is necessary because the code will later change the color of each sphere individually. If all the mesh instances referenced the same material, all spheres would change color simultaneously.

The Listing 3 code also builds up an array named pos that references the 27 meshes, with one entry for each sphere. The winner-determination algorithm uses the pos array to check if a player has won (and to reset the game). The computer player's code also uses the pos array extensively to determine if the computer player is currently under threat or should make an offensive move.

Each of the sphere mesh objects has an additional attribute named claim that is initialized to UNCLAIMED. This attribute changes to RED or GREEN when a user makes a committed move at the associated playable position/sphere.

Figure 5 shows the playing-position numbering scheme that's generated by the code in Listing 3. Each number represents the index of that playing position (sphere mesh) in the generated pos array. The winner-determination algorithm uses sets of these indices to determine if a player has won.

Figure 5. Playing position numbering scheme
Image shows playing position numbering scheme

Determining a winner

The game has 49 possible winning combinations of three positions each. You can enumerate them manually by examining Figure 5.

In tictacthreed.html, the wins array contains an enumeration of all winning combinations. To determine if a player has won, the checkWin(playerColor) function goes through each combination and sees if all three spheres in the combination have the player's color. The winner is determined by examining the claim attribute of each sphere in the combination; this attribute is set to a player's color when a player clicks the selected sphere. Listing 4 shows the code for checkWin().

Listing 4. The checkWin() function
function checkWin(color) {
    var won = false;
    var breakEx = {};
    try {
     wins.forEach( function(wincomb) {
      var count = 0;
      wincomb.forEach( function (idx) {
        if (pos[idx].claim == color)
           count++;
      })
      if (count === 3) {
        won = true;
        throw breakEx;

      }

    })
   } catch (ex) {
    if (ex != breakEx) throw ex;

   }
    return won;

}

In the highlighted code in Listing 4, checkWin() short-circuits the forEach() loop by throwing an exception after a winner has been determined and returns true status to the caller.

Picking objects in a 3D scene using a 2D mouse

Another important user-interaction technique for 3D is object picking, which is the selection of objects within the 3D scene. In the tic-tac-toe game, the input device is a 2D mouse. The user actually clicks on the canvas where the 3D scene is rendered. Because the rendering changes when the user orbits around the scene, the 2D coordinates of the mouse must be mapped dynamically (upon a mouse click) into the 3D coordinate space of the scene to determine which object(s) is being selected.

In 2D graphics, mouse selection is performed via hit testing. Object picking is a form of hit testing in 3D. Three.js simplifies object picking by offering a projector helper class that can transition from a 2D canvas (x,y) point into the 3D world of the scene, taking account of the properties of the current camera (direction at which it is pointing, perspective, and so on).

Three.js also has a RayCaster class that can cast a ray into the 3D scene and determine if the ray intersects with a specified collection of 3D objects in the scene.

In the tic-tac-toe game, hit testing is performed during screen update. The mouse-move event listener saves the x and y coordinates of the mouse into a global variable.

During hit tests, the first object intersecting with the Raycaster is highlighted in yellow (RGB Hex 0xffd700) to tell the user that the position is playable. Listing 5 shows the code in tictacthreed.html that performs this hit testing.

Listing 5. Hit testing
function updateControls() {

var vector = new THREE.Vector3( mouse.x, mouse.y, 0.5 );
    projector.unprojectVector( vector, camera );
var ray = new THREE.Raycaster( camera.position,
    vector.sub( camera.position ).normalize() );
var hits = ray.intersectObjects( pos );

    if (mouse.clicked)  {
     ...
    }
    else { /* mouse move */
     if ( hits.length > 0 )
     {
        ...
     }
     else
     {
      ...;

     }
   }

}

Graphical sprites

Graphical sprites are basic elements used in 2D graphic animation. They are typically rectangular blocks of pre-rendered graphics optimized for fast display and animation on a 2D accelerated hardware platform. In the context of 3D rendering, sprite refers to a rectangular flat planar 2D geometry containing 2D drawing/graphics — such as a 2D text label in a 3D scene

After hit testing, the hits variable contains a list of spheres within the scene (from pos) that the THREE.RayCaster has found intersecting. If the list isn't empty, the first object in the array returned is the object first intersecting with the cast ray (the "topmost" sphere).

When a user clicks a playable position, the sphere at that position changes to the player's color. The sphere's claim attribute is also updated to reflect the player's move. Listing 6 shows the relevant portion of tictacthreed.html.

Listing 6. Updating the spheres' attributes
function updateControls() {

      var vector = new THREE.Vector3( mouse.x, mouse.y, 0.5 );
       projector.unprojectVector( vector, camera );
      var ray = new THREE.Raycaster( camera.position,
            vector.sub( camera.position ).normalize() );
      var hits = ray.intersectObjects( pos );
      if (mouse.clicked)  {
      if ( hits.length > 0 )
      {

        hits[0].object.material.color.setHex((currentMOVE == RED) ? 0xff0000 : 0x00ff00);
        hits[0].object.claim = currentMOVE;
        updateWin(currentMOVE);
      
}

      mouse.clicked  = false;
    }
    else {
    ...
    }

2D on-screen status displays over 3D scenes

Billboarding in 3D

To display text labels inside a 3D scene instead of as an overlay, you can use billboarding with Three.js. Billboarding is a technique used in 3D rendering for signage or graphical sprites that always face the viewer no matter where the camera is placed. Billboarding is typically used to label in-scene objects within 3D visualizations — for example, the organs of a 3D-rendered human body — to ensure that the viewer can read the label while navigating around the scene.

As you know, all 3D scenes are rendered as a sequence of 2D images within an HTML5 canvas element. To achieve the semitransparent 2D text overlay effect for on-screen status displays, you can create a CSS3-styled DOM element (such as a <div> or <span>) with semitransparent text and place it on top of the canvas element in which the 3D scene is rendered.

For the game's on-screen status display, tictacthreed.html uses CSS and HTML DOM manipulations and contains no WebGL calls. Listing 7 shows the relevant code.

Listing 7. Status-display code
var gamestatus = document.createElement('div');
gamestatus.setAttribute("id", "status");
gamestatus.style.cssText =
  "position:absolute;width:300;height:200;color:rgba(255,255,255,0.3);" +
  "background-color:rgba(0,0,0,0);top:10px;" +
  "left:20px;font:bold 48px arial,sans-serif";
gamestatus.innerHTML = "Move: RED";
div.appendChild(gamestatus);

Listing 7 shows how the stylized <div> is programmatically created and added to the rendering canvas. An id attribute is also added to the <div> element, giving it an ID of status.

Now you can update the on-screen status at any time by modifying the <div>'s innerHTML attribute. The typical code used within the game to update the on-screen status is:

document.getElementById("status").innerHTML = "new status message";

Resetting the game

When a winner is determined, the global gameWon variable is set to true. Subsequent mouse clicks while the game is in this state are short-circuited to call the resetGame() function, shown in Listing 8.

Listing 8. Code to reset the game
function resetGame() {
    pos.forEach( function(position) {
    position.claim = UNCLAIMED;
    position.material.color.setHex(0xffffff);
    document.getElementById("status").innerHTML =
                 ((currentMOVE == RED) ? "Move: GREEN" : "Move: RED");
    currentMOVE = ((currentMOVE == RED) ? GREEN: RED);

  });
 }
  ...

function updateControls() {
    ...
    if (mouse.clicked)  {
      if (gameWON) {
        resetGame();
        gameWON = false;
        mouse.clicked = false;
        return;
      }
      if ( hits.length > 0 )
      {
       ...

The resetGame() function changes all the playable positions to UNCLAIMED again and reverts all the spheres' colors to white. It also updates the on-screen display and gives the loser of the game the first move in the new game.

Now it's time to move on to the next example and try creating a WebGL application using the SceneJS framework.


Prototyping a 3D big data navigation UI

Most of the familiar data-navigation interfaces that you work with on the web or on your mobile device are 2D. Every user knows how to work with UI elements like scrolling lists, drop-down combo boxes, and toggle buttons. GUIs have changed little during the past few decades in this respect. Now, with the availability of ubiquitous 3D, you can experiment with designing new 3D-centric data-navigation UIs that are more natural for the application at hand.

This next example explores the use case of navigating through a large data source across a number of data axes. The prototype UI enables at-a-glance navigation over 10,000 consolidated data points — each point representing monthly summary metrics over 100 months (almost 9 years) at each of 100 data centers. Figure 6 shows this prototype, sjbdvis.html (see Download), in action.

Figure 6. 3D UI prototype for navigating a big dataset (in Safari on OSX)
Image shows 3D UI prototype for navigating a big dataset (in Safari on OSX)

In Figure 6, the UI comprises two axes on the x-z plane. On one axis lie the names of data center locations (ranging from AA to AZ, BA to BZ, CA to CZ, DA to DV) — 100 locations. On the other axis is the date of the consolidated metrics, ranging from January 2004 to April 2012 — 100 months.

Rapid navigation over 10,000 consolidated data points

A 3D rectangular bar is rendered for each (month, data center) combination — a total of 10,000 bars. The value of each bar can range from 1 to 100.

Two thresholds are set that determine the color of the bar rendered. A bar is rendered in green unless its value exceeds the HighThreshold. Any bar with value between HighThreshold and PeakThreshold is rendered in yellow. Bars with values higher than PeakThreshold are rendered in red. You can imagine how the PeakThreshold red bars might indicate an exceptional condition that merits further examination.

The ability to visualize the relative values (heights) of adjacent months (bars) can help to identify certain trends or conditions that numeric reports, 2D graphs, or heat maps cannot possibly convey. The ability to visualize the relative value differences between data center locations can also be helpful during analysis and investigation.

Using the mouse, you can orbit around the display in 3D (just as you did with imatlight.html) to take a closer look at any part of the dataset.

As shown in Figure 7, when you move the mouse over the bars in sjbdvis.html, the large superimposed on-screen display provides feedback on the current month and year, and the name of the data center selected.

Figure 7. sjbdvis.html with on-screen feedback display (in Chrome on Windows)
Image shows sjbdvis.html with on-screen feedback display (in Chrome on Windows)

When you locate a problematic data point requiring further investigation, you can click it to fetch the detailed daily report for that month. (In this prototype, no actual data is available.) A pop-up window indicates the month, year, and data center that the reports should be fetched for, as shown in Figure 8.

Figure 8. Clicking on a data bar in sjbdvis.html (in Firefox on Windows)
Image shows clicking on a data bar in sjbdvis.html (in Firefox on Windows)

The highlighted code in Listing 9 shows how the dataset (simulating 10,000 consolidated data points) is generated randomly for the prototype.

Listing 9. Generating the dataset
<!DOCTYPE html>
<html lang="en">
<head>
    <title>developerWorks WebGL SceneJS Big Data Visualization Example</title>
    <meta charset="utf-8">
    <script src="./scenejs.js"></script>
<script>
  var mouse = {x: 0, y: 0, updated: false, clicked: false};
  var monthLabels =
   ["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"];
  var yearLabels =
   ["2004","2005","2006","2007","2008","2009","2010","2011","2012","2013"];
  var PeakThreshold = 97;
  var HighThreshold = 93;

   var data = [];
   var node2Data = {};

   function initData() {
     for (var dim=0; dim<100; dim++)  {
        var axis = [];
        var datacenter = String.fromCharCode(65 + Math.floor(dim /26)) +
            String.fromCharCode(65 + (dim % 26));
        for (var i=0; i<100; i++) {
            var val = Math.floor(Math.random() * 100 + 1);
            var dpoint = { 
               location: datacenter,
               year: yearLabels[ Math.floor(i /12)],
               month: monthLabels[ i % 12],
               value: val};

            axis.push(dpoint);
        }
        data.push(axis);

     }
   }
...

The data is created as an array of JavaScript objects named data[]. Each of the array elements has the following structure:

{
   location: datacenter location,
   year: year,
   month: month,
   value: value of metrics
}

In production, this data can be fetched from a back-end data source.

In sjbdvis.html, the on-screen display information is dynamically updated as you work with the UI. The display consists of semitransparent 2D text that's superimposed over the 3D scene, created using the same technique shown in the preceding example.

SceneJS for large data-driven visualizations

sjbdvis.html is built using SceneJS. As you know from Part 2 of this series, SceneJS can render highly complex scenes and is specialized to handle data-driven applications. Meshes to be rendered can be defined in a SceneJS scene graph (tree) of JSON-compatible JavaScript object nodes, then parsed and rendered by SceneJS. After the scene graph is parsed, IDs associated with each mesh can be used to access and manipulate it, in a style similar to dynamic HTML5 programming. This aspect of SceneJS makes it suitable for big data visualization applications.

The "core" scene graph, coreScene, is statically defined and contains a simple directional light that points from the upper left toward the rendered meshes. The light's id is lights1. This ID enables the node to be accessed directly from code later. Listing 10 shows the code for coreScene.

Listing 10. coreScene definition
var coreScene = {
     nodes:[
         {
             id: "light1",
             type:"lights",
             lights:[
                         {
                             mode:"dir",
                             color:{ r:1.0, g:1.0, b:1.0 },
                             diffuse:true,
                             specular:true,
                             dir:{ x:-5, y:-2, z:-5 },
                             space:"view"
                         }
             ]

         }
     ]
};;

Programmatically creating a 10,000-bar scene

The final scene of 10,000 bars is created based on the empty coreScene by programmatically creating and inserting child nodes under light1.

First the light1 node is located using the asynchronous getNode() method call of the scene. When the light1 node is located, the bars are created one at a time and added to light1 as children, using its addNode() method. Listing 11 shows the code for adding the child nodes.

Listing 11. Adding the bars to the light1 node
scene.getNode("light1", function(light) {
   var zpos = 0;
   data.forEach(function(dim) {
     zpos++;
     var xpos = 0;
     dim.forEach(function(dpoint) {
       var curVal = dpoint["value"];
       var curColor = ((curVal > PeakThreshold) ? { r: 1, g:0, b:0} :
          ((curVal > HighThreshold) ? {r:1, g:1, b:0} : {r:0, g:1, b:0}));
       var nodeName = "n" + zpos + "m" + xpos ;
       node2Data[nodeName] = dpoint;
       light.addNode(
           { type: "name",
             name: nodeName,
             nodes: [
                {
                type: "material",
                color: curColor,
                nodes:[   {
                         type:"translate",
                         y: curVal/10,
                         x: xpos++,
                         z: zpos,
                         nodes:[
                         {
                          type:"prims/box",
                          xSize: 1,
                          ySize: curVal/10,
                          zSize: 1
                         }
                         ]
                     }
                     ]
                   }
                   ]
                 }
           );
       });  // dim.forEach
   });  // data.forEach
});  // scene.getNode("light1")

The highlighted code in Listing 11 shows the nodes structure of each 3D bar created. Each bar is an instance of SceneJS's primitive cube geometry (prims/box) and has a wrapper material node giving it color. A translate node is used to position it into place. Table 1 summarizes the key fields and how their values are determined.

Table 1. Key fields used in creating each 3D bar
Node typeFieldDescription
NamenameThe name node is used to enable "picking" (or 3D selection) of each node. The unique name for each bar is algorithmically generated based on the bar's position on the x-z plane.
MaterialcolorThe material node gives each bar its individual color. The color is used to render the bar as determined by the value of its associated data point, dpoint["value"]. HighThreshold and PeakThreshold are used to determine whether green, yellow, or red is used.
TranslatexThe x position increments by 1 unit with every bar drawn within a date/time series. Each bar rendered is 1 unit square on the x-z plane and represents a monthly average result.
TranslateyThe y position of the bar is determined by the data point value dpoint["value"]. It is divided by 10 to scale the range down.
TranslatezEach series of data, representing a different data center, occupies a different z position.
prims/boxySizeThe ySize or height of the bar is determined by the data point value dpoint["value"]. It is divided by 10 to scale the range down.

As the nodes of the 3D bars are generated, the code also populates the node2data hash. This hash is used for quickly obtaining data point information when a node name is available during object picking. It is also possible to create a data field within the SceneJS node to contain this data. However, retrieval of the node to get at the data requires more work than a hash lookup.

Camera orbiting in SceneJS

The camera controlling the rendered view, like other objects in SceneJS, is represented by a node in SceneJS's scene graph. Like Three.js, SceneJS has support for orbiting the camera around the scene using a mouse. To enable camera orbiting, substitute the conventional camera type with the type:"camera/orbit" plugin included with the SceneJS distribution. The scene-creation code in sjbdvis.html wraps the coreScene (highlighted — the 10,000-bar scene) with an orbiting camera node:

scene = SceneJS.createScene({
        canvasId: "shapecanvas2",
        nodes:
          [
            { type:"cameras/orbit",
              look:{ x: 80, y:0 },
              yaw: 0,
              pitch: -20,
              zoom: 200,
              zoomSensitivity:10.0,
              nodes: [coreScene]
             }
          ]
        });

With the orbiting camera node in place, mouse movement for rotating and zooming is handled by the SceneJS libraries. The look, yaw, and pitch fields can be used to control the initial orientation and positioning of the camera. The zoom field determines how far away initially the view appears to be from the rendered scene.

Object picking in SceneJS

Support for hit testing of rendered objects is readily enabled in SceneJS. The key is to wrap each pickable object in a type:"name" node. Every one of the rendered bars in sjbdvis.html is wrapped in a type:"name" node with a unique name and, hence, can be picked from the scene. In sjbdvis.html, object picking is used to perform live update of the on-screen status display as the user moves the cursor over the rendered 3D bars. Every pickable 3D bar has the following wrapping:

{ type: "name",
  name: unique node name,
  nodes: [
    ...
  ]
}

The current mouse position is captured in the mousemove event handler of the canvas with:

canvas.addEventListener('mousemove',
 function(event) {
   mouse = {x: event.clientX, y: event.clientY, updated: false, clicked: false};

 },
 false);

The event handler returns immediately after updating the global mouse position/state variable to keep things responsive. The global mouse variable captures the position and state of the mouse as it is moved around the bars. Object picking and the associated update of the on-screen display is performed during SceneJS scene refresh ticks via:

scene.on("tick", function() {
   if (!mouse.updated) {
     scene.pick(mouse.x, mouse.y);
     mouse.updated = true;
   }
});

On screen updates, the object picking is performed based on the currently saved position of the mouse. If the user's mouse is over one of the bars (or the user clicks on one of the bars), that particular bar object is picked by the SceneJS runtime, and a SceneJS pick event is emitted, making a callback to indicate a hit:

scene.on("pick",
  function (hit) {
    var name = hit.name;
    var datapoint = node2Data[hit.name];
    document.getElementById("dmonth").innerHTML = datapoint["month"];
    document.getElementById("dyear").innerHTML = datapoint["year"];
    document.getElementById("dcenter").innerHTML = datapoint["location"];
    if (mouse.clicked)  {
      //simulate dive into  detailed data
      alert("Show daily detailed reports for " + datapoint["month"] + ", "
        + datapoint["year"] +  " at data center location "
        + datapoint["location"]);
       mouse.clicked = false;
     }

  });

Extending SceneJS's data-driven nature in sjbdvis.html

The relative ease with which you can programmatically modify rendered geometries (based on the value of data points) using SceneJS opens up the possibility of dynamic update of the displayed statistics. One can imagine adding a "networked live data feed" to drive updates of sjbdvis.html — creating a real-time monitoring console for managing 100 data centers across up to 100 different metrics.

The hit callback argument contains the name of the type:"name" node that was picked by the user (in hit.name). This name is used to look up the associated data point via the nod2data map, and the data point values are used to update the content of the CSS-styled <div> DOM elements that represent the on-screen status display. If the mouse is clicked, an alert is popped up showing the data point requiring further analysis — simulating a data-fetch event generated by the UI.


Using alternative 3D input devices

Just one more thing about the final example in this series. The PC mouse really shows its age when interacting with 3D scenes. To supplement 2D mouse input data with 3D cues, you must click the various buttons while dragging the mouse. Ideally, 3D position information is directly entered via a 3D-capable input device.

The Leap Motion Controller is such a device. This widely available device is a small sensor box connected via USB 3 to your PC or Mac. 3D position and gesture information are detected through the use of a software driver working together with the device's hardware infrared lights and cameras array. This device can accurately sense the position and motion of hands, fingers, and tools that are held within range of its sensing area.

Figure 9 shows a user interacting in 3D with a version of imatlight.html (named ilpmatlight.html in the code Download), displayed on a large TV screen and modified to work with the Leap Motion Controller. By moving one hand in 3D space, you can pan, rotate, and zoom the camera in the Three.js scene, viewing the scene from varying perspectives.

Figure 9. User interacting in 3D with scene via Leap Motion Controller
Image shows user interacting in 3D with scene via Leap Motion Controller

The official JavaScript support software for Leap Motion Controller is leapjs. It consists of a node.js server that communicates with a native driver and sends sensor information to the local JavaScript (browser) client via WebSocket. In addition to leapjs, threeleapcontrols is a set of camera and object controls that work with the Leap Motion Controller and Three.js. Listing 12 shows the code changes made in ilpmatlight.html to support 3D hand-position-controlled input.

Listing 12. Adding Leap Motion 3D input support to ilpmatlight.html
<!doctype html>
<html>
<head>
  <title>developerWorks WebGL Three.js 3D Interactive Lights 
     and Shadows Effect Example with Leap Motion Controller</title>
  <script src="Three.js" ></script>
  <script src="leap.min.js"></script>
  <script src="LeapCameraControls.js"></script>
  <script type="text/javascript">
  
 
  function draw3D()  {

     var controls;
    Leap.loop(function(frame) {
      pyramid1.rotateY(Math.PI/180);
      sphere.rotateY(Math.PI/180);
      cube.rotateY(Math.PI/180);
      multi.rotateY(Math.PI/480);
      controls.update(frame);
      renderer.render(scene, camera);
    });
    


    var geo = new THREE.CylinderGeometry(0,2,2,4,1, true);
    var pyramid1 = new THREE.Mesh(geo, new THREE.MeshPhongMaterial({color: 0xff0000}));
    pyramid1.position.set(-2.5, -1, 0);

    ...
    var camera = new THREE.PerspectiveCamera(  45, 1024/500,0.1, 100);       
    camera.position.z = 10;
    camera.position.y = 1;

    controls = new THREE.LeapCameraControls(camera); 

    var multi = new THREE.Object3D();
    pyramid1.castShadow = true; sphere.castShadow = true; 
    ...
    var div = document.getElementById("shapecanvas2");      
    var renderer = new THREE.WebGLRenderer();
    renderer.setSize(1024,500);
    renderer.setClearColor(0x000000, 1);
    renderer.shadowMapEnabled = true;
    div.appendChild( renderer.domElement );
  }
  </script>
 </head>
 <body onload="draw3D();">
 <span id="shapecanvas2" style="border: none;" width="1024" height="500"></span>
 <br/>
  </body>
</html>

Dawn of mainstream 3D computing

An entire generation of computer users who grew up playing 3D games is poised to join the computing mainstream. They are hungry for novel and innovative application of 3D-related technologies to increase their productivity and improve their daily computing experience. Also in this decade, a vast amount of research capital is being poured into big data visualization technology, 3D scanning, and 3D printing — perhaps legitimizing computing in 3D for the first time in history. Combine this trend with the ever-increasing 3D hardware rendering power of today's mobile devices — and the ability to harness this power simply and effectively using WebGL — and JavaScript developers are now at the forefront of an exciting wave.


Download

DescriptionNameSize
Sample codewebGL3dl.zip339KB

Resources

Learn

  • WebGL: Visit the WebGL home page on the Khronos Group site, and read the latest working draft of the WebGL Specification.
  • Check out Sing Li's 3D development with WebGL, Part 3 code libraries.
  • Three.js by mrdoob: Study the documentation, try the getting-started tutorial, and work through a variety of examples.
  • SceneJS by Lindsay Kay @xeoLabs: Browse the examples, tutorials, documentation, and more.
  • Leap Motion Controller: Check out the quintessential 3D input device. The Leap Motion Controller can track the 3D movement and gestures of two hands, finger positions, and handheld tools.
  • leapjs is the official JavaScript library for the Leap Motion Controller. Study the documentation and samples for more 3D interaction possibilities.
  • threeleapcontrols: Torsten Sprenger's threeleapcontrols integrates the Leap Motion JavaScript library with Three.js, offering a 3D input API that is as straightforward to use as Three.js's 2D mouse-based THREE.OrbitControls.
  • Oculus Rift 3D Visualization Head Tracking Device: This helmet-like 3D viewer device promises immersive 3D world virtual reality experiences for its users.
  • WebGL Quick Reference: Take advantage of a handy cheat sheet for WebGL API syntax and concepts at a glance.
  • Can I use WebGL?: This valuable site tracks up-to-date browser support for WebGL by versions.

Get products and technologies

Discuss

  • Get involved in the developerWorks community. Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Web development on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Web development
ArticleID=961726
ArticleTitle=3D development with WebGL, Part 3: Add user interaction
publish-date=02042014