Synchronized multimedia plays an important role in modern communications strategies. In broad strokes, the coordinated and ordered presentation of video, audio, still images, text, and other elements offers a dynamic, alternative, and editable approach in a world where the competition for an audience is intense. In addition, the presentation of elements in parallel can appeal to several different audiences at the same time.
Imagine you work in the realm of music education. As producer of a master class in piano technique, you present to a wide audience on large video screens around the world. The class is a discussion of a well-known passage that is technically difficult. The audio is the sound of the piano interspersed with the voice of the presenter. Part of your audience wants to see the pianist and presenter. Another segment knows very well what the pianist looks like but wants to follow the music score on screen, zooming in closely during some passages. Yet another segment wants a close up of the pianist's hands (sometimes left, sometimes right) as the difficult passage is discussed in detail. Synchronized multimedia can handle all of this through software—some elements sequentially and others in parallel—given that lots of screen real estate is available.
If the situation were simpler, then a straightforward script playing from a list of media files is enough. However, in complex situations, a more comprehensive package is needed. Synchronized Multimedia Integration Language (SMIL) is a W3C specification that expresses the required instructions for this kind of presentation in XML format which allows for great complexity and sophistication: starting, stopping, overlapping, and interleaving components as required. IBM® developerWorks has covered SMIL 2 in the past(see Resources). The SMIL specification was expanded to version 3, adding even more interesting techniques and flexibility to enable multimedia producers to compete more effectively in grabbing the attention of all those eyeballs and ears.
Status of tools
Developers need to be able to play with the new version of the language to become familiar with what is new and how it works. The W3C site provides a list of tutorials to get the beginner started. You need an editor or authoring tool and a player, so a list of authoring packages and SMIL players, both commercial and open source, is also available on the W3C website (see Resources). However, many of these tools are not yet up to speed with SMIL 3. No doubt work is proceeding frantically in the background to make many of them version 3 capable, but what is the developer who wants to play at the cutting edge to do? Right now, in the context of SMIL 3, you can use a plain-text editor for editing and Ambulant as a player, or you can use your own preferred tools.
The editor: Your choice
A truly visual editor with drag-and-drop and timeline capability is not yet here for SMIL 3, which will discourage many users. However, because the SMIL specification is based on XML, which is composed in plain-text format, any reasonably capable editor will do.
A good choice in a pure SMIL context is Eclipse (see Resources). To open .smil files directly
in the XML editor, ensure that the mime type
*.smil is registered in Window >
Preferences > General > Content Types and Window
> Preferences > General > Editors > File
Also, Eclipse is able to use the SMIL DTD to control the editing of the underlying code in a SMIL context.
The player: Ambulant
Ambulant (see Resources) is an open source player that is quite close to a complete SMIL 3 implementation. The program is open source and available on the three major platforms: Linux®, Microsoft® Windows®, and Mac OS® X.
Ambulant comes in two formats: stand-alone and browser plug-in. A developer should choose the stand-alone version. Even though browser plug-ins can end up as the user engine of choice, the stand-alone version has much better reporting and debug capabilities (which are important to a developer as the risk of error is much higher and feedback has a high priority). Note that, even though the stand-alone and the browser plug-in versions are maintained concurrently, you might get different behavior from the two versions due to implementation differences.
Installing and setting up Ambulant in Linux
For the purposes of this article, I used version 2.2 of Ambulant on OpenSUSE 11.2.
The Third Party Tools are components necessary for the compilation of Ambulant. These components are specially provided since, although they are commonly available, they need to be tweaked or edited to dovetail with Ambulant. They are provided in their altered (and correct) format in a subdirectory called /third_party_packages and will be selected preferentially by the make process.
Building Ambulant on Linux is a two stage process. First build the third party packages with the command:
python build-third-party-packages.py linux
Once this is complete, the usual
./configure, make, and
make install sequence will
build the standalone Ambulant reader. See the INSTALL file in your
downloaded package for full instructions.
Setting up the XML validator and other options
The first time you run Ambulant, you will probably receive a message that the XML validator is not set up. In the window that appears, select File > Preferences and then select the following:
- Auto for XML validation. Always might be available but is not a good choice because some state declarations do not require the same checking and validation can fail incorrectly.
- Debug for log level.
- Xerces or Expat for a parser, whichever you prefer if you have a choice. Xerces can provide more information when things go wrong
- Other options according to your needs.
After you set up the XML validator, Ambulant should be able to run a file with a .smil extension using File > Open. There are some example files in the Extras/Welcome/ directory.
Working with Ambulant
There are some things you should know as you work with Ambulant.
As a developer, you need information when things go wrong. Therefore, I recommend that you make a copy of a .smil file that you know works, add an error to that file using a plain-text editor, and try to run the file with the deliberate error. When it fails, you can examine the error output from View > Log Window.
Note that, if there is no file to load when the Ambulant window first opens, the window size is minimal; you might have to look for it. It is only when you open a valid file that the window resizes to encompass the main region declared in the file.
Repeated opening and closing of Ambulant from the command-line interface (CLI) destroys the error buffer each time. Sometimes this is an advantage, and sometimes it's a problem.
If you suspect that Ambulant has corrupted itself and needs to be rebuilt, you can do this by removing the build working directories, re-extracting the source files, and recompiling.
A profile is a label which indicates to the device on which Ambulant is running what components should be loaded to use its resources most efficiently. A fully-loaded PC can easily load everything, but smaller devices can be helped by providing a subset of the normal components.
You indicate the profile in the root element of the SMIL file in the
manner indicated in Listing 1, as the baseProfile attribute of the
Listing 1. Specifying the
... <smil xmlns="http://www.w3.org/ns/SMIL" version="3.0" baseProfile="Language"> ...
Until the software catches up fully with the specification, it is
best to specify the
Language in your experiments because other profiles
may not yet be implemented.
Currently, Ambulant does not remember the previous path used, so it might be best to work from a known local working directory and reload to avoid the need to drill down into lower directories. Another option is to run it from the command line.
The SMIL 3 test cases
When using examples from web sources, it is easy to navigate away from code samples that conform strictly to SMIL 3. A more reliable and informative approach is to use the SMIL 3 test cases listed on the W3C recommendation page.
The test cases are touchstones that enable you to see what SMIL 3 can do. Here is a suggested procedure to explore these new capabilities:
- Download the test cases that are new to SMIL 3 (see Resources).
- Unzip the files into a known directory.
- Make the directory containing the files the current working directory.
- Drill down through the files subdirectory to one of the following
test case groups:
With the current directory set to the /files directory in the expanded test cases, you have all the test cases arrayed one directory below in their subdirectories. From there, it is a straightforward call to AmbulantPlayer_gtk with a test case as a single argument as in Listing 2.
Listing 2. Command-line launching
$ AmbulantPlayer_gtk /Media/FancyPanZoomDemo.smil
- Exit the Ambulant player after each test case. Doing so clears the error buffer, leaving it fresh for the next test case, and keeps a known basic .smil file handy to make adjustments with cut and paste as required.
As with any fast-developing domain you might find that some of the test cases do not work immediately as you expect. The examples in the following table have been specially selected as likely to provide an immediate positive experience.
Two types of changes are in the SMIL 3 recommendation. One type involves the movement of code between engine modules and does not directly affect top-level SMIL coding. The purpose of that type of change is to support different profiles, avoiding the loading of parts of the engine that are not required where resources are limited, such as on handheld devices.
The other type of change results in changes to elements and attributes, which are of interest to the SMIL developer.
Table 1 lists and describes some selected test cases that appear to work correctly on at least one machine.
Table 1. Selected test cases
|ContentControl||Adds a |
|Metainformation||Can now be used in the |
|State||Allows the presentation to be dependent on the state of the system being used and allows some control over those state conditions. An interesting test is test-05-send.smil, which involves writing and reading XML files. The case chosen is that of an instance of Ambulant sending messages to another instance that will run later.|
|Structure||Identifies version and profile in the |
Back to the master class
Revisiting the piano master class proposed earlier, you can now make some plans for the presentation employing some of the features of SMIL 3. An assortment of audio and video files, both moving and still, in various formats including .ogg, .mp3, .mov, .jpg and .mpeg, can be interleaved with careful timing.
The first part of the introduction is followed by a video clip of how
the passage was treated by a famous pianist using historic film footage,
followed by the end of the introduction. These three parts are clearly
sequential, and each of them can use the entire screen, so you need use
only the one screen
region. However, you also
need to display crawling text across the screen using various TextStyling and
TextMotion techniques to put the crawling text in its own region to
show the title and credits. This is run parallel to the video.
Then the class begins in earnest with four screen regions defined in
layout element: one for the presenter in
overview, one for the score, and two more for the left and right hands.
These components all play in parallel, and all four are in parallel with
the sound track. Use
MediaPanZoom in the
score region to focus on annotations made in the margin of the score and
to follow the notation as it is played.
Finally, the conclusion releases the four regions and returns to a format similar to the introduction. Throughout the entire class, status variables choose the correct language audio file to play with a switch statement, and metainformation related to copyright and encodings is stored in the body of the SMIL document.
The result is something between the fully baked (meaning the process cannot be reversed easily) and printed film, which is much less adaptable to the audience, and the fully interactive, totally fluid, console approach, where a director who is constantly monitoring the output is able to make changes on the fly.
SMIL 3 is an effort to add useful functionality and modularize the engine for use on a wide variety of platforms in preparation for a world of many devices with different sizes, types, and capabilities. In a production environment, developers need complete and reliable tools, but in the meantime, Ambulant is good as a learning aid for SMIL 3. Ambulant has its shortcomings, but these traps will, of course, disappear over time. While the stand-alone version of Ambulant is excellent for developers, the eventual choice of users will likely be the browser plug-in version.
- Eclipse Check out the open source community for Eclipse. Its 60-plus projects focus on building an extensible development platform, runtimes and application frameworks to build, deploy, and manage software across the entire software lifecycle.
- Get up to speed with SMIL 2 (Anne Zieger, developerWorks, September 2002): Read this article for information about SMIL 2 and an XML-based approach to integrating multimedia into web content.
- Synchronized Multimedia: Visit the W3C lists of authoring and player packages and tutorials for SMIL..
- SMIL 3.0 Test Suite: Get more information about the collection of new testcases for the new features introduced in SMIL 3.0 and former test cases from the SMIL 2.1 Test Suite for the unchanged features of SMIL 3.0 from the W3C.
- XML area on developerWorks: Get the resources you need to advance your skills in the XML arena.
- My developerWorks: Personalize your developerWorks experience.
- IBM XML certification: Find out how you can become an IBM-Certified Developer in XML and related technologies.
- XML technical library: See the developerWorks XML Zone for a wide range of technical articles and tips, tutorials, standards, and IBM Redbooks. Also, read more XML tips.
- developerWorks technical events and webcasts: Stay current with technology in these sessions.
- developerWorks on Twitter: Join today to follow developerWorks tweets.
- developerWorks podcasts: Listen to interesting interviews and discussions for software developers.
Get products and technologies
- Ambulant: Explore an open-source media player with support for SMIL 3.0.
- Testcases for new features in SMIL 3.0: Download the testcases for new features from the W3C. The cases cover Animation, ContentControl, Layout, Media, Metainformation, Namespace_and_Doctype, SMILText, State, Structure, and Timing_and_Sync.
- IBM product evaluation versions: Download or explore the online trials in the IBM SOA Sandbox and get your hands on application development tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
- XML zone discussion forums: Participate in any of several XML-related discussions.
- developerWorks blogs: Check out these blogs and get involved.