In the previous article, you gathered together and installed all the pieces required to develop your Mac-mini-hosted multimedia application. This article shows you how to build the first stage of the media player software. It will still handle JPEG images only, but will be easily extensible to handle other media types.

Before I dive into this, however, I'd like to thank everyone who submitted feedback on the previous article. As a result of your comments, I've decided to add at least one more article to the end of this series, dealing with security in embedded Linux™ applications.

Graphical requirements for an image viewer

We now resume our regular programming. You need to be able to do three things, at minimum, to make a credible still-image viewer appliance:

1. Draw bitmapped color graphics to the display.
2. Enumerate graphics files available on the hard disk or other attached storage device (many commercial appliances would use flash media -- removable or internal -- for storage).
3. Decode graphics files.

Of course, other features are also desirable. One almost essential feature is the ability to scale images to fit in the screen's physical display area. Additionally, a highly enjoyable cosmetic bonus is to provide some different transition effects for getting new images onto the screen.

Hardware graphics support

I'll begin with graphics support. The Mac mini's hardware and the software bundle you loaded in the last article give you several different ways of manipulating the screen contents at a pixel level. The canonical approach would probably be to use X, which is in fact installed on your hard disk if you followed the instructions in the previous article exactly. I'll address using X later, but for now just consider that route as being too complex. Actually, it's not unbearably onerous, but there are good reasons for describing easier methods first, as you'll see.

A much simpler graphics route than X is to use the kernel framebuffer driver directly. This conveniently allows you to access video framebuffer memory directly. The display is pre-initialized into an APA (all-points-addressable) mode by the kernel; all you need to do is query the driver to determine the display's characteristics, and map the framebuffer address into your process' address space.

The oft-quoted disadvantages of using the framebuffer are:

• Considerable device dependence: Your application needs to be able to handle all the work of translating from whatever internal color representation you use into the framebuffer's native memory mapping format. All the framebuffer driver can really do is tell you what it expects; it can't handle the manipulation for you.
• Generally lower performance: Most modern graphics chipsets have features, such as 3D acceleration, hardware flood-fill, hardware line draw and so on, that aren't exploited by the simple framebuffer approach. Native X drivers (where available) are generally quite up-to-date with support for these acceleration features, so X-based applications will automatically enjoy any performance benefits. Note, however, that many embedded platforms don't have hardware acceleration features -- so X is actually a performance drag on such platforms.
• Workload: A corollary to the previous point is that the application developer has the workload of writing or porting any required graphics primitives -- font rendering, polygon drawing, palette manipulation, and so on.

When considering the specialized case of embedded applications, the first bullet point above is often a mere semantic argument. Practically by definition, in an embedded application, you're choosing (and freezing) the hardware to be used with your software. The only time you're likely to worry about device independence is if you're contemplating porting the code to a new platform with different memory to (sub)pixel mapping.

Likewise, the second point is often inapplicable to embedded development. In fact, the main reason I want to introduce you to the framebuffer method before gallivanting off into the world of pre-built GUI code is because it's the lowest common denominator, and the majority of embedded Linux devices with a graphical interface are built on top of the framebuffer device (even if they happen to be running X; hybrid interfaces are quite common. In fact, this series will show you how to develop a hybrid interface.).

Accessing the framebuffer

The framebuffer device is accessed principally through the device nodes /dev/fb* (most applications will assume /dev/fb0). Mode-switching can be accomplished through ioctls, or from the command line using the fbset utility. This utility, in turn, relies on the configuration file /etc/fb.modes, which specifies the timing and default color depth for various video modes.

You can specify a desired boot-time video mode in your PowerPC®-based system by modifying the yaboot parameter file to add a video specifier to the kernel options. If you come from an x86 background, you'll be pleased to hear that the yaboot loader is practically identical to LILO in terms of configuration and usage; the configuration file lives, by default, at /etc/yaboot.conf, and once you have edited it, you can transfer the configuration data to the boot loader itself by running the ybin utility. To set a specific video mode explicitly at boot time, simply add append="video=OPTIONS" to the end of the yaboot stanza that loads your kernel, and run ybin to commit the change (more on how to format those OPTIONS in a moment). For example, a yaboot.conf for the Mac mini might look like this:

boot=/dev/hda6
init-message="Hit TAB for image list, or wait..."
partition=8
timeout=30
install=/usr/lib/yaboot/yaboot
magicboot=/usr/lib/yaboot/ofboot
delay=10
enablecdboot

image=/boot/vmlinux-2.6.10-1.ydl.1
label=linux
read-only
initrd=/boot/initrd-2.6.10-1.ydl.1.img
root=/dev/hda8
append="video=radeonfb:1024x768-16"

(For the "shadow" iMac system I set up, the video driver is atyfb, since that machine is based around the ancient Mach64 video chip. For timing reasons, you should also specify the video mode as atyfb:vmode:17,cmode:16 on the iMac -- see below).

You can format the parameters given on the "video=" line using two methods: either video=driver:hresxyres-depth (where hres is the physical X-resolution, yres is the physical Y-resolution, and depth is the color depth), or video=driver:vmode:V,cmode:C (where V is the video mode number and C is the color depth code). Note that the vmode codes are arbitrary numbers, originally defined in some ancient Apple specification for PRAM contents. You'll find them documented in the Linux kernel at drivers/video/macmodes.h.

Having told you all this, please be warned that you may encounter trouble if you tinker with the boot-time video parameters, and the reason for this is pure and simple black magic. Some Mac hardware -- and I'm thinking here specifically of the iMac backup system I mentioned in the previous article (see Resources) -- is downright recalcitrant about its video parameters. This behavior is partly due to various strange limitations of the integral monitors in Apple's all-in-one systems, and partly due to unknown issues in the PLL calculation code in the kernel.

By the way, if you happen to be installing on a machine where Linux can't get a good default video mode, you can use the parameter "video=ofonly" (ofonly means use Open Firmware mode only). This is conceptually similar to using the VESA BIOS modes on an Intel®-based PC.

If you want to leave the boot-time video configuration the way the Linux install set it up, you can simply use the fbset program to change to a more desirable video mode at your leisure. Begin by adding the following stanza to the end of your /etc/fb.modes file:

mode "1024x768-universal"
geometry 1024 768 1024 768 16
timings 12735 160 32 28 1 96 3
hsync high
vsync high
endmode

After you save this file, you can set your system into a 1024x768, 16bpp video mode using the command fbset 1024x768-universal. If you're wondering why I developed the above timings, by the way, the answer is that the integral monitor in the iMac and the LCD monitor I'm using on the Mac mini both have severe problems locking onto any unusual syncrates. I couldn't get any of the default (kernel) video modes working properly on the iMac in particular; the only mode that would work is the 1024x768, 8bpp palettized mode that both machines started in. So I dumped out that mode's timing values using the fbset command (with no command-line parameters), changed the bit depth to 16, and thereby found a happy medium setting that works with both machines.

The code I'll be working with assumes that you have already set the machine into a compatible (16bpp) video mode, though it doesn't actually care what the resolution is. In order to ensure that your machine gets into the right mode automatically on startup, add the following lines to /etc/rc.d/rc.local, immediately before the "touch..." line:

fbset 1024x768-universal setterm -blank 0

The second line, by the way, simply disables automatic blanking of the screen. If you didn't do this, the display would automatically go blank after five minutes unless you manipulated the keyboard or mouse.

Back to top

Scribbling on the screen

Now that you've got the graphics mode set up nicely (well... almost -- you'll see what I mean at the end of this article), look at what you need to do in order to write data to the screen. All you really need is two pieces of information: where the video memory is, and how it's laid out. You can get this information through a couple of ioctl calls that tell you the visible size, bit depth, virtual size, and other pertinent information. Refer to graphics.c in the source code archive referenced at the end of this article. Briefly, the steps are:

1. Open the framebuffer device /dev/fb0.
2. Get the "fixed" and "variable" screen information (fb_fix_screeninfo and fb_var_screeninfo) -- this includes physical and virtual dimensions, bits per pixel, and physical address of video memory.
3. Use mmap(2) to map the framebuffer memory into this process's address space.
4. Allocate enough main memory for an offscreen buffer for scratch purposes.
5. Disable the text-mode cursor.

With reference to the last point, if you take a quick peek at the end of the GR_InitGraphics function, you'll find the following hack:

// Disable cursor
handle = open(FB_TTY, O_RDWR);
if (0 < handle) {
	write(handle, "\033[?1c", 5);
	close(handle);
}

If you don't have code like this in your system somewhere (note: you could just as easily use a line like echo -e '\033[?1c' in your startup files) then you'll be left with a blinking soft-cursor over the top of your graphics.

The reason the code snippet above is a hack is that it relies on arcane knowledge that the framebuffer device you're using (fb0) is associated with the console device tty0, which isn't necessarily true. (And, by the way, nothing prevents this program from being spawned from a different virtual terminal!) The assumptions made here are, however, always going to be true, because this is an embedded application where you can control how the program is loaded and the environment it will see.

While on the topic of graphics.c and graphics.h, also observe that I've defined the unit of measure, as it were, for pixels as a typedef PIXEL, and I've further defined a macro RGB2PIX that converts an 8:8:8 RGB triad into the native screen memory format (a 16-bit unsigned short integer in this case). This provides a rudimentary level of compile-time device abstraction, which will be useful for you if you intend to port this code to another platform.

Back to top

Finding files

Now that you've got the graphics subsystem working, you need to find the files on your hard disk. This functionality is accomplished by the code in filescan.c -- principally, FL_Scan(). That function scans for supported files in the /web directory and populates the slideshow array. This is a flat array of dynamically-allocated structures. At the moment, each structure only contains a filename, but this is a placeholder for later, when you'll add user-defined attributes (slide duration, and so on) to each image in the slideshow. Note also that I've implemented a separate function to identify if a given filename is playable based on a case-insensitive extension match, and to assign an enumerated type designator. This might seem like an unnecessary frill, but the feature will be useful later when you support other media types.

The main loop for this application works as follows:

• Run FL_Scan() to enumerate images. If none found, wait ten seconds and try again.
• Run through the slideshow displaying each image; pause for ten seconds between images.
• Repeat indefinitely.

Now to decode some JPEGs! For simplicity's sake, I use the JPEG library shipped with Yellow Dog. This library is based on the standard IJG JPEG 6b reference sources and is easy to use. Please note that I wouldn't normally recommend that you use this library unmodified in a deeply embedded system, principally because it makes heavy use of dynamic memory management (malloc/free). However, this application doesn't really qualify as "deeply embedded," which is why I felt justified in using the standard library (as well as some of my own dynamic memory allocation).

For a smaller system, you might need to think about the memory issue more carefully. In a previous life, I worked on several commercially fielded digital multimedia appliances. I used the exact same Independent JPEG Group source, but I wrote a small custom memory manager that allocated all the decompressor's RAM requirements out of a single pool that was reset every time the codec was invoked. This was all implemented on a rather unusual system architecture running a proprietary home-rolled operating system, and based on a PA-RISC microcontroller with 384KB of available RAM for all requirements (stacks, buffers, global variables, and so on). The Mac mini system is far less constrained!

Refer to codecs.c for information on how to invoke the JPEG decompressor. A certain amount of scaffolding is involved, most of which centers around assembling a setjmp return point for any errors that occur deep in the bowels of the JPEG library. The essential steps are documented very well in the example C program linked in Resources. Note that reading in a single scanline at a time is officially deprecated for efficiency reasons. The nature of JPEG encoding is such that the original file is decoded in eight-scanline chunks (actually, 8x8 blocks), so each time you call jpeg_read_scanlines, it either just copies the next scanline out of a pre-decoded buffer, or fetches the entire next eight scanlines.

In practice, through profiling, I've found that almost all of the decode and rendering time is spent either in the JPEG math (which you can't avoid), or in external user-supplied code doing scaling, dithering, and other functions. That's where all the inner loops are, and fetching one scanline at a time or eight will not significantly impact overall rendering speed unless some very unusual circumstances surround your application.

Back to top

Taking it for a spin

So, now you can drop a couple of JPEG files into the /web directory and run the ibmslides program. On a normal system, you'll see the images display one after the other (note that they are not scaled or centered). If you connect through ftp and drop a few more images into the system, you'll see them appear in the next rotation. Everything is well and good... or is it? Oh, no, it isn't! On the iMac, the colors are all wrong!

There's a whole sub-story of frustration and reverse-engineering in there, by the way, and I'll cut to the explanation without going through all the experiments I performed. On the old iMac, the framebuffer device reports that it's running in a standard 5:6:5 16bpp video mode. This is a horrid lie, however -- it's actually running in a very strange and utterly nonstandard mode. The most significant bit of each word is used for some unusual purpose; pixels where that bit is set appear black. The other color components don't behave as normal either; red, green, and blue cover the right spread of bits, but the onscreen display intensity isn't linearly related to what you write into those bits. It almost seems that the DAC in the Mach64 chip is still in a palettized video mode (using a CLUT) rather than using direct color mode.

If at first you don't succeed...

How do you fix this? You could either roll up your sleeves and patch the kernel in a manly fashion, or you could take the weasel's way out and use something else to initialize the video subsystem. Weasels are adorable and grossly misrepresented creatures, and here I'm going to emulate their fine example. I'll use the XFree86 X11 server to initialize the display!

Actually, in all seriousness, I planned to move this application from being a vanilla framebuffer program to a hybrid X + framebuffer program for other reasons anyway -- this annoying phenomenon merely moves up the schedule a bit.

For the moment, I'm just going to give you a magical solution without explaining it very much. If you look in the source archive, you'll find a graphics file called blank.xbm. This is an empty bitmap file created with the bitmap(1) utility, and you'll use it to blank out the onscreen cursor.

Before doing anything else, edit /etc/X11/XF86Config, and change the Screen section to read as follows:

Section "Screen"
        Identifier      "Screen0"
        Device          "Card0"
        Monitor         "Monitor0"
        DefaultDepth    16
        SubSection "Display"
                Depth           16
                Modes           "1024x768"
        EndSubSection
EndSection

Now copy blank.xbm to /etc (or anywhere else convenient), and then use the following command line (this assumes that the ibmslides program is in the current directory):

X & sleep 10 ; DISPLAY=:0 ; export DISPLAY ; twm & xsetroot -cursor
/etc/blank.xbm /etc/blank.xbm ; xset -dpms s off ; ./ibmslides

Presto -- perfect images!

So by this time you have a working PowerPC Linux installation with various daemons running, and you should now have a basic understanding of how to use the framebuffer device to implement graphics applications on Linux. In the next article, I'll explain why this magic works and what's going on. Then I'll spend more time showing you how to add support for scaling and a simple scripting language.

Back to top

Downloads

Contact the author for downloads.

Resources

• See the other articles in this series.
  
  
• Check out the HOWTO for programming the framebuffer. Unfortunately, it's rather old and x86-centric, but there's a little useful beginner information to be gleaned from there.
  
  
• The best resource money can buy for most of the code discussed here is the Linux kernel itself. You can always download the latest kernel from the Linux kernel home page.
  
  
• You can read the canonical sample usage of the IJG code online.
  
  
• You'll find information about the JPEG standard, and related standards such as JBIG and JPEG2000, at the ISO JPEG standard committee's home page.
  
  
• Another Mac Media project is CenterStage.
  
  
• The battery article was also discussed on slashdot.
  
  
• Interested in figuring out how many pictures you can store? Be sure to look into the tradeoffs between JPEG size and quality.
  
  
• The IBM Image Library Applications team has done a lot of research on capturing, storing, and displaying images. It might be overkill for this application, but you can find a lot of interesting information there.
  
  

About the author

Rate this article

Comments

Back to top

Table of contents

• Graphical requirements for an image viewer
• Scribbling on the screen
• Finding files
• Taking it for a spin
• Downloads
• Resources
• About the author
• Comments

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers