 | Level: Intermediate Joshua Fruhlinger (dw@jfruh.com), Editor and Writer, Freelance
18 Dec 2007 "Test on multiple browsers" has been a mantra ever since there have
been multiple browsers to test on. Testing them all—especially these
days—is impossible. But you can come a lot closer than you may think. In this article, learn a variety of techniques for cross-browser testing, from the very thorough to the quick and dirty. The choice you make will depend on your resources, but this is an issue you can't ignore.
We all know—or we all should know—that we need to
test our Web applications on multiple browsers and operating systems. But that's a task
that's often easier to agree to in the abstract than to actually undertake. When you
start thinking about the many combinations of browsers and operating systems that your
customers might be using to access your site, a complete suite of cross-browser tests might start to seem like a drain on your development resources.
What? There's more to life than IE?
Developers in the early '00s might have gotten away with only testing on Internet
Explorer, but those days are long gone. IE accounts for about 80% of browser visits
today. That's a large majority, but not an overwhelming one—certainly no
company can afford write off 20% of its customer base out of hand!
You also shouldn't assume that your particular audience has browser habits that mirror
the world of Internet users at large. Browser and OS habits vary by interest and
country. Many sites aimed at specialized audiences will be more likely to see
non-Windows visitors and non-Internet Explorer browsers. For instance, about 60% of the
developerWorks site's visitors use Internet Explorer, with about 35% using Firefox. For the Web development zone in particular, the numbers are 51% using Firefox and 41% using Internet Explorer.
Fortunately, a number of tools and tricks make the process a little
easier—and cheaper. In this article, you'll see a variety of strategies
you can use for testing; you can pick and choose what will work best for you, based on
your needs and resources. I'll begin by explaining some of the more thorough ways you can test, but you'll see that there are alternatives if time and resource constraints prevent you from testing as deeply as you might like.
Browser juggling
|
Bells and whistles
Don't forget that not everyone will have two of the technologies that many Web
applications depend on—Flash and JavaScript—activated on
their browsers. No matter which of the techniques I discuss in this article you end up
using, you should test with both JavaScript and Flash on and off. (Most of the hosted
services offer an easy way to toggle these options.) This goes for cookies, too—turn them off, clear them, do whatever you can to simulate the potential state of a browser visiting your site.
JavaScript is of course the basis of Ajax applications. Cross-platform testing of Ajax code can get kind of gnarly; the bottom line is that the closer you can get to the real user experience, with fewer layers of hardware or browser emulation, the better. The trickiness of debugging Ajax apps across browsers and the difficulty of getting them properly indexed by search engines are two of the major impediments to their adoption. See the links in Resources for more information, including a link to developerWorks' own Ajax resource center.
|
|
The most obvious, most thorough—and, generally, the most
expensive—method for testing across multiple browsers and OSes is to
actually test across multiple browsers and OSes. In many ways, the browser is the
easiest part of the equation, as most of the heaviest hitters are available for free.
Remember, though, that in all likelihood you're going to want to test not just several
different browsers, but several different versions of the same browser. As a developer, you almost certainly have the latest and greatest version of the browsers you use on your computer—this is one of the reasons the "test on multiple browsers" mantra got started in the first place—but a quick look at your site's browser statistics will probably show that at least some of your customers don't.
Running multiple versions of the same browser concurrently can be a bit tricky, as the
install process for most browsers overwrites any previous versions. Fortunately, there
are workarounds. For Firefox, it's generally as simple as saving the executable of each
browser under a different name, and then being sure to launch the profile manager the
first time you start each version so that profiles aren't shared. Figure 1 shows two
versions of the Firefox browser—the latest 2.0.0.10 release and a creaky old 0.7 version—running side by side.
Figure 1. Multiple versions of
Firefox running simultaneously
For Internet Explorer, the process is a little trickier. Windows really doesn't
want you to do this; to get around its recalcitrance, you need to tweak some registry
settings. TredoSoft, a small software firm, has gained a great deal of visibility by
releasing a free, handy tool for installing multiple versions of IE and instructions for mucking about with the correct registry keys and DLLs; see Resources for a link (and for links to other tools and expansions on the tips I'll mention here, though such links don't necessarily constitute endorsements from me or from developerWorks).
|
Test in real life
Not only should you test various browsers with various configurations;
it goes without saying (or should) that test machines (or, in a more
modest shop, trusted remote user-testers) should have their own
connections to the Internet. In other words, don't forget to test a
real-world connection, so that you're not fooling yourself by loading
images and scripts locally. Throw in any and all real-life factors you
can add in—pop-up and ad blockers, personal firewalls, and other
software users are likely to have on their machines, which may affect
the way your pages are displayed; this will help you to get a better
idea of the real end-user experience. The University of Washington's
Web Tripwire Toolkit (see Resources) can help you see when weirdness is caused by interference, rather
than browser implementations.
|
|
Of course, before you can get all of these browsers up and running, you need to figure
out where to get them! Microsoft® doesn't offer Internet Explorer 5 for download from its site, for instance, and it isn't obvious on the Mozilla Foundation's page where you can get Firefox 1.0. Fortunately, enterprising Web citizens have created archives of older executables. The volunteer-driven Web community at evolt.org has an extensive collection of old browsers of every flavor you can imagine, from IE to Opera and more, though there are some gaps in the more recent Mozilla releases; those can be made up at file aggregation sites like filehippo.com and its kin, which you can find without too much trouble. And for Apple's Safari, the Multi-Safari project, a one-developer labor of love, has conveniently packaged up earlier versions of Safari such that they can be run concurrently without fuss. (See Resources for links to all of these.)
Saving screen space
One practical problem you might run into while fiddling with multiple browsers is
window clutter. While for the most part that's unavoidable, you can at least
consolidate things a little bit by using a couple of handy tools. IE Tab is a
Firefox plug-in that allows you to run an Internet Explorer instance from within a Firefox tab. It's not emulation—it really uses the IE rendering engine, which means that what you see is what you really get in Explorer, though it also means that it only works on Windows. Figure 2 shows it in action.
Figure 2. Using IE Tab
RightLynx is a plug-in for Internet Explorer and Firefox that pops up a window when you right-click that shows you what the current page looks like in Lynx so that you don't have to keep an active terminal window open; a Web service loads the page in Lynx and then sends the resulting text back to your computer. Both these tools are free of charge and simple to install, and you can find links in the Resources section.
Lynx: So old it's new
You may have done a double-take when you saw Lynx mentioned in the previous section.
Lynx (for those of you too young to remember) is a text-based browser first developed in
the early 1990s for use on the UNIX® command-line (though there are versions for DOS, and thus for Windows, as well). Figure 3 shows the Web development zone as seen in Lynx.
Figure 3. Lynx in
action
Testing your site in Lynx might at first seem like a pointless exercise—after all, just about any person visiting your site is going to have access to a modern graphical browser. But the stripped-down, bare-bones version of your site that you'll see in Lynx is a good approximation of what your site looks like to one of your most important visitors: the Web crawlers and spiders that index your site for the search engines that will provide much of your traffic. Lynx can be a great tool for making sure that important search terms aren't hidden away behind Ajax widgets.
Virtualization and emulation
Of course, browsers are only half the battle in your testing; you'll probably want to
test across multiple operating systems as well. This is of course a much more expensive
proposition than just downloading some free browsers onto your development machine. The
most thorough way to go about it is obviously to actually have a Windows machine, a Mac,
a Linux® box, and physical versions of any other systems you want to test. That's
liable to break your Web development's budget for a year, though—but
fortunately, there a number of ways to turn one computer into multiple test platforms.
One of the most straightforward ways is to create dual-boot setups, where multiple OSes
are installed on a single machine, with a choice being made at startup as to which one
to load. Generally speaking, the OSes are all saved on separate hard drive partitions,
but so-called LiveDistros are stored on CDs or thumb drives to avoid the hassle of hard drive formats. Dual booting has historically been associated with Linux and other open source distributions, but the advent of x86-based Macs mean that many machines can now dual boot Windows and OS X.
The downside of dual booting is that switching between OSes is a time-consuming process. This is not the case in virtualization, which involves multiple OSes running simultaneously. Once the province of powerful servers, this tool is now well within the reach of anyone with a standard-issue desktop PC. If you have any intention of serving users with more than one OS, you'll want a test machine with some form of virtualization on board. In Figure 4, I'm using Virtual PC on a Power Mac to look at the developerWorks Web architecture zone in four different browsers on two OSes simultaneously.
Figure 4. Four browsers at once:
Safari on OS X, Firefox on OS X, Firefox on Windows XP, and Internet Explorer
A number of tools are available, including VMWare and Virtual PC, that make it easy to run Windows and most other x86-compatible OSes on any generic PC. One OS acts as a host, and the others run within the virtualization application. This makes it possible to test your applications on, say, Windows Vista, Windows XP, and the latest Ubuntu distro in adjacent windows on your screen, without rebooting. You'll still have to buy all the different OSes that aren't free, but at least you only need one piece of hardware.
You'll note that Mac OS X wasn't among the sample OSes I listed in the previous
paragraph. Unfortunately, OS X won't run on anything but Mac hardware, but there are
virtualization options available for the Mac. Thus, if you're looking for the broadest
coverage available on a single machine, you may want to consider using an Intel® Mac and one of the two major virtualiztaion products available for it, VMWare Fusion or Parallels Desktop (see Resources for links). With such a setup, you can test OS X, Windows, Linux, and any other operating systems that run on x86 hardware. The downside, of course, is that neither the software nor the hardware is free, or even particularly cheap.
Going mobile
If your Web site or application will be targeted at mobile users—or even
if you expect significant numbers of users from mobile devices—obviously
your ideal is to test on actual mobile devices. Unfortunately, the mobile device market
is much more fragmented than the PC market, making it difficult and cost prohibitive to buy and maintain a huge stable of physical test devices. Fortunately, you can also use a desktop computer to emulate mobile platforms.
Many specific mobile platforms have free SDKs available that include device emulators
that run on a PC desktop. For instance, you can download the SDK for Google's Android
platform (see Resources for a link). Getting access to the
browser is a little clunky—you need to install the Android plug-ins into
the Eclipse development environment and then launch one of the sample applications—but once you do, you'll have access to a browser based on WebKit, the open source framework that underlies both Safari and the browser on Nokia's S60 smartphones. Figure 5 shows the browser emulator in action.
Figure 5. The browser in the
Android SDK
The market for mobile browsers is again rather fragmented, but many of the higher-end
smartphones are now using modified versions of their desktop cousins—the iPhone uses Safari, and Opera Mobile is used on many smartphones, and you can do much of your testing for the phone browsers directly on the desktop, though the correlation isn't perfect. (iPhoney is an OS X app that uses the standard desktop version of Safari, but at least gives you a sense of how your site might look in the iPhone's restricted screen space; see Resources for a link to the free download.)
Of course, many phones are still out there that access the Web through the "walled
garden" of WAP. Innumerable free WAP emulators are available online that simply run in a browser window if you need to test a WAP site.
In general, the mobile space is still the trickiest to get a really good, broad
coverage through in-house testing. But you do have other options, as I'll show you in the next section.
Outsourcing the testing
You may have reached this point in this article and be feeling a certain amount of despair. Perhaps your deadline is tomorrow. Perhaps you are not convinced. Perhaps you are on a strict budget, either time-wise or dollar-wise. Perhaps your IT department frowns upon you installing old browser versions on your computer or tweaking the registry keys. What are you to do? Will you be unable to test on anything other than the latest versions of Firefox and IE?
You do still have one other option: You can use a hosted service to do the testing for
you. You supply the URL, and they show what that URL looks like on a variety of
platforms. You lose the ability to really whale away on a site from your own test
machine, but you get access to quite a broad array of platforms for very little cost—and sometimes for no cost at all.
Browsershots.org exemplifies what you can get from the free version of this sort of service. Figure 6 shows the different options you have available to you (which don't include mobile platforms or Lynx). For each box you check, you get a PNG file with a screenshot of the URL you specify for that browser/OS combination. The process can take a half hour or more, so you'll want to keep it running in the background while you do something else, but you can cover the important bases for free.
Figure 6. Customizing results
from Browsershots.org
When it absolutely, positively
has to be tested overnight—and tested everywhere
If you have the budget to pay, you can get a bit more advanced. For instance, for a
monthly fee, Browsercam.com gives you access to a broader range of platforms and
browsers—including some mobile platforms—and these pages are dynamic, allowing you to test Ajax apps and other JavaScript-based pages. Some services, like Litmusapp, can integrate with your testing and development workflows, offering versioning support and ways to publish test results to external clients.
These sorts of services are also well suited to the mobile world, where new devices are always popping up faster than most people can keep track. The DotMobi Virtual Developer Lab is a good example of a hosted service that focuses on the mobile space; it offers access to hundreds of different mobile devices and lets you manipulate them in a number of ways. The interface for both is still not the same as dealing with a real live device, but in many cases it's as close as you can expect to get; and as it focuses exclusively on mobile devices, it would serve as a complement to either your own internal tests or another service for desktops.
Conclusions
Ideally, every Web application should be tested to ensure that it will work perfectly
on every browser that might access it. But with the fragmentation of the browser market
and the increasing importance of the very fluid world of mobile platforms, that's a
practical impossibility. Still, you can come closer than you might think. You can use a wide array of tools for cross-platform Web testing. Whether you have the resources for a workstation with several virtualized OSes, or can only run your application through a hosted service, you have the capacity to see how your application will work in different user environments—and you owe it to your users to make it work as well in as many places as you can.
Resources Learn
Get products and technologies
-
Android SDK: Google's mobile development toolkit is free to download and includes an Eclipse plug-in that provides a mobile browser emulator.
-
evolt browser archive: An incredibly rich archive of older browsers for you to use in downloading and testing. Some gaps from recent years, but includes all versions of Internet Explorer up to 6.
-
Firefox on filehippo.com: Every version of Firefox since 1.0 is available for download here.
-
Multi-Safari: Download multiple versions of Apple's Web browser and run them simultaneously.
-
IE Tab: Run Internet Explorer as a tab in Firefox.
-
RightLynx Lynx Preview Tool: See what your pages look like in Lynx without leaving your graphical browser. Works with Firefox and Internet Explorer.
-
Lynx: Go back to the future with this text-based browser.
-
Virtual PC and VMWare offer generic x86 virutalization options.
-
Parallels Desktop and VMWare Fusion: Popular virutalization products for the Intel Mac.
-
WapTiger: See how WAP pages look right on your PC.
-
iPhoney: See how your sites will look on the coveted iPhone.
-
Browsershots.org: A free hosted cross-platform testing service.
-
Browsercam.com: A paid service that allows you to interact remotely with your Web page on a variety of platforms.
-
Litmusapp: A paid hosted service that can integrate with your development workflow.
-
DotMobi Virtual Developer Lab: This paid service offers access to hundreds of different mobile devices for testing.
About the author | | | Joshua Fruhlinger is a freelance writer and editor. He lives in Baltimore with his wife Amber and his cat Hoagie. |
Rate this page
| |
