Also published on https://aboutsoftwaredev.wordpress.com/2017/04/17/on-testing-web-application-across-multiple-environments/
Users have the freedom of choosing which browser to access the web. Add to that the variety of operating systems and the potential need of also making web pages layout compatible (responsive) with devices like tablets and smartphones and we get the picture of how laborious and challenging performing functional tests for a web application might become.
Deciding how many variations to regard on testing might depend on different goals - whether to target a specific niche, to initially comprise only the combinations being used for most users or something else. Regardless, testing is constrained by the resources available:
- People: the number of team members fully or partly dedicated (developers who also build the application) to tests and their skills
- Infrastructure to run tests
- Schedule: how much time can be allocated to working on tests
Starting really simple, developers can test on browsers they have available in their workstations. Depending on the project strategy, that might be enough. For instance, a startup building its first MVP might want not even know whether the ideas being put in place will be well accepted by the market, so collecting feedback quickly becomes key to determine whether to continue or to quit (Fail Fast approach).
Chrome, Firefox and Safari are available for different desktop and mobile Operating systems, potentially comprising a very large set of users. Even without a Windows environment, teams can test on Internet Explorer using the virtual machines Microsoft made available - see this link.
For mobile, tests can be performed on real devices like tablets and smartphones or through simulators - there are some options for Android; for IOS, there is Xcode, available in macOS only, comprising a good set of Apple devices. There is also the option of using responsive mode of desktop browsers, where the screen dimensions can be reduced to mimic a device.
As the number of tests increase, and environment variations to run those on, automation becomes key. Selenium is one of the tools available for automating functional tests, being capable of interacting many different browsers. It is free and it provides APIs that are already being used by a large number of programming languages.
By the way, in my current project, I use Nighwatch.js, a Node.js based tool that invokes the Selenium APIs and provides a framework for defining and running test scripts.
To comprise different build versions of browser, plus the variety of operating systems, more machines (physical and virtual) can be setup to cover a set of variations. By the way, Selenium also has a subproject called Selenium Grid, that comes with tooling for constructing a grid of machines for performing tests.
Or, rather than building machines, there are also (paid) services on Cloud with infrastructure ready to be provisioned on demand, setup to run functional tests in different browsers and operating systems. Sauce Labs and BrowserStack are two examples of such services. The level of test execution concurrency and frequency is determined by what plan, among many being offered by those services, can be afforded by the team.
Besides the tools mentioned here, surely there are many others. Feel free to leave comments.