Test automation for SAP packaged applications

from The Rational Edge: Read how Arsin, an IBM "Ready-for-Rational" partner, provides a testing solution for SAP packaged applications that leverages IBM Rational Functional Tester in combination with Arsin's own products. This content is part of the The Rational Edge.


Danis Yadegar, President and CEO, Arsin

More than fifteen years ago Danis Yadegar founded Arsin Corporation, which in 2007 became part of Semantic Space Technologies. Arsin continues as an industry leader in enterprise software test and test automation. As president and CEO, he is responsible for Arsin's long-term strategy and day-to-day operations. He has more than 25 years experience in all aspects of software design, development, and testing with special emphasis on integration, scalability testing, database, and user interface development.

Sarat Addanki, Vice President, Strategic ERP Solutions, Arsin

Sarat Addanki is Vice President, Strategic ERP Solutions at Arsin. He has thirteen years of experience in the development, implementation, and testing of ERP applications. For the past eight years he has been strictly focused on SAP, and he has pioneered the development of testing strategies and test automation solutions for complex SAP implementations, which includes McKesson Corporation. At Arsin, he designed a suite of SAP test automation solutions to accelerate testing of SAP implementations. He is a Project Management Professional (PMP) as certified by the Project Management Institute.

15 February 2008

Also available in Chinese

image of gearsSAP Packaged Applications allow you to rapidly configure and customize business processes as your environment changes, under the assumption that you need a testing solution that can be configured and customized as quickly as your SAP landscape. In this article, we will show you how you can use your IBM® Rational® Functional Tester toolset along with tools from IBM Ready-for-Rational partner, Arsin. Arsin's QA Mapper, Effecta Validation Engine, and the Arsin Support and upgrade toolkit allow you to develop a reusable, repeatable, and easy to maintain SAP Test Automation regression library for your SAP landscape, including custom applications and Inbound/Outbound interfaces.

We will discuss:

  1. A structured approach to SAP testing
  2. SAP current test automation paradigm and its challenges
  3. The need for a new solution for SAP test automation
  4. How Arsin Packaged Test Automation for SAP integrated with IBM Rational Functional Tester (RFT) helps address these challenges.

We will examine the functionality of QA Mapper, Effecta validation Engine, and the Arsin Support and upgrade toolkit, in conjunction with RFT, to collect the test requirements, define and build the test cases, build the test procedures, and execute and analyze the reports. Use of RFT and Arsin's tools enables you to greatly expand your test scope, significantly compress your test schedule, and reduce testing costs.

A structured approach to SAP testing

With over 45,000 tables, more than 100,000 fields, and millions of relations between each of them, SAP implementations pose some of the most intriguing and difficult challenges in the QA universe. The thickly netted system is extremely integrated and is typically linked to every business process in the enterprise. To tackle such an immense system, QA engineers must approach SAP applications with care.

With more than a decade of experience in testing SAP systems for a large client base in myriad industry verticals, we have developed a test maturity model assessment and improvement framework to bring about an organized and a structured approach to SAP testing. This framework has a three pronged approach, which offers process improvement, knowledge management, and test automation, as follows:

  1. Process improvement. Process improvement deals with the assessment of the current Test Maturity Model and developing a plan to improve the Test Maturity Model to the next level and then implement it. A mature test process that has standardized templates, well-defined processes, clear protocols, and no bottlenecks provides for a complete and comprehensively tested SAP system. By comparing the current test maturity model with industry standards and identifying the gaps and focusing on them, test maturity can be improved.
  2. Knowledge management. Knowledge management deals with institutionalizing QA knowledge collected over time. Traditional SAP systems testing relies on the functional and technical consultants of the SAP system for subject matter expertise to deal with various instances. In this phase, test artifact libraries are built for critical business process for regression. The following test artifacts are documented:
    • Test Requirements
    • Test Cases
    • Test Procedures
  3. Test automation. After test artifacts have been documented in the regression library during the knowledge management phase, they are ready to be automated. However, before they are automated, these test artifacts are analyzed for feasibility of automation, the effort required for automation, the frequency of use of the business process, and the longevity of the business. After the decision to automate using Arsin's QA Mapper as the test artifact repository, execution components are developed using RFT, and Validation Components are configured using Arsin's Effecta Validation Engine to execute them automatically.

The remainder of this discussion focuses on the test automation aspect of the Structured SAP Testing Approach. Our belief is that RFT in conjunction with Arsin's QA Mapper and Effecta Validation Engine makes SAP testing thorough, comprehensive, easy, and cost effective.

Importance of test automation in SAP implementations

The SAP landscape is continuously changing, as a result of changes to SAP modules from SAP, business process changes within the client company, changes to the system environment, changes to applications interfacing with SAP, and a multitude of regulatory compliance mandates.

Figure 1 illustrates the interdependencies that are constantly in motion within the SAP environment.

variety of changing conditions, centered by regulatory compliance

Figure 1: Interdependencies within the SAP environment

In order to keep up with these changes, SAP systems must be thoroughly tested. With every change, there is a regression library of test cases that needs to be executed to ensure stability. Each test requires time and effort when executed manually; by comparison, automated test take a very small fraction of the time and effort to execute. Automation also helps makes most of the test assets reusable.

Current SAP testing solutions and their limitations

The existing SAP testing model on the market today makes a very rudimentary use of automation, in terms of:

Validation: In most cases, GUI tools that are available are used to automate test execution, which is only about 25% of the total testing effort. Validation represents more than 75% of this effort, and scrubbing the data using GUI test automation tools is difficult. A certain level of validation is possible through GUI based test automation tools, however it takes a long time to script this validation and any change requires a lot of coding following the first implementation.

Data management: Traditionally, data used for testing is captured and maintained in spreadsheets. Searching and sorting through this data is difficult, as is maintaining the consistency of data across users and locations. This difficulty is compounded by ever increasing volumes of test data to be maintained. In addition, there is no intelligent association between SAP metadata and its corresponding test data.

Managing change: Changes to SAP implementations occur during reconfiguration or the addition of custom-built components (programs). In these situations, the scripts for automated test execution need to be changed regularly, which is difficult. Moreover, when using GUI tools for automation, 75% of the effort needs to be constantly re-worked to keep up with the changes to the SAP system.

Addressing these limitations

The limitations above described call for a new solution that can address these issues. We offer a solution that combines the following components:

  • Arsin QA Mapper
  • Arsin Effecta Validation Engine
  • IBM Rational Functional Tester

As shown in Figure 2, the main components of the solution are the QA Mapper (the test artifact repository), RFT (the engine for automated execution of test cases), and Effecta (the automatic validation engine). The metadata for SAP programs and transactions are stored in QA Mapper as Execution Components, which are read by RFT to execute the transaction. RFT drives the execution components to execute a test case with a wrapper script that is developed to use the meta data from the QA Mapper. When a test case is executed, this script pulls up the test procedure that is linked to the test case and executes the components/transactions. For the validation, the key information, such as the sales order number or the delivery number, is passed to the Effecta validation engine, which then pulls up the actual values corresponding to those keys from the SAP database and then compares them with the expected results.

Boxes labeled as components left to right

Figure 2: The main components of the Arsin SAP testing solution

This solution shown in Figure 2 addresses the limitations posed by the earlier model. Now we will examine this solution in detail.

Test data management

In the QA Mapper product, test data is managed in a relational database, which makes it easy to search, sort, and maintain consistency across multiple users. QA Mapper also maintains data in terms of data sets that may be reused. These data sets are created separately so that they may be reused in various test cases. QA Mapper enables the creation of project specific and secure test data sets which can be easily maintained through a web-based interface. Additionally, the creation of input test data is accelerated through automatic importing of master data from the system under test.

Managing change

The solution provides completely customizable and configurable components that do not need any scripting changes. The use of the wrapper script and metadata over the actual data means that for any additional fields to the screens, only metadata needs to be added to the database. This feature saves much time and effort.


Arsin's validation test engine, called Effecta (see Figure 2, right) drives the automated validations for each test case the QA team is required to execute. Validation takes as much as 75% of the total testing effort in a manual effort scenario. Automation via Effecta reduces this time and produces detailed reports on exactly which validations failed, thus simplifying the debugging process in case of a failure. The tool also generates audit trackable, repeatable, and scalable QA test results.

Business object configuration and customization

The procedure for automating validations involves creation of business objects and validation components. A business object is an aggregation of all the tables that are used or affected in a transaction in SAP, and the relation between these tables through the fields. It forms the connection platform that identifies which fields from one table relate to fields in another table. Business objects are easily configurable and extendable. When a transaction has to bear new functionality or include new tables, the basic business object can be extended by adding the requisite tables and the new relations, as shown in Figure 3.

Screen image showing business object configuration

Figure 3: Basic business objects can be extended by adding the requisite tables and the new relations.

Validation component creation, configuration, and customization

A validation component is a collection of rules that refer to the fields that must be validated to ensure stability and consistency of the SAP system. Validation components use business objects to form the platform that interrelates the underlying SAP tables. The validation rules in these components compare the data in the table fields with the expected data from the different data sources, including table fields and fixed values.

Validation components are maintained by configuration and do not have any scripting involved. The customization involves changing rules, which in turn are the operations of selecting the table fields to be validated and the expected source for each of them.

On completing the validation, the Effecta engine provides a detailed report, as shown in Figure 4, which indicates exactly which fields failed the validation rules. This helps in bug tracking and fixing.

Image of a test validation report

Figure 4: A detailed report indicates exactly which fields failed the validation rules.
Click to enlarge

Inbound/outbound interface validation

Apart from providing validation for transactions in SAP, Effecta also provides inbound and outbound interface validation. The transfers in these interfaces are usually in the form of Intermediate Documents (IDocs), flat files, etc. When these are to be validated, Effecta extracts data from them and validates them against the tables and fields in SAP.

Production support and support packs validation

When testing production support and support packs during an upgrade there are two scenarios: one before the change has been done and one after the change. The aim of testing in these cases is to make sure that the system remains stable because of the change.

In these cases, when transactions are executed, tables are updated, documents and IDocs are created. These documents, tables, and fields are to be compared before and after the change. For a transaction such as the creation of a sales order, there may be fifteen tables and between ten and fifteen fields that are to be compared in each table. Comparing these manually requires considerable time and effort. Effecta has automated tools to compare tables, IDocs and documents that drastically reduces time and effort in these cases. These tools also provide the tester with a detailed report stating the results.

Arsin is currently working on building the Effecta validation engine for Oracle and Sterling platforms as well.


The benefits of using automation in SAP testing are many. Test automation enables increased test coverage, which in turn reduces cycle time and enables efficient bug detection early in the development cycle. Since test automation is designed for reusability, routine tasks are eliminated and total cost of ownership is reduced. Test automation is far more precise and consistent and features standardized reporting, enabling clear test analysis across the QA environment. Test Automation can be deployed with a minimum effort.



developerWorks: Sign in

Required fields are indicated with an asterisk (*).

Need an IBM ID?
Forgot your IBM ID?

Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.


All information submitted is secure.

Dig deeper into Rational software on developerWorks

ArticleTitle=Test automation for SAP packaged applications