A/B testing

How A/B Testing Works

When a new visitor arrives to a website tracked by Digital Analytics, they are assigned a random A/B testing number between 0 through 99 regardless of whether they are in a test or not. This value is established in the Digital Recommendations personalization cookie and stays with the visitor until the cookie is deleted. A/B tests are associated with a site zone. When a visitor arrives at a site zone where an active A/B test is running, a single Digital Analytics element tag is sent to Digital Recommendations with the name of the test and name of the test group the visitor is in. To report on the results of an A/B test, Digital Recommendations creates report segments by using the appropriate element tag variables as criteria. You can use Top Line, Merchandising, and other Digital Analytics reports that support report segment filtering to determine the results for each test group.

Test Groups and Multiple Zone Tests

A Digital Recommendations A/B test is associated with a site zone. However, to achieve some testing scenarios, configure multiple A/B tests for simultaneous operation. In general, test only one hypothesis or scenario at one time. Testing multiple hypotheses one time makes it difficult to determine which change was responsible for a lift or drop in conversions.

When you use multiple Digital Recommendations A/B tests to test a single hypothesis, split the percentage for each test equally. For example, if you are running an A/B test on zone 1 and the test has a 50/50 split across a Control group and a Test A group. If you run a second A/B test on zone 2, make sure that the test has a 50/50 split across a Control group and a Test A group. All visitors who are assigned a test number are in the Control group for both tests. All visitors who are assigned a test number between 50 through 99 are in the Test A group for both tests.