A/B Testing

Use A/B testing of your communications to ensure your content meets your brand's needs.

Overview

parcelLab allows for easy A/B testing of communications both in a "sticky" and a "non-sticky" way. A sticky test is where users stay in a test even if they are no longer in the test's target group. For a non-sticky test, users are removed from the test when they do not fit the test's target group.

Configuring A/B Testing

The following sections describe the configuration required to perform A/B testing at message level and content level.

A/B Testing on Message Level

Our team can setup your A/B testing on message level. This means your individual customer groups will receive different communication flows altogether, allowing for different messages and a completely different flow where required.

Not all messages need to be included in the A/B test.

A typical scenario is the invite to multiple review platforms. You will have the normal communications flow excluded from the A/B test, but the Delivered messages divided up by review platform, which can be any of our supported platforms.

Trust Platforms

In this case, all communications would stay the same but you would have independent message IDs for the Delivered message (for example: DeliveredTrustPilot, DeliveredTrustedShops, and DeliveredEkomi), each assigned an equal amount of the orders.

Message level A/B tests support the following levels of stickiness:

• Not sticky - each individual email is sent out randomly out of the groups.

• Order-sticky (this is the default setting) - all messages of a specific order (including multiple parcels) will be assigned to the same group and therefore have the same communications flow.

• Customer-sticky - a customer will permanently be sticky to his A/B test group even across multiple orders.

A/B Testing on Content Level

A/B testing on content level allows you to display individual content blocks easily using placeholders.

Content level A/B tests support the following levels of stickiness:

• Not sticky - each individual email is sent out randomly out of the groups.

• Order-sticky (this is the default setting) - all messages of a specific order (including multiple parcels) will be assigned to the same group and therefore have the same communications flow.

You can display that content both in a template as well as a in a message using the appropriate conditional placeholders.

<stickyness>-ab-test-<number of groups>-groups/group-<group id>

You need to set the following parameters in this placeholder:

• <stickyness> - can be either random (that is: not sticky), or sticky to have it order-sticky.

• <number of groups> - tells the system how many A/B test groups you want to have and can be between two, three, four, five, and six.

• <group id> - is the actual group you want to address with this content block (that is: here the ID is only the digit like group-3).

Content Level Testing Example

In this example, the test is order sticky, there are five A/B test groups and groups two and five have specific content blocks.

{{#sticky-ab-test-five-groups/group-2}}
  <p>You are in group 2 of 5!</p>
{{/sticky-ab-test-five-groups/group-2}}

{{#sticky-ab-test-five-groups/group-5}}
  <p>You are in group 5 of 5!</p>
{{/sticky-ab-test-five-groups/group-5}}