A/B Tests are a great way to measure the impact of your adoption efforts with Chameleon accurately. You can create and compare different versions of the same Experience and decide which is more impactful for your goals.

From the Dashboard, you can run a Control Group (Chameleon vs. nothing) or A/B Test (different variations). But you can also test the other Experiences manually. We recommend starting with a Control Group test and iterating until your Experience is effective before testing Experience variations. 

🔐 A/B Testing available for Growth & Enterprise

📍 Ready to power your Tours, Embeddable Banners, (Tooltips, Microsurveys, Launchers)

📩 Contact to discuss your plan needs

When creating your Experience, you can choose to run a Control Test for your selected audience and exclude a percentage of users who should not see it. This will enable you to assess the effectiveness of your Experience and understand how it impacts your users' goals.

Build your Tour or Banner as usual, and from the Testing panel, you can start a Control Group test. Set the proportion of users that should be part of the control group and decide when your experiment ends.

You can also test multiple variants of the same Tour or Banner with Control Testing. When building out your Experience from the Build panel, you can also create a new variant, and pick to:

In the same Testing panel, you can switch between variants and pick the variant that should be displayed to your users during the test.

Every user identified by Chameleon is automatically assigned a random number (Testing ID property) with a value between 0 and 100. This value is persistent for each user and can be leveraged for targeting and experimentation.

To select a control group, Chameleon randomly chooses a number between 0-100 as the start of the range for the group. The size of the group will determine the end range for the group (including going back to 0 after 100.) The corresponding users within this range will form the group.

E.g. if you choose a 20% control group for a Tour, the Testing IDs for the group could be 87.3 to 7.3. If you do this for another Tour, the group may be 12.9-32.9.
​

See in your Dashboard the results of all Experiments. You can change your Tour or Embeddable Goal after you start an Experiment, and Chameleon will simply start a different Experiment for you to keep the results clear.

An event will be logged whenever a user is first identified to be within the Target Audience and is liable to see this Experience. This event* -- "Chameleon Experiment entered" -- will be available within all your connected analytics integrations.

Within Mixpanel, the event name is "Experiment Started" to better match Mixpanel's experimentation analysis framework. 👉 See our integration guide to how to identify different Tour Variants in Mixpanel.

For this event, the following properties will also be logged:

The "Group" property values are either:

Other events (e.g. "Tour started") will also be logged as normal once a user starts interacting with your Tour. You will see all these within your analytics platform (e.g. Amplitude, Heap, Mixpanel, Google Analytics) and can use this to further analyze the conversion or relative impact of your Chameleon experiment. 

You can also perform multi-variate testing on your Tours or Banners and show different versions to users to determine which version performs better.

Build your Experience as usual, and from the Dashboard's Build panel, 'Create a Variant' for your Tour or Banner from the dropdown and pick between:

With each variant, you'll be able to edit the Steps and all configurations as usual, including reordering or deleting Steps in the Dashboard. In the Builder, you'll be able to switch between each variant from the top bar to make adjustments or preview how each displays.

You can also rename your Variants from the Dashboard by clicking the pencil button next to each name. This change will be reflected everywhere in the Dashboard so you can track experiments easily.

You can create variants with a different number of Steps and configurations. Once you have your content and style set for each variant, go to the Test panel in the Dashboard to define your A/B test. You can:

Publish your Tour live next. In the Analytics tab, you'll see a section dedicated to your experiments where you can review how each variant is performing.

Here too, you can switch between different Experiments to review the results.

Review your results in the Dashboard and send them to any connected analytics integrations to better understand which is more successful and manages to drive users toward achieving their goals.

With each Experiment you start, Chameleon analyzes how each variant is performing and rates it with a 'Confidence score'. This takes into account how many users engaged with your Tour during your Experiment.

The 'Confidence score' is a good way to understand what you need to work on to improve either the quality of your experiment (e.g. audience size) or the configurations of your Experience (e.g. if not enough users meet your goal).

You can also manually test different Experience variations. You'll have to define your Test Segments and assign them to the appropriate Experience version when creating your variations. Here's how to do it 👇

To create a Test Group, simply add an extra "sampling filter" to your Segment that will select a random sample of users of the desired size. You can still target users based on other conditions, such as user properties, events, data sources, etc. 

To add the sampling filter:

In the above example, users that have a Testing ID value between 50 and 100 would be targeted. This would constitute 50% of the users within the group defined by the other segment filters. 

To target 10% of users, you could use either:

You can use the same filter configuration in another Segment (by re-creating the filter) to target the same user group. This enables you to run multiple A/B tests on the same user group. 

To test two variations of the same Experience: