Overview

A/B Testing consists of a randomized experiment with two flow variants, A and B, which are run simultaneously to determine which variant improves your business metrics.

You should add A/B Test in your flows when you want to test a hypothesis about a flow setup you think will optimize your payment results. For example, if you think the payment connection X will have a higher authorization rate and lower costs when compared to payment connection Y, you can create a variant A with the connection X and variant B with the connection Y, run the experiment and evaluate the results. Even if you think your hypothesis is obvious, we highly encourage you to test it before making significant changes in your flows.

Transactions of the flow under an A/B Testing are randomly sampled, at transaction level, between the variant A and B. In addition, some transactions can be allocated to a Control Group Flow that will work as a sanity check to evaluate whether the sampling is biased.

Create an A/B Test

You need at least one Flow published to create an A/B Test. Then you define the hypothesis you want to test in that Flow, the percentage of the total volume you want to allocate to the experiment, and the Flow setting of the variant B that will reflect your hypothesis. The only thing that the variant A and B have to be the same is the Filter. Except for that, you can remove, add, change the order of any element in the Flow for your variant B.

You also have to publish your A/B Test to start your experiment with actual transactions. In case you want to start an experiment, please contact us at golive@tuna.uy. We will let you know when your A/B Test is published, but you will be able to see the changes in the Flow menu.

You can monitor the performance of all your A/B Test with the Flow Metrics. If you click on the Control Group Flow, located above the variant A and B, you will find more details about your experiment.

tip

Be careful about not jumping o a conclusion too quickly. You need statistical relevance for conclusive results. Thus, it might need some time before you have enough transactions for your experiment.

Once your experiment has statistical relevance, you can promote the Flow variant B of your experiment to process all transactions of that Flow or keep using the Flow variant A.