A/B Testing Email Campaigns
Campaign Deputy supports A/B Testing for your email campaigns. We call each different email a Variation and Variant starting with Variant A. We recommend an email list size of at least 2,000 deliverable emails or to use the Send to All option for smaller lists.
To enable A/B Testing, slide the A/B Testing slider to the right. Your current From, Subject, and Body will be copied to Variant A. If you Disable A/B Testing, you will lose changes made to any of your Variants.
You can add a new Variant by clicking the Add Variant button. Variants are lettered alphabetically starting at A. Variant A can never be removed.
- Send to All – Evenly sends to all of your emails. If you have 101 emails and 2 variants, a variant will be randomly chosen for the 101 email.
- Test Open Rate – This will decide the winner based on the unique open rates. The calculation will include delivered emails and the open rate per variation.
- Test Click Rate – This will decide the winner based on the unique click rates. The calculation will include delivered emails and the click rate per variant. Clicking the Unsubscribe link does not count towards the click rate.
Determining Winning Variant
After your set Test Duration we will calculate the winner. The timing will not be exact. The time durations start after all emails have been sent and the system may delay calculating the winner if there are several email opens or clicks arriving right before the calculation.
Suppose you have an email list of 4,000 emails and a Test Size of 20%. Variant A delivers 160 emails, while Variant B delivers 195 emails, due to bounces or full mailboxes.
Variant A has 48 opens of the 130 delivered emails.
Variant B has 50 opens of the 195 delivered emails.
Variant A is the winner as it had more opens compared to total delivered rate with an open rate of 36% compared to Variant B’s open rate of 25%.
We also use a Confidence Interval of 95% to determine the winner.