In Kit, you can A/B test email content to learn which format or email template drives higher engagement and clickthrough rates.
This feature is only available for Broadcasts and Creator Pro plan users.
Setting up an A/B content test
To set up a email content A/B test for your Broadcast, click the flask icon next to the subject line in the Broadcast editor:
Alternatively, turn on the Run an A/B Test toggle in the right sidebar.
You'll see the A/B test options. Select Email Content.
You'll then see a new navigation bar showing Version A and Version B of your email:
Both Versions A and B contain the email content you've already created. Click the Version A and Version B toggles to edit their content as you'd like. We recommend being specific about the layout or call to action you want to test.
Setting the test duration
Under the Run an A/B Test section in the right sidebar, click Edit A/B test to adjust your A/B content test's duration:
Your test can run between 30 and 240 minutes, with 240 minutes being the default. Click the curved arrow icon to use this default duration.
Use the slider to adjust the test duration in 30-minute intervals, or type a custom test duration into the minutes field on the right.
Hit Save to save your A/B test settings. You can then send or schedule the Broadcast, and the test will run automatically.
To edit your A/B test settings, click the flask icon next to the subject line, or the Edit A/B test button in the right sidebar.
If you change your mind about running an A/B test, you can turn it off by disabling the Run an A/B Test toggle. Version B of your email will be deleted.
How does the A/B content test work?
When you test email content:
We'll send Versions A and B of your Broadcast to two 15% segments of your recipients (i.e., 30% of your total recipients).
After the testing period, we'll determine which version has a higher click rate.
We'll then automatically send this winning version to the remaining 70% of your recipients (i.e., those not involved in the initial test).
To ensure more statistically significant test results, we recommend having at least 500 subscribers to run an A/B content test with.
How long should your A/B test be?
Our suggested A/B test duration is 240 minutes, which you can use by clicking the curved arrow icon in the duration setting. But these pointers may be useful if you want to set your own duration:
If you're sending to a smaller list, we suggest setting a longer A/B test duration to give more recipients time to open your email. You will get a more meaningful test result this way.
That said, if your email is more time-sensitive, set a shorter A/B test duration. This is so that all recipients receive your email as close to the initial send time as possible.
Experiment with different test durations to learn what works best for you.
A/B test reporting
The Analytics page of your Broadcast's reports will show the A/B content test results for Versions A and B of your email. Each version's stats will continuously update as your test goes on.
After the test is complete, the winning version will be highlighted in green and display a trophy icon.
The aggregate stats for your Broadcast—as sent to all recipients—will also become available on the Overview page at the end of the test.
FAQs
Where in Kit is A/B content testing available?
A/B content testing is available for Broadcasts at this time.
Can I A/B test email content and subject lines at the same time?
No. You can run only one type of A/B test at a time.
Can I use different email templates for versions A and B?
Yes. Each version can use a completely different layout and design.
How is the winner chosen?
The version with the higher click rate within the test group is automatically selected.
Can I end an A/B test early?
Yes. To do this, go to the Broadcast's Analytics report.
Click the three vertical dots on the right, followed by Cancel A/B test.
What happens if I cancel my A/B test?
If you cancel your A/B test before it completes sending, your Broadcast will have been sent to only your test recipients.
To send the Broadcast to your remaining recipients, click the Recipients page to view the subscribers who have already received the Broadcast.
Select all of these subscribers, and then add a temporary Tag (e.g., a Tag called "Cancelled A/B test") to them.
Duplicate the Broadcast. Next, send it to your initial recipients list (i.e., all 100% of them) except for the subscribers who have the temporary Tag you just created.
You can delete the temporary Tag after that.