Unable To Choose A Winner For A Sitecore Email A/B Test

Unable To Pick Winner

On a rare occasion, you may experience this odd issue in EXM; when it comes to sending an email campaign that has taken part in an A/B test. Specifically, after the test has been run and when having to select a winner manually, you're unable to do so. The buttons are simply greyed out. The status of the email campaign might also indicate it is in the Paused state. Before you decide whether this issue is the one you are indeed having, let's walk through the symptoms in more detail.

Important Note - This solution involves Developer and/or Administrator level with backend access to the SQL Master database. Whether it be in an on-prem or Azure environment.

The Symptoms

A few things are visible when this issue is present so it's rather easy to deduce if this solution will work for you before performing it.

Symptom #1 - Greyed Out Buttons

If your A/B test has run its course in EXM and after going to the Delivery tab of your Email Campaign neither of the buttons labeled Choose as Winner can be clicked. Even after selecting the Resume button at the bottom of that tab does nothing to resolve the issue.

Symptom #2 - Status Is Paused

One of the first things people will try is to click the Resume button at the bottom of the Delivery tab. The status will be updated but if it doesn't finish, the status, in this case, will go back to Paused. In our case, we set the minimum test size to be 20%. And based upon the Sent Progress value, that has been met. However, we've noticed that in cases where a campaign fails to resume or the winner can't be chosen, the total number of recipients and those skipped will equal 20%. Sitecore is including the skipped value.

Check The Logs

It's important to understand as much as you can about what's happening in the backend as well as what we can visibly see. That's why enabling DEBUG level logging in EXM is important to ensure nothing else may be causing certain emails to not be sent. You can do this by going to /App_Config/Sitecore/EmailExperience/Sitecore.EmailExperience.Core.config on your Single Instance or Content Management instance and scrolling down to the bottom of the file. At the bottom, you'll find the logging information. Update the level to DEBUG.

<log4net>
    <logger name="Sitecore.EXM" additivity="false">
    ​   <level value="DEBUG" />
    ​   <appender-ref ref="ExmLogFileAppender" />
​    </logger>
    ​<appender name="ExmLogFileAppender" type="log4net.Appender.SitecoreLogFileAppender, Sitecore.Logging">
        ​<file value="$(dataFolder)/logs/Exm.log.{date}.txt" />
        ​<appendToFile value="true" />
        ​<layout type="log4net.Layout.PatternLayout">
        ​   <conversionPattern value="%4t %d{ABSOLUTE} %-5p %m%n" />
        ​</layout>
        ​<encoding value="utf-8" />
    ​</appender>
​</log4net>

The Solution

This solution is primarily for those who have lists of emails in the hundreds or thousands. If your email campaign list is less than 100, it might simply be best to review, manually, all the emails in the list and see if there are any noticeable issues with them. Manually fixing the emails, or even deleting them from the list itself, may resolve your issue.

If there don't appear to be any logs indicating other issues and no emails are immediately obvious causing the test to fail, then we can proceed with the solution. You will need access to your SQL Server that hosts the Master database.

Performing The Fix

Log in to your SQL Server and pull up the Master database. You'll want to open up a fresh Query window to perform the fix. Prior to running the SQL query, you need to gather a few things such as the Email Campaign GUID and the current value in the database for the Test Criteria.

Email Campaign GUID

You can obtain this from the URL while editing the Email Campaign. In SQL queries it will be the value for ItemId.

Test Criteria Value

This value you can determine two ways. It's made up of a combination of two integers. {A}|{B}. The first value, {A}, is the number of emails that would need to be sent based upon the percentage chosen for the Size of the Test. For example, if your total list size was 500 and you chose 20%, this value would be 100 It would take 100 emails sent before the winner could be chosen. The second value, {B}, represents the percentage chosen. i.e. If you chose, 20%, then this value is 20. We can confirm this by doing a SQL SELECT using the Email Campaign GUID, shown below. Note: The FieldId value shown below is always '2DF7DE6E-A412-4A98-B8C4-B7AB7032634A'.

SELECT Value FROM SharedFields WHERE ItemId = 'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX'
AND FieldId = '2DF7DE6E-A412-4A98-B8C4-B7AB7032634A'

If we used the specs above, then this value would show as 100|20.

Update Test Criteria Value With New Value

The value for {A} will always stay the same. So if we got 100|20 as a value, we keep the 100. The value for {B} is what we will change. Depending on the number of skipped emails showing, the new value of {B} will need to be changed accordingly such that the number of emails that have been sent triggers the completion. If 10 emails were skipped, then this value should be changed to 18, representing 18% sent.

UPDATE SharedFields
Set Value = '{A}|{B}'
WHERE ItemId = 'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX'
AND FieldId = '2DF7DE6E-A412-4A98-B8C4-B7AB7032634A'

For example:

UPDATE SharedFields
Set Value = '100|18'
WHERE ItemId = 'E0E92CFD-EB3D-48D3-BD1C-9A338FF4ED4D'
AND FieldId = '2DF7DE6E-A412-4A98-B8C4-B7AB7032634A'       

The Final Step

Given the value has been updated through SQL rather than UI, Sitecore will continue to use the cached value of the item. This can be updated going to the Sitecore Admin cache page and refreshing the caches there: https://<your sitecore instance>/sitecore/admin/cache.aspx

Once the cache has been cleared reload EXM and go back into your email campaign. You should see that you are now able to click the button Choose as Winner. This is because the number of emails sent equals the required percentage. In our example, 90 emails sent represents 18% of the 500 emails in the list.

A Patch?

At the time of writing this article, there is no known patch for this issue. If one becomes available I will update the article with references to it. Our hope is you only need to use this solution on rare occasions.