A/B testing analytics are very similar to general User Activity analytics so read up on the basics.
The metric tiles within Analytics are split into three different sections: Goals, Favorites and Significant Changes. If you are testing a Message, you will also see out-of-the-box Message or Campaign analytics within the A/B Test reporting.
Whichever metric you have selected (via the dropdown above the chart, or the metric tiles below) will be displayed on the graph. The graph defaults to display Daily Active Users. Above the graph you will find a notation of the control and all of the variants, as well as the percent change each variant experienced compared to the control.
Below each graph you will find tabs representing each variant. All metric tiles represent the percent change, relevant to the control group. Navigate between these tabs to see the percent change for metrics across variants.
When looking through your metric tiles, color coding indicates a positive or negative change for each metric. Once a metric change has reached statistical significance, the color will be changed to Green or Red. Green indicates a positive change in the metric while Red indicates a negative change in the metric.
For any A/B Test or Holdback Message, you can select as many goals as you need. Since all data will be captured and analyzed for this test by default, goals can be added or removed from the test at any time, without any loss of historical data.
If you are running a test that includes a message, you will automatically get the default messaging metrics within the A/B Test. These metrics show statistical significance as well, making it very quick and easy to tell if one message variant is encouraging more engagement than another.
Just like every report in Leanplum, your favorite metrics will be shown for quick analysis. Favorites can be changed and updated at any time, from any report. Use this section to save your most important metrics which can be used to analyze the results of any test or campaign.
Significant Changes Metrics
Significant Changes is a section that Leanplum will automatically populate with any metric that has undergone a statistically significant change as a result of this test. This helps avoid unforeseen consequences. For example, a test may positively optimize some goals you pre-selected, but negatively effected other metrics that you did not think would be impacted by this test.