Analyzing A/B Tests and Messages by Unique Metrics

What it Does

Analyzing results by unique metrics is an important part of measuring the success of your A/B tests and messages in Leanplum.

Our normal metrics use "daily users" to calculate cumulative averages. Over the span of a 10-day test, for example, the same user might do X on day-1 and do X again on day-2. Our standard metrics will count these as two occurrences of X with two daily users. Unique user metrics, however, will group these occurrences together as two occurrences of X by one unique user, which gives you a far more accurate picture of your results.

Unique metrics enable customers to answer the question: How many occurrences of X per *unique* user happened over the course of my experiment or message. For example:

  • How many rides per unique user were taken over the duration of my experiment?
  • How many purchases per unique user occurred over the duration of my experiment?
  • How many level completes per unique user occurred over the duration of my experiment?

 

Supported Unique Metrics

Metric Definition
Unique Users Unique users
Sessions per unique user Total sessions / Unique users
Time in app per unique user Total time in app / Unique users
{{Event}} occurrences per unique user Total {{event}} occurrences / Unique users
{{Filtered metric}} occurrences per unique user Total {{filtered metric}} occurrences / Unique users
{{State}} occurrences per unique user Total {{state}} occurrences / Unique users

 

Unique user event metrics are automatically added to your report for any Message Goals. Unique metrics can be added manually as Goals or Favorite metrics via the metric chooser by navigating to occurrences > occurrences per unique user.

Unique User Metric Limitations

Unique user metrics are currently only available for some reports. If you try to view a unique metric when one or more of the following scenarios are true, you will see an error message overlaid on top of the graph and "N/A" on the Unique User metric tile.

  • Viewing a User Activity report (outside of A/B test or message analytics)
  • Date range does not include the start date of the A/B test or message
  • If any Filter or Group is added
  • If actual values are selected (unique reports should be viewed cumulatively).

The error message overlaid on top of the graph says "Unique user metrics are currently only available for some reports. Some parameters of your report need to be modified to view this report." For example:

Clicking "click here" on the error message will automatically reset any unsupported parameters from your report and allow you to view the unique metric.

 

We plan to add support for additional unique metrics in the future. Please reach out to support@leanplum.com or directly to your customer success manager with any questions or to to be whitelisted for Unique Metrics.





Was this article helpful?
Have more questions? Submit a request