New this week (11/24/2016): Impression and Held Back Events for A/B tests

Both of these new events allow you target users based on their prior A/B tests, namely whether they were involved in a test (Impression Event) or if a message was not delivered to them because their A/B test group had that particular message disabled (Held Back Event).

Target users involved in prior A/B Tests

For all A/B tests going forward, an Impression Event will be tracked to determine whether or not a user entered the test.

If the A/B test has an impression criteria set, then the event is tracked upon the impression criteria being triggered. See here to learn more about impression criteria and when to use it.

If no impression criteria is set, the event is tracked immediately upon the user being placed in the test.

You can leverage Impression Events as targeting criteria for your messages and A/B tests. For example, adding a target of “occurrences of event ABC Impression is greater than 0” will only allow users who were included in A/B test .a123 into your test.

Impression events appear in all event lists on the dashboard and can be identified by “.aAB_TEST_ID”. For example, for an A/B test with name ABC, the event name would be “ABC Impression". Additionally, the variant ID is tracked as an event parameter. Going forward, impression events also appear in the raw data export with the same naming convention.

Target users that did not receive a message due to an A/B test

For all messages in A/B tests going forward, a Held Back Event will be tracked to help determine why a user did not receive a message (despite matching the targeting criteria). Now, all users in an A/B test group that have a message disabled will receive a Held Back Event for that message to signify that they would have received the message but it was "held back" for the A/B test.

Like all Leanplum Events, you can leverage Held Back Events as criteria for messages and A/B tests. For example, if you want to re-send a message ("Welcome Message") to users that did not receive it as part of the A/B test, you could add a target of "occurrences of Welcome Message Held Back is greater than 0."

Held Back events appear in all event lists on the dashboard and can be identified by "<message name> Held Back". For example, a message with a name "Onboarding" would potentially have a Held Back event titled “Onboarding Held Back." 

Was this article helpful?
Have more questions? Submit a request