Incrementality is like heaven: no-one knows how to get there

Is incrementality for mobile user acquisition impossible?

Or … is it the best thing in mobile attribution since unique and indestructible and easily available device identifiers (remember UDIDs)?

We’ve been writing about next-generation attribution on the Singular blog for a few months now, including media mix modeling, post-IDFA user acquisition, the end of last-click measurement, and the future of mobile measurement. Incrementality is a big part of that conversation as well because the impetus for all these conversations is the loss of signal that privacy measures — necessary though they may be — are causing for marketers.

The mobile measurement question

How will marketers measure, attribute, and optimize marketing in a privacy-safe ecosystem?

And let’s be honest, this is not just any old marketing we’re talking about. This is not selling real estate or sports drinks or Lululemon pants. Mobile user acquisition is perhaps the fastest-paced marketing niche around, where shortening the distance between stimulus and response is critical to campaign optimization.

But I’ve been putting off talking about incrementality for months now.

I’ve been intrigued by what I’ve been hearing from AppLift veteran Maor Sadra’s new startup INCRMNTAL, as well as what Brian Krebs, the CEO of MetricWorks, has had to say about incrementality as a key form of mobile marketing measurement.

I just haven’t felt ready.

Recently, however, I had a conversation with Moshi Blum for the Mobile Heroes podcast I do with Peggy Anne Salz for Liftoff.

He’s the VP of Beach Bum, a mobile game studio owned by Voodoo, was a general manager for Adjust (never heard of that company), led user acquisition for Viber, and more. And he knows incrementality, with the blood, sweat, tears, and scars to show for it, along with pretty much every other form of mobile measurement from both the high-volume practitioner side as well as the measurement provider side.

He’s kinda been there, done that on a lot of different levels. And he’s pretty realistic about the challenges and opportunities in marketing measurement.

In fact, if you remember Winston Churchill’s famous quote about democracy being the worst of all forms of government except for the rest, you’ll recognize the inspiration behind Blum’s view of last-touch attribution:

“Last touch attribution is the worst way to measure your marketing campaigns … except all other metrics of measuring your marketing campaigns.”

– Moshi Blum, VP Marketing at Beach Bum

Why is incrementality hard?

So why is incrementality, which is intended to show you the additional or incremental results of your marketing campaigns, so notoriously hard?

Because causes and effects are mixed up, and the relationships between individual causes and effects are spaghettied into difficult-to-separate masses. Also, many effects are over-determined, which means that they don’t have a single cause but multiple factors are working together to create an effect. (And multi-touch attribution perks up its ears …) Everything is changing all of the time as multiple departments in your organization are building product, releasing features, kicking off campaigns, posting to social, crafting offers, building creative. And shocker: the world is changing, as macro-level systems like weather and economy intersect with microcosms of individual situations and moment-by-moment states like hunger, desire, boredom, time, attention, and more.

So much so that separating out incremental impact can seem impossible.

“Over the experience we had with trying to understand how to calculate it or bring it even further from installs to revenue, from revenue to paying users, from paying users to understanding how much of what I spent on Google or Facebook or Apple or any other ad network is actually contributing to my bottom line of profit … that’s something that I found absolutely or almost impossible to get.”

– Moshi Blum, VP Marketing at Beach Bum

So incrementality for mobile user acquisition is hopeless?

Not exactly.

Every measurement methodology has its limitations. As Blum says in his quote above, last touch attribution — which he uses for Beach Bum currently — is the worst. Except, it’s also the best.

What incrementality can do?

Incrementality has some significant benefits. You could, for example, be adding a brand new app to your portfolio. With limited or no pre-existing campaigns, you can fairly easily check incrementality via different platforms, channels, and partners. (And, maybe even apply those learnings to your other campaigns for established, busy, and continuously growing apps.) In other circumstances you can pause most or all of your efforts on an app, put all your eggs in one basket, and check the results. While you know you’ve got some existing organic and some persistent lag from prior campaigns, you’ll get a useful read on a channel that you might have been wondering about.

Not only that, you’ll get a sense for the interplay between channels, especially as you see audience overlap between them. Here’s how Brian Krebs put it in a chat I had with him:

“The analogy I hear often is the fishing poles in the stream, right? It’s the same group of fish, each media source you’re adding is just another fishing pole.

And the critical thing here is not really to optimize your marketing based on what the last touch happens to be, the ads that happened to get the last touch. It’s really optimizing the media mix, which is optimizing the perfect number of fishing poles and the perfect mix of fishing poles in that stream.”

– Brian Krebs, CEO of Metricworks

It’s part of the attribution mix

The key is layering and weaving.

Layering in different measurement methodologies as needed. Weaving them together when and where appropriate. Not necessarily relying on just one but using them all to build up a multifaceted and modeled version of reality that is based as much as possible on deterministic and granular data and as much as necessary on probabilistic and aggregated information.

As we recently wrote about the future of mobile measurement:

“That future involves building out varying views of reality and integrating them intelligently into a single source of truth. At Singular, we’re looking at marketing performance from known and aggregated spend data, from deterministic last-click measurement, from probabilistic aggregated results data, from first-party data, and from other sources. All of those have their unique perspective on what is actually happening as marketers market, whether putting dollars to work or investing in organic promotion. Each of them has value.

But then they also need to coalesce into a single source of truth to provide a simpler modeled view of reality.”

– Ron Konigsberg, chief growth officer at Singular

Which means there’s a place for incrementality.

It’s not in micro-measurement of the details of a marketing campaign or the performance of one creative over another, or even the relative efficacy of one sub-campaign over another. That’s almost impossible, Blum says, and I think he’s right.

But there is an occasional role in getting good insight whether a campaign adds accretive value or not, or whether a channel is adding valuable fishing poles to the stream or even — could it be — fishing in a stream that few other channels access.

Incrementality also has specific value for specific channels like Apple Search Ads, where you can check organic volume on keywords and competing keywords. There, Blum says, it’s easier to measure your impact; whether you’re “buying your own traffic” (AKA wasting ad spend on already-were-going-to-install organic users), or defending your keywords from competitors, or actually creating a would-you-believe-it brand new install that wouldn’t have happened any other way.

(Note, that’s “easier,” not “easy.”)

Love it or hate it, last-click works

That fits where it fits, but most of the time, Blum says, he’s simply focusing on expanding growth through channels that perform well according to last-click mobile attribution data, whether that’s GAAD/AAID on Android or SKAdNetwork postbacks on iOS.

Where incrementality seems to fit best in mobile marketing is not as a day-to-day measurement methodology but as a monthly or more likely quarterly check-up on channel quality.

And that’s when you do the full meal deal test.

“What you’re doing is you’re really running a randomized controlled trial like you would in a pharmaceutical company … taking a population, dividing it up into two separate groups randomly — that’s key here — into a control group and an experiment group, or a treatment group, or a test group, whatever you want to call it. And that treatment group is the one that sees ads. The control group does not.”

– Brian Krebs, CEO of Metricworks

Clearly, that’s extra work. And because you’re likely pausing other activity while doing this kind of test — and potentially doing it for multiple channels — it takes time and has significant opportunity cost for apps that need to grow fast.

But it is a worthwhile investment, from time to time.

Just not the silver bullet we might wish it could be in an era of less signal and less hard data.

We can help

Working on incrementality? Need a full suite of data from cost to attribution to modeling to probabilistic? Singular can help.

Book some time to chat today.