Taming the chaos of personalisation through the foundations of experimentation

I’ll admit, I have a complicated relationship with Personalisation. I’m using a capital P here because I’m not talking about just the application of personalisation.

Don't get me wrong. When personalisation is planned and delivered well, the impact it has on a user’s experience can be second to none.

But no, the complicated relationship I’m talking about is with the hype levels that often drive teams to try jumping into personalisation yet never getting the most from it.

It’s from the fact that personalisation has been “the next big thing” for so long. It's the way it seems to hit 5 spaces of buzzword bingo all by itself. And the way that personalisation is often done from a fear of falling behind rather than really considering what it can provide.

Okay, therapy session over, that’s enough about complicated relationships.

Again, done right, personalisation provides so much value. There’s a reason it comes up so often for every organisation, that the netflixes of the world attribute so much of their success to helping individual customers find exactly what they need at the right moment in time.

That makes perfect, logical sense to us. Have a tailored conversation with your customers as they visit your site, hit them with the message they need and wham. You can get great, bespoke customer experiences PLUS a lift in sales? It’s EXCITING.

Maybe too exciting?

As much as it’s great when the need for personalisation stems from a genuine customer use case - all too often it actually starts with a great case study at a conference or a consultant touting it as the solution to all of your problems.

From there the tools get bought and set up, the team gets a bit of training, and away we go.

Chaos is the word that typically comes to mind when I think of this stage.

The excitement that comes in early on is both thrilling and easy to get swept away in. We dive into creating experiences for every single customer group we can think of, hoping they will hit just the right notes.

Unfortunately, that same excitement makes it incredibly easy to find ourselves with a few traits likes these for our personalisation program that we’d rather avoid.

1. Disjointed experiences

Often, especially in the context of web, personalisation means different things to different people. Often even people within the same organisation.

Each time we walk into a room with a new team talking personalisation, we have to ask what springs to mind with that team.

Are they picturing a product recommendations engine? An onboarding experience that evolves step-by-step? Remembering the inputs from a form? Putting a picture of a small blonde boy in the hero for your Swedish visitors?

Even with only a handful of people around a business running different forms of personalisation to the individual cohorts they care the most about, it’s easy to see how this can get messy quickly.

2. Deliveries that miss the big picture

Decentralised teams diving straight in can create another problem - a series of personalised experiences that solve lots of small problems but don’t trace back to overall business outcomes or the biggest problems customers are facing.

When you’re getting into minute segments of users, it’s dangerously easy to zero in on a painstaking set of metrics or changes that only matter to a couple of people. What’s harder is to zoom all the way out to understand how it ties back to the overall impact of a personalisation program.

This especially tends to happen when these experiences are created one at a time, from siloed teams and individual practitioners.

3. Results are hard to measure

Teams new to personalisation will also often launch without a measurement plan to understand success, suddenly running personalisations out to 100% of everyone who fits the criteria for a segment.

All we’ve seen from all these case studies is that personalisation works every time so surely we can just launch it and it’s going to work right? …Right?

At the same time, there can be little thought put into whether or not the way a personalised experience was delivered was the right one. The message is more tailored, sure, and maybe even it really is measurably more effective than an unpersonalised experience.

What if it could be even better or address an even bigger need for that set of users?

What’s that in the sky? A bird? A plane? No, it's experimentation!

^{I couldn't find any experimentation superhero comics but surely there’s something out there}

There’s a reason that, at Drumline, we often classify personalisation under the umbrella of experimentation. There’s a huge overlap in methodology, skillsets, and even the tools that are often used between the two.

There’s also a reason personalisation-first tools often talk about having AB testing capability and experimentation-first tools often have personalisation features.

Solving the problems we’ve set out above is something that experimentation handles incredibly well from the challenges around measuring success to tying your program back to business goals that senior leaders care most about.

Using experimentation to build a strong foundation

The habits we form to set up a strong roadmap, ensure we have robust problem statements backing each test, plan how we measure success - these are all foundations that are heavily emphasised and rewarded when delivering experimentation.

Those same habits also help to move past the issues that often pop up in personalisation programs.

Let's look at the benefits of an effective backlog, for instance, as a communication and planning tool.

It’s not rocket science - a single place where you can keep track of what’s been released, what’s in progress, and what’s planned for the future provides a broad overview that is useful for everyone. It’s critical in ensuring cohesiveness, even where several teams might be responsible for their own experiments they’d like to run.

Yet it’s something that’s still missed for many teams that are running personalisation.

What about tying a program back to larger business and customer goals? That’s part of the principles for experimentation too!

As the mad scientists we are, we’re often planning a series of tests we want to set them out in a way that combines their learnings to tackle a larger, overarching problem. When we first map out problems we want to tackle, we may have 3 or more potential solutions for any problem and each test can build on the learnings of the test before.

Applying this to personalisation sets us up to plan ahead and demonstrate a better level of traceability back to the big challenges.

Without planning in bulk and in advance, it’s far too easy to fall into bad habits with either ticky tacky little changes that don’t support the big picture. Ultimately, your program needs to be driving success for the big goals your company has.

That doesn’t mean every test or personalisation has to be revenue generating but they also can’t be trying to solve problems that are too small.

Measure, measure, measure

The thing is, even to this point we haven’t touched on the primary reason we can argue that personalisation should fall under the umbrella of experimentation.

It isn’t just that they have a small overlap through process and tools - the real reason comes down to the fact that running personalised experiences as experiments is the single best way to understand the impact you’re having. How else do you know if your personalisation efforts are effective if you aren’t testing the difference of personalised against your standard experiences!

How do you know the version of a personalised experience you’ve created is the best one without testing other variations? If we’re already set up to experiment, personalise, and experiment on our personalisations, then we’re in prime position to drive great outcomes.

We can let the goals of our organisations and our customers drive the way we deliver rather than just needing to personalise because we think it is a cool thing to do.

How do you see the two?

After all that, I’d love to know - how do you see the differences and overlap between personalisation and experimentation?

Whether this has brought you around to seeing the two more closely interlinked or whether you see them as entirely different, these are important conversations to have as they improve the ways we use all of the tools available to us. Drop a note below or reach out directly, I’m always happy to talk shop.