Bringing an Experimentation Mindset to Personalization

COVER PAGE

Bringing an Experimentation Mindset to Personalization

Delivering more impactful and personalized experiences through experimentation

Bringing an Experimentation Mindset to Personalization 2

Personalization has been a hot marketing topic for the last several

years. In fact, these days, it’s one of the most frequently discussed

marketing trends. And when you think about it, that makes perfect

sense. After all, who wouldn’t want to make their customers’

experiences feel more customized and engaging? Connecting

individually is becoming dramatically more important, particularly

as the millennial market matures and has more spending power.

However, personalizing experiences in an effective way isn’t as

easy as it might seem. It requires significant thought, planning, and

technology to engage customers and get a return on your investment.

The good news is, we believe there’s a solution to this personalization

challenge—a solution that’s both feasible and scalable for most

businesses. That solution requires leveraging data and applying

an experimentation mindset in order to personalize. To devise the

right approach to a personalization campaign, you need to iterate

and experiment on everything. It’s essential to make decisions

about your digital presence based on data and information, rather

than best guesses. We also believe that, in order to personalize,

you need to test and learn. That means gathering data about your

customer interactions, determining what works and what doesn’t,

understanding why, and then personalizing messages in a way that

will ensure success.

INTRO

To devise the right approach

to a personalization

campaign, you need to

iterate and experiment on

everything.


For those less familiar with the business benefits of personalization,

let’s review why personalization is a business critical element of

customer engagement and retention. According to Forrester, 89%

of digital strategy professionals say that personalizing the customer

experience has been an investment priority for the last 12 months.1

And it’s not just an organizational trend. There is also data to indicate

that 78% of US online adults have chosen, recommended, or paid

more for a brand using personalized experiences or services.2

Today, being able to personalize depends on your ability to closely

examine your customers’ journeys and understand how they engage

with your products and brand. The first step in this direction is to map

the customer journey and then determine each of the components

or attributes that you want to personalize. Understanding where a

personalized experience can impact a customer’s experience is key

to your success.

It’s undeniable that personalization is valuable, but how do you

actually accomplish it? That’s the question this white paper is here to

explore. It will address the challenges of personalizing, the thought

process behind it, and how effective experimentation can ultimately

help you achieve your business goals.

Digital Strategy Professionals prioritize investing in personalized customer experiences

Customers wanting personalized experiences or services

1 Forrester, “Vendor Landscape: Personalization Solution Providers, Q3 2017”. October 20, 2017.2 Forrester, “Vendor Landscape: Personalization Solution Providers, Q3 2017”. October 20, 2017.

Understanding where a

personalized experience

can impact a customer’s

experience is key to your

success.

89%

78%


To start, let’s clarify a bit of terminology. Experimentation is both

a process and a mindset, one that enables organizations to

continuously test and iterate on different hypotheses. There are

different ways to proceed with this. Two of the more common ways

are to run A/B tests and to run personalization campaigns. When

making decisions, it’s important to maintain an experimental mindset,

regardless of your type of test or hypothesis. Also, in order to be

successful with any type of personalization and experimentation, you

need to continuously test and learn. Personalization without constant

experimentation can result in wasted time and resources, and in

some cases can be detrimental to the business. Personalization can

negatively impact your customer experience and business metrics if it

isn’t properly tested and informed by data.

ITERATE YOUR WAY TO A PERSONALIZED EXPERIENCE

Personalization without

constant experimentation

can result in wasted time and

resources, and in some cases

can be detrimental to the

business.


A/B testing is all about testing to understand what most customers

want from their experience. It’s a critical component of testing

and businesses can benefit greatly from its use. In essence, what

you’re doing is optimizing for the majority of people. A/B testing

gives you the largest and most immediate wins. However, once you

have experimented in this way and seen the benefits, you can then

continue to test and learn in additional ways. Personalization is one

of those ways. If you analyze the data, you will find that different

experiences will work better for different groups of customers,

which may lead to new hypothesis for new groups. Every customer

is unique, and each one will be drawn to different experiences.

Personalization creates a custom experience for each individual

based on data. Once you segment out the experience that you’ve

optimized for, we’re then able to see that the winning experience was

far better for certain groups while it was possibly worse for others.

The reality is that the winning experience isn’t the same for everyone.

However, by segmenting, and then personalizing for each segment,

you can engage individually with customers and make sure that every

single segment (and thereby individual) is having a better experience.

But the question still remains—how do you know that your

customers are having a better experience? You can track all sorts

of metrics online. You can spend countless hours mapping each

individual customer journey. While the time and effort you invest in

understanding what really resonates with each of these segments

is important, without experimentation the process would be

overwhelming. Experimentation isn’t simply the key to optimizing

this personalized experience. It’s the only way to develop a

personalization strategy that works. It’s the only way to iterate and

discover what genuinely resonates with prospects and customers.

Embracing the experimental mindset is vital to your success when

personalizing.

The reality is that the winning

experience isn’t the same for

everyone.


Let’s walk through an example of how the process works. One of our

customers, AeroMexico, is working to improve conversions through

their checkout process. They often witness people nearly completing

the flight purchasing process only to abandon their carts at the last

moment. The airline believed if they were more upfront about the

total price of each flight package, they would decrease the number of

people abandoning their flight purchases and increase conversions.

They began by testing that assumption. Based on their A/B test, they

saw significant increases in conversion. Additionally, upon viewing

the results, they decided to personalize the experience by making it

easier for customers who had abandoned the process to return to the

precise flights that they had searched for and purchase them later.

They conducted this experiment using Optimizely X Personalization

and saw an increase in revenue and sales conversions, demonstrating

that they effectively streamlined the checkout process for their

customers. This example illustrates how a personalization experiment,

derived from an A/B test, enables you to benefit from both the initial

A/B uplift and the subsequent gain from continuing to engage with

customers in a personalized way.

BA


There may be no limit to the number and variety of possible tests you

can perform. The reality is you’ll never know until you try. Even after

experimenting, and iterating, your customer experience may not be

100% perfect. Iteration is key to your success. Amazon, Facebook,

Google, and Netflix are all known for having extremely personalized

experiences, significantly more so than the typical retailers and

media companies. What’s their secret? They all experiment on every

aspect of their personalization strategy. Netflix famously ran a contest

where anyone could submit a better recommendation algorithm and

they would test it. Google tests every new search algorithm against

the former one. Facebook and Amazon experiment on every part

of their user experience to learn about their users’ reactions. All of

these organizations have created a culture of experimentation and

are seeing results. Experimentation gives organizations the data to

know what’s working, and what isn’t, when engaging with customers.

Organizations can then use that data to improve on the experiment

and iterate. This process of continuous experimentation and constant

feedback becomes even more important for personalization, because

customer engagement is at the very core of personalization.

By experimenting with

personalized use cases, it

will quickly become evident

when your ideas aren’t

working. For example, your

copy might be resonating

with most demographics,

but when you change it

to seem more youthful,

your 65+ audience reacts

negatively. By combining

personalization with an

experimentation mentality,

this becomes very clear,

very quickly.


There are a broad host of challenges that organizations run into

when implementing personalization strategies. One of them is that

you really have to know your audience and have the data to support

personalization. If you don’t, your personalization campaigns won’t

work. There are other starting points. You can certainly iterate and

experiment your way to success by trying different things across the

board. It is possible to personalize based on behavior. But the more

data that you gather about your customer base and how you want to

segment them, the better off you are.

Here are a few fundamentals that you need to have in place before

you start to truly personalize your experience for customers.

Develop a deep understanding of your

audienceSegment your audience based on

factors like demographic, contextual,

and behavioral factors

Have a plan to identify where you

can personalize

Know what metrics you are trying to

optimize for/personalize for

THE IMPORTANCE OF DATA


Working through each of these above components will ensure that

you have a cohesive personalization plan. Additionally, it’s important

to note that you need to set up repeatable frameworks to define

and identify audiences. This is not a one-and-done exercise. Your

plan must be flexible enough to change as the individuals your

organization targets change. If any of these elements is missing, it can

be challenging to accurately engage your audience, to see a return

on your personalization campaign investment, and to effectively build

a personalization campaign.

There are a variety of aspects that you can choose to personalize

as well. Some of them may be visitor behavior, while others are

demographics like gender or age. You may also have many different

options regarding where on your site you want to personalize. Once

these four key components have been identified, you are ready to

personalize.

With a data-oriented personalization approach, it is important to

take a strategic and analytical approach towards personalization.

Trying to oversegment and dig too deep initially, before you get

traction and results, can lead to significant audience challenges

and misperceptions. This is another reason why iteration is key to

personalization—by experimenting with your messaging for each

audience, you gather more data and achieve better long-term impact.

Challenges can be

amplified when you aren’t

looking at the data. Layers

of mistakes about how

you are engaging with

audiences can have a

multiplier effect, causing

a greater impact, and not

always in a positive way.

Fortunately, Optimizely’s

personalization is enabled

by Stats Accelerator. That

means you can determine

statistical significance for

such situations as quickly

as possible. As a result,

you’ll be able to engage

with all of your prospects

and customers, and not

just some of them.


When people think about personalization, they often imagine creating

one-to-one experiences. They envision individualized customer

experiences predicated on their detailed customer journey map.

Individual personalization is an admirable goal. However, when

we refer to effective personalization and experimentation, we are

typically talking about segmenting and creating actionable audiences,

at least to begin with. Maybe you segment into a higher volume

audience (e.g., all female shoppers in northern California), or a high

value audience (females with purchasing behaviors that make them

your top 1% of customers). Either way, the goal is to personalize for

business impact. Often, at least initially, when you’re experimenting

and personalizing, you’re not addressing specific individuals, but

instead larger audiences where you learn what works for specific

groups of individuals. This gives the business substantial ROI, and is a

natural activity as your experimentation maturity grows.

High Volume Audience

High Value Audience


The crux of personalization is this premise—the right answer isn’t

the same for everyone. Each person will have different reactions,

budgets, and expectations that shape their experience. By tapping

into those individual characteristics of a person, you can build

customer loyalty by acknowledging that your customers are all

different. Customers appreciate that. A/B testing is fantastic at

finding the best experience on average and can help you achieve a

solid ROI. However, to truly engage with all of your customers, you

need to personalize. Gartner notes, “[a]n effective personalization

strategy enhances customer experience, leading to reduced costs

as well as increased revenue from greater customer satisfaction,

loyalty, and advocacy.” 3 The combination of these factors means

that personalization is an essential strategy to organizations trying

to engage with customers. It is the combination of iterating on

personalized experiences, though, that will enable organizations

to see true results. Let’s spend some time looking at how

personalization is done, depending on an organization’s familiarity

with experimentation.

ITERATING TO (NEAR) PERFECTION

3 Gartner, “Use Personalization to Enrich Customer Experience and Drive Revenue”. 20 December 2017.

5-15% of Traffic,

2-5 Campaigns

30-60% of Traffic,

10-20 campaigns

75-100% of Traffic,

25+ campaigns


When an organization first starts out with personalization, we

would expect them to be tackling relatively simple audiences

and personalization campaigns. Typically, these audiences are

natively available and have simple defining conditions. Often, we

see organizations begin by creating only 2-3 audiences defined by

complexity, reach, and how actionable each audience is. At first, we

would expect tactics or use cases like image swaps and changing of

modules. This phase of personalization should only reach a limited

population, around 5-15%, and is intentionally limited. What you are

doing is building the personalization muscle. Experimenting with

personalization on smaller groups to ensure that you understand

how to structure personalized campaigns allows you to engage with

the process and learn. We typically see customers do this for 2-5

experiments.

In the early stages, it’s often smart to start with relatively simple use

cases. These use cases allow us to develop that personalization

muscle, while not overcomplicating aspects of the campaign, and

still positively impacting the business. Examples include tests like

leveraging previous customer behaviors and tailoring the site based

on those behaviors when customers return. So, if a customer is on an

e-commerce site, and looks at shoes, when they come back to the

site after navigating away from it, they will see a homepage tailored to

shoe purchasers, and not generic shoppers. This is a basic behavioral

test, but it can really help amplify your messaging to ensure that your

site resonates with customers. A customer was actively looking for

something, so it’s important to use that behavioral information and

generate targeted messages during subsequent engagements.

Step 1


Once you’ve built your organization’s personalization muscle, you

would expect to increase both the number of campaigns (roughly

10-20) as well as the audience reach (about 30-60%). You would

also expect organizations at this stage of personalization maturity

to have integrated with first- and third-party data, and to have some

integration of testing and personalization work flows. By this time, you

should be targeting intersecting audiences (e.g., an iPad user in New

York, rather than someone who is in New York or an iPad user). We

would also expect to see very interactive elements, dramatic changes,

and more one-to-one personalized recommendations and content.

The types of experiments you conduct will also change once your

personalization maturity grows. Depending on your business,

there are a variety of use cases you can explore. One example

would be using purchase history data to engage with product

recommendations. You have data suggesting what customers look

at after purchasing specific products, and you can use that to target

individuals with similar purchasing behaviors. Another example is

experiments related to symmetric messaging. This is where you

personalize your site and engagement channels to ensure that they

all tell the same story. Because digital marketers utilize multiple

channels to engage with customers, it is important to ensure

everything is aligned. For instance, if a woman searches for handbags

on Google and is directed to your website, you want to make sure

she can find handbags. Symmetric messaging can significantly help

increase your ROI and reduce CPL from acquisition campaigns. It

is one example of a relatively simple, yet important experiment that

drives the business forward.

Step 2


With organizations that exhibit the greatest level of personalization

maturity, we would expect to see fully-integrated systems and

extremely iterative testing. Data modeling and predictive analytics

should drive new audience strategies, and use cases would undergo

relentless iteration to ensure that the organizations are driving new

audience strategies. Here, we would expect them to personalize for

75-100% of the audience, in over 25 personalization campaigns per

year, with each of them undergoing continuously iteration. These

personalization campaigns would be focused around granular

audiences that have been discovered using analytics and modeling,

and old audiences that have been iterated on and improved upon.

This is where having extremely detailed data is integral to the

testing process. Organizations need to utilize and leverage their

significant customer data at this stage. An example of an experiment

during this maturation phase would be looking at customers who

have abandoned their carts, examining the purchase behaviors of

those who have completed purchases, and then personalizing new

checkout promotions based on those behaviors. Because Optimizely

is PCI compliant, it is entirely feasible for a company to personalize

at every step in the checkout process. While experimenting, you can

also test new offers, as well as cross-sell and upsell other products,

based on similar customer purchasing patterns. Of course, this

situation involves multiple levels of customer data and poses some

potentially complicated outcomes, which is why it is better suited for

more mature organizations.

Most of these examples can be more complex, or less complex,

depending on the data aspects you want to experiment with and

your overall business goals. The important thing to remember is to

hypothesize about what a good personalization experiment would be,

and then to learn everything you can while you conduct it.

Step 3


As we’ve mentioned, it is important to adopt an experimentation

mentality and try different things on your journey towards

personalization. It is, without a doubt, the potent combination of

personalization and experimentation that allows organizations to

create the kind of digital experiences that engage their customers

and impact their businesses. Personalization without experimentation

is risky. But by embracing them both, your organization can transform

itself. And the ways in which you can personalize and experiment

are endless—from symmetric messaging to creating end-to-end

personalized experiences. By exploring them all, your organization

will be able to discover what engages customers, apply what it

learns to future experiments, build agile customer experiences, and

make itself more competitive than ever. Wherever you are on the

path to personalization and experimentation, Optimizely can help by

providing the services, tools, and results that will move your business

forward. Visit optimizely.com/personalization to learn more about our

products and understand how Optimizely can help you get started on

the path to personalization.

CONCLUSION

Optimizely can help by

providing the services, tools,

and results that will move

your business forward.

ABOUT OPTIMIZELY X PERSONALIZATION

Optimixely X Personalization enables you to connect with customers on a personal level by delivering targeted content in real time and personalizing based on demographic, contextual and behavioral information.

Learn more about how Optimizely can help you get started on the path to personalization at optimizely.com/personalization

Bringing an Experimentation Mindset to Personalization

Documents