How Eye Tracking Research Helps You Run Better A/B Tests

Visual Engagement Metrics: Your Optimization Secret WeaponVisual Engagement Metrics: Your Optimization Secret Weapon

Visual Engagement Metrics » Your Optimization Secret Weapon

Visual Engagement Metrics: Your Optimization Secret WeaponVisual Engagement Metrics: Your Optimization Secret Weapon

Agenda● Quick conversion research overview ● What is visual engagement research ● How to set up an eye tracking study ● Case Studies from Sticky ● Case Studies from ConversionXL Institute

○ Ecommerce Product Pages○ Visual Cues on a Landing Page

● Q&A

Visual Engagement Metrics: Your Optimization Secret Weapon

Visual Engagement Metrics: Your Optimization Secret Weapon

What’s a Winning Optimization Process?● Run as many tests as possible at all times (every day without a test

running on a page/layout is regret by default)● Win as many tests as possible● Have as high impact (uplift) per successful test as possible

Visual Engagement Metrics: Your Optimization Secret Weapon

There are many frameworks. Here’s ours.

Visual Engagement Metrics: Your Optimization Secret Weapon

ResearchXL● Finding opportunity areas

○ Digital analytics○ Technical analysis○ Heuristic analysis

● Finding possible solutions (and more opportunity areas)○ Qualitative surveys○ User testing○ Mouse tracking / session replay

Visual Engagement Metrics: Your Optimization Secret Weapon

Marrying qualitative & quantitative dataYou’re trying to answer a few questions:

● Who is my customer?● How are they interacting with my site?● What are their possible problems?● How can we potentially fix them?

Visual Engagement Metrics: Your Optimization Secret Weapon

PXL Prioritization Model

Visual Engagement Metrics: Your Optimization Secret Weapon

Continuous innovation in customer understanding

● Better and better ways to answer “how are users interacting with my site?”

● These insights help form better A/B tests (and make you more money)● Introducing Sticky.AI

Visual Engagement Metrics: Your Optimization Secret Weapon

What is Visual Engagement Analysis?It’s essentially a visual display of the user experience at a scale large enough to infer trends and insights.

Visual Engagement Metrics: Your Optimization Secret Weapon

Types Visual Engagement Analysis

<6% correlation with mouse movement and eye tracking

Scroll Click/Hover Eye Tracking

Visual Engagement Metrics: Your Optimization Secret Weapon

How Visual Engagement Analysis Helps

● See if people get stuck● See if important elements are

being missed● Know if something is being

read● Know in what order elements

are being seen● All used to optimize layout,

creative, CTA placement, and much more.

Important branding is not seen

Majority of visual attention focused on celebrity face

Visual Engagement Metrics: Your Optimization Secret Weapon

What is Eye Tracking?Sticky is leader in web-based eye tracking

Visual Engagement Metrics: Your Optimization Secret Weapon

Why Visual Engagement is AwesomeData is displayed as an aggregate which is a valuable way to understand the overall experience of visitors

Correlates with >70% of action, purchase intent, likeability, recall and click

Accurate on device measurement using built-in camera

Scalable implementation and analysis

Quick data visual output - available in just hours

Simple drag and drop setup, participant interface, no install required

500,000,000 gaze points measured

amazon.com ( press Ctrl key 3 times to turn highlighting on/off)

display ad replacement (Look for the smiley face)

Innovative Technology

Visual Engagement Metrics: Your Optimization Secret Weapon

Analyzing Norwegian Airlines’ Visual Engagement

Visual Engagement Metrics: Your Optimization Secret Weapon

How to Run a Visual Engagement Study

Visual Engagement Metrics: Your Optimization Secret Weapon

According to this map, there’s too much going on.

Visual Engagement Metrics: Your Optimization Secret Weapon

The improved version includes smaller images and a more

prominent call to action

Visual Engagement Metrics: Your Optimization Secret Weapon

Results

The CTA button is now the main focus after the redesign:

● 1st thing seen

● Seen 10% more

● Seen 2X longer

● Button is noticed 2X

Visual Engagement Metrics: Your Optimization Secret Weapon

UX Research with Eye-Tracking to Create Better Tests

We combine visual engagement metrics with surveys that determine things like recall, perception, and recognition to conduct better tests and find more opportunity areas.

Information Processing Theory

Incoming info (e.g., website homepage)

sensory memory

working memory

long-term memory

Visual Engagement Metrics: Your Optimization Secret Weapon

UX Research with Eye-Tracking to Create Better Tests

We combine visual engagement metrics with surveys that determine things like recall, perception, and recognition to conduct better tests and find more opportunity areas.

Information Processing Theory

Incoming info (e.g., website homepage)

sensory memory

working memory

long-term memory

Eye-Tracking

Post-task Survey

Visual Engagement Metrics: Your Optimization Secret Weapon

Examples - 1. Optimizing the eCommerce Product page2. Visual Cue Study

Ecommerce Product Page - Three part study:

1. How Product Image Size Impacts Value Perception2. How Product Image Size Affects Attention and Engagement3. How Does Product Copy Format Affect User Engagement?

Visual Engagement Metrics: Your Optimization Secret Weapon

How Product Image Size Impacts Value Perception

● Hypothesis 1: Larger Image = Higher Value Perception● Hypothesis 2: Hypothesis 1 might change with different product classes

Visual Engagement Metrics: Your Optimization Secret Weapon

How Product Image Size Impacts Value Perception

experience good

search good

Visual Engagement Metrics: Your Optimization Secret Weapon

How Product Image Size Impacts Value Perception

experience good

search good

+ ~$2

+ ~$15

Visual Engagement Metrics: Your Optimization Secret Weapon

How Product Image Size Affects Attention & Engagement

● Hypothesis: A large product image will garner more attention than a smaller one.

Visual Engagement Metrics: Your Optimization Secret Weapon

How Product Copy Format Affects User Engagement

● Hypothesis: Product information will be viewed differently depending on how it’s formatted

● 9 Variations based on description format - Paragraph, Bullets, Nothing (control)

Visual Engagement Metrics: Your Optimization Secret Weapon

What can we do with all this data?

● Quantify high impact opportunity areas (copy format, image size, etc.)● Develop stronger hypotheses

○ “Increasing product image size will increase order volume”

● Improve testing prioritization/strategy○ “Higher lift for techy type products”

Visual Engagement Metrics: Your Optimization Secret Weapon

Visual Cues StudySettle debates with data, not years old case studies

Which visual cue works best?

Visual Engagement Metrics: Your Optimization Secret Weapon

We studied 6 variations

● Human looking away form● Human looking towards form● Arrow● Triangular ● Line● Prominent form

Visual Engagement Metrics: Your Optimization Secret Weapon

Visual Engagement Metrics: Your Optimization Secret Weapon

Visual Engagement Metrics: Your Optimization Secret Weapon

Visual Engagement Metrics: Your Optimization Secret Weapon

Visual Engagement Metrics: Your Optimization Secret Weapon

Visual Engagement Metrics: Your Optimization Secret Weapon

Visual Engagement Metrics: Your Optimization Secret Weapon

Results?

Visual Engagement Metrics: Your Optimization Secret Weapon

Other studies using same approach:

Social Proof

Value Proposition

Pricing Page Plan Order

Trust Seals

Homepage Banner Ads

Ecommerce Benefit List Placement

Online Reading Patterns

Visual Engagement Metrics: Your Optimization Secret Weapon

Conclusion: Tying it All Together

Visual engagement studies lower the cost of A/B testing by focusing your design changes on the areas that need it most, making the live-traffic split experiments more effective.

Visual Engagement Metrics: Your Optimization Secret Weapon

Questions?

Visual Engagement Metrics: Your Optimization Secret Weapon

CXL-STICKY-WEBINAR

Conversion optimization training program