Your Solution Design Reference - ObservePoint · organizations including Search Discovery and Numeric Analytics. His breadth of experience ... 1 The Forrester Wave™: Data Governance

Your Solution Design Reference:From Strategy to Set-up to Scalability in 7 Steps

2

Introduction and about the authorIn today’s Big Data world, where companies are drowning in data but often starving for insight, organizations can no longer afford to simply live with an analytics deployment slapped across digital properties by web developers or the IT department as an afterthought.

Companies jockeying for position in a highly competitive space, seeking to win consumers who are more empowered than ever before, must treat their data like an enterprise asset—perhaps the most valuable one they have.

It’s time for a business objectives-driven approach to deploying your data collection technologies, and this starts by creating a solution design reference (SDR).

Jason CallJason caught the “digital marketing bug” over ten years ago when his music went viral, and he became the first unsigned artist to reach a million downloads on the internet. Since then, he has devoted his career to mastering analytics and providing actionable insights for hundreds of clients, spanning many industries and verticals. Before working at ObservePoint, Jason held roles in Operations, Business Intelligence, Data Visualization and Optimization for respected organizations including Search Discovery and Numeric Analytics. His breadth of experience spans many facets of the analytics space, but he is most passionate about data governance, and is a leader and critical member of ObservePoint’s Professional Services team.

Senior Analytics Data Expert

3

Table of ContentsIntroduction and About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

What a Solution Design Reference Is, Why You Need One & How to Create One in 7 Steps . . . . . . . . . . . . . . . . . . . .4

1. Ownership of the SDR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2. Location and Format of the SDR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8

3. Collecting Business Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

4. Defining Metrics to Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

5. Building the SDR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

6. Coding the SDR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

7. Maintenance and Quality Assurance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4

What is a Solution Design Reference?Referred to by myriad names, including: variable map, tag plan, solution design, tech spec, site reference and more, a solution design reference (SDR) is the documentation of your digital analytics strategy and process—a foundational piece of your data governance framework.

A solution design reference is the blueprint for your analytics implementation.

An SDR takes business requirements identified by stakeholders throughout your organization and maps them to metrics that can be measured across your digital properties.

The SDR helps analytics architects know where, when and how to implement code.

5

Why You Need an SDRAn SDR creates a baseline for data governance processes, to ensure organizations are maximizing the ROI of their MarTech stack and collecting high-quality data to make strategic business decisions.

“Data Governance programs have a way of starting and then fading away. This creates peaks and valleys in an organization’s ability to engage relevantly with their analytics program. Forrester finds that 45% of organizations govern data inconsistently across business units.”1

Documentation is essential to collecting and governing high-quality data. Without an SDR:

• Business initiatives fall through the cracks• Analytics efforts fail• Code degrades• Data quality suffers

1 Customer Ecosystems Demand Outcome-Oriented Data Governance, Forrester Research, February 2016

6

07STEP

Defining Metrics to Measure

Location of the SDR

Coding the SDR

Building the SDR

Ownership of the SDR

Collecting Business Requirements

06STEP

05STEP

04STEP

03STEP

02STEP

01STEP

Maintenance andQuality Assurance

Building a Solution Design in 7 StepsThis white paper details 7 steps that support building and deploying a robust SDR across analytics teams and organizations.

Designed to build on one another, each step steers your team towards greater data governance capability in a simple and scalable way.

1. Ownership of the SDR2. Location of the SDR3. Collecting Business Requirements4. Defining Metrics to Measure5. Building the SDR6. Coding the SDR7. Maintenance and QA

Whether you are building a solution design in-house or utilizing the services of an expert agency, ensuring each of the above areas is addressed sets your team up for success.

7

Step One: Ownership of the SDRThere are two critical points of leadership for establishing the SDR and creating a culture of true data governance in an organization.

Data Governance Leader

With direct ownership of data governance and quality in an organization, a data governance leader should have the correct background and authority, including:

• A strong technical background• Familiarity with MarTech solutions• Knowledge of the platforms their digital properties are built on• Organizational seniority• Understanding of the business value of data• Thorough data orientation

Industry research indicates that organizations with dedicated data leader, what Forrester calls a Cheif Data Officer, are top performers:1

1 Top Performers Appoint Chief Data Officers, Forrester Research, August 2015

Data Governance Council

In supporting the data governance leader, a data governance council should be composed of business leaders from all areas of the company who understand their unique business objectives and the company mission. A data governance council should:

• Incorporate leaders from Marketing, Sales, IT, Operations, Finance, Human Resources, etc.

• Respond to requests of the data governance leader• Meet at least quarterly to review the SDR• Be a strategic priority

The key to having an effective data governance program is to prioritize and maintain it.

A regular meeting of the data governance council might include the following:

• Review quality assurance notes on the latest implementation• Discuss any outstanding actions on your implementation, such as

inactive tags• Review the business requirements and discuss potential additions

Top Performers Have A Data Governance LeaderTop Performers Low Performers

10%+ Revenue Growth <4% Revenue Growth

54% 33%

8

Step Two: Location and Format of the SDRTreat your SDR as a living, breathing document.

DO’S DONT'S Visibility Store your SDR somewhere that it can

be easily accessed by all stakeholders.Allow the document to get buried on someone’s computer.

High Touch Keep the SDR in a format that allows collaboration and frequent updates—this fosters a sense of community around data governance.

Use ambiguous labels, difficult formats or inappropriate access limitations.

Versioning Keep the SDR in a place that allows you to track changes, such as Google Docs/Sheets, Dropbox, Box or within a platform like ObservePoint.

Save the document in a static file.

“Governance needs to become more agile by means of dynamic collaboration.”1

1 The Forrester Wave™: Data Governance Stewardship Applications, Q1 2016, February 2016

9

Step Three: Collecting Business RequirementsEvery business has unique objectives and data needs. An SDR should always be aimed at understanding these needs in order to provide an actionable and accurate measurement and implementation plan.

Example: If you would like to know the channels visitors use to reach your website, you can include that as one of your business requirements.Answering the initial questions around identifying your business requirements usually involves asking another question like, "What is their objective for being on our site?" These secondary questions will lead you to determine specific use cases.

Example: You might be wondering, “Are visitors to the site finding the content that they are looking for?” Or even just, “What are visitors looking for?”You can quickly generate a list of unorganized questions, and then work to focus your collection. This will save you more time throughout the next steps of building your SDR.

Identifying your business requirements begins with business questions:

1. Why are we doing this?2. What do we want to know about the user?

Your business requirements should articulate, specifically, what you want to know about the user experience (UX).

10

USE CASE BUSINESS QUESTION

Marketing Acquisition Which marketing sources refer the most/least visits? (e.g., search, email, social, etc.)

Marketing Acquisition What percentage of email traffic results in a conversion event?

Campaigns Which specific marketing campaigns led to newsletter sign-ups?

Campaigns Which campaign channels are most effective at driving new visitors to the website?

Navigation How often do visitors return to the website?

Navigation What are the top exit pages?

Products & Content What are the top traffic pages?

Products & Content What product categories are most viewed on the website?

Conversion Events What percentage of visitors complete a conversion event?

Conversion Events What were the total number of conversion events during the specified time period?

Using a category system makes it easy to quickly prioritize the collected business questions.

John Lovett, Analytics Demystified

Analytics Demystified experts recommend categorizing the business objectives your stakeholders want to know and then mapping each question back to the appropriate category, or use case.

So, your collection document might look something like this:

Step Three: Collecting Business RequirementsEvery business has unique objectives and data needs. An SDR should always be aimed at understanding these needs in order to provide an actionable and accurate measurement and implementation plan.

Marketing Acquisition

Campaign

Product Demo Webinar

Newsletter

ConversionEvents

11

Step Four: Defining Metrics to MeasureAsk yourself, “What story does this metric tell? How can it help leadership make the right decisions?”

Every metric should be mapped to an end goal.

Once the data governance council has established what they want to discover, analysts and developers should work together to define appropriate metrics to generate actionable insights.

One of the challenges in this is that while analysts may have a solid understanding of how the data that is unique to each business goal should be analyzed and represented, they are often working with stakeholders who may not.

Generating example reports helps address this common disconnect.

It is tempting to skip this step because it is time-consuming, but generating mock analytics reports has several advantages, in that it:

• Validates that your team is collecting the right information• Shows stakeholders they will be able to make decisions about their digital asset with

these reports• Helps your analytics team understands the business requirements• Provides an opportunity to refine your metrics• Lessens the likelihood that your team will have to go back and implement tracking that

was overlooked during initial discovery

“Are my visitors finding the content that they are looking for?” If a user is looking for something on your site, they will most likely utilize the internal search function. Businesses must consider user search actions and how they can be measured.

The second part of the business requirement is:

“Did they find what they were looking for?” This can be measured by whether or not their search yielded results. The metric that combines these two use cases would be an internal search that yielded results.

Another sample question to consider is:

“Are visitors getting the information they need from the frequently asked question section of my website?” A corresponding user action could be clicking a link to the FAQ page or expanding a FAQ field. You can monitor the frequency of clicks on certain FAQs to understand what your visitors want to learn more about.

12

Step Five: Building the SDRAn SDR concisely outlines all the events and coding details to help your data governance council and designated programmers maintain coherency and consistency with your metrics.

ObservePoint - Solution Design - Scope - Q2When to Set Where to Set NotesExampleVariable DescriptionVariableUse-CaseStatus

Active

Active

Active

Active

Active

Active

Active

Active

Active

Active

Active

Active

Active

Frequently Asked Questions

Internal Searchwith Results

Internal Searchwith Results

Internal SearchWith Results

Internal SearchWith Results

Internal SearchWith Results

Internal SearchWithout Results

Internal SearchWithout Results

Internal SearchWithout Results

Internal SearchResults Click

All Pages

All Pages

All Pages

prop20

event10

eVar28

eVar29

prop11

event70

event79

eVar28

eVar29

prop11

event1

pageName

Channel

event

event

page load

page load

page load

page load

page load

page load

page load

page load

page load

page load

page load

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

http://www.domain.com/directory/page.html

Should be able to capture text in <div>FAQ Interactions (c32)

Internal Search (e10)

Internal SearchTerm (v28)

Internal SearchResults (v29)

Internal SearchPaths (c11)

Internal SearchResult Click (e70)

Zero SearchResults (e79)

Internal SearchTerm (v28)

Internal SearchResults (v29)

Internal SearchPaths (c11)

Page View Event

Pages

Site Sections

[faq description]

[search term]

[0-∞]

[0-∞]

[search term:page name]

[search term:page name]

[search term]

‘event1’

‘event79’

‘event70’

‘event10’

[page name]

[section name]

Once you have decided on a specific event for each metric, you can begin assembling your SDR.

Make sure that you give your SDR a name that is clear and concise, along with some context, like the website name and some sort of timestamp.

A comprehensive SDR is a matrix of use cases and variables. It should contain similar information to the chart provided.

13

3. VariableThe analytics variable that will be collecting the data, whether it be a prop, an evar or an event.

6. When to SetDoes the variable fire on page load, or is the event triggered with a click?

8. NotesYou can also include columns for QA notes and a Pass/Fail column if the tag is not working correctly to be used by quality assurance and for review during data governance meetings.

1. StatusTells you if the tag is active or relevant to the current implementation, or whether you plan on using it in the future.

5. ExampleAn example of what you expect the values to look like.

4. Variable DescriptionA quick explanation of the variable.

7. Where to SetA URL, stating where this variable will be placed.

2. Use CaseHow the user is interacting with the site.

About AnnotationWhen it comes to annotation, such as in using quotes to denote an expected finite value or brackets to denote a dynamic tag, make sure that your data governance council and your programmers are all on the same page.

Step Five: Building the SDRAn SDR concisely outlines all the events and coding details to help your data governance council and designated programmers maintain coherency and consistency with your metrics.

14

Step Six: Coding the SDRA completed SDR will provide programmers all the necessary information to code an effective analytics implementation that produces actionable data.

The SDR will serve as a reference point for all who collaborate on the project in the short and long terms.

Once you have documented your SDR, your team can really dive in and start programming the implementation in a staging environment.

This will usually be done by an IT team, but with the amazing power of some tag management systems, marketers can also build analytics rules themselves, easing demand on the IT department.

If you don’t have the resources to code your implementation in-house, don’t be afraid to work with a contractor or an agency that specializes in analytics implementations. The only thing worse than no data is bad or misleading data—and you could save yourself a lot of frustration and headaches if you outsource this essential task to an experienced analytics firm.

Data Platform Engineer

SDR

Senior ManagerAnalytics Digital

Manager Programmatic

Analytics

15

Missing, misfiring or duplicate tags can result in incomplete or misleading data, which destroys confidence in your data for decision-makers. Always double- and triple-check your tagging integrity, even when using a tag management system.

Step Seven: Maintenance and Quality AssuranceValidate your analytics implementation against your SDR during staging, development and after release.

When to Perform QA:• In a staging environment

before releasing your implementation

• After releasing your implementation

• Periodically to ensure that any updates or changes have not affected the functionality of your implementation

Automated Quality Assurance

Tag management systems have made it extremely easy to build and deploy code, both for programmers and marketers. If not properly regulated, however, this can lead to issues with your implementation. Massive amounts of tags deployed on multiple sites, with subdomains and different languages operating on different browsers, can make monitoring and maintaining all of these implementations a very sticky job—one that takes thousands of hours of manual spot-checking and validation.

Thankfully, automation is possible.

With an automated QA solution, rule-based scans can be used to run audits on thousands of pages to ensure that your analytics tags are firing correctly, according to the expectations established in your solution design.

Active critical-path monitoring notifies you if your critical paths are functioning properly so that users can move through your site towards conversion.

Constant validation allows you to keep your baseline true.

16

100 Page Audit

61

Audit Summary

Audit Score100

908070605040302010

0

Audit of 100 pages completed on 06/23/2016

Audit Score Over Time

06/21/2016

100%Pages Tagged

100Pages Audited

84%Status 200 - OK

33JavaScript Error Pages

9.6 secAverage Load Time

Primary Tags

454Duplicate Tags

0Failed Vendor Rules

6Failed Business Compliance

OPTIONS

Help

Step Seven: Maintenance and Quality AssuranceValidate your analytics implementation against your SDR during staging, development and after release.

Example of an automated audit from a website ranking in the top 2,000 most popular websites, according to ranking.com. This audit covered 100 high-traffic pages.

Automated analytics tools save you time and manpower for maintaining an effective analytics implementation. Quick takeaways:

• 454 duplicate tags indicate inflated data

• 6 failed business compliance rules means 6% of pages aren’t collecting data as expected.These rules can be set on the variable level—in this case PageName is missing.

• Only 84% of pages with status 200, leaving 16% of pages likely suffering from excessive redirects, risking SEO scores. Another possibility is that some of these pages are returning 404 errors, meaning your visitors can’t find content.

• 33 pages with JavaScript errors puts a third of the pages at risk for not firing tags and collecting data properly.

• 9.6 seconds for average load time is well above the three-second acceptable average, suggesting potential high-traffic loss and unfired analytics tags.

This quality assurance audit took about 5 minutes from set-up to completion and provides the URLs where issues occur so they can be addressed immediately to promote your data to trustworthy quality.

17

Further Enhance Your SDR with Automation

Just as automated audits of your analytics implementations save valuable time and resources, there are some automated offerings that can augment the creation, storage and usability of your newly created SDR.

The Digital SDR is an excellent free solution that integrates with the Adobe API and ObservePoint API to document your analytics implementation.

The output is a matrix of all your variables in an SDR format. This document is easy to consume, share and edit. Best of all, it is all done automatically, saving you valuable time and the pain of manual work.

An automated solution like this can be used to document your existing analytics implementation by populating the information from your site into a Google Sheet in minutes.

This newly generated SDR can then be given to your web analytics governance council for review so that they can adjust strategy and action as needed.

Step Seven: Maintenance and Quality AssuranceValidate your analytics implementation against your SDR during staging, development and after release.

Solution Design Reference Builder

SDR builder allows you to create and manage a map of all your analytics variables in a Google Sheet.

18

Summary1. Ownership of the SDR2. Location of the SDR3. Collecting Business Requirements4. Defining Metrics to Measure5. Building the SDR6. Coding the SDR7. Maintenance and QA

Building a successful analytics implementation is essential to understanding and responding to user behavior and maximizing the functionality of your website. Tag management systems, automated quality assurance and other analytics tools have simplified the process of deploying and maintaining analytics implementations.

The best way for your organization to accomplish your goals is to really take control of your data governance and develop a SDR outlining all of the business requirements that you hope to achieve through your implementation.

Doing so emphasizes that your marketing data is an enterprise asset—one that drives your competitive advantage and ultimately your bottom line.

Defining Metricsto Measure

Location of the SDR

Coding the SDR

Building the SDR

Ownership of the SDR

Collecting BusinessRequirements

Maintenance andQuality Assurance