Sitecore xDB Explained

A N U N D E R S TA N D A B L E E X P L A N AT I O NO F H O W x D B W O R K S

THE SITECORE xDBEXPLAINED FOR MARKETERS

SITECORE EXPERIENCE PLATFORM (XP)

Sitecore XP consists of…• Sitecore Content Management System

(CMS)• Sitecore Experience Database (xDB)• Sitecore Experience Marketing applications

• Experience Analytics• Experience Profile• Experience Optimization• List Manager• Path Analyzer• and more…

SITECORE EXPERIENCE DATABASE (xDB)

Sitecore Experience Database (xDB)• New digital marketing management

features

• Creates a unified view of customers• Collects interactions from multiple channels• Connects interactions to create detailed views

• Makes data available in real-time • Used to engage with

contacts & customers

• Marketers can optimize engagement value

SITECORE xDB DATA STORAGE

Sitecore xDB stores customer analytics data• xDB is a data repository for analytics data• It uses a special type of data storage setup• Multiple types of data are stored together• Facilitates data mining & related research

SITECORE xDB CONTACTS

Contacts in xDB• Persons who interact with your organization

by using one or more devices

• Used as a container to store customer information• The devices they use• Their online interactions• Social network activity• Email campaigns• Website visits

SITECORE xDB ACCOUNTS

Accounts in xDB• Used to organize contacts into groups

• Usually based on company or organization• Identified by Business Name & Country

• Business units are connected to an account• Each account has at least

one business unit• Business units are identified

by Business Name, Country,Region or City

SITECORE xDB DEVICES

Devices in xDB• Contacts use devices

to interact with sites• Browsers• Smart phones• iPads & other tablets

• Device details are stored in collection DB

• Device is linked to contact for each session• One device can be linked to multiple contacts • Multiple devices can be linked to one contact

Client & customer interactions• How a contact engages with organizations• Occurs over a specific media or channel• Involves a two-way exchange of information• More trust than passively

viewing web pages• Demonstrates a higher

level of commitment• Contact provides

identifying information• Login• Email

SITECORE xDB INTERACTIONS

SITECORE xDB CUSTOMER EXPERIENCE

xDB customer experience• Record of all online interactions stored in

xDB• Interactions between organization & customer• Exists for the full duration of the relationship

xDB architecture options• On-premise server solutions• Single standalone server • Multi-server scalable setup

• Hybrid server solutions• Partly on the premises &

partly in the cloud• Example: MongoDB in cloud

• Full-scale cloud solutions• All Sitecore components installed in the cloud• This includes all Sitecore CMS & xDB components

SITECORE xDB ARCHITECTURE OPTIONS

NoSQL DATABASES

Not only SQL (NoSQL) databases• Developed for apps with special needs• Structured data storage• Unstructured data storage• Combination of structured & unstructured data

• There are multiple types of NoSQL databases• Key-value stores• Graph databases• Document databases

NoSQL DOCUMENTS DATABASE

NoSQL documents database• Uses text documents to store data• All data is stored as human-readable text• Data values are nested in a tree structure• Document format is similar to JSON or XML

• Any data item can be different from all others

• Related data are stored together

• Use MongoDB for thexDB NoSQL database

MongoDB DATABASE

MongoDB database• NoSQL document database used by xDB• Open-source • More info is at https://www.mongodb.org/

MongoDB BENEFITS

MongoDB key benefits• High performance & availability

• Great for Agile development• Very flexible • Schema can be

updated on-the-fly

• Can store many typesof data together

• Automatic scaling

SITECORE xDB DATABASES

Sitecore xDB databases• SQL Server was not fast enough for volume

of data collection required by high-traffic sites

• xDB uses two types of databases• A collection database on MongoDB (NoSQL)• A reporting database on MSSQL Server

• xDB databases are in addition to Sitecore DBs

• Can be run on separate servers or one server

Collection database powered

by MongoDB

Reporting database powered

by MSSQL

xDB COLLECTION DB CONFIGURATION

Setting up the xDB collection database• Use MongoDB for the collection database• Use a dedicated server for performance• Verify the MongoDB working set fits in memory• Use solid state physical drives

• Consider a cloud-based or hybrid xDB• Add servers to scale MongoDB horizontally• Horizontal partitioning

of the DB is “sharding”• Helps to spread the load

across multiple servers

COLLECTION DB DATA COLLECTION

The collection DB collects analytics data• NoSQL documents database for analytics• Should be implemented using MongoDB• Initial storage for

all collected data• Collects data from

all your websites• Contacts• Devices• Interactions

COLLECTION DB DATA AGGREGATION

Collection DB performs aggregation tasks• Connects to the collection & reporting DBs

• Extracts data from the collection database

• Groups & reduces raw analytics data

• Transforms analytics data for reporting apps

• Transfers data from collection DB to reporting DB

Collection database

powered by

MongoDB

Reporting database

powered by MSSQL

COLLECTION DB DATA PROCESSING

Collection DB performs processing tasks• Performs update tasks on collection DB• Keeps reporting DB in sync with collection

DB• Does routine maintenance on collection DB • Rebuilds reporting DB upon request

xDB REPORTING DATABASE

Sitecore xDB reporting database• Reporting database is a data warehouse• Stores analytics data from diverse sources• Gets aggregated data from collection DB• Data optimized for Sitecore Experience

apps• Experience Analytics• Experience Profile• Path Analyzer• List Manager• and more…

REPORTING DB CONFIGURATION

Setting up the xDB reporting database• Use SQL Server for the reporting database• Use MSSQL 2008 R2 SP1 or higher • Install SQL Server Management Studio

• Use a dedicated server to improve performance• Install generous amounts

of physical memory• Use solid state

physical drives

REPORTING DB ORGANIZATION

Reporting database organization• Performs the role of the old Analytics DB• Uses Data Warehouse (DW) schema• Data is in Fact & Dimension tables• Fact table contains the data to be analyzed• Measurements• Metrics

• Dimension tables support Fact tables• Text fields that contain descriptive attributes• Used to analyze the data in fact tables

SESSION STATE

Session state overview• Allows storage of user-specific analytics

data• Works for a specified time window• Useful for personalizing content delivery• Stores session details in collection database• The contact• The device • The interaction

SESSION STATE MODES

Session state configuration settings• Addresses limitations of HTTP requests• Ties contact’s requests to previous requests• Tracks user session across series of requests

• Session variables stored on a per-user basis• Associates all requests

from same browser• User data is stored as

user navigates web pages

SESSION STATE INPROC MODE

Session state In Process (InProc) mode• Default session state for xDB• Only suitable for standalone installations• Mostly used for test & development• Stores session state data in memory• Uses the IIS

application pool to store data

• Bad approach for scalability & reliability

THE IIS APPLICATION POOL

IIS application pool (AppPool) overview• A group of one or more URLs in IIS• Can be serviced by a single worker process• Can be serviced by a set of worker processes

• All in-proc state data lost on AppPool restart • App Pool restarts

on a scheduled basis• Webmaster has limited

control of restarts

xDB OUT-OF-PROCESS PROVIDERS

Out-of-process session state providers• Store session state values & variables in

DBs• Session state preserved if app is restarted• Store data in the collection DB by default• Session state store provider for

MongoDB• Session state store provider for SQL

Server• Support the Session_OnEnd event• Required for multi-server environments• Session state

is availableacross a Web farm

SESSION STATE SERVER

Session state server• Database that stores the session state• May not be a separate physical server• Contains the data required to serve a

session• Identifying the contact• Previous behavior

of the contact• Characteristics

of the contact

xDB SESSION STATE TYPES

There are two types of session state in xDB• Private session state contains all the

information related to an individual visit/interaction

• Shared session state contains contact & device information that can be shared over multiple sessions on a server or cluster

xDB PRIVATE SESSION STATE

xDB private session state• The standard ASP.NET session state data• Collects data related to a specific

interaction• Separate sessions are created when…• Contact uses multiple devices• Contact uses multiple browsers on same device

• Saves data to the collection DB by default• Pages visited• Events triggered• Goals converted

xDB SHARED SESSION STATE

xDB shared session state• This is a new xDB-specific session state• Collects data on a contact & device(s)• Saves data to the collection DB by default• Engagement automation states triggered• Contact details• Devices used

SESSION STATE SESSION ID

Session state session identifier• Unique ID used to track individual sessions• Stored in a non-expiring session cookie• Browser stores the session cookies• Sends to server with each HTTP request

• Server checks each pagerequest for a SessionID• Starts a new session

if no SessionID value• Sends new SessionID

to browser with response

COOKIELESS SESSIONS

Cookieless sessions• Web apps have other ways to store

Session IDs without using cookies • Device Profile• URL

SESSION STATE LENGTH

Session state length• A session is active as long

as it continues to use the same SessionID value

• A session expires when itexceeds the timeout value

• The default timeout value is 20 minutes• Requests made with an expired SessionID

value result in a new session unless the Regenerate expired session ID is checked

• Checkbox is disabled for cookie settings

SESSION END

Session end• Used by xDB to track website visits reliably• Data is flushed to the collection database• Data is scheduled for processing

• Supported by inProc session state mode• Supported by xDB session state providers

xDB REPORTING ARCHITECTURE

The xDB reporting architecture• Reporting service queries to fetch report

data• Data from collection DB (MongoDB)• Data from reporting DB (MSSQL Server)• Data from CRMs & other custom data sources

• Some visitor data is onlystored in collection DB

• Some data is not aggregated to reporting DB

xDB PROCESS FLOW

xDB PROCESS FLOW

How data flows through the xDB1. A contact interacts with an xDB website2. The visit may be redirected to a cluster3. The visit may be redirected to a server 4. Contact stays connected to the same cluster & server even

if they switch browsers or devices5. xDB stores details of the contact, interaction & device in

either private or shared session state6. On SessionEnd, analytics data is flushed to the collection

database & scheduled for processing7. Customer data in the collection database is aggregated for

use by the segmentation index8. Online visit interaction data is reduced, grouped & stored in

the reporting database9. The reporting service queries the reporting & collection

databases to fetch data for use in reporting applications

xDB STANDALONE ENVIRONMENT

xDB standalone installation• xDB supports a standalone, all-in-one setup• All components installed on the same server• Most suitable for development & testing• Not recommended for production sites

• One Sitecore server with all xDB components• Content databases (master, core & web)• Collection database (MongoDB)• Reporting database (SQL Server)

• No need to share session state across servers• All session state can be stored in memory• Helps reduce traffic between the app & DBs

xDB VERTICAL SCALING

Vertical scaling is used for SMB organizations• Install xDB components on separate servers• Add CPU & memory to server(s) as needed• Only upgrade xDB components that need it

• Content delivery• Collection• Reporting

xDB HORIZONTAL SCALING

Horizontal scaling is for large organizations• Organizations that handle lots of web traffic• Groups that need reliability & high availability

• Deploy components across multiple servers• Content management• Content delivery• Collection• Processing

• Use clusters to maintain high availability• Use load-balancing to optimize

performance• Use multiple geo-distributed collection

points

xDB AUTHORITATIVE REFERENCES

xDB Sitecore documentation• xDB Overview and Architecture Reference• http://sdn.sitecore.net/upload/sitecore7/75/

xdb_overview_and_architecture_sc75-usletter.pdf

• xDB Configuration Guide Reference• http://sdn.sitecore.net/upload/sitecore7/75/

xdb_configuration_guide_sc75-usletter.pdf

MongoDB AUTHORITATIVE REFERENCE

MongoDB online reference manual• http://docs.mongodb.org/manual/

SITECORE xDB SUMMARY

Fireworks are pretty but they only last a few seconds

xDB is not about dazzling your contacts & customers

It’s a set of online marketing tools that enable

you to develop lifelong relationships with them

CAVEAT EMPTOR

• There are many ways to implement Sitecore xDB• Experienced vendors can propose alternative

solutions• Review development needs with a Sitecore Partner• Review proposed Sitecore sites with all stakeholders

• This presentation is intended to be academic• To enhance public knowledge of issues related to

xDB• To promote better understanding of Sitecore xDB

• Author cannot guarantee accuracy or fitness foryour site• Author makes no representations or warranties• Author disclaims any liability if you act on this data

THE INSIDE CORNER, INC.

• The Inside Corner, Inc. (TIC) is owned by Craig Nelson

• TIC is a Sitecore partner & web consultancy

• TIC is located in Benicia, CA (San Francisco Bay Area)

• A U.S. point of contact for teams of Sitecore specialists

• We create new Sitecore sites & enhance existing ones

TIC OWNER - CRAIG NELSON

• Sitecore Specialist• 10 years of system administration experience• 10 years as a Microsoft Certified Trainer• 5 years of ASP.NET development experience• 4 years of Sitecore development experience• 3 years as a Sitecore technical instructor• Sitecore architecture & configuration expert• Performance & SEO optimization specialist

MORE SITECORE TOPICS FROM CRAIG

All Sitecore-related slide shares from Craighttp://www.slideshare.net/cwnelson/presentationsSitecore XP8 User Interfacehttp://www.slideshare.net/cwnelson/sitecore-xp8-comparisonHow to Assure Success for a New Sitecore Site (full slide deck)http://www.slideshare.net/cwnelson/creating-new-sitecore-sites-fullImplementing Routine Sitecore Maintenancehttp://www.slideshare.net/cwnelson/sitecore-maintenance-tipsTroubleshooting Sitecore Performancehttp://www.slideshare.net/cwnelson/troubleshooting-sitecore-performanceSitecore 7 – New Features for Usershttp://www.slideshare.net/cwnelson/sitecore-7-new-featuresSitecore Performance Tips for Visitor Engagementhttp://www.slideshare.net/cwnelson/sitecore-performance-tips

LET US CREATE YOUR SITECORE SITE

Invite Craig to connect with you on LinkedIn• http://www.linkedin.com/in/craigtic

Add Craig to your Circles on Google Plus• https://plus.google.com/114214657038667808289

Follow Craig on Twitter • https://twitter.com/TheInsideCorner

Contact Craig to get help with creating a new site• Craig Nelson – Sitecore Certified Developer &

Partner• craig[at]ticdevs.com• Mobile - 707.319.7446Visit Craig’s Sitecore instructional site• http://www.ticdevs.com

THE SITECORE xDBEXPLAINED FOR MARKETERS

Thank you for taking the timeto view this presentation

Please share with others & add a comment if you think it is useful

Thanks! - Craig