History Many teams have made great efforts to use relational databases in projects only to have to supplement them in order to meet performance demands.

History

Many teams have made great efforts to use relational databases in projects only to have to supplement

them in order to meet performance demands.

Common 3-tier Architecture

Physical Architecture• Web Farm

• Caching Server

• CDN - Media

• Application Server

• Search Engine Server

• Database Server

• SANS or Disk Array

• In the 90s Microsoft development best practices, code samples and designs started with designing the database first

• Eventually ORMs NHibernate 2004, Linq to SQL Nov 2007, and Entity Framework August 2008

The Path:Education, Experiences, Vendor Influences

• ORMs solve the problem of interacting with the database by mapping domain objects to relational tables (CRUD)

• Effects with NHibernate focus more on domain (EF eventually follows)

• We have taught BAs about CRUD and other technology terms to cope with constraints

Performance of SELECT mapping over 500 iterations - POCO serializationMethod DurationRemarksHand coded (using a SqlDataReader) 47msDapper ExecuteMapperQuery<Post> 49msServiceStack.OrmLite (QueryById) 50msPetaPoco 52ms Can be fasterBLToolkit 80msSubSonic CodingHorror 107msNHibernate SQL 104msLinq 2 SQL ExecuteQuery 181msEntity framework ExecuteStoreQuery 631ms

http://code.google.com/p/dapper-dot-net/

Performance of SELECT mapping over 500 iterations - typical usageMethod DurationRemarksLinq 2 SQL CompiledQuery 81ms Not super typical involves complex codeNHibernate HQL 118msLinq 2 SQL 559msEntity framework 859msSubSonic ActiveRecord.SingleOrDefault 3619ms

http://code.google.com/p/dapper-dot-net/

Common Performance Tweaks• Indexes

• De-normalizing Tables

• Pinning Tables

• Full Text Search

Scaling Up or Out?• Scale Up with SQL Server is cheaper than

scaling out (licensing)

• SQL Azure

Our Efforts• ORMs learning curve

• 30 to 40 % of the project for saving data

• In pursuit of Normalization / Normal Form

Normalization in reality isn’t normalization at all!• First step in performance: Caching,

caching, and more caching

Why did we do this again?

Changing your perspective• Simpler design tends to be better

• MVC is a successful UI pattern because it is a simpler solution than say web forms

What are we doing today?• What is caching? A key/value database

• Taking advantage of caching

Document Database is one solution• Mongo, CouchDB, RavenDB etc…

Changing your Model to deal with performance• Is this the answer?• No silver bullet• Great to have in the tool belt• One view one model

Advantages of document database

Craigs List – scheme change took over a month to process using MySQL Farm

Question:

How is the data being used and what is the data being used for?

Event based architecture –

events happen in a system

capture the changes

CQRS - Command Query Separation

Separating your reads from your writes

At its heart is a simple notion that you can use a different model to update information than the model you use to read information. This simple notion leads to some profound consequences for the design of information – Martin Fowler

Writes (Commands)• Are commands• Service Bus• Handlers on different machines• Publish event to db• Publish event to cache• Publish event to search engine

• Events stored in event store• A true audit log

• Reply events to populate cache• Reply events to find exact state of user!

Events as a Storage MechanismSave data as binary in SQL Server or JSON in document database

From Jonathan Oliver’s Event Store

CREATE TABLE [dbo].[Commits]( [StreamId] [uniqueidentifier] NOT NULL, [StreamRevision] [int] NOT NULL CHECK ([StreamRevision] > 0), [Items] [tinyint] NOT NULL CHECK ([Items] > 0), [CommitId] [uniqueidentifier] NOT NULL CHECK ([CommitId] != 0x0), [CommitSequence] [int] NOT NULL CHECK ([CommitSequence] > 0), [CommitStamp] [datetime] NOT NULL, [Dispatched] [bit] NOT NULL DEFAULT (0), [Headers] [varbinary](MAX) NULL CHECK ([Headers] IS NULL OR DATALENGTH([Headers]) > 0), [Payload] [varbinary](MAX) NOT NULL CHECK (DATALENGTH([Payload]) > 0), CONSTRAINT [PK_Commits] PRIMARY KEY CLUSTERED ([StreamId], [CommitSequence]));

Events as a Storage MechanismSnapshots to lower the level of mining on multiple events

From Jonathan Oliver’s Event Store

CREATE TABLE [dbo].[Snapshots]( [StreamId] [uniqueidentifier] NOT NULL, [StreamRevision] [int] NOT NULL CHECK ([StreamRevision] > 0), [Payload] [varbinary](MAX) NOT NULL CHECK (DATALENGTH([Payload]) > 0), CONSTRAINT [PK_Snapshots] PRIMARY KEY CLUSTERED ([StreamId], [StreamRevision]));

Events as a Storage Mechanism

Aggregate: A collection of objects that are bound together by a root entity, otherwise known as an aggregate root. The aggregate root guarantees the consistency of changes being made within the aggregate by forbidding external objects from holding references to its members.

Example: When you drive a car, you do not have to worry about moving the wheels forward, making the engine combust with spark and fuel, etc.; you are simply driving the car. In this context, the car is an aggregate of several other objects and serves as the aggregate root to all of the other systems.

Reads (Query)• Reads on cache machines

• Reads on search indexes

• Sharding available on some cache solutions

• Transactions available on some solutions

Effects• Effects on business BAs – now talking about

business

• Simpler development – more productivity

• Need scalability add machine for reads, writes, or bus

• Audit log – exactly what the user was doing

• Build cache

• Build development environment

• Task Based UI for writes

Physical Parts• Service Bus

• Caching

• Web Server

• Servers for Handlers

• Database

Task Based User Interface• Single URL application

• More complicated

• Pub / Sub with Node or SignalR

Resources

Gregg YoungUdi DahanJonathan OliverRinat Abdullin

http://distributedpodcast.com/http://www.cqrsinfo.com/http://martinfowler.com/bliki/CQRS.htmlhttp://www.udidahan.com/2009/12/09/clarified-cqrs/http://abdullin.com/cqrshttps://github.com/joliver/EventStorehttp://ncqrs.org/http://ncqrs.codeplex.com/releases/view/43035