SQLUG event: An evening in the cloud: the old, the new and the big

• intended audience : – SQL DBA’s, SQL Server enthousiasts, non Azure Database developers

• goal : – Enlight the most valueable aspects on Windows Azure SQL Database and all related material

An evening in the cloud:

The old, the new and the big

Techmike2KX.wordpress.com

@Techmike2KX

[email protected]

Me.About();

A quick intro to Windows Azure

SaaSPhysical Virtual

A Continuous Offering From Private To

Public Cloud

IaaS PaaS

Cloud ServicesPackaged Software

Storage

Servers

Networking

O/S

Middleware

Virtualization

Data

Applications

Runtime

You

man

age

Infrastructure(as a Service)

Storage

Servers

Networking

O/S

Middleware

Virtualization

Data

Applications

Runtime

Managed by vendor

You

man

age

Platform(as a Service)

Managed by vendor

You

man

age

Storage

Servers

Networking

O/S

Middleware

Virtualization

Applications

Runtime

Data

Software(as a Service)

Managed by vendor

Storage

Servers

Networking

O/S

Middleware

Virtualization

Applications

Runtime

Data

Cloud PlatformsPublic Cloud Platforms

Private Cloud Platforms Infrastructure as a service (IaaS)

Platform as a Service (PaaS)

For Scalable Web Apps For Data-Driven Business Apps

For Collaboration Apps

Microsoft

Others

Dynamic infrastructure Toolkit for System

Center (Hyper-V Cloud, AppFabric Server)

Dynamic DataCenter Toolkit for Hosters

(Hyper-V Cloud)

Hosters

Windows Azure Platform

VMWare vSphere (ESX, vCenter)

Hosters

VMWare vCloud Initiative (ESX, vCenter)

Amazon Web Services

GoogleAppEngine

Saleforce.comForce.com

GoGrid Cloud Hosting, RackSPace, others

Dynamics CRM / xRM Online Office 365

Windows Live

Google Docs

Amazon Web Services

Cloud services Web sites Virtual machines

Applicationbuilding blocks

StorageBig data

Caching

CDN

Database

Identity

Media

Messaging

Networking

Traffic

DBSQL databaseRelational SQL Server Engine in the CloudClustered for high availabilityFully Managed ServiceSQL Reporting support

A SQL Database Architecture in Azure

A Server Is Not A Machine

SQL Server

A Machine

SQL Database

Server

A TDS Endpoint

How It Works

ArchitectureClient Layer - Used by application to communicate directly with SQL Database.Services Layer – Gateway between Client layer and Platform layer.Platform Layer – Includes physical servicers and services that support the Services layer.Infrastructure Layer – IT administration of the physical HW and OS.

Infrastructure Layer

PHPWCF Data Services

SQL ServerApplications

and Tools

ODBC ADO.NET

Tabular Data Stream (TDS)

Client Layer

Services Layer

Provisioning

Billing and Metering

Connection Routing

Provisioning

Billing and Metering

Connection Routing

Provisioning

Billing and Metering

Connection Routing

…

…

…

TDS+SSL

Platform Layer

SQL Server

SQL Database

Fabric

Management Services

SQL Server

SQL Database

Fabric

Management Services

SQL Server

SQL Database

Fabric

Management Services

…

…

…

Database Replicas and Failover

Replica 1

Replica 2

Replica 3

DB

Single Database Three Replicas

Single Primary

Replica 4

!

Provisioning a SQLDatabase in Azure

Server ProvisioningServer DefinedService head that contains databasesConnect via automatically generated FQDN (xxx.database.windows.net)Initially contains only a master database

Provision Servers InteractivelyLog on to Windows Azure Management PortalCreate a SQL Database serverSpecify admin login credentialsAdd firewall rules and enable service access

Automate Server ProvisioningUse Windows Azure Platform PowerShell cmdlets (or use REST API directly)wappowershell.codeplex.com

Create Database…Use Familiar TechnologiesTransact-SQL

Languages.NET Framework (C#, Visual Basic, F#) via ADO.NET

C / C++ via ODBC

Java via Microsoft JDBC provider

PHP via Microsoft PHP provider

FrameworksOData, Entity Framework, WCF Data Services, NHibernate

ToolsSQL Server Management Studio (2008 R2 and later)

SQL Server command-line utilities (SQLCMD, BCP)

CA Erwin® Data Modeler

Embarcadero Technologies DBArtisan®

SQL Server ComparisonFocus on logical vs. physical administration

Database and log files automatically placed

Three high-availability replicas maintained for every database

Tables require a clustered index

Maximum database size is 150 Gb

Unsupported SQL Server FeaturesUse command, linked servers, distributed transactions, distributed views, distributed queries, four-part names

Service Broker

Common Language Runtime (CLR)

SQL Agent

Native Encryption

*Backup / Restore

Enhanced ToolingSQL Database Management PortalWeb designers for tables, views, stored procs

Interactive query editing and execution

SQL Server Data Tools (SSDT)Visual Studio IDE for database development

Includes modern designers and projects with declarative, model-driven development

Develop and test in both connected and disconnected states

Platform targeting for both SQL Server (2005 and above) and SQL Database

Get it free with Web PI, with SQL Server 2012 and with Visual Studio 11

Database Deployment

Data-Tier Application Framework (DAC Fx)Alternative to traditional script based approach

Dramatically simplifies deployment, migration and versioning of databases

Provides a single unit of deployment for schema (dacpac) or for schema + data (bacpac)

Supports automatic versioning of database schemas

Supports platform targeting for both SQL Server (2005 and above) and SQL Database

Build from scratch or extract from existing db

How To Get The Latest DAC FxWith SQL Server Data Tools

With SQL Server 2012 Management Studio

With SQL Database Import/Export Service

Via sqldacexamples.codeplex.com

Feature pack supportWhat’s Supported

ConstantsConstraintsCursorsExtension of spatial data types and methods through CLRIndex management and rebuilding indexesLocal temporary tablesReserved keywordsSpatial data and indexesStored proceduresStatistics managementTransactionsTriggersTables, joins, and table variablesTransact-SQL language elements such as

Create/drop databasesCreate/alter/drop tablesCreate/alter/drop users and logins

User-defined functionsViews

What’s not SupportedCommon Language Runtime (CLR)Database file placementDatabase mirroringDistributed queriesDistributed transactionsFilegroup managementGlobal temporary tablesSQL Server configuration optionsSQL Server Service BrokerSystem tablesTrace Flags

T-SQL• supported :

http://msdn.microsoft.com/en-us/library/ee336270.aspx

• partially supported : http://msdn.microsoft.com/en-us/library/ee336267.aspx

• unsupported : http://msdn.microsoft.com/en-us/library/ee336253.aspx

There Are Two Ways To Secure A Database:

Within The Database

On The Server

Server Benefits

Server identity and access controlSQL authentication supported (No Integrated authentication)

The Admin login is similar to sa

Connect to master to administer logins

loginmanager: Server-Level security role for creating logins

dbmanager: Server-Level security role for creating databases

Database BenefitsDatabase identity and access controlLogins require an associated user account

The Admin login is automatically associated with dbo

The dbo has full rights in the database

Manage users with CREATE / ALTER / DROP USER commands

Add users to roles via sp_add_rolemember to grant privileges

Utilize schemas where appropriate

SQL Database Firewall

Securing your dataIP Address-based access control for SQL Database

Rules can be defined at the server and database

No IP authorized by default

Configurable using the SQL Database Portal and REST API

Option to disable/enable access from applications hosted in Windows Azure

Services Layer

Services Layer

Internet

SQL Database Firewall

Application ConnectivityConnecting To SQL Database1. TDS (Tabular Data Stream) protocol over TCP/IP supported

2. SSL required

3. Use firewall rules to connect from outside Microsoft data center

ASP.NET EXAMPLE:

Considerations And Best Practices1. login: [login]@[server]

2. Idle connections

3. Long running transactions

4. DoS guard

5. Failover events

6. Throttling

7. Connection pooling and Retry logic

8. Latency introduced for updates

9. No cross-database dependencies

<connectionStrings><addname="AdventureWorks"connectionString=

"Data Source=[server].database.windows.net;Integrated Security=False;Initial Catalog=ProductsDb;User Id=[login];Password=[password];Trusted_Connection=False;Encrypt=true;"

providerName="System.Data.SqlClient"/></connectionStrings>

BI Capable SQLDatabase in Azure

SQL ReportingVisualize your dataSQL Server Reporting Services technology as a serviceIdeal for operational reporting against SQL Database dataEnterprise-ready with automatic support for HADesigned to scale elastically with demandRapid Provisioning

Architecture – What’s Under The Hood

Load Balancer

Gateway GatewayGateway

RS InstanceMonitor/Comm[SLA & Usage][Performance][Traces & Logs]

RS Engine

RS InstanceMonitor/Comm[SLA & Usage][Performance][Traces & Logs]

RS Engine

RS InstanceMonitor/Comm[SLA & Usage][Performance][Traces & Logs]

RS Engine

SQL Database

RS GPMTenant Data

Operations

Directory Services

Allocation + Service

Customer Service

SLA Conformation

Load Balancing

Billing/Usage

Monitoring[Alerting]

[Performance][Traces & Logs]

http://server.reporting.windows.net/reports/report.rdl

SSRS Azure SSRS

Tooling BIDS, Report Builder BIDS, Report Builder

Data Sources Assorted Data Sources SQL Database

Report Management Report Manager or SharePoint Windows Azure Developer Portal, URL Browsing

Developer Custom Data Sources, assemblies, etc.

No extensibility yet

Security Model Windows Authentication SQL Database Authentication

Feature Comparison

The cloud and Beyond

SQL Data Sync

Data SynchronizationMicrosoft Sync Framework technology as a service

No code setup and configuration

Synchronize datasets hosted in SQL Database or SQL Server

Customizable to fit your business needs

SQL Databas

e (US)

SQL Databas

e (Hub)

SQL Server

(OnPrem)

SQL Databas

e (WE)

SQL Data Sync

Components And Architecture

HTTPS

TDS

SQL Database

Windows Azure Blobs

Data Sync Agent

Windows Azure PortalAdmin UI

HTTPS

Service Data• Configuration - Servers, Sync

Groups, Schemas, Agents, etc.• Tasks queue - sync, get schema,

provision, etc.• Logging - task results

Service Data• Temporary batch files of

uploaded data changes

Batch Files• Temporary files containing

changed data to upload

Windows Service

UI

Admin UI Handler

Cloud Task Execution

Task Scheduler

- Get tasks- Task results

Browser

Agent Task Execution

TDS

User DatabasesAdded database objects:• Change Tracking tables• Sync meta-data tables• Triggers & sprocs

User DatabasesAdded database objects:• Change Tracking

tables• Sync meta-data

tables• Triggers & sprocs

Key:Windows Azure Web or worker roles

Syncing Between SQL Server And SQL Database

SQL Server

Application

On-Premises

SQL Databa

se

Application

Cloud

E.g. Migration periodE.g. Different apps sharing same dataE.g. DR

Multiple Locations (e.g. branch)Share database between locationsAggregate data in cloud

One-way sync to cloudOne-way sync from cloudTwo-way sync

Sync

Sync

SQL Server

Application

Syncing Between SQL Databases

SQL Databa

se

Application

SQL Databa

se

Application

SQL Databa

se

Application

SQL Databa

se

Application

Sync

Sync

Sync

Cloud

Scale-out via multiple copies of dataSeparate reporting & OLTP workloads

Geographically located web applications

Data Sync And Reporting

SQL Databa

se

Application

SQL Databa

se

Reporting

One-way Sync

Different Indexing

Filtered Subset of Data

Sync

SQL Data Sync And Windows Azure Traffic Manager

US

SQL Databa

se

Application

SQL Databa

se

Application

Sync

Europe

SQL Databa

se

Application

Asia

Sync

Traffic ManagerControl traffic distribution Azure-hosted

applications

DNS-based

Several Distribution Options

Behavior And Database ImpactSync ProvisioningChange Tracking via triggers and side-tables

Minimal DML overhead

No change to user tables

Stored Procedures and meta-data tablesTransactional Consistency“Net changes”; not “all changes”

Batches split into transactions to apply

PK/FK ordering preserved

Transaction boundaries at source not preservedE.g. Could get new Order applied in separate transaction to associated new OrderDetail

BatchingData change split into batches for transfer

Spring is in the Cloud

PaaS SaaSPhysical Virtual IaaS

A Continuous Offering From Private to

Public Cloud

Windows Azure SQL Database

SQL Server and Windows Azure VMSupported VersionsSQL Server 2012, 2008 R2, 2008

Supported FeaturesAll SQL Server features supported except failover clustering *

SQL Server ProvisioningCloud-first using stock images, bring your own server / VHD, capture cloud images

SQL Server LicensingPay by the hour or migrate your own license via Software Assurance ** * AlwaysOn Availability Groups not supported until GA

** Windows Azure Compute and Storage charges also apply

Virtual Machine Sizes and SQL Editions

Each Persistent Data Disk Can be up to 1 TB

VM Size CPU Cores Memory Bandwidth# Data Disks SQL Edition

Extra Small Shared 768 MB 5 (Mbps) 1 Express

Small 1 1.75 GB 100 (Mbps) 2 Standard

Medium 2 3.5 GB 200 (Mbps) 4 Standard

Large 4 7 GB 400 (Mbps) 8Standard / Enterprise

Extra Large 8 14 GB 800 (Mbps) 16Standard / Enterprise

Running SQL Server in a Windows Azure VMStorage RecommendationsDo not use write cachingAvoid using OS drive for large databasesConsider putting database and transaction log files on separate drivesConsider putting tempdb on the non-persistent cache disk (D:\)

Database RecommendationsConsider using database page compression to reduce I/O

High Availability RecommendationsConsider latency between primary and replica when choosing sync mode

SQL Server or Windows Azure SQL Db? SQL Server

(IaaS)Windows Azure SQL Database

(PaaS)

Development

Migrate Existing Apps Develop New Apps

Management

Full Control Managed Service

Compatibility

Full SQL Server Capabilities Based on SQL Server TechnologyShared TechnologyNetwork transport (Tabular Data Stream)

SQL dialect (Transact-SQL)Data access APIs (ADO.NET, ODBC, JDBC)

Development tools (SQL Server Data Tools)Management tools (SQL Server Management

Studio)

SQL Federation

Database ScalabilityScale to hundreds of nodes via database shardingMulti-tenancy via flexible repartitioningOnline split operations to minimize downtimeAutomatic data discovery regardless of changes in how data is partitioned

Database ScalabilityScale-upSingle database that houses all the data of an applicationHard to handle peak loadOK with exponential incremental cost

Scale-OutMultiple databases spread over multiple independent nodesCost effective, commodity class hardwareTypical patterns: Sharding and Horizontal Partitioning

User Traffic

Single Database

LB LBFront Tier Middle Tier

SQL Azure

SQL Federation

SQL Databas

eLarger User Traffic

FrontTierFrontTierFrontTierFrontTier

FrontTierFrontTierFrontTierMiddle

Tier

Database ElasticityExtend the scalability model to the database tierAdd and remove SQL Database nodes via database partitioning (Federations)Scale on demand to your traffic without any downtime

Why Use SQL Federation?Database ScalabilityCreate an elastic database tier that can expand and contract with your applications workload without downtime

Gain practically unlimited scale by harnessing 100s of SQL Database nodes

SQL Databas

e

Why Use SQL Federation?Simplified Multi-tenancyBuild Multi-tenant Solutions

Efficient management of tenant placement and re-placement

Multiple-tenants per database Single tenant per database Multiple databases per tenant

Simplified Development and AdministrationRobust programming and connectivity model with native tooling

Same great programming model using existing tools

Who Are SQL Federation for? A Few ExamplesWeb Scale DB Solutions

Multi-tenant Saas ISVs

Workloads with Spikes, Bursts, Peaks, etc…

NoSQL Applications

Windows Azure Connectivity Options

Data SynchronizationSQL Data Sync

Application-Layer Connectivity & Messaging

Service Bus

CLOUD ENTERPRISE

Secure Machine-to-Machine Network

ConnectivityWindows Azure Connect

Secure Site-to-Site Network Connectivity

Windows Azure Virtual Network

Secure Site-to-Site Network Connectivity

Windows Azure Virtual Network

Windows Azure Virtual Networks

Your “virtual” branch office / datacenter in the cloudEnables customers to extend their Enterprise Networks into Windows AzureNetworking on-ramp for migrating existing apps and services to Windows AzureEnables customers to run “hybrid” apps that span cloud and their premises

A protected private virtual network in the cloudEnables customers to setup secure private IPv4 networks fully contained within Windows AzureIP address persistenceInter-service DIP-to-DIP communication

Virtual Network ScenariosHybrid Public/Private Cloud

Enterprise app in Windows Azure requiring connectivity to on-premise resources

Enterprise Identity and Access Control

Manage identity and access control with on-premise resources (on-premises Active Directory)

Monitoring and Management

Remote monitoring and trouble-shooting of resources running in Windows Azure

Advanced Connectivity Requirements

Cloud deployments requiring persistent IP addresses and direct connectivity across services

DNS ScenariosWindows Azure DNS Scenarios Use your own DNS Scenarios

A. Client-server applications using persistent VMs

B. Stateless roles with persistent VMs

C. Hybrid connectivity with on-premise (DNS on-premise)

D. SharePoint with custom DNS (persistent VM)

Persistent VM Role

SQL Reporting Service

Persistent VM Role

SQL Analysis Service

Persistent VM Role

SQL Service

Internet

IIS Web ApplicationWeb Role 1

IIS Web ApplicationWeb Role 2

IIS Web ApplicationWeb Role 3

Persistent VM Role

SQL ServiceLB

On-Premises Machine

Active Directory

Active Directory

SQL ServiceDomain joined to On-

Premises Network

On-Premises Machine

Business Components &

Entities

On-Premises Machine

UI Process Components

Web Tier

Active Directory

Internet

Persistent VM Role

SharePoint FrontEnd

Persistent VM Role

SharePoint FrontEnd

Persistent VM Role

Search and Indes

SQL Service

Persistent VM Role

DC DNS

Persistent VM Role

Persistent VM Role

SQL

Persistent VM Role

SQL

Local DNS

SQ

L Mirro

ring

LB

Open User Access

(Website)

Warning! Hot and New!!

Released last week : new featuresLinked Server and Distributed

Queries Allow server linking like in On Premises' situations

1. Only link to Windows Azure SQL Database not vice versa!!

2. Run queries over the database from within your on premises situation

3. Virtually all DB ‘s can be linked (except for the MASTER DB)

Released last week : new featuresRecursive Triggers

As it states : triggers can recurse (as in Sql Server 2012 box product (see BOL for more info))

USAGE EXAMPLE:

ALTER DATABASE ADVENTUREWORKS2012 SET RECURSIVE_TRIGGERS ON|OFF;

Released last week : new featuresDBCC SHOW_STATISTICS

Displays displays current query optimization statistics for a table or indexed view.

USAGE EXAMPLE:

DBCC SHOW_STATISTICS ( table_or_indexed_view_name , target )

[ WITH [ NO_INFOMSGS ] < option > [ , n ] ]

< option > :: =

STAT_HEADER | DENSITY_VECTOR | HISTOGRAM | STATS_STREAM

Released last week : new featuresFIREWALL RULES ON

DATABASE LEVEL• Previously set only at the server level, either

through the management portal or via T-SQL commands

• Now, firewall rules can be additionally set at the more granular database level

• Different rules on different DB on the same logical server possible

Released last week : new featuresFIREWALL RULES ON DATABASE LEVEL

USAGE EXAMPLE / HOWTO:

sp_set_database_firewall_rule-- create new database-level firewall setting for only IP 0.0.0.4

exec sp_set_database_firewall_rule N'Example DB Setting 1','0.0.0.4','0.0.0.4'

-- update database-level firewall setting to also allow IP 0.0.0.5

exec sp_set_database_firewall_rule N'Example DB Setting 1','0.0.0.4','0.0.0.5'

New table available for storing these : sys.database_firewall_rules additionally to the sys.firewall_rules

Bigger, Better, Bolder!

An Exotic OverviewBIG DATA RELATED PRODUCTSHADOOP on Windows Azure

“Cloud Numerics”

“Data Explorer”

“Data Hub”

“Trust Services”

StreamInsight

Codename “Cloud Numerics”Microsoft Codename "Cloud Numerics“ is an HPC Based computing model making use of Windows Azure SQL Database with following capabilities:

• Parallel workloads• Scale-out enabled functions ranging from Descriptive Statistics, to Fourier

Transforms, to Linear Algebra and more• Improved IO enabling parallel reads and writes from a variety of data sources,

including Azure Blob storage, and formats

Call to Action

http://WindowsAzure.com• Guidance

• Training• Technical Docs• Pricing, Licensing

Resources

• www.windowsazure.com• Windows Azure Training Kit• MSDN WA SQL Database home : http://

msdn.microsoft.com/en-us/library/windowsazure/ee336279.aspx• BI :

https://www.windowsazure.com/en-us/home/features/business-analytics/

• SQL : https://www.windowsazure.com/en-us/manage/services/sql-databases/

• Pricing, Licensing : https://www.windowsazure.com/en-us/pricing/calculator

Thank you for your attention!Thank you for your attention!