Upgrade to 2008 Best of PASS

Upgrade your Databasewith only 15 Minutes of Downtime.

Chris Shaw ([email protected])

DBA240 • Upgrade Case Study

The Case

DBA240 • Upgrade Case Study

Critical updates, could impact public safety.

No more than 15 minutes of downtime.

3 databases

• 2 were updated on a regular basis

• 1 was updated once a week

Total size was 300 GB

Using SQL Litespeed for compression

Database Maintenance was limited to backups (No Re-indexing to

speak of)

Outages are extremely hard to come by.

Full Backups taken every Sunday, Diffs taken every night, T-Logs every

20 min.

Migration to SQL 2008 in a Mission Critical

Environment

Agenda

About 888

Upgrade Methodology

Upgrade Tools

Upgrade Process

Demo – Real-Time Migration

Agenda

About 888

Upgrade Methodology

Upgrade Tools

Upgrade Process

Demo – Real-Time Migration

888.com is a global online gaming company.

Our purpose is to provide quality entertainment for people who enjoy gambling.

Giving them the opportunity to do so in a safe, fun, fair, regulated and secure environment.

About 888

Our products

50 Production Instances

300 Development Instances

400 Databases

100 TB Of Data   

24*7 Availability

99.95 Uptime

888 Database Environment

R&R Of DB Infrastructure Team

Installations Maintenance

Backups Indexes Statistics Data retention

Monitoring Performance Tuning Version Uploads Security

SQL 2008 Project Summary

50 production clustered instances were upgraded

300 development and QA instances were upgraded

All SQL server hosts upgraded windows 2008 R2 OS

Cut licensing costs by more than $500,000

Reduce load from IO subsystems by using SQL server compression

Minimal interference to the operation

DB Infrastructure Team

Ilanit Gat

Back officeFinance

Web

Ronny Sapir

Bingo3th party

Guy Hochman

DWHGaming

Adi Cohen

Guru

Agenda

About 888

Upgrade Methodology

Upgrade Tools

Upgrade Process

Demo – Real-Time Migration

In Place Upgrade

Advantages Simple No connection string changes No need to copy system objects – system DBs are

upgraded No need to copy the user databases

Disadvantages No control over the process A standby server is mandatory for a quick rollback

Side by Side Upgrade

Advantages Full control over the process A quick rollback OS upgrade can be included

Disadvantages The necessity of copying the system objects Connection string changes pose an overhead

In place Side by side

Control over the process

No Yes

OS Upgrade No Yes

Rollback Slow Fast

Copy system objects No Yes

Connection string change

No Yes

Copy user databases No Yes

In Side Upgrade

In Side Upgrade

Control over the process Yes

OS Upgrade Can be upgraded

Rollback Change back connection strings

Copy system objects Copy system objects efficiently

Connection string changes Eliminate connection string changes

Copy user databases Copy user databases efficiently

Copy System Objects Linked servers Jobs Configurations Logins and Credentials Server master key

By The Book: Generate Export/Import Scripts

Q: Can I copy system databases across different versions?

Copy system databases across different versions of SQL servers

System DBs System DBs

Connection String Change

By the book: Stop application Change connection strings Start application

Downtime Timeline

Q: Can we do it in one step and faster?

Stop allapps

Start all apps

DBsRO

2000/2005

DBs RW

2008

Backup log

Restore log

Downtime starts

Downtime ends

Conn string

changes

Server Identification Hostname / Virtual-name IP Port Service Name

Connection Examples: hostname\INST1 Virtual-name\INST1 (in clusters) Virtual-name, port IP, port

Q: What can I change?

Change the SERVER Identification

2005· REAL-APR-SQL\APR· 10.10.10.193· Port 1433

2008· REAL-APR-SQL2\APR· 10.10.10.198· Port 1433

Downtime Timeline

Stop allapps

Start all apps

DBsRO

2000/2005

DBs RW

2008

Backup log

Restore log

Downtime starts

Downtime ends

Conn string

changes

Copy User Databases

Storage Level Move LUNs Clone LUNs

Database Level Database Mirroring Log shipping - Works at the database level

Agenda

About 888

Upgrade Methodology

Upgrade Tools

Upgrade Process

Demo – Real-Time Migration

Installation Challenge #1

SQL 2008 Basic

InstallationSP1 CU6

Number of installations

Slipstream

Robocopy

Means to integrate packages of updates, patches or service-packs into one installation pack.

Slipstream Advantages

66% less installations

Less restarts & less failovers

Saves time

Installation Challenge #1

SQL 2008 Basic

Installation

SQL 2008 SP1CU6 SP1 CU6

Number of installations

Installation Challenge #2

SQL 2008 Full

Installation

Installation Duration

Silent Installation

Installation that does not display messages or windows during its progress

Silent Installation Advantages

Easy to use – less parameters than GUI

Saves time

Install many instances with one script

Coherence and uniformity in all environments

Silent Installation

Installation Challenge #2

SQL 2008 SP1CU6

SQL 2008 SP1CU6

Installation Duration

Copy User Databases

• Microsoft Log Shipping • Works at the database level

• 888 AutoRestore• Works at the instance level

In-House Tool – Autorestore

Auto Restore Implementation  

An in-house tool that facilitates the control and management of restores from a central location

Uses transaction log backups from the source servers

Supports parallel restore operations

Works on an instance level (several databases) and not on a per-database basis (Mirroring)

Agenda

About 888

Upgrade Methodology

Upgrade Tools

Upgrade Process

Demo – Real-Time Migration

Pre-Migration

Run Windows cluster validation

Install a new instance on a new cluster

Perform a failover

Network – check open ports

Open UDP 1434 or create aliases

Preparation Stage

Pre-Migration

Instance 2005

Instance 2008

Instance2008

Windows 2008

Restore server objects by restoring the system databases

Windows 2003

Instance2000\2005

Preparation Stage

Restore Restore Upgrade

Pre-Migration

Auto-Restore

Copy user databases using instance level log shipping (Auto-Restore)

Windows 2003

Instance2000\2005

Windows 2008

Instance2008

Preparation Stage

Pre-Migration

Simulation Stage

Application sanity checks

System health checks – OS ,Storage, Host, Network

Migration Process (scripts, time)

Compression Preparations

Migration Day

1. Stop SQL server agent

2. Set all databases to read-only

3. Run last log backup for all databases

On 2000/2005 Instance

Actual Migration

4. Restore last logs with no recovery

5. Check if all the logs were restored from the old server

6. Restore with recovery for all databases

7. Set all databases to read-write

8. Fix orphan logins

9. Change compatibility to 100

On 2008 Instance

Actual Migration

10. Take offline the old instance group

11. Take offline the new instance group

12. Delete the old and the new virtual IP and name DNS records from the DNS server

Changing Instance Identification

Actual Migration

13. Clear the DNS server’s cache

14. Give the new server the old server’s virtual IP & name

15. Flush DNS cache clients of new server, Arpcache

16. Start the new instance group

Changing Instance Identification

Post-Migration

1. Perform Sanity Checks

2. Check NOC Alerts

3. Check SSIS, MSDTC and Job statuses

4. Run a Profiler’s Error Trace

5. Check SQL Error Log & Event Log

6. Check Performance Baseline

7. Run Full Backups

Compression

Compression Advisory – in-house utility Compression Scripts

Agenda

About 888

Upgrade Methodology

Upgrade Tools

Upgrade Process

Demo – Real-Time Migration