BPR476 Windows Sharepoint Services And Sharepoint Portal Server: Data Protection, Disaster Recovery, Site Archival And Beyond Joel S. Oleson Joel S. Oleson Technologist Technologist Microsoft Corporation Microsoft Corporation
BPR476 Windows Sharepoint Services And Sharepoint Portal Server: Data Protection, Disaster Recovery, Site ArchivalAnd Beyond
Joel S. OlesonJoel S. OlesonTechnologistTechnologistMicrosoft CorporationMicrosoft Corporation
OverviewMicrosoft IT Case StudyMicrosoft IT Case Study
MS SharePoint EnvironmentMS SharePoint EnvironmentIT Pain PointsIT Pain PointsData Protection / BackupsData Protection / BackupsDisaster RecoveryDisaster RecoveryData Recovery / RestoreData Recovery / Restore
Demo TextCopyDemo TextCopyIT ImprovementsIT Improvements
Chofu
Dublin
Singapore
SharePoint Deployment
Redmond
Stats:Stats:10 WSS Web Farms, 6 SPS Web Farms10 WSS Web Farms, 6 SPS Web Farms7 SQL Clusters; 4 Search clusters, 8 Index boxes7 SQL Clusters; 4 Search clusters, 8 Index boxes2.5 TB of database content2.5 TB of database content~320 SPS 2003 Portals~320 SPS 2003 Portals50,000+ Site Collections50,000+ Site Collections80,000+ User Profiles80,000+ User Profiles
Americas; Europe, Middle East, Africa (EMEA); Americas; Europe, Middle East, Africa (EMEA); South Pacific; Far EastSouth Pacific; Far East
4 Regional Intranet Deployments4 Regional Intranet Deployments3 Regional Extranet Deployments3 Regional Extranet Deployments
Americas Intranet Hardware
MSWeb Shared Services
2 WFE
2 Search
2 Index
2 WFE
Portal 100
2 WFE
Portal 200
2 WFE
Upgraded WSS Farm
SQL (A/P)
Team SQL Cluster
SharePoint Portal Server 2003 SharePoint Services
2 WFE
Team Farm
2 WFE
Host Header
SQL (A/P)
Portal SQL Cluster
Regional SharePoint
SQL (A/P)
Consolidated SQL Cluster
Intranet Deployment
SQL 2000
Extranet Deployment
Shared Services and Portal Farm
2 WFE
2 Search
2 Index
2 WFE
Corp Hosting
2 WFE
Host Header
2 WFE
Extranet WSS
IT SharePoint Storage4 TB SAN (DB)4 TB SAN (DB)2 TB per SQL Cluster 2 TB per SQL Cluster
(DB) (DB)
Plan to Reclaim 50%Plan to Reclaim 50%
Site Backups Site Backups
Data (29%)Data (29%)
DB Dump (29%) DB Dump (29%)
Site Dump (29%)Site Dump (29%)
Transaction Logs &Transaction Logs &System databases (13%)System databases (13%)
Backup Infrastructure
WSS Backup / Restore Legend
1.0 DB Dump to SAN (attached to SQL Backend)2.0 – 2.2 Site Collection Backup to SAN3.0 – 3.1 Site Collection / DB Backup to Tape4.0 Restore of Site Collection to any Share4.1 Restore of DB to Disk (a. for Disaster & Recovery b. Restore files from DB)
Dump
specific site
SQL Backend
Dump Dump Dump
Tape BackupSite Collection WinDisk
2.0 2.1 2.2
DB DB DB
SAN
Site Collection
Database WinDisk DB Dump WinDisk
WSS Frontend
1.02.0
Backup
3.03.14.1
4.0
Tape Content
Service Growth # of MySites
05,000
10,00015,00020,00025,00030,000
Jul Aug Sep Oct Nov Dec Jan Feb Mar
# of WSS Sites
010,00020,00030,00040,00050,00060,000
Oct Nov Dec Jan Feb Mar
# of WSS Extranet Sites
0
500
1000
1500
2000
2500
Jul Aug Sep Oct Nov Dec Jan Feb Mar
# of Portals
050
100150200250300350
Jul Aug Sep Oct Nov Dec Jan Feb Mar
# of TB in Sharepoint
0
0.5
1
1.5
2
2.5
Jul Aug Sep Oct Nov Dec Jan Feb Mar
# of TB in fileshare
0.0
2.0
4.0
6.0
8.0
10.0
Jul Aug Sep Oct Nov Dec Jan Feb Mar
IT Pain PointsTapeTape
Tape Backups long runningTape Backups long running300-500MB/min300-500MB/minHigh NIC utilization by Tape backupHigh NIC utilization by Tape backup
DatabaseDatabaseDatabase backups long running Database backups long running
1TB 10hr (100 GB/hr)1TB 10hr (100 GB/hr)SPSBackup backup duplication of databasesSPSBackup backup duplication of databases
Site CollectionSite CollectionSTSADM doesn’t scale well over the network STSADM doesn’t scale well over the network
~300GB in 24/hr (12MB/min)~300GB in 24/hr (12MB/min)STSADM full 2-3 days (1TB)STSADM full 2-3 days (1TB)
Restore management painsRestore management pains
Data ProtectionWhat to backupWhat to backupDatabase BackupsDatabase BackupsToolsTools
SPS ToolsSPS ToolsWSS ToolsWSS ToolsSTSADM demoSTSADM demoThird Party Backup VendorsThird Party Backup VendorsScripts/Dev IdeasScripts/Dev Ideas
Disk to TapeDisk to TapeHot Swappable FarmHot Swappable Farm
SharePoint Storage ObjectsDatabase
Site Collections
Document Libraries and Lists
Files/Documents and List Items
What to BackupDatabasesDatabasesWSS/SPSWSS/SPS
CONFIG DBCONFIG DB
STS_W3SVC#_#STS_W3SVC#_#SPSSPS
PROF DBPROF DB
SERVDBSERVDB
SITEDBSITEDB
*External to Db*External to DbSystem StateSystem State
SPS: Indexes & SPS: Indexes & WebStore Doc LibWebStore Doc Lib
IIS Meta baseIIS Meta base
Web Part PackagesWeb Part Packages
Home DirectoriesHome Directories
Database BackupsDatabase to dump on diskDatabase to dump on disk
DatabaseBackup Optimization
Database sizesDatabase sizes25-50GB Database Storage (>500 25-50GB Database Storage (>500 sites)sites)Number of databases (>300)Number of databases (>300)
Cost SavingsCost SavingsDirect to Media?Direct to Media?Backup compression/SpeedBackup compression/Speed
SQL LiteSpeed – 25-75% compression SQL LiteSpeed – 25-75% compression expectedexpected
Snapshots or ClonesSnapshots or Clones
SharePoint Portal ToolsSPSBackup.exeFull Farm BackupFull Farm BackupSpsbackup.exeSpsbackup.exe /all /file <Backup path and file>/all /file <Backup path and file>Central Portal Index Backup:Central Portal Index Backup:
Spsbackup.exeSpsbackup.exe /portal http://portal /service index /file \\svr\shr\backup /overwrite /portal http://portal /service index /file \\svr\shr\backup /overwrite
DatabasesDatabasesBackward-compatible Doc LibBackward-compatible Doc LibSPS IndexesSPS IndexesXML manifestXML manifest
schema versionschema versionportal GUID portal GUID Path to DB backup filePath to DB backup fileResultResult
Windows SharePoint Services ToolsSTSAdm.exeSTSAdm.exeExample: stsadm.exe –o backup –url Example: stsadm.exe –o backup –url
http://team/sites/sitename -f sitename.bakhttp://team/sites/sitename -f sitename.bakHelp: stsadm.exe –help backupHelp: stsadm.exe –help backup
WSS sites – full fidelity w/ securityWSS sites – full fidelity w/ securityMy sites - content backed upMy sites - content backed up
Personalization, web part customizations, and profile Personalization, web part customizations, and profile changes are not preserved in migrationschanges are not preserved in migrations
Sites under portals – full fidelity w/ security Sites under portals – full fidelity w/ security Smigrate.exe Smigrate.exe
Files & Lists copied to disk with manifest Files & Lists copied to disk with manifest Note: Security on sites not preserved from WSS to WSSNote: Security on sites not preserved from WSS to WSSFrontPage2003 Backup/RestoreFrontPage2003 Backup/Restore
SharePoint BackupThird Party VendorsAvePoint *AvePoint *
DocAve – SPS 2003 – Item level backup/restore/archive with DocAve – SPS 2003 – Item level backup/restore/archive with multiple migration options (supports SQL & Webstore)multiple migration options (supports SQL & Webstore)Downloadable Demo AvailableDownloadable Demo Available
Computer Associates (CA) *Computer Associates (CA) *BrightStor ARCserve Backup BrightStor ARCserve Backup
Backup/Restore SharePoint databases hosted on the various SQL Backup/Restore SharePoint databases hosted on the various SQL servers distributed across the enterprise.servers distributed across the enterprise.Advanced document level backup and restore supportAdvanced document level backup and restore support
CommVault *CommVault *GA, Galaxy- SPS 2003 database level backup and recovery GA, Galaxy- SPS 2003 database level backup and recovery productproduct
Enterprise wide backup and recovery of SPS 2003 data (SQL and Enterprise wide backup and recovery of SPS 2003 data (SQL and indices). Site and sub-site backup and recovery; Webstore backup indices). Site and sub-site backup and recovery; Webstore backup supportsupport
Product in Beta for “Doc Level” backup/restoreProduct in Beta for “Doc Level” backup/restoreVeritas *Veritas *
Backup Exec 10.0 (Late Q3 04) - SPS Server Level backupBackup Exec 10.0 (Late Q3 04) - SPS Server Level backup
* Here at Tech ED 2004* Here at Tech ED 2004
Hot Swappable SQLVia SQL Log Shipping
Mirror ServersMirror ServersSet DB Full or Bulk Logged recovery modeSet DB Full or Bulk Logged recovery modeMirror DataMirror Data
Full Backup of FarmFull Backup of FarmFull Restore of FarmFull Restore of Farm
Enable SQL Log shippingEnable SQL Log shippingSITE, SERV, PROF & Content dbs SITE, SERV, PROF & Content dbs (NOT CONFIG)(NOT CONFIG)
Set DBs on Mirror to Standby or No RecoverySet DBs on Mirror to Standby or No RecoveryConfigure FrequencyConfigure FrequencyCreate Farm, Customization/Portal & Virtual Server Create Farm, Customization/Portal & Virtual Server configconfig
Recommend: Periodic execution of “delete portals and Recommend: Periodic execution of “delete portals and detach databases; reconnect them to mirror farm detach databases; reconnect them to mirror farm config DB” config DB”
Incremental Site BackupIncremental Site BackupBackup/Restore SiteBackup/Restore Site
Site ArchivalCleanup Storage
Analyze Site UsageAnalyze Site UsageLast ModifiedLast ModifiedIIS usageIIS usage
Identify sites to “cleanup”Identify sites to “cleanup”Backup targeted sitesBackup targeted sitesDelete unused SitesDelete unused Sites
Monitoring BackupsSharePoint Backup Relevant MOM Events
WSS MOM PackWSS MOM PackDatabase Capacity ReachedDatabase Capacity ReachedDatabase Warning ReachedDatabase Warning Reached
SPS MOM PackSPS MOM PackSPSBackup.exeSPSBackup.exe backup success/fail backup success/fail
Indexes or databaseIndexes or databaseSQL Server 2000 MOM PackSQL Server 2000 MOM Pack
SQL Server database and transaction log SQL Server database and transaction log space warnings and errors.space warnings and errors.SQL Server failed backupsSQL Server failed backupsSQL Server failed agent jobsSQL Server failed agent jobs
Backup MediaDisk to TapeDisk to Tape
TapeBackup Optimization
Network Upgrade - Over NetworkNetwork Upgrade - Over NetworkPrivate NetworkPrivate NetworkSAN TapeSAN Tape
1GB/Min (db dump to tape)1GB/Min (db dump to tape)Max drive or share sizeMax drive or share size
100-300 GB for quicker restart100-300 GB for quicker restart
Script/Dev ideasIncremental Site Backup – Incremental Site Backup – backup changed backup changed sitessitesDocument backup – Document backup – Web folder backup of Web folder backup of documents or object model iterating through documents or object model iterating through documentsdocumentsIncremental Document backupIncremental Document backup – Backup – Backup changed documents using Object Modelchanged documents using Object ModelArchive – Archive – Use visits w/ modify date to Use visits w/ modify date to determine what sites should be archived determine what sites should be archived (backed up to disk) then deleted(backed up to disk) then deleted
Data RecoveryDisaster RecoveryDisaster RecoveryRecovery Scenarios & RequestsRecovery Scenarios & RequestsRecovery FlowRecovery Flow
DatabaseDatabaseGranularGranular
Demo: Textcopy Demo: Textcopy Doc restore from Content DB Doc restore from Content DB
Portal RestorePortal RestoreService LevelsService Levels
Disaster Recovery: ServerWeb Front End – Web Front End –
A. Restore failed drives and system state (metabase) A. Restore failed drives and system state (metabase) B. Reinstall, remove/re-add topology reconfigure, re-add web parts, B. Reinstall, remove/re-add topology reconfigure, re-add web parts, recreate virtual servers, extend and map to existing virtual serversrecreate virtual servers, extend and map to existing virtual servers
Search – Search – A. Restore failed drives and system state (metabase) propegateA. Restore failed drives and system state (metabase) propegateB. reinstall, remove/add back to topology and propagateB. reinstall, remove/add back to topology and propagate
Index – Index – A. Restore failed drives and system state (metabase) restore Index A. Restore failed drives and system state (metabase) restore Index using using spsbackup.exespsbackup.exereinstall, remove/add back to topology, restore indexes using reinstall, remove/add back to topology, restore indexes using spsbackup.exespsbackup.exe, and propagate, and propagate
SQLSQL – Restore failed databases (except Config db) – Restore failed databases (except Config db)Disconnect reconnect content databasesDisconnect reconnect content databases
Backward Compatible Doc LibBackward Compatible Doc Lib – –Disconnect Doc libs, remove from topology & farmDisconnect Doc libs, remove from topology & farmRe-add new server to farm & topologyRe-add new server to farm & topologyRestore using Restore using SPSBackup.exeSPSBackup.exe restore restore
Recovery of Hot Swap SQL Via SQL Log ShippingSQLSQL
Close Open ConnectionsClose Open ConnectionsChange role of Secondary SQL to Primary Change role of Secondary SQL to Primary (or keep dbs read only until recovery of (or keep dbs read only until recovery of primary)primary)
SharePointSharePointAs needed Create Portal from DBs As needed Create Portal from DBs (connect)(connect)Disconnect/Reconnect Content DbsDisconnect/Reconnect Content Dbs
WINS/DNS to point to new Farm VIPWINS/DNS to point to new Farm VIP-Start a Full Crawl-Start a Full Crawl
Recovery ScenariosDatabase -Database - most reliable disaster recovery most reliable disaster recovery scenarioscenario
Site Collections – Site Collections – STSAdm.exe STSAdm.exe restorerestoreDatabase restoreDatabase restore
Document Library, List, Doc recovery Document Library, List, Doc recovery – – Database restoreDatabase restoreSTSAdm.exeSTSAdm.exe restore restoreDocumentDocument- - Textcopy or other API can make retrieval Textcopy or other API can make retrieval from DB possiblefrom DB possible
Restore Requests
Default 15000 Sites MaxPer Content DB
Avg 20GB
>1% of Restore Requests
Includes Sub sites Workspaces
Avg 30-65 MB
41% Restores68/167 site68/167 site
Doc Libraries, list items, lists, events,Surveys, forms, etc..
Avg 1.5 MB
49% Restores82/167 doc82/167 doc
Database RestoreTape to Disk to Online dbTape to Disk to Online db
Database RecoverySITE / STS_W3SVC#_#SITE / STS_W3SVC#_# – restore database, – restore database, remove from config db using admin UI, re-add remove from config db using admin UI, re-add using admin UIusing admin UI
PROF DB – PROF DB – restore profile database, start full restore profile database, start full import, remove objects missing from import import, remove objects missing from import using UIusing UI
SERV DB – SERV DB – restore services database, bring restore services database, bring online (keep in sync w/ SITE DB)online (keep in sync w/ SITE DB)
Granular RecoveryPortal and below
Restore relevant databases (SITE, SERV, PROF, Restore relevant databases (SITE, SERV, PROF, STS_W3SVC#_#)STS_W3SVC#_#)
Restore to standby environment using Restore to standby environment using SPSBackup.exeSPSBackup.exe /Restore tab GUI /Restore tab GUI
STSAdmSTSAdm for site collection migration for site collection migration
SmigrateSmigrate for area or sub site migration (reapply for area or sub site migration (reapply security)security)
TextCopyTextCopy for doc restore for doc restore
Portal Restore From DBDBs Only (no index)
Central AdminCentral AdminClick - Create a portal siteClick - Create a portal siteChoose - Restore a portal siteChoose - Restore a portal siteSelect Servers and databasesSelect Servers and databasesChoose Virtual Server and specify URLChoose Virtual Server and specify URLExtend and MapExtend and Map
Granular RecoverySite/Sub siteSite CollectionSite Collection
Backed up Backed up Stsadm.exe –o restore –url Stsadm.exe –o restore –url http://http://sitenamesitename –f –f filename –overwritefilename –overwriteRequires 2Requires 2ndnd content db content db
No Site BackupNo Site BackupRestore content databaseRestore content databaseAttach database to Attach database to standbystandby virtual server/Lab virtual server/Lab environmentenvironmentBackup site using Backup site using stsadm.exestsadm.exeRestore using Restore using stsadm.exestsadm.exe with overwrite option with overwrite option
Sub site - use Sub site - use smigratesmigrate to move data to to move data to production with overwriteproduction with overwrite
TextCopyTextCopyRestoring document from DBRestoring document from DB
Demo – TextCopyIdentify Database NameIdentify Database NameUSE @WSS_configUSE @WSS_configSELECT nameSELECT nameFROM sites s, databases dFROM sites s, databases dWHERE s.databaseid = d.databaseidWHERE s.databaseid = d.databaseidAND urlpath = @urlpathAND urlpath = @urlpath
Identify Document/FileIdentify Document/FileUSE [X1_SITE]USE [X1_SITE]SELECT content SELECT content FROM docsFROM docsWHERE dirname = @dirnameWHERE dirname = @dirnameAND leafname = @leafnameAND leafname = @leafname
> textcopy /s server /u @user /P @password /d “database" /t docs /c content > textcopy /s server /u @user /P @password /d “database" /t docs /c content /W "select * from docs where leafname= ‘@leafname’ and dirname= /W "select * from docs where leafname= ‘@leafname’ and dirname= ‘@dirname’ /F c:\temp\leafname /O /Z‘@dirname’ /F c:\temp\leafname /O /Z
Service LevelsCharge back – Charge back – charge the user cost charge the user cost center for restorecenter for restoreSponsor – Sponsor – ask for corporate sponsor on ask for corporate sponsor on restore requestrestore requestQuantify cost of restoreQuantify cost of restoreFull Portal or Virtual ServerFull Portal or Virtual ServerCap off Max Site Collection sizeCap off Max Site Collection size
IT ImprovementsTapeTape
SAN Tape/Fiber Attached BackupSAN Tape/Fiber Attached BackupThroughput 1.2 GB/Min, 2-3xThroughput 1.2 GB/Min, 2-3x
Split drive into ~300GB shares for Tape BackupSplit drive into ~300GB shares for Tape BackupGB Network upgrade on SQL boxesGB Network upgrade on SQL boxes
DatabaseDatabaseReduce database sizes Max 25-50GBReduce database sizes Max 25-50GBReduce DB to >300DB per SQL instanceReduce DB to >300DB per SQL instanceSPSBackup.exeSPSBackup.exe to backup indexes only to backup indexes only
Site CollectionSite CollectionIncremental BackupsIncremental BackupsCleanup/AutoDelete – ArchiveCleanup/AutoDelete – Archive
Restore process flow better understoodRestore process flow better understood
© 2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
Q1:Q1: Overall satisfaction with the sessionOverall satisfaction with the session
Q2:Q2: Usefulness of the informationUsefulness of the information
Q3:Q3: Presenter’s knowledge of the subjectPresenter’s knowledge of the subject
Q4:Q4: Presenter’s presentation skillsPresenter’s presentation skills
Q5:Q5: Effectiveness of the presentationEffectiveness of the presentation
Please fill out a session evaluation on CommNetPlease fill out a session evaluation on CommNet