Sponsors Gold Silver Bronze Building the ‘Perfect’ SharePoint 2010 Farm Best Practises from the Field Michael Noel Convergent Computing
Dec 23, 2014
SponsorsGold
Silver
Bronze
Building the ‘Perfect’ SharePoint 2010 Farm
Best Practises from the Field
Michael NoelConvergent Computing
Brisbane 2011
Michael NoelBrisbane
Sydney
Skippy
Canberra
Katoomba
Melbourne
HungryQuokkas
Adelaide
12 (11)Apostles
Perth
Bondi
Great to be back in Beautiful Australia!
Tasmania
Brisbane 2011
What we will cover• Examine various SharePoint 2010 farm architecture best
practises that have developed over the past year• Examine SharePoint Best Practice Farm Architecture• Understand SharePoint Virtualisation Options• Explore SharePoint DR and HA strategies using Database
Mirroring• Explore other common best practises (RBS, SSL, NLB)• Examine best practice security for SharePoint• A large amount of best practises covered (i.e. Drinking
through a fire hose,) goal is for you to be able to take away at least 2-3 useful pieces of information that can be used in your environment
Brisbane 2011
ARCHITECTING THE FARM
Brisbane 2011
Web
Service Apps
Data
Architecting the FarmUnderstanding the Three Tiers of SharePoint Infrastructure
Brisbane 2011
• ‘All-in-One’ (Avoid)
DB and SP Roles Separate
Architecting the FarmSmall Farm Examples
Brisbane 2011
• 2 SharePoint Servers running Web and Service Apps
• 2 Database Servers (Clustered or Mirrored)
• 1 or 2 Index Partitions with equivalent query components
• Smallest farm size that is fully highly available
Architecting the FarmSmallest Highly Available Farm
Brisbane 2011
• 2 Dedicated Web Servers (NLB)
• 2 Service Application Servers• 2 Database Servers
(Clustered or Mirrored)• 1 or 2 Index Partitions with
equivalent query components
Architecting the FarmBest Practice ‘Six Server Farm’
Brisbane 2011
• Multiple Dedicated Web Servers
• Multiple Dedicated Service App Servers
• Multiple Dedicated Query Servers
• Multiple Dedicated Crawl Servers, with multiple Crawl DBs to increase parallelisation of the crawl process
• Multiple distributed Index partitions (max of 10 million items per index partition)
• Two query components for each Index partition, spread among servers
Architecting the FarmScaling to Large Farms
Brisbane 2011
• Previously a third party product ($$$$)• More reasonable pricing now• Highly tuned and specialised search engine
for SharePoint and also as an enterprise search platform
• Replaces SharePoint 2010 Native Search if used
• ‘Net new’ features built-in.
Architecting the FarmFAST Search
Brisbane 2011
Feature
SharePoint Foundation
2010
Search Server 2010 Express
Search Server 2010
SharePoint Server 2010
FAST Search Server 2010
for SharePoint
Basic search X X X X XBest Bets X X X XSearch Scopes X X X XCrawled and Managed Properties X X X XQuery Federation X X X XQuery Suggestions X X X XRelevancy Tuning by Document or Site Promotions
X X X XShallow Results Refinement X X X XWindows 7 Federation X X X X
Architecting the FarmFAST Search – Comparison Matrix – Slide 1 of 2
Brisbane 2011
Feature
SharePoint Foundation
2010
Search Server 2010 Express
Search Server 2010
SharePoint Server 2010
FAST Search Server 2010
for SharePoint
People Search X XSocial Search X XTaxonomy Integration X XMulti-Tenant Hosting X XVisual Best Bets XSimilar Results XDuplicate Results XSearch Enhancement based on user context XSort Results on Managed Properties or Rank Profiles
X
Deep Results Refinement XDocument Preview XRich Web Indexing Support X
Architecting the FarmFAST Search – Comparison Matrix – Slide 2 of 2
Brisbane 2011
VIRTUALISATION OF SHAREPOINT SERVERS
Brisbane 2011
• Dedicated hosts for SharePoint Virtual Guests• No Software on Host OS! (Except A/V or Backup)• Don’t overallocate memory (ballooning) or
Processor (2:1 ratio max)
Virtual Hosts
• Ensure proper amount of IO (0.75 IOPs / GB)• Allocate Passthrough/RDM disk for best perf• If using virtual disks, use fixed-sized, not
dynamically expandingDisk
• Aggregate multiple NICs on host for the guest networks
• Allocate Passthrough/RDM NICs for best perfNetwork
• Web Role is best candidate, but be cautious if using multiple app pools (800MB/pool)
• Service App systems generally good candidates• Use caution with the database role!
Virtual Guests
Virtualisation of SharePoint ServersCaveats – Be Sure to Understand Virtualisation Concepts
Brisbane 2011
vCPU RAM (Bare Minimum)
RAM (Recommend)
RAM (Ideal)
Web Only* 2 6GB 8GB 12GB
Service Application Roles Only
2 6GB 8GB 12GB
Dedicated Search Service App
2 8GB 10GB 16GB
Combined Web/Search/Service Apps
4 10GB 12GB 18GB
Database* 4 10GB 16GB 24GB
Virtualisation of SharePoint ServersVirtual Guest Processor and Memory Guidelines
Brisbane 2011
Allows organisations that wouldn’t normally be able to have a test environment to run one
Allows for separation of the database role onto a dedicated server Can be more easily scaled out in the future
Virtualisation of SharePoint ServersSample 1: Small Single Server Environment / No HA
Brisbane 2011
High-Availability across Hosts
All components Virtualised
Uses only two Windows Ent Edition Licenses
Virtualisation of SharePoint ServersSample 2: Two Server Highly Available Farm
Brisbane 2011
Highest transaction servers are physical
Multiple farm support, with DBs for all farms on the SQL cluster
Virtualisation of SharePoint ServersSample 3: Mix of Physical and Virtual Servers – Best Perf
Brisbane 2011
Virtualisation of SharePoint ServersSample 4: Scaling to Large Virtual Environments
Brisbane 2011
• Processor (Host Only)– <60% Utilisation = Good– 60%-90% = Caution– >90% = Trouble
• Available Memory – 50% and above = Good– 10%-50% = OK– <10% = Trouble
• Disk – Avg. Disk sec/Read or Avg. Disk sec/Write– Up to 15ms = fine– 15ms-25ms = Caution– >25ms = Trouble
• Network Bandwidth – Bytes Total/sec– <40% Utilisation = Good– 41%-64% = Caution– >65% = Trouble
• Network Latency - Output Queue Length– 0 = Good– 1-2= OK– >2 = Trouble
Virtualisation of SharePoint ServersVirtualisation Performance Monitoring
Brisbane 2011
1. Create new Virtual Guest (Windows Server 2008 R2)2. Install SP2010 Binaries. Stop before running Config
Wizard3. Turn Virtual Guest into Template, modify template to
allow it to be added into domain4. Add PowerShell script to run on first login, allowing SP
to be added into farm or to create new farm
End Result - 15 minute entire farm provisioning…quickly add servers into existing farms or create new farms (Test, Dev,
Prod) on demand
Virtualisation of SharePoint ServersQuick Farm Provisioning using VMM/Virtual Center
Brisbane 2011
QUICK FARM PROVISIONING WITH VMM 2008 R2
Demo
Brisbane 2011
DATA MANAGEMENT
Brisbane 2011
• Start with a distributed architecture of content databases from the beginning, within reason (more than 50 per SQL instance is not recommended)
• Distribute content across Site Collections from the beginning as well, it is very difficult to extract content after the face
• Allow your environment to scale and your users to ‘grow into’ their SharePoint site collections
Data ManagementDistribute Data Across Content DBs and Site Collections
Brisbane 2011
Brisbane 2011
• BLOBs are unstructured content stored in SQL• Includes all documents, pictures, and files stored in
SharePoint• Excludes Metadata and Context, information about
the document, version #, etc.• Until recently, could not be removed from SharePoint
Content Databases• Classic problem of structured vs. unstructured data –
unstructured data doesn’t really belong in a SQL Server environment
Data ManagementBinary Large OBject (BLOB) Storage
Brisbane 2011
• Can reduce dramatically the size of Content DBs, as upwards of 80%-90% of space in content DBs is composed of BLOBs
• Can move BLOB storage to more efficient/cheaper storage• Improve performance and scalability of your SharePoint
deployment – But highly recommended to use third party
Data ManagementGetting your BLOBs out of the Content DBs
Brisbane 2011
SQL DATABASE OPTIMISATION
Brisbane 2011
SQL Database OptimisationContent Databases Distributed Between Multiple Volumes
DB-AFile 1
DB-BFile 1
Volume #1
DB-AFile 2
DB-BFile 2
Volume #2
DB-AFile 3
DB-BFile 3
Volume #3
DB-AFile 4
DB-BFile 4
Volume #4
Tempdb File 1 Tempdb File 2 Tempdb File 3 Tempdb File 4
Brisbane 2011
SQL Database OptimisationContent Databases Distributed Between Multiple Volumes
• Break Content Databases and TempDB into multiple files (MDF, NDF), total should equal number of physical processors (not cores) on SQL server.
• Pre-size Content DBs and TempDB to avoid fragmentation
• Separate files onto different drive spindles for best IO perf.
• Example: 100GB total Content DB on Four-way SQL Server would have four database files distributed across four sets of drive spindles = 25GB pre-sized for each file.
Brisbane 2011
• TempDB is critical for performance• Pre-size to 20% of the size of the largest content
database.• Break into multiple files across spindles as noted• Note there is a separate TempDB for each physical
instance• Note that if using SQL Transparent Data Encryption
(TDE) for any databases in an instance, the tempDB is encrypted.
SQL Database OptimisationTempDB Best Practises
Brisbane 2011
HIGH AVAILABILITY AND DISASTER RECOVERY
Brisbane 2011
• Clustering is Shared Storage, can’t survive storage failure, makes Mirroring more attractive
• Clustering fails over more quickly• Mirroring is not supported for all databases,
but Clustering is• Both Clustering and Mirroring can be used at
the same time (Instance to Instance)
High Availability and Disaster RecoveryData Tier – Clustering vs. Mirroring
Brisbane 2011
• Introduced in SQL 2005 SP1• Greatly improved in SQL 2008 and now SQL 2008 R2• Available in Enterprise and Standard (Synchronous only) editions• Works by keeping a mirror copy of a database or databases on two
servers• Can be used locally, or the mirror can be remote• Can be set to use a two-phase commit process to ensure integrity
of data across both servers• Can be combined with traditional shared storage clustering to
further improve redundancy• SharePoint 2010 is now Mirroring aware!
High Availability and Disaster RecoveryData Tier – SQL Database Mirroring
Brisbane 2011
• Single Site• Synchronous
Replication• Uses a SQL
Witness Server to Failover Automatically
• Mirror all SharePoint DBs in the Farm
• Use a SQL Alias to switch to Mirror Instance
High Availability and Disaster RecoveryData Tier – Database Mirroring Model #1 – Single Site
Brisbane 2011
• Two Sites• 1-10 ms
Latency max• 1Gb
Bandwidth minimum
• Farm Servers in each location
• Auto Failover
High Availability and Disaster RecoveryData Tier – Database Mirroring Model #2 – Cross-Site with HA
Brisbane 2011
• Two Sites• Two Farms• Mirror only
Content DBs• Failover is
Manual• Read-only
Mode possible
• Must Re-Attach and Re-Index
High Availability and Disaster RecoveryData Tier – Database Mirroring Model #2 – Remote Farm
Brisbane 2011
Synchronous Mirror Support
Asynchronous Mirror Support
Configuration XCentral Administration content XContent Databases X XUsage and Health Data Collection
Business Data Connectivity XApplication Registry service * (BDC Upgrade)
Subscription Settings service * (PowerShell Enabled) XSearch – Search Administration XSearch - Crawl XSearch - Property X
High Availability and Disaster RecoveryData Tier – Database Support for Mirroring – Slide 1 of 2
Brisbane 2011
Synchronous Mirror Support
Asynchronous Mirror Support
User Profile - Profile XUser Profile - Synchronisation
User Profile – Social Tagging
Web Analytics - Staging
Web Analytics - Reporting XSecure Store X XStage XManaged Metadata XWord Automation Services XPerformancePoint X
High Availability and Disaster RecoveryData Tier – Database Support for Mirroring – Slide 2 of 2
Brisbane 2011
High Availability and Disaster RecoveryTwo Node/Two Instance Cluster – Take Advantage of both servers
Brisbane 2011
High Availability and Disaster RecoveryNetwork Load Balancing
• Hardware Based Load Balancing (F5, Cisco, Citrix NetScaler – Best performance and scalability
• Software Windows Network Load Balancing fully supported by MS, but requires Layer 2 VLAN (all packets must reach all hosts.) Layer 3 Switches must be configured to allow Layer 2 to the specific VLAN.
• If using Unicast, use two NICs on the server, one for communications between nodes.
• If using Multicast, be sure to configure routers appropriately
• Set Affinity to Single (Sticky Sessions)• If using VMware, note fix to NLB RARP
issue (http://tinyurl.com/vmwarenlbfix)
Brisbane 2011
• Best Practice – Create Multiple Web Apps with Load-balanced VIPs (Sample below)– Web Role Servers
• sp1.companyabc.com (10.0.0.101) – Web Role Server #1• sp2.companyabc.com (10.0.0.102) – Web Role Server #2
– Clustered VIPs shared between SP1 and SP2 (Create A records in DNS)
• spnlb.companyabc.com (10.0.0.103) - Cluster• spca.companyabc.com (10.0.0.104) – SP Central Admin• spsmtp.companyabc.com (10.0.0.105) – Inbound Email VIP• home.companyabc.com (10.0.0.106) – Main SP Web App
(can be multiple)• mysite.companyabc.com (10.0.0.107) – Main MySites Web
App
High Availability and Disaster RecoveryWindows Software Network Load Balancing Recommendations
Brisbane 2011
SHAREPOINT INSTALLATION
Brisbane 2011
• Good to understand how to install SharePoint from the command-line, especially if setting up multiple servers.
• Allows for options not available in the GUI, such as the option to rename databases to something easier to understand.
• Use PowerShell with SharePoint 2010• Sample scripts available for download…
SharePoint InstallationScripted Installations
Brisbane 2011
Function Configure-SPSearch {PARAM($AppPool, $FarmName, $SearchServiceAccount)
$searchServiceInstance = Get-SPEnterpriseSearchServiceInstance -localStart-SPEnterpriseSearchServiceInstance -Identity $searchServiceInstance
$dbName = $FarmName + "_SearchServiceApplication"
$searchApplication = New-SPEnterpriseSearchServiceApplication -Name "$FarmName Search Service Application" -ApplicationPool $AppPool -DatabaseName $dbName$searchApplicationProxy = New-SPEnterpriseSearchServiceApplicationProxy -name "$FarmName Search Service Application Proxy" -SearchApplication $searchApplication
Set-SPEnterpriseSearchAdministrationComponent -SearchApplication $searchApplication -SearchServiceInstance $searchServiceInstance
$crawlTopology = New-SPEnterpriseSearchCrawlTopology -SearchApplication $searchApplication$crawlDatabase = Get-SPEnterpriseSearchCrawlDatabase -SearchApplication $searchApplication
New-SPEnterpriseSearchCrawlComponent -CrawlTopology $crawlTopology -CrawlDatabase $crawlDatabase -SearchServiceInstance $searchServiceInstance
while($crawlTopology.State -ne "Active"){
$crawlTopology | Set-SPEnterpriseSearchCrawlTopology -Active -ErrorAction SilentlyContinueif ($crawlTopology.State -ne "Active"){
Start-Sleep -Seconds 10}
}
$queryTopology = New-SPenterpriseSEarchQueryTopology -SearchApplication $searchApplication -partitions 1$searchIndexPartition = Get-SPEnterpriseSearchIndexPartition -QueryTopology $queryTopologyNew-SPEnterpriseSearchQueryComponent -indexpartition $searchIndexPartition -QueryTopology $queryTopology -SearchServiceInstance $searchServiceInstance
$propertyDB = Get-SPEnterpriseSearchPropertyDatabase -SearchApplication $searchApplication
Set-SPEnterpriseSearchIndexPartition $searchIndexPartition -PropertyDatabase $propertyDB
while ($queryTopology.State -ne "Active"){
$queryTopology | Set-SPEnterpriseSearchQueryTopology -Active -ErrorAction SilentlyContinue
if ($queryTopology.State -ne "Active"){
Start-Sleep -Seconds 10}
}}
SharePoint InstallationSamples Scripts – http://tinyurl.com/SPFarm-Config
Brisbane 2011
SharePoint InstallationSome Manual Service Apps Still Required
• Due to complexity and/or bugs, certain Service Apps will need to be manually configured in most cases.
• This includes the following:– PerformancePoint Service Application– User Profile Service Application– Web Analytics Service Application
Brisbane 2011
SECURITY
Brisbane 2011
SharePoint SecurityLayers of Security in a SharePoint Environment
• Infrastructure Security and Best Practises– Physical Security– Best Practice Service Account Setup– Kerberos Authentication
• Data Security– Role Based Access Control (RBAC)– Transparent Data Encryption (TDE) of SQL Databases– Antivirus
• Transport Security– Secure Sockets Layer (SSL) from Server to Client– IPSec from Server to Server
• Edge Security– Inbound Internet Security (Forefront UAG/TMG)
• Rights Management
Brisbane 2011
Service Account Name Role of Service Account Special PermissionsCOMPANYABC\SRV-SP-Setup SharePoint Installation Account Local Admin on all SharePoint servers (for
install of SP).COMPANYABC\SRV-SP-SQL SQL Service Account(s) – Should be separate
admin accounts from SP accounts.Local Admin on Database Server(s) (Generally, some exceptions apply)
COMPANYABC\SRV-SP-Farm SharePoint Farm Account(s) – Can also be standard admin accounts. RBAC principles apply ideally.
N/A
COMPANYABC\SRV-SP-Search Search Account N/ACOMPANYABC\SRV-SP-Content Default Content Access Account Read rights to any external data sources
to be crawledCOMPANYABC\SRV-SP-Prof Default Profiles Access Account Member of Domain Users (to be able to
read attributes from users in domain) and ‘Replicate Directory Changes’ rights in AD.
COMPANYABC\SRV-SP-AP-SPCA Application Pool Identity account for SharePoint Central Admin.
DBCreator and Security Admin on SQL. Create and Modify contacts rights in AD OU used for email.
COMPANYABC\SRV-SP-AP-Data Application Pool Identity account for the Content related App Pool (Portal, MySites, etc.) Additional as needed for security.
N/A
SharePoint SecurityInfrastructure – Sample List of Service Accounts
Brisbane 2011
• When creating any Web Applications in Classic-mode, USE KERBEROS. It is much more secure and also faster with heavy loads as the SP server doesn’t have to keep asking for auth requests from AD.
• Kerberos auth does require extra steps, which makes people shy away from it, but once configured, it improves security considerably and can improve performance on high-load sites.
• Should also be configured on SPCA Site! (Best Practice = Configure SPCA for NLB, SSL, and Kerberos (i.e. https://spca.companyabc.com)
SharePoint SecurityInfrastructure – Enable Kerberos when using Classic-Auth
Brisbane 2011
• Role Groups defined within Active Directory (Universal Groups) – i.e. ‘Marketing,’ ‘Sales,’ ‘IT,’ etc.
• Role Groups added directly into SharePoint ‘Access Groups’ such as ‘Contributors,’ ‘Authors,’ etc.
• Simply by adding a user account into the associated Role Group, they gain access to whatever rights their role requires.
User1
User2
Role Group
SharePoint Group
SharePoint SecurityData – Role Based Access Control (RBAC)
Brisbane 2011
• New in SQL Server 2008
• Only Available with the Enterprise Edition
• Seamless Encryption of Individual Databases
• Transparent to Applications, including SharePoint
SharePoint SecurityData - Transparent Data Encryption (TDE)
Brisbane 2011
SharePoint SecurityData - Use SharePoint-Aware Antivirus (3rd Party or FPS)
Brisbane 2011
• External or Internal Certs highly recommended
• Protects Transport of content• 20% overhead on Web Servers• Can be offloaded via SSL offloaders if
needed• Don’t forget for SPCA as well!
SharePoint SecurityTransport - Secure Sockets Layer (SSL) Encryption
Brisbane 2011
• By default, traffic between SharePoint Servers (i.e. Web and SQL) is unencrypted
• IPSec encrypts all packets sent between servers in a farm
• For very high security scenarios when all possible data breaches must be addressed
SharePoint SecurityTransport – IPSec from Server to Server
Brisbane 2011
SharePoint SecurityEdge – Forefront Unified Access Gateway
Brisbane 2011
• AD RMS is a form of Digital Rights Management (DRM) technology, used in various forms to protect content
• Used to restrict activities on files AFTER they have been accessed:– Cut/Paste– Print– Save As…
• Directly integrates with SharePoint DocLibs
SharePoint SecurityRights Management - Active Directory Rights Management Services
Brisbane 2011
For More Information
• SharePoint 2010 Unleashed from SAMS Publishing (http://www.samspublishing.com)
• Windows Server 2008 R2 Unleashed and/or Hyper-V Unleashed (http://www.samspublishing.com)
• Microsoft ‘Virtualizing SharePoint Infrastructure’ Whitepaper (http://tinyurl.com/virtualsp)
• Microsoft SQL Mirroring Case Study (http://tinyurl.com/mirrorsp )
• Failover Mirror PowerShell Script (http://tinyurl.com/failovermirrorsp )
• SharePoint Kerberos Guidance (http://tinyurl.com/kerbsp)
• SharePoint Installation Scripts (http://tinyurl.com/SPFarm-Config)
• Contact us at CCO.com
Thanks for listening!
Remember to submit your feedback so you can go into the raffle draw at the end of the day! And don’t forget that
you have to be at the draw to claim your prises!
SponsorsGold
Silver
Bronze
Brisbane 2011
Thanks for attending!Questions?
Michael NoelTwitter: @MichaelTNoel
www.cco.comSlides: slideshare.net/michaeltnoel