©2011 Quest Software, Inc. All rights reserved.. Chris McNulty SharePoint Strategic Product Manager February 2012 Boston Tech Meetup SharePoint Performance – Best Practices from the Field
Dec 13, 2014
©2011 Quest Software, Inc. All rights reserved..
Chris McNulty
SharePoint Strategic Product Manager
February 2012
Boston Tech Meetup
SharePoint Performance – Best Practices from the Field
©2011 Quest Software, Inc. All rights reserved. Confidential.
2
©2011 Quest Software, Inc. All rights reserved..
2
AmericasEMEAAPJa
Sales/MrkgR&DSupport
178 CountriesAll VerticalsGlobal 200SMB
Database, Monitoring, Data Protection, User Workspace/Virtualization, Windows (SharePoint, AD, Messaging), Identity Mgmt
60 Offices, 3 HQs
3600+ Employees
100,000+ Customers
Quest Market Presence
Multiple Business Lines
©2011 Quest Software, Inc. All rights reserved. Confidential.
3
©2011 Quest Software, Inc. All rights reserved..
3
4
©2011 Quest Software, Inc. All rights reserved..
Chris McNulty• SharePoint Strategic Product
Manager at Quest Software
• 10+ years with SharePoint
• 20 years consulting (led KMA SharePoint practice) and financial services technology (Santander, John Hancock/Manulife, GMO, State Street)
• MBA in Inv Mgmt from Boston College
• Write and speak often on Microsoft IW technologies (blogs & books)
• MCSE MCTS MSA MVTSP MCC
• Hiking, cooking, playing guitar, colonial history, photography
• My family: Hayley, three kids (17, 8, 5) and my dog Stan
6
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Agenda• The dilemma
• Architecture, Design & Planning
• Installation and Upgrade• Post Installation Best Practices
• Service/Feature Placement
• Support• Monitoring and Optimization
• Backup
• PowerShell
• Development Functions
• Optimization
• Patching
• SQL Maintenance
• Best Practices
7
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Presentation Governance
• Out Of Scope• Deep Dives (e.g. PowerShell, BI, Upgrade, SQL DBA)
• Development
• Customization
• Design & Architecture
• Power User (e.g. Library Customization, Designer Workflows, etc.)
• Rules• Move fast, PowerPoint is shared – http://
slidesha.re/xTcZYq
• Questions – time permitting during session
• Any time after session – email etc. - @cmcnulty2000
Microsoft SharePoint Server 2010 … the bright frontier
Eastern Long Island, July 4, 2010
Architecture and Design
11
©2011 Quest Software, Inc. All rights reserved..
Server Farm – Web Front End
• Typical Roles:
• http services
• Search query
• Scaling
• Add servers to load balanced cluster
• Performance Optimization
• RAM
• Easily virtualized
11
12
©2011 Quest Software, Inc. All rights reserved..
Server Farm - Application Server
• Typical Roles:
• Search index/crawl
• Excel calculation
• User profiles
• Managed Metadata
• Scaling
• Add search servers and partitions
• Move shared services to dedicated servers
• Performance Optimization
• CPU
12
13
©2011 Quest Software, Inc. All rights reserved..
Server Farm - Database
• Typical Roles:
• Data storage
• SQL Reporting
• Scaling
• Add storage capacity
• Performance Optimization
• Disk I/O
13
14
©2011 Quest Software, Inc. All rights reserved..
Sizing - Single Server
• Typical Roles:
• Small teams
• Small pools of documents
• Considerations
• Performance & fault tolerance less of a concern
• SQL & Web on same system
• Search not a core function
15
©2011 Quest Software, Inc. All rights reserved..
Sizing - Medium Farm
• Typical Roles:
• 100-10,000 users
• 10,000 – 1MM documents
• Scenarios
• Enterprise portal
• Large scale collaboration
• Broader applications platform
• Larger external search pool
• Mix and match internal external front end servers on common content databases
16
©2011 Quest Software, Inc. All rights reserved..
Sizing - Large Farm
• Typical Roles:
• Large distributed enterprise users (10000+)
• Large pools of documents (>1MM)
17
©2011 Quest Software, Inc. All rights reserved..
Office 365 Enterprise Plans
E1 E2 E3 E4
SharePoint Online
√ √ √ √
Office Web Apps
√ √ √
Local Copy of Office Professional 2010 Plus
√ √
Forms Services, Vision Services, Access Services
√ √
Monthly cost per user
$10 $16 $24 $27
Sizing – No Servers – Office 365
• Constraints/Unavailable• Custom, non sandbox solutions
• Power Pivot
• SQL Server Reporting Service Integration
• Business Connectivity Services (OK for web services- based remote data in O365 BCS.)
• FAST Search Server Integration
• Web Analytics
• Site collections greater than 100GB
18
©2011 Quest Software, Inc. All rights reserved..
Logical Components
• High capacity!
• Maximums
• 250,000 sites per site collection
• 5,000 site collections per content DB
• 200GB max content DB (single site collection)
• >200GB post SP1
• 300 Content DBs per web application
• 30MM documents/library
• 2GB document size
• 2011 News• 14TB Demo
19
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Disk SizingContent Search
Initial Content Size XXX GB External Crawl Size YYY GB
Initial User Pool U
User Collab Size .25GB
n YR Growth Rate – Archive Rate
G%
End Content Size XXX (1+G)n = ECS End Search Size YYY (1+G)n = ESS
End User Collab Size .25 * U * (1+G)n = EUCS
Content DBs ECS + EUCS
Search DBs .05 * (ECS + EUCS + ESS)
Search Index Files .05 * (ECS + EUCS + ESS)
• Inputs: Size of SharePoint content and non-SharePoint content included in search
• For DBs, don’t forget transaction logs, disk dumps (if used for backup) which can add 1-3X.
• In SAN or virtual environments, not all disk need be provisioned early
20
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Memory Sizing
• Web Front End (WFE)• 8GB minimum
• 12GB larger
• 16GB max
• Application• 8-12GB
• SQL• 8-24GB
• HP Sizing Tool• http://
h71019.www7.hp.com/activeanswers/Secure/548230-0-0-0-121.html
• Don’t forget about the swap files (1-2X size of RAM)
21
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Psst…about SQL…
• Sidebar on SQL Disk optimization• RAID 1: Mirroring (Wastes disk)
• RAID 5: Parity Bit (write performance hit)
• RAID 10 Stripes across mirrors (costly)
• Physical location of data, log, temp and/or backup files. If virtualization or SAN technology doesn’t isolate the disks, not much performance gain,
• Performance optimization/fault tolerance by:• RAID1 on boot disks
• RAID5 on data disks
• RAID10 on log disks
• No RAID, or RAID 5 on backup disks
• RBS reduces size (and count) of content databases but doesn’t reduce size of total storage
• IOPS, IOPS, IOPS!
22
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Database sizing
• Automatic database growth:• 50-100MB clumps – not by percentage.
• Using a small size leads to more frequent, but smoother, steady state growth.
• Presize tempdb to about 20% the size of the single largest content database.
MonitoringMonitoring and Operations
24
©2011 Quest Software, Inc. All rights reserved..
Proactive Issue Resolution
• Developer Dashboard• Empower developers and
users
• Integrated Health Analyzer• Runs when necessary
• Alerts anomalies
• Fixes when it can
• Web Analytics• User usage
• Resource usage
25
©2011 Quest Software, Inc. All rights reserved..
Logging, Monitoring, and Alerts
• Unified Logging
• Out-of-the-box reports
• Richer Web Analytics
• Open Schema
• SCOM Integration
• PLUS• Developer Dashboard
• Health Analyzer
ULS Logs
Windows Events
Page requests
Feature Logging
Health data
Logging DB
27
©2011 Quest Software, Inc. All rights reserved..
Developer Dashboard Improve customized solutions with the Developer Dashboard
SQL Queries Performance
Memory Usage
Data-Request Trace
28
©2011 Quest Software, Inc. All rights reserved..
Monitoring – WSS_Logging
• Query Database Views Directly
• Requires Timer Jobs Enabled• Diagnostic Data Provider: Trace Log
• Diagnostic Data Provider: Event Log
• ULS Configuration Matters
• Database will GROW!
• Aggregates from ALL Servers
• Sample: • SELECT * FROM [WSS_Logging].
[dbo].[ULSTraceLog] WHERE CorrelationID = '04377DAE-C2FD-4DBE-A57E-101B3005059E'
30
©2011 Quest Software, Inc. All rights reserved..
Development Support –Sandbox• CA | Application Management
• Manage Quotas and Locks• (Also where you can set a site collection read-only)
• Daily quota is 300 points • includes AbnormalProcessTerminationCount,
CPUExecutionTime, CriticalExceptionCount, InvocationCount, PercentProcessorTime, ProcessCPUCycles, ProcessHandleCount, ProcessIOBytes, ProcessThreadCount, ProcessVirtualBytes, SharePointDatabaseQueryCount, SharePointDatabaseQueryTime, UnhandledExceptionCount, UnresponsiveprocessCount
• Reconfigure values via PowerShell (unlikely)
• Examine them using:• [Microsoft.SharePoint.Administration.SPUserCodeService]::Local.ResourceMeasures
31
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Development Support – Three RegionsDevelopment
• often internal to developers
• problem reproduction that require advanced inspection tools (e.g. Visual Studio) are done here
• permissions can be looser, may have multiple environments for multiple developers
• sensitive data from production cannot be copied here without masking or customer signoff
• changes here can be deployed ad hoc
Staging/Test
• no Visual Studio, no MS Office
• match/mirror production as closely as possible; match hardware/system performance as closely as practical
• security permissions match production
• any sensitive data copied here stays under production-grade controls
• test accounts should be created in a separate OU if possible
• changes here can only be delivered and deployed from source control and according to production release methods
Production
• optimized hardware configurations
• highly secure• no use of user rotating
password accounts as service accounts
• changes here can only be delivered and deployed from source control and according to production release methods
32
©2011 Quest Software, Inc. All rights reserved..
demoMonitoringDeveloper DashboardHealth Analyzer
Optimization
Optimization
34
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Optimization
• IIS Bit Rate Throttling (Video)• Part of IIS Media Services 2.0 or greater
• Download http://www.iis.net/download/BitRateThrottling
• Configured through IIS Manager
• Large File Upload Support:• Central Admin: 600MB Upload Size
• IIS7 Connection Timeout 65536 seconds
• Web app web.config = change line to • <httpRuntime maxRequestLength="2097151" executionTimeout="999999" />
• in 14hive\template\layouts\web.config, change above line to same
• Finally, as per http://support.microsoft.com/kb/944981/en-us add section to web.config to allow 600MB content length (it’s a new requirements in IIS7).• <system.webServer>
• <security>
• <requestFiltering>
• <requestLimits maxAllowedContentLength="629145600"/>
• </requestFiltering>
• </security>
35
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Optimization
• Disk-based BLOB Caching• Local store for audio/video, PDF other frequent read only files
• Edit in Web.config (C:\Inetpub\wwwroot\wss\virtualdirectories\...)
• <BlobCache location="" path="\.(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled="false" />
• Location = Local Disk Location
• maxSize = GB
• Enabled = true
• Different from RBS/EBS!
• For publishing sites
36
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Patching – Process through August 2011
Patch
• SharePoint Foundation
Patch
• SharePoint Server
Deploy
• Run SharePoint Products and Technologies Wizard• (Or psconfig)
• Sequential Application to Central Admin, Application Server(s), Web Front End Servers
37
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Patching – Process After August 2011
Patch
• Separate patch longer needed – single patch CU now available
Patch
• SharePoint Foundation• OR SPF/Server• OR SPF/Server/Project Server
Deploy
• Run SharePoint Products and Technologies Wizard• (Or psconfig)
• Sequential Application to Central Admin, Application Server(s), Web Front End Servers
38
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Patching – Notes
• Test before installation!!!
• Cumulative Updates every two months
• Service Pack every 6-18 months
• Service Pack 1 REQUIRES at least the June 2011 CU
• December 2011 CU • iOS 5 Mobile Safari support for Performance Point
• Fixed Administrator updates to user profile pictures
• Check my blog for latest: http://www.chrismcnulty.net/blog/Lists/Categories/Category.aspx?CategoryId=5&Name=Version-Build Numbers
39
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
SQL Maintenance
• Backups• Local Disk – easy but storage intensive
• Agents – remote, requires extra software
• RBS Maintenance• BLOB Orphans
• Log Sizing• Full logged (default) generates huge t-logs
• Simple doesn’t but prevents point in time restore
• Maintenance Plans
40
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Metrics and Results
• IOPS <400• Improve disk I/O or reduce content database sizes
• Processor/CPU utilization >60%• You may need to add servers or redistribute services to reduce the load.
• Available memory <50%• Again, add RAM.
• Disk average reads/writes >15ms. • ~66.67 IOPS Common on virtualized SQL servers. That’s slow, and predicts
reduced performance even for smaller content databases on a SQL server. Use physical disk or move the volume to dedicated storage.
• Sustained network bandwidth higher than 25%. Network latency output queue >0• Network can’t keep up with users or the server. Add physical switched Ethernet
ports, adding servers to divide the load.
• Or move a virtualized guest server to a new host.
Best Practices
42
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Troubleshooting – Top Support Questions
• Users Receive “Cannot Connect to Configuration Database” Web Page• SharePoint farm account is locked out
• No one can upload anything but site is up• Database disk volume is full – check transaction logs, backups
• In virtualized environment, host file systems may be full
• I can’t find a document I think I should see; Someone can’t see a file I just uploaded• Security and permission variations
• Document “movement” (a/k/a ECM) try search by name or Document ID. Check ECM logs/audits
• Confirm permissions, and make sure document is checked in (Required properties may be missing)
43
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Troubleshooting – More Support Questions
• Repeated requests to re-enter Windows credentials• Add to Local intranet zone, add site, custom level, automatic login with
current user name and password (it’s the last thing in the item list)
• OR Trusted sites
• My workflow didn’t start• Recycle timer service
• “FixSharePoint.exe” = IISReset & Timer Service Recycle
• I’m not seeing the right search results• Confirm that crawls are running and complete by checking crawl logs;
restart a full crawl if crawls finish OK
• I need a file back that I deleted• Recycle Bin Recovery
• Use Backup & Restore
44
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Miscellaneous Tips and Tricks I• Bad web parts on page
• ?Contents=1 on end of URL
• HyperV & DiskPart• From the run menu type "diskpart.exe" to enter the command line utility to resize
disk partitions in Windows Server 2003.
• The command list volume will show you all the available volumes.
• Select your volume as shown below. select volume 1 corresponds to the "D" volume.
• Extend the volume with the extend command; the partition will be immediately extended under the Disk Management snap in.
• User Profile Sync• DirSync permissions in AD are essential (Replicating Directory Changes )
• PSCONFIG for a stuck wizard• psconfig.exe -cmd upgrade -inplace b2b -wait
• 14HIVE\bin on PATH
• How big is that site collection anyway?• Site Settings | Site Collection Analytics | Scroll down to Inventory and choose
Storage Usage
45
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Miscellaneous Tips and Tricks II• How do I make a bigger template?
• stsadm -o setproperty -propertyname max-template-document-size -propertyvalue size_in_bytes
• Save site as template• To save any site as a template, just go to Site Actions | Site Settings | Save Site
as template. This saves the site as a standard .WSP package in the Site Collection Solutions Gallery (/_catalogs/solutions/Forms/AllItems.aspx)
• Auditing• In Central Admin | Security, configure Information Management Policy (/_
admin/Policyfeatures.aspx). Make sure Auditing is Available
• At the Site Collection Audit Settings screen (in Site Collection Administration) select the events you want to audit
• Site Setting | Site Collection Administration | Audit Log Reports
• Service Account Management in Central Admin• Security | Managed Service Accounts
• When all else fails – web.config!!!!
Seven Deadly Sins for the SharePoint Professional
47
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Seven Deadly Sins
• No SQL maintenance plans
• Default names for every database (WSS_CONTENT_12345abc…)
• No patching
• One environment for everything
• One acct for everything
• Single server install with SQL Express
• Runaway content database size
Seven Virtues for the SharePoint Professional
49
©2011 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Seven SharePoint Virtues
• Security Applied via AD Groups and SharePoint Groups
• Review System Health
• Test Restore and Recovery
• Monthly Web Analytics Review – Usage, Storage, Search
• PowerShell instead of STSADM
• Governance
• Use ECM, MMS, Clients, Archiving and Training to Keep Content in SharePoint, reduce accidental duplication and keep searching and browsing fresh
50
©2011 Quest Software, Inc. All rights reserved..
•Questions
•Contact Me
•SharePint
•Prizes
©2011 Quest Software, Inc. All rights reserved. Confidential.
52
©2011 Quest Software, Inc. All rights reserved..
52
TEC USER GROUP DISCOUNT – (The Experts Conference - San Diego April 29-May 2)
Attend the conference for $1375 this is $300 off current price and $500 off full price! Open to all SPUG
membersemail [email protected]
$200 VISA Gift card when you arrive in San Diego**be one of the first 10 user group members to register by March 5
2 Key Notes – Bill Baer of Microsoft Michael Lotter – founder of www.sharepointsaturday.org
Three Themes-SharePoint Governance, -the future of SharePoint and -SharePoint business intelligence - featuring SQL Server 2012 Denali.
Other Speakers include:Chris McNulty, Ben Curry, Scott Jamison, Susan Hanley, Chris Beckett, Sadie Van Buren, Zlatan Dzinic, Shadeed Eleazer, Bob German, Jason Himmelstein, Dan Holme, Chris Givens, Derek Cash-Peterson, Sean McDonough, Phillip Wicklund
Email [email protected] for details – see www.facebook.com/tecsharepoint and www.theexpertsconference.com for
more info
©2011 Quest Software, Inc. All rights reserved. Confidential.
53
©2011 Quest Software, Inc. All rights reserved.
53
Increase SharePoint Adoption!
• Quest AttachThis– A simple Outlook add-in that uploads email attachments to
SharePoint and replaces them with secure links in a way that is intuitive to end-users
– Encourage users to use SharePoint collaboration while still using Outlook as their everyday business application.
– BETA starts Monday, February 27th – Granular Attachment Permissions - Users will be able to set granular
permissions for the document attachment that is to be uploaded into SharePoint based on the recipient list of the email or they can choose to assume the permissions of the site where the document is to be stored.
– Enhanced Client User Interface - Users will experience an intuitive user interface for AttachThis that enables them to send the attachment as a SharePoint hyperlink, regular attachment or both as well as select the SharePoint location and set document permissions based on the email recipient list.
– Simple Client Install and Configuration - AttachThis has a very simple to use, step-by-step installer/configurator for easy installation and implementation.
Join the Beta – www.Quest.com/AttachThis_Community
©2011 Quest Software, Inc. All rights reserved. Confidential.
54
©2011 Quest Software, Inc. All rights reserved.
54
A user sends an email with a document attachment to multiple co-workers for their edits and feedback.
When the user hits send, the attachment gets uploaded into SharePoint and a link now replaces the document attachment in the email.
The recipients receive the email with a link to the document on SharePoint.
Collaboration Today – Email Based with AttachThis
V1
V1
55
©2011 Quest Software, Inc. All rights reserved..
Thank you…
• Contact• Email [email protected]
• Blog http://www.chrismcnulty.net/blog
• Also http://www.sharepointforall.com
• Twitter: @cmcnulty2000
• LinkedIn:http://www.linkedin.com/in/cmcnulty
• Upcoming:• Feb 2012 -- SPTechCon San Francisco
• Mar 2012 – SharePoint Conference Heartland (Columbus OH),SharePoint Conference AUS, SharePoint Connections (Las Vegas)
• April 2012 – SharePoint Saturday Twin Cities; The Experts Conference San Diego
• May 2012 – New England SPUG
• June 2012 – New Hampshire SPUG
• July 2012 – SharePoint Saturday NYC
©2011 Quest Software, Inc. All rights reserved..
57
©2011 Quest Software, Inc. All rights reserved..
Questions?Get more information at
www.Quest.com/ControlYourChaos