Lenovo® System x3650 M5 with HGST Ultrastar® SAS SSDs 9TB ... · New Data Warehouse Features in Microsoft® SQL Server® 2016 SP1 With the release of SQL Server 2016 SP1, Microsoft

www.SanDisk.com

Lenovo®Systemx3650M5withHGSTUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEditionBasedontheSQLServer®2016DataWarehouseFastTrackReferenceArchitecture(DWFTRA)

WHITEPAPER

LenovoSystemx3650M5withUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEdition

2

TableofContents

ExecutiveSummary..........................................................................................................................................3

Audience................................................................................................................................................................3

AbouttheLenovoSystemx3650M5................................................................................................................4

HGSTUltrastar®SSD1600MM...........................................................................................................................4

NewDataWarehouseFeaturesinMicrosoft®SQLServer®2016SP1................................................................5

AbouttheDataWarehouseFastTrackReferenceArchitecture.........................................................................5

ReferenceArchitecture.....................................................................................................................................5

ServerUEFI/BIOSConfiguration........................................................................................................................6

GeneralSettings....................................................................................................................................................6

OperatingSystemSettings................................................................................................................................6

WindowsConfiguration–PowerSettings.........................................................................................................6

StorageConfiguration............................................................................................................................................6

StoragePool:PhysicalDisks..............................................................................................................................7

StoragePool:VirtualDiskandVolume..............................................................................................................7

SQLServerSettings................................................................................................................................................7

DatabaseConfiguration.....................................................................................................................................7

tempdbConfiguration.......................................................................................................................................7

MemoryAllocation............................................................................................................................................7

LocalSecurityPolicy..........................................................................................................................................7

SQLServer2016ConfigurationParameters......................................................................................................8

MeasuredPerformance:ReferencePoint.........................................................................................................8

SQLServerDataWarehouseFastTrackCertification........................................................................................9

Summary........................................................................................................................................................10

BillofMaterials..............................................................................................................................................11

LenovoSystemx3650M5...............................................................................................................................11

LenovoSystemx3650M5withUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEdition

3

ExecutiveSummary

Thiswhitepaperdetailstheserver,storage,andsoftwareconfigurationforaMicrosoft®SQLServer®-basedDataWarehouseusingtheLenovo®Systemx3650M5withcertainHGST-brandUltrastar®12GSASSSDs,whichMicrosofthascertifiedforaFastTrackRatedUserCapacityof9TB.

ThegoalwiththisconfigurationwastoprovideahardwareandsoftwareconfigurationthatmaximizedsavingswhilesimultaneouslyprovidingtheperformancetopassFastTrackcertification.ThiswasachievedbyinstallingasingleCPUinthetwo-socketx3650M5system,usingthemoreaffordableSQLServer2016SP1StandardEdition(SE)andWindowsServer®2016StandardEditionsoftware,andscalingthestorageconfigurationtobalanceperformance,capacity,andcost.

ThegoalofaDataWarehouseFastTrackReferenceArchitectureistoachieveanefficientresourcebalancebetweenSQLServerdataprocessingcapabilityandcomponenthardwarethroughput,andtoprovidecustomersstandardandprovensystemarchitecturesoptimizedforarangeofenterprisedatawarehousingneeds.

SQLServer2016SP1SEexposesfeatureswhichwereformerlyavailableonlyinEnterpriseEdition(EE),suchascolumnstorein-memorytechnology,datacompression,andtablepartitioning.ThesefeaturesareleveragedinthisreferencearchitectureandqualifyuseofSQLServerSEforFastTrack-certification.

ThisconfigurationusesStandardEdition.ComparedtoanEnterpriseEditionconfiguration,licensingsavingsexceedthecostofthehardware.

Thesystemcharacterizedinthisdocumentdeliversdatawarehouseperformancecomparabletoamuchmoreexpensivesystem:80%rowstorerelativethroughput,and122%columnstorerelativethroughputonthisSQLServerStandardEditionconfiguration,comparedtothe2-socketreferenceconfigurationrunningSQLServer2014EnterpriseEdition.

AlthoughthisconfigurationusedonlyfourHGSTUltrastar1.6TBSSDs,ithasaRatedUserCapacityof9TB.AddinganotherSSDresultedinatestratingof12TB,whichcanalsobeobtainedbyusingaStorageSpacesresiliencysettingof“simple”insteadof“parity.”Thus,a33%greaterRatedCapacitycanbeachievedbyaddingasingle1.6TBdeviceorbyprovidingavailabilitybeyondthestoragelayer(notuncommonfordatawarehousestores).

Thissystemscaleseasilytohandlelarger,moredemandingdatawarehouseworkloadsbysubstitutingaprocessorwithmorecores,populatingthesecondsocket,addingsystemmemory,addingstorage,orupgradingtoSQLServer2016EnterpriseEdition.LenovoSystemx-basedsystemswithvalidatedFastTrackRatedUserCapacitiesofupto145TBhavebeenpublished.

Audience

Thisdocumentisforindividuals(BusinessIntelligencedatabase,andsystemarchitects;DBAs;reportdevelopers;ITdirectors)involvedindecisionmakingwhoarelookingforguidancewhendesigningenterprisebusinessintelligenceandanalyticsapplications.

LenovoSystemx3650M5withUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEdition

4

AbouttheLenovoSystemx3650M5

TheLenovoSystemx3650M5ServerdeliversthelatestperformanceandexpandabilityintheLENOVO2Urackportfolio.Reliability,serviceability,andnearcontinuousavailability,backedbyacomprehensivewarranty,makeitidealforanyserverenvironment.

LenovoSystemservers,suchastheSystemx3650M5server,featurethelatestIntel®Xeon®E5-2600v4series(Broadwell-EP)processors.Withmorecoresandmorememory,thenewM5familyisfast.ThegreatlyincreasedprocessingpowerisprovidedbythelatestE5-2600v4processors.IncorporatingflashfromHGSTimprovesproductivitythroughdataconsolidation,availability,performance,andscalability.Thesesolid-statedevicessimplifyDWFTstorageconfigurationandmaintenanceversustheuseofSANs,whichdemandmoremanagementoverheadandaresignificantlymoreexpensive.LenovoSystemx3650M5serversincludethefollowingfeatures:

• Twotimesthememorycapacityofpreviousgenerationprocessors,with24DIMMsocketsinthex3650M5

• Supportfor64GBTruDDR4MemoryLRDIMMs,upto1.5TBofmemoryinthex3650M5

• Newstoragetechnologies,suchasenterpriseflashfromHGST

Acomprehensive,easy-to-understandreviewofoptionsandfeaturesisavailableinthisauthoritativereference:LenovoSystemx3650M5(MachineType8871)ProductGuidehttps://lenovopress.com/lp0068-lenovo-system-x3650-m5-machine-type-8871

HGSTUltrastar®SSD1600MMSASSSDs

Tomeettoday’sever-increasingneedsforreliability,endurance,andperformanceindemandingenterpriseenvironments,thesecondgenerationUltrastarSSD1600MM12Gb/sSASSSDisasolidchoice.ItistheresultofdecadesofprovenenterprisestorageexpertiseinSASdesign,reliability,firmware,customerqualification,andsystemintegration.TheUltrastarSSD1600MMSSDdelivers10drivewritesperday(DW/D)forfiveyears,anditoffersabroadrangeofencryptionoptions.

TheUltrastarSSD1600MMdoublesinterfacespeedfromthepriorgeneration,providinghighperformanceformission-criticalapplicationworkloadsforhyperscale,cloud,andvirtualizeddatacenters.Itoffersafeature-rich,robustdesign,includingfulldatapathprotection(T10-DIFsupport),temperature-basedpowercontrol,SED

LenovoSystemx3650M5withUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEdition

5

instantsecureerase,anMTBFof2.5millionhours,andpower-failprotection.ThisSSDisbackward-compatiblewith6Gb/sSASandoffersasinglefirmwarebinaryplatformforseamlessintegrationandreducedqualificationtimes.Formoreinformation,refertotheSSD1600MMdatasheet:http://www.hgst.com/sites/default/files/resources/US_SSD1600MM_ds.pdf

NewDataWarehouseFeaturesinMicrosoft®SQLServer®2016SP1

WiththereleaseofSQLServer2016SP1,Microsoftannouncedtheavailabilityofacommonprogrammingsurfaceacrosseditions.FeaturesformerlyavailableonlyinEnterpriseEditionwereexposedinalleditions,includingcolumnstorein-memoryindexes,datacompression,andtablepartitioning,arenowavailableinStandardEdition.Thesystemcharacterizedinthisdocumentleveragesthesecapabilities,provingouttheperformancebypassingFastTrackcertificationinascalablearchitecture.FormoreinformationoncolumnstoreindexesinSQLServer2016SP1,see:

Developerschoice:ColumnstoreindexinStandardandExpressEditionwithSQLServer2016(SP1)https://blogs.msdn.microsoft.com/sql_server_team/columnstore-index-standard-and-express-editions-with-sql-server-2016-sp1

ThearticledocumentsthatcolumnstorequeriesinStandardEditionarelimitedtotwothreads.Thislimitation—areasonablecompromiseforsubstantiallyreducedlicensingcosts—didnotthwartthissystemfromachievingFastTrackcertification.

AbouttheDataWarehouseFastTrackReferenceArchitecture

TheSQLServerDataWarehouseFastTrackreferencearchitecture(DWFTRA)providesascalableframeworktoachievemaximumperformancefromSMP-basedservers.FastTrackeliminatesthecomplexityofsizingserverswithdatawarehousesbyprovidingasetofdataconsumptionratesthatproperlybalancesperformancebetweenthestoragesubsystem,CPU,andmemory.

MoreinformationonSQLServerDWFTRAcanbefoundhere:http://www.microsoft.com/en-us/server-cloud/data-warehouse-fast-track.aspx

ReferenceArchitecture

ThefollowingtableshowstheconfigurationdetailsfortheLenovoSystemx3650M5withUltrastarSSDsDataWarehouseFastTrackReferenceArchitecture.

Server LenovoSystemx3650M5

OperatingSystem MicrosoftWindowsServer2016StandardEdition

CPU Intel®Xeon®[email protected](1S/8C/16T)

PCI-ESlots UptoninePCIe3.0slotsSlots4,5,and9arethefixedslotsonthesystemboard,andtheremainingslotsdependontherisercardsinstalled.

OSDrives 2x1.2TB10K12GbsSAS2.5inHDD(mirrored)

DataDrives 4x1.6TBUltrastarSSD1600MM12GbsSASSSD(StorageSpaces)

RAM 128GB

LenovoSystemx3650M5withUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEdition

6

ServerUEFI/BIOSConfiguration

• TurboBoost:PowerOptimized

• OperatingMode:MaximumPerformance

• C-Status:Disable

• Power/PerformanceBias:PlatformControlled

• PlatformControlledType:MaximumPerformance

• Hyper-Threading:Enable

GeneralSettings

OperatingSystemSettings

TheoperatingsystemusedforthisDataWarehouseFastTracktestwasMicrosoftWindowsServer2016StandardEdition.Typicalinstallationstepswereusedtoinstalltheoperatingsystemwithdefaultvalues,followedbyWindowsUpdates.

WindowsConfiguration–PowerSettings

TheHighPerformanceplanwaschosentoeliminateCPUthrottling.

StorageConfiguration

Amirroredpairof10KSASHDDshostedtheoperatingsystem.

Four1.6TBUltrastarSSD1600MMSSDswereconfiguredusingStorageSpaces.ThistechnologyenablesWindowsServertovirtualizestoragebygroupingindustry-standarddisksintostoragepoolsandthencreatingvirtualdisks(StorageSpaces)fromtheavailablecapacityinthosestoragepools.

Twovirtualdiskswerecarvedfromthephysicalmedia.SQLServeruserdatabasedataandtempdbdatawerehostedonavirtualdiskconfiguredwithsimpleresiliency.Userdatabaselogandtempdblogfileswerehostedonavirtualdiskconfiguredwithparity.

NotethattheonboardRAIDcontrollerwasdisabled.Thenon-RAIDI/OcontrollerdescribedintheBillofMaterialswasrequiredtoprovidediskconsistencyandfacilitatestoragespacesconfiguration.

TheperformanceprovidedbyUltrastarSSDsminimizesinvestmentcostsbyeliminatingtheneedforexternalstorageandrelatedperipherals.SQLServerwaitstatisticsconfirmedthatstoragewasnotaperformancebottleneckatanytime.

LenovoSystemx3650M5withUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEdition

7

StoragePool:PhysicalDisks

Slot BUS PhysicalDisk RawCapacity StoragePool

3

SAS UltrastarSSD1600MM 1.6TB SQLdata

SQLlog4

5

6

StoragePool:VirtualDiskandVolume

Name Layout Provisioning Capacity StoragePool

UserDataSimple Fixed 3.6TB SQLdata

TempdbData

UserLogParity Fixed 200GB SQLlog

TempdbLog

SQLServerSettings

DatabaseConfiguration

A1TBdatawarehouseschemawascreatedforbenchmarkingusingtheFastTracktoolkit.TheTPC™-H-likeschemawascomprisedofeightusertablesdistributedacrosstenfilegroups.

tempdbConfiguration

Eight512MBdatafilesanda32MBlogfilewereconfiguredforthetempdb.Thesefilesprovidedsufficientcapacitytoavoidgrowthevents.

MemoryAllocation

Ofthe128GBofavailableservermemory,118GBwasallocatedtoSQLServer.ThisstandardizedconfigurationisdefinedbytheFastTracktestcriteriatodrivediskactivity,especiallyduringtherowstorequeryruns.Actualproductionconfigurationsmaybenefitfromadditionalmemory.

LocalSecurityPolicy

TheSQLServerserviceaccountwasgrantedthefollowingprivileges:

• Enablelockpagesinmemory

• Performvolumemaintenancetasks

LenovoSystemx3650M5withUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEdition

8

SQLServer2016ConfigurationParameters

Parameter Setting Description

MemoryAllocation 118GB ThisvalueisrequiredbyFastTrackprotocols.MemoryisdeliberatelyconstrainedtodriveI/Opressureonthestoragesubsystem.

MaxDegreeofParallelism

16 WhenSQLServerrunsonacomputerwithmorethanonemicroprocessororCPU,itdetectsthebestdegreeofparallelism(thenumberofprocessorsemployedtorunasinglestatement).Forthisconfiguration,itisequivalenttothelogicalprocessorsavailablefromasinglesocket.

StartupParameter -T834 Testsonthisconfigurationleveragingthistraceflagyieldedstatisticallyequivalentresultscomparedtothedefault(notraceflag).

Whenset,SQLServerusesWindowslarge-pagememoryallocationsforthebufferpool.Thistraceflagcanimprovethroughputratesfordatawarehousingworkloads.

MeasuredPerformance:ReferencePointDuringFastTrackDatabasevalidation,Microsoft’sReferencePointtooldrivesmultipleconcurrentqueryworkloadsdesignedtoidentifybottlenecks.Thetoolestablishesthekeyperformancemetricsinthetablebelow,forasingle-socketconfiguration.

Asnotedpreviously,theconfigurationdocumentedherehasaRatedUserCapacityof9TB.However,addinganadditionalHGSTUltrastar1.6TBSSDresultedinatestratingof12TB.Likewise,thisresultwasobtainedbyusingaStorageSpacesresiliencysettingof“simple”insteadof“parity.”

Thebasereferenceconfigurationfromwhichrelativethroughputwascalculatedisa2-socketsystem.Itisratedfor25TBandusedtheDWFTv4methodologywithSQLServerEnterpriseEdition.TheresultsdocumentedhereforasingleprocessoronSQLServerStandardEditioncomparefavorablyorexceedtheperformanceofthereferenceimplementationusingasignificantlymoreexpensiveEnterpriseEdition.

ScanRateType Scanrate Description

RatedUserCapacity 9TB RepresentstheoptimalFastTrack-certifieddatacapacityoftheconfiguration.Itallowsfor5:1compressionwith10%recommendedfreespace.25%capacityisreservedfortempdb,whilesomememoryandthroughputbasedlimitsarealsoapplied.

RowStoreRelativeThroughput

80 PercentagethroughputofthisconfigurationincomparisontotheFTDWreferenceconfiguration.Thisresultalmostmatchedthereferenceconfiguration,usingonlyasinglesocket.Thereferencearchitectureisa25TBdual-socketconfiguration.

ColumnstoreRelativeThroughput

122 PercentagethroughputofthisconfigurationcomparedtotheFTDWreferenceconfiguration.

MaximumUserDataCapacity

11TB Calculated,basedontotaldiskcapacity.Allows5:1compression.ThisfactorsintherecommendedfreespacebutignoresthethroughputlimitsappliedtotheRatedUserCapacity.

RowStoreMeasuredThroughput(Q/Hr/TB)

107 Rowstorebenchmarkqueriescompletedduringtheone-hourmeasurementinterval,normalizedtothe1TBdatabase.

ColumnstoreMeasuredThroughput(Q/Hr/TB)

794 Columnstorecombinedbenchmarkqueriescompletedduringtheone-hourmeasurementinterval,normalizedtothe1TBdatabase.

LenovoSystemx3650M5withUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEdition

9

DataWarehouseFastTrackCertification

LenovoSystemx3650M5withUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEdition

10

Summary

Together,LenovoandWesternDigitalemployeesspenthundredsofhourstestingtoensuretheSQLServerDWFTRAsolutionprovidedoptimalreliabilityandperformance.ThesetestspushedtheLenovoSystemx3650M5topeakperformancewithoutfailure.Thereliabilityandperformanceexperiencedduringtestingiswhatcanbeexpectedinproductionenvironments.

AlthoughthisconfigurationusedonlyfourHGSTUltrastar1.6GBSSDs,ithasaRatedUserCapacityof9TB.AddinganotherSSDresultedinatestratingof12TB,whichcanalsobeobtainedbyusingaStorageSpacesresiliencysettingof“simple”insteadof“parity.”Thus,a33%greaterRatedCapacitycanbeachievedbyaddingasingle1.6TBdeviceorbyprovidingavailabilitybeyondthestoragelayer(notuncommonfordatawarehousestores).

Thesameconfigurationsatisfiedbothrowstoreandcolumnstoreconfigurations,deliveringhighphysicalreadthroughputintherowstoreconfigurationexceeding2.0GB/s,andhighqueryratesinthecolumnstoreconfigurationat794Q/Hr/TB.

ThisperformancewasachievedusingMicrosoftWindowsServer2016StandardEdition(insteadofDatacenterEdition),hostingSQLServer2016SP1StandardEdition(insteadofEnterpriseEdition).UsingtheStandardEditionsminimizedlicensingcostsby3.8x.ThisconfigurationincreasedsavingswhileprovidingtheperformancetoachieveFastTrackcertification.Withthisconfiguration,licensingsavingsexceedthecostofthehardware.

Thearchitecturescaleseasily,includingaddingaprocessorwithmorecores,populatingthesecondsocket,addingmorememory,addingadditionalstorage,orupgradingtoSQLServer2016EnterpriseEdition.

Insummary,thesystemcharacterizedhereoffersaffordability,reliability,performance,scalabilityallinaconfigurationvalidatedbyMicrosoft.

LenovoSystemx3650M5withUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEdition

11

BillofMaterials

LenovoSystemx3650M5

Thetablebelowliststhecomponentsofthereferencearchitecturesystem.Itshowstherecommendedoptionsforhighperformancenetworkingneededfordataloading,backupandrestore,andredundancy(powersupplies,etc.).

Quantity FeatureCode Component Description

1 8871-AC1 Server 9TBDWFT:LenovoSystemx3650M5

1 ATES Processor IntelXeonProcessorE5-2667v48C3.2GHz25MBCache2400MHz135W

4 ATCB Memory 32GBTruDDR4Memory(2Rx4,1.2V)PC4-19200CL172400MHzLPRDIMM

4 AS7J SSDFlashStorage(database) 1.6TB12GSAS2.5"MLCG3HSEnterpriseSSD

1 N2215 I/OController(non-RAID) SAS/SATAHBA

1 A3PN NIC Mellanox®ConnectX®-32x40GbE/FDRIBVPIAdapter

2 AT8C BootDevices 1.2TB10K12GbpsSAS2.5"G3HSHDD

1 A5G6 x3650M58x2.5"HSHDDAssemblyKit(SingleRAID)

1 5977 SelectStoragedevices;noIBM®configuredRAIDrequired

2 ATDY Systemx3650M524x2.5"BasewithoutPowerSupplyBDW

2 A5EW PowerSupply Systemx900WHighEfficiencyPlatinumACPowerSupply

1 9206 Nopreloadspecified

1 6311 2.8m,10A/100-250V,C13toIEC320-C14RackPowerCable

1 A5FW LenovoSystemGen-IIUniversalSlidesKit

1 ATG5 LenovoSystemx3650M5WWPackaging

1 A5R6 Systemx3650M5PCIeRiser2(2x8FH/FL+1x8FH/HLSlots)

1 ATE4 Systemx3650M5PlanarBDW

1 A59E LenovoSystemx3550M5LabelGMB

1 A5FP Systemx3650M5PCIeRiser1(2x8FH/FL+1x8FH/HLSlots)

01GX-366 3YRTechInstallParts24x7x4

Optionsandfeaturesmaybereviewedinthisauthoritativereference:LenovoSystemx3650M5(MachineType8871)ProductGuidehttps://lenovopress.com/lp0068-lenovo-system-x3650-m5-machine-type-8871

LenovoSystemx3650M5withUltrastar®SASSSDs9TBDataWarehouseFastTrackReferenceArchitectureonSQLServer2016SP1StandardEdition

12

Specificationsaresubjecttochange.©2017WesternDigitalCorporationoritsaffiliates.Allrightsreserved.WesternDigital,SanDisk,theSanDisklogo,andUltrastarareregisteredtrademarksortrademarksofWesternDigitalCorporationoritsaffiliatesintheUSand/orothercountries.IntelandXeonaretrademarksofIntelCorporationoritssubsidiariesintheU.S.and/orothercountries.Lenovo®isatrademarkofLenovointheUnitedStates,othercountries,orboth.Microsoft,Windows,SQLServerandWindowsServerareeitherregisteredtrademarksortrademarksofMicrosoftCorporationintheUnitedStatesand/orothercountries.Allothermarksarethepropertyoftheirrespectiveowners.MellanoxandConnectXareregisteredtrademarksofMellanoxTechnologies,Ltd.IBMisatrademarkofInternationalBusinessMachinesCorporation,registeredinmanyjurisdictionsworldwide.OneMBisequaltoonemillionbytes,oneGBisequaltoonebillionbytes,oneTBequals1,000GBandonePBequals1,000TBwhenreferringtoHDD/SSDcapacity.AccessiblecapacitywillvaryfromthestatedcapacityduetoformattingandpartitioningoftheHDD/SSDdrive,thecomputer’soperatingsystem,andotherfactors. DPL1701EN20170628