Relational Engine Relational Engine Enhancements Enhancements James Hamilton James Hamilton Language Processing & Exec Dev Language Processing & Exec Dev Manager Manager SQL Server SQL Server Microsoft Corp Microsoft Corp [email protected][email protected]5-206 5-206
48
Embed
Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp [email protected].
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Business OperationsBusiness OperationsAvailabilityAvailabilityScalabilityScalabilityPerformancePerformanceProgrammabilityProgrammabilityISV drivenISV drivenInternet/CommerceInternet/Commerce
Data WarehousingData WarehousingLarge DatabasesLarge DatabasesComplex QueriesComplex Queries
SQL Server 7.0 SQL Server 7.0 Best TCO in the industryBest TCO in the industry Most Complete DW offeringMost Complete DW offering Handle 95% of all ERP customers needsHandle 95% of all ERP customers needs
(SAP, Baan, Psoft…)(SAP, Baan, Psoft…) >50% share on Internet Shopping Sites>50% share on Internet Shopping Sites Largest database on the InternetLargest database on the Internet
http://terraserver.microsoft.comhttp://terraserver.microsoft.com Best SAP POS B-mark on ANY platformBest SAP POS B-mark on ANY platform Best TPC-C price/perf in IndustryBest TPC-C price/perf in Industry Best TPC-C per/processor perf in industryBest TPC-C per/processor perf in industry Achieving 99.9x% availability TODAY!!!Achieving 99.9x% availability TODAY!!!
(Pennzoil, Barnes & Noble, E-Plus, Terraserver, more…)(Pennzoil, Barnes & Noble, E-Plus, Terraserver, more…) Over 8.5M seats sold in FY ’99Over 8.5M seats sold in FY ’99
SQL Server 7.0: SQL Server 7.0: Relational EngineRelational Engine State of the art QP for OLTP, DSSState of the art QP for OLTP, DSS Intra-Query ParallelismIntra-Query Parallelism Advanced join technologyAdvanced join technology Caching of query plansCaching of query plans UnicodeUnicode Distributed queriesDistributed queries Full-TextFull-Text Auto Management (memory, stats)Auto Management (memory, stats)
SQL Server 2000: SQL Server 2000: Relational Engine Relational Engine (Customer Feedback)(Customer Feedback)
ProgrammabilityProgrammability XML supportXML support ManageabilityManageability
Cascaded DRICascaded DRI User Defined FunctionsUser Defined Functions Column Level CollationsColumn Level Collations Instead Of TriggersInstead Of Triggers New DatatypesNew Datatypes FullText EnhancementsFullText Enhancements Extended PropertiesExtended Properties ChecksumChecksum Session contextSession context
Cascaded DRICascaded DRI
Automatic cascading of Deletes Automatic cascading of Deletes andand Updates Updates from PK to FK tablesfrom PK to FK tables
ANSI Standard Restrict and Cascade semanticsANSI Standard Restrict and Cascade semantics
CREATE TABLE country ( CREATE TABLE country ( country_name NVARCHAR(75) NOT NULL PRIMARY KEY ) country_name NVARCHAR(75) NOT NULL PRIMARY KEY )
CREATE TABLE employee ( CREATE TABLE employee ( employee_name NVARCHAR(75) NOT NULL,employee_name NVARCHAR(75) NOT NULL, nationality NVARCHAR(75) NOT NULL REFERENCES country nationality NVARCHAR(75) NOT NULL REFERENCES country ON UPDATE CASCADE ON UPDATE CASCADE ON DELETE NO ACTION,ON DELETE NO ACTION, passport_number VARCHAR(25) NOT NULL, passport_number VARCHAR(25) NOT NULL, PRIMARY KEY (nationality, passport_number))PRIMARY KEY (nationality, passport_number))
CREATE TABLE renewal_reminder ( CREATE TABLE renewal_reminder ( nationality NVARCHAR(75) NOT NULL,nationality NVARCHAR(75) NOT NULL, passport_number VARCHAR(25) NOT NULL, passport_number VARCHAR(25) NOT NULL, date_of_expiry datetime NOT NULL,date_of_expiry datetime NOT NULL, FOREIGN KEY (nationality, passport_number) REFERENCES employeeFOREIGN KEY (nationality, passport_number) REFERENCES employee ON UPDATE CASCADEON UPDATE CASCADE ON DELETE CASCADE )ON DELETE CASCADE )
Shiloh: User Defined FunctionsShiloh: User Defined Functions
ApplicationApplication User-defined view updatabilityUser-defined view updatability Before triggersBefore triggers
Trigger executed INSTEAD of Insert, Trigger executed INSTEAD of Insert, Delete, Update actionDelete, Update action
Trigger can be created on view or tableTrigger can be created on view or table Inserted/Deleted tables still availableInserted/Deleted tables still available Statement LevelStatement Level
Instead Of Trigger: ExampleInstead Of Trigger: Example
CREATE TRIGGER AuIns on AuthorsView INSTEAD OF CREATE TRIGGER AuIns on AuthorsView INSTEAD OF INSERTINSERT
Table TypeTable Type Table valued UDFsTable valued UDFs
BigIntBigInt 8 byte integer8 byte integer
SQL_VariantSQL_Variant Can store any base type (except LOB)Can store any base type (except LOB) Useful for open schema, annotationsUseful for open schema, annotations
Full Text EnhancementsFull Text Enhancements
Change TrackingChange Tracking Maintain list of changes to indexed dataMaintain list of changes to indexed data
Image FilteringImage Filtering Support for MS Office documents, HTMLSupport for MS Office documents, HTML Custom filters (3rd party)Custom filters (3rd party)
Failover ClusteringFailover Clustering Column Level Linguistic analysisColumn Level Linguistic analysis
Columns can have associated languageColumns can have associated language Top-N-By-RankTop-N-By-Rank
Relational To XMLRelational To XML SELECT … FOR XMLSELECT … FOR XML
FOR XML RAW (return an XML rowset)FOR XML RAW (return an XML rowset) FOR XML AUTO (exploit RI, name matching, etc.)FOR XML AUTO (exploit RI, name matching, etc.) FOR XML EXPLICIT (maximal control)FOR XML EXPLICIT (maximal control)
Annotated SchemaAnnotated Schema Maps between XML and relational schema expressed in XMLMaps between XML and relational schema expressed in XML
TemplatesTemplates Encapsulated parameterized queryEncapsulated parameterized query XSL/T supportXSL/T support XPATH supportXPATH support
Direct from DOMDirect from DOM MSXML component hosted in SQL Server kernelMSXML component hosted in SQL Server kernel
Direct URL access (SQL “owned” virtual root)Direct URL access (SQL “owned” virtual root) SELECT … FOR XMLSELECT … FOR XML Annotated schemaAnnotated schema TemplatesTemplates
XML To RelationalXML To Relational
Bulk load using XMLBulk load using XML Updategram (web delivery Updategram (web delivery
post Shiloh)post Shiloh) Templates and Annotated SchemaTemplates and Annotated Schema DOMDOM
Insert document into SQL Server DOMInsert document into SQL Server DOM Select from DOM rowset and insert Select from DOM rowset and insert
Support multiple SQL Servers Support multiple SQL Servers instances on single machineinstances on single machine
Applications:Applications: Application hosting Application hosting Server consolidationServer consolidation SQL Runtime (MSDE)SQL Runtime (MSDE) Used internally in implementation of Used internally in implementation of
shared disk Failovershared disk Failover
Multi-InstanceMulti-Instance
One default Instance (typically server name)One default Instance (typically server name) Multiple named instances (server_name\instance)Multiple named instances (server_name\instance) Named instances are all version 8.0 or aboveNamed instances are all version 8.0 or above 7.0 or 8.0 supported as default database7.0 or 8.0 supported as default database
Multiple Instances of server per systemMultiple Instances of server per system Database level collations Database level collations Microsoft installer based setupMicrosoft installer based setup Reduced start-up time, working setReduced start-up time, working set Simplified licensingSimplified licensing Access integrationAccess integration
Partitioned ViewsPartitioned Views
A partitioned view is a UNION ALL view A partitioned view is a UNION ALL view over two or more tables which have over two or more tables which have partitioning columns with disjoint partitioning columns with disjoint intervalsintervals
Example:Example:
CREATE TABLE CUSTOMER_1K (…, C_W_ID INTCREATE TABLE CUSTOMER_1K (…, C_W_ID INT NOT NULL NOT NULL CHECK (C_W_ID BETWEEN 1 AND 1000) CHECK (C_W_ID BETWEEN 1 AND 1000), , PRIMARY KEYPRIMARY KEY (C_W_ID, …) (C_W_ID, …)
……CREATE TABLE CUSTOMER_10K (…, C_W_ID INTCREATE TABLE CUSTOMER_10K (…, C_W_ID INT NOT NULL NOT NULL CHECK (C_W_ID BETWEEN 9001 AND 10000) CHECK (C_W_ID BETWEEN 9001 AND 10000), , PRIMARY KEYPRIMARY KEY (C_W_ID, …) (C_W_ID, …)
CREATE VIEW CUSTOMER ASCREATE VIEW CUSTOMER AS
SELECT …, C_W_ID FROM CUSTOMER_SELECT …, C_W_ID FROM CUSTOMER_1K1K UNION ALLUNION ALL … …
SELECT …, C_W_ID FROM CUSTOMER_SELECT …, C_W_ID FROM CUSTOMER_1OK1OK
CREATE V IEW CUSTO M ER AS SELECT <local> ...custom er1K UNIO N ALL ... SELECT nodeNK...custom erNK
node1K node2k node4k node5k nodeNk
Netw ork Sw itchFast Interconnect
...
CREATE V IEW CUSTO M ER AS SELECT node1K...custom er1K UNIO N ALL ... SELECT <local>...custom erNK
Each node in the ‘cluster’ has a single commonEach node in the ‘cluster’ has a single common logical ‘view’ of the tables across all other nodeslogical ‘view’ of the tables across all other nodes
1.1. Each table in the application is split across the ‘cluster’Each table in the application is split across the ‘cluster’
2.2. Each table is partitioned into disjoint intervals using a Each table is partitioned into disjoint intervals using a CHECK constraintCHECK constraint
3.3. All nodes are linked using OLEDB linked serversAll nodes are linked using OLEDB linked serverssp_addlinkedserver node1sp_addlinkedserver node1sp_setnetname node1, ‘MACHINE NAME’ …sp_setnetname node1, ‘MACHINE NAME’ …
4.4. Views are created for each ‘table’ on each node to link Views are created for each ‘table’ on each node to link together the partitions stored across all nodestogether the partitions stored across all nodes
PerformancePerformance
Index EnhancementsIndex Enhancements Indexed ViewsIndexed Views Other Index ImprovementsOther Index Improvements
ParallelismParallelism Query Optimizer enhancementsQuery Optimizer enhancements Distributed QueriesDistributed Queries Large memory supportLarge memory support Lots of other enhancementsLots of other enhancements
Indexed ViewsIndexed Views
Decision support workloadsDecision support workloads Content of view is persisted, maintainedContent of view is persisted, maintained View may contain joins, aggregations View may contain joins, aggregations
(some restrictions)(some restrictions) Allows storage of partial resultsAllows storage of partial results Automatically used by optimizer Automatically used by optimizer Example : Example :
Create View Store_Sales as Select StoreId, SUM(qty) Create View Store_Sales as Select StoreId, SUM(qty) From Sales Group By StoreIdFrom Sales Group By StoreId
Create Clustered Index I1 on Store_SalesCreate Clustered Index I1 on Store_SalesSelect StoreId, Qty from Sales where StoreId = ‘100’Select StoreId, Qty from Sales where StoreId = ‘100’
Index Tuning Wizard SupportIndex Tuning Wizard Support
Index EnhancementsIndex Enhancements
Indexes on Computed Columns Indexes on Computed Columns (PK, Unique constraints)(PK, Unique constraints)
Indexes with Asc, Desc columnsIndexes with Asc, Desc columns
Example : Example : Select a, b, c Select a, b, c
From TFrom T
Where a > 10Where a > 10
Order By a asc, b descOrder By a asc, b desc
RE creates, maintains indexesRE creates, maintains indexes Indexes supported on bit columnsIndexes supported on bit columns
ParallelismParallelism
Enhancements to query parallelismEnhancements to query parallelism More efficient plansMore efficient plans Bitmap filteringBitmap filtering Multi-threaded access to spoolsMulti-threaded access to spools Reduced parallel sort stallsReduced parallel sort stalls Elapsed time costingElapsed time costing Integrated deadlock detectionIntegrated deadlock detection
Insert/Update/DeleteInsert/Update/Delete Index creationIndex creation DBCCDBCC
Query OptimizerQuery Optimizer
Cardinality estimationCardinality estimation Exploit new statisticsExploit new statistics Improve many computationsImprove many computations Handle extreme skewHandle extreme skew
More granular statisticsMore granular statistics Dynamic bucket boundariesDynamic bucket boundaries
Star JoinsStar Joins Snowflake SchemaSnowflake Schema More ‘trivial’ plan casesMore ‘trivial’ plan cases
Distributed QueriesDistributed Queries
New providers – Active DirectoryNew providers – Active Directory™™, , Exchange 2000Exchange 2000
OLEDB Interfaces (Statistics, Constraints)OLEDB Interfaces (Statistics, Constraints) Cost based optimizationCost based optimization
Obtains statistics from remote sourceObtains statistics from remote source Smarter remoting logicSmarter remoting logic
More SQL dialectsMore SQL dialects Minimal SQLMinimal SQL More granular capabilitiesMore granular capabilities
Scale-up memory limited applicationsScale-up memory limited applications Up to 64GB on Intel x86 CPUs (PAE)Up to 64GB on Intel x86 CPUs (PAE) Uses Windows 2000 AWE SupportUses Windows 2000 AWE Support Single Buffer CacheSingle Buffer Cache Physical/Virtual Re-Mapping of BuffersPhysical/Virtual Re-Mapping of Buffers sp_configure optionsp_configure option
SQL Server 2000: SQL Server 2000: Relational Engine Relational Engine (Customer Feedback)(Customer Feedback)
ProgrammabilityProgrammability XML supportXML support ManageabilityManageability
Built on top of Built on top of MicrosoftMicrosoft®® Cluster Server Cluster Server Multi-Instance (no virtual layer)Multi-Instance (no virtual layer)
Fixed known issues with 7.0Fixed known issues with 7.0 Installation (Setup)Installation (Setup) Perfmon countersPerfmon counters ToolsTools Licensing informationLicensing information BugsBugs
AvailabilityAvailability
EnhancementsEnhancements Multiple IP addressesMultiple IP addresses Rolling upgradesRolling upgrades Full-TextFull-Text Disaster RecoveryDisaster Recovery Windows 2000 Data Center supportWindows 2000 Data Center support All components are cluster awareAll components are cluster aware EASY to add/remove nodesEASY to add/remove nodes