DAT319 XML In The Database The XML Data Type In SQL Server 2005 (Code Named "Yukon") Michael Rys Program Manager SQL Server XML Technologies Microsoft.

DAT319DAT319

XML In The DatabaseXML In The DatabaseThe XML Data Type In SQL Server 2005 The XML Data Type In SQL Server 2005 (Code Named "Yukon")(Code Named "Yukon")

DAT319DAT319

XML In The DatabaseXML In The DatabaseThe XML Data Type In SQL Server 2005 The XML Data Type In SQL Server 2005 (Code Named "Yukon")(Code Named "Yukon")

Michael RysMichael RysProgram Manager Program Manager SQL Server XML TechnologiesSQL Server XML TechnologiesMicrosoft CorporationMicrosoft Corporation

XML CharacteristicsXML CharacteristicsXML CharacteristicsXML Characteristics

Self-describing:Self-describing:<doc id="d1">This is an <doc id="d1">This is an <important>example</important>.</doc><important>example</important>.</doc>

Complex dataComplex dataTrees, recursive, graphTrees, recursive, graphStructured Data: Highly regular, homogeneous structuresStructured Data: Highly regular, homogeneous structuresSemi-structured Data: Heterogeneous, sparse dataSemi-structured Data: Heterogeneous, sparse dataMarkup Data: Documents/content markupMarkup Data: Documents/content markup

Document ordering Document ordering Schema/Type systemSchema/Type system

Schema-lessSchema-lessOptional Schema: Semi-structured, structuredOptional Schema: Semi-structured, structured

ExtensibleExtensibleAnnotations, multiple schemas (late binding)Annotations, multiple schemas (late binding)

XML ScenariosXML ScenariosXML ScenariosXML Scenarios

Document ManagementDocument ManagementOffice XML DocumentsOffice XML Documents

Data/view separationData/view separationDocuments, style sheets, transformsDocuments, style sheets, transforms

Data ExchangeData ExchangeBusiness to business (B2B), business to consumer Business to business (B2B), business to consumer (B2C), application to application (A2A)(B2C), application to application (A2A)XML is ubiquitous, extensible, platform independent XML is ubiquitous, extensible, platform independent transport formattransport format

MessagingMessagingSimple Object Access Protocol (SOAP)Simple Object Access Protocol (SOAP)

Mid-Tier CollaborationMid-Tier Collaboration→→Transport, Store, and Query XML dataTransport, Store, and Query XML data

SQL Server 2000 XML SupportSQL Server 2000 XML SupportOverviewOverviewSQL Server 2000 XML SupportSQL Server 2000 XML SupportOverviewOverview

Server supportServer supportFOR XML– generate XML from tablesFOR XML– generate XML from tables

OpenXML – generate relational rowset from XML OpenXML – generate relational rowset from XML

Mid-tier supportMid-tier supportXML views (annotated mapping schemas XSD)XML views (annotated mapping schemas XSD)

Templates Templates

UpdateGrams/BulkLoadUpdateGrams/BulkLoad

Access methodsAccess methodsHTTP SOAP (via mid-tier ISAPI)HTTP SOAP (via mid-tier ISAPI)

ADO, OLE DB; ADO .NETADO, OLE DB; ADO .NET

Why XML Datatype?Why XML Datatype?Why XML Datatype?Why XML Datatype?

XML for relational dataXML for relational data

Rel Rel XML XML

Native Native XML Store XML Store in RDBMSin RDBMS

+ XML dt+ XML dt

XML in SQL Server 2000XML in SQL Server 2000(including SQL XML)(including SQL XML)

XML, relational and semistructured dataXML, relational and semistructured data

Native XML Native XML andand

semistructured datasemistructured data

DocumentDocumentMgmtMgmt

Rel Rel XML XML

XML in SQL Server “Yukon”XML in SQL Server “Yukon”(including SQL XML)(including SQL XML)

What’s New In SQL Server What’s New In SQL Server 2005?2005?What’s New In SQL Server What’s New In SQL Server 2005?2005?

New primitive XML datatypeNew primitive XML datatypeUnifying relational and XML storageUnifying relational and XML storage

Deep integration (trigger, replication, Deep integration (trigger, replication, bulkload, security, data access …)bulkload, security, data access …)

XML Schema CollectionsXML Schema Collections

Indexing of XML datatypeIndexing of XML datatype

XQuery and update on XML datatype XQuery and update on XML datatype (see DAT327)(see DAT327)

What’s New In SQL Server What’s New In SQL Server 2005?2005?What’s New In SQL Server What’s New In SQL Server 2005?2005?

FOR XML (Highlights only)FOR XML (Highlights only)Assignable, composable and thus nestableAssignable, composable and thus nestable

New PATH modeNew PATH mode

TYPE directive returns XML datatype TYPE directive returns XML datatype instanceinstance

ROOT node directiveROOT node directive

Element-centric RAW mode with row Element-centric RAW mode with row namingnaming

XML Or Relational?XML Or Relational?XML Or Relational?XML Or Relational?

XMLXML RelationalRelationalFlat Structured DataFlat Structured Data Hierarchical Hierarchical Structured DataStructured Data

Not First Class: PK-Not First Class: PK-FK with cascading FK with cascading deletedelete

Semi-structured DataSemi-structured Data Not First ClassNot First Class

Mark-up DataMark-up Data Not First Class: FTSNot First Class: FTS

Order preservationOrder preservation Not First ClassNot First Class

RecursionRecursion (Recursive query)(Recursive query)

Architectural OverviewArchitectural OverviewArchitectural OverviewArchitectural Overview

XML ParserXML ParserXMLXML

ValidationValidation

XML datatypeXML datatype

(binary XML)(binary XML)

SchemaSchema

CollectionCollection

XMLXML RelationalRelational

XML SchemataXML Schemata

OpenXML/nodes()OpenXML/nodes()

FOR XML with FOR XML with TYPE directiveTYPE directive

RowsetsRowsets

query()query()

modify()modify()

XML Datatype AndXML Datatype AndXML Schema CollectionsXML Schema Collections

Native SQL typeNative SQL typeUse for column, variable or parameterUse for column, variable or parameter

Can representCan representXML 1.0 documentsXML 1.0 documentsXML 1.0 fragments (0 to n element nodes and text XML 1.0 fragments (0 to n element nodes and text nodes at top)nodes at top)Can be constrained by XML Schema collectionCan be constrained by XML Schema collection

Methods on XML data type (see DAT327)Methods on XML data type (see DAT327)query(), value(), exist(), modify(), nodes()query(), value(), exist(), modify(), nodes()

Storage of XML instancesStorage of XML instancesAs LOB (2 GB) in efficient binary representation As LOB (2 GB) in efficient binary representation XML encoding transformed to UTF-16XML encoding transformed to UTF-16

Well-formed and validation checksWell-formed and validation checks

XML DatatypeXML DatatypeXML DatatypeXML Datatype

XML Schema SupportXML Schema SupportXML Schema SupportXML Schema Support

XML Schema (World Wide Web Consortium XML Schema (World Wide Web Consortium [W3C] standard)[W3C] standard)

Rich mechanism for type definitions and validation Rich mechanism for type definitions and validation constraintsconstraints

Can be used to constrain XML documentsCan be used to constrain XML documents

Benefits of typed data Benefits of typed data Guarantees shape of dataGuarantees shape of data

Allows storage and query optimizationsAllows storage and query optimizations

XML type systemXML type systemStore XML schemas in system metadataStore XML schemas in system metadata

Does not preserve annotationsDoes not preserve annotations

XML Schema CollectionsXML Schema CollectionsXML Schema CollectionsXML Schema Collections

Creating an XML Schema collection Creating an XML Schema collection CREATE XML SCHEMA COLLECTION S1 AS '<xs:schema CREATE XML SCHEMA COLLECTION S1 AS '<xs:schema targetNameSpace="http://www.ms.com/xdb" …/>')targetNameSpace="http://www.ms.com/xdb" …/>')

Constraining XML documentsConstraining XML documentsCREATE TABLE T(x XML(S1))CREATE TABLE T(x XML(S1))CREATE TABLE T(x XML(CONTENT S1))CREATE TABLE T(x XML(CONTENT S1))CREATE TABLE T(xDoc XML(DOCUMENT S1))CREATE TABLE T(xDoc XML(DOCUMENT S1))

Retrieving XML Schema collectionsRetrieving XML Schema collectionsSELECT XML_SCHEMA_NAMESPACE(N'dbo', name) SELECT XML_SCHEMA_NAMESPACE(N'dbo', name) FROM sys.xml_schema_collectionsFROM sys.xml_schema_collections

Deleting an XML Schema collectionDeleting an XML Schema collectionDROP XML SCHEMA COLLECTION S1DROP XML SCHEMA COLLECTION S1

Adding new types/elements/schemata at topAdding new types/elements/schemata at topALTER XML SCHEMA COLLECTION S1 ADD ‘<schema … ALTER XML SCHEMA COLLECTION S1 ADD ‘<schema … targetNamespace= “http://www.w3.org/namespace/”> targetNamespace= “http://www.w3.org/namespace/”> …</schema>’…</schema>’

Cannot change existing components and content Cannot change existing components and content models (watch for wildcard sections!)models (watch for wildcard sections!)

Retyping an XML instanceRetyping an XML instanceALTER TABLE ALTER TABLE T T ALTER COLUMNALTER COLUMN x x XMLXML (S1) (S1)

works from untyped to typed and from one schema works from untyped to typed and from one schema collection to another collection to another

Does not work with XML index present (drop index Does not work with XML index present (drop index first) first)

XML Schema CollectionsXML Schema CollectionsEvolutionEvolutionXML Schema CollectionsXML Schema CollectionsEvolutionEvolution

XML IndexXML IndexXML IndexXML Index

Create primary XML index on XML columnCreate primary XML index on XML columnCREATE PRIMARY XML INDEX idx_1 ON docs (xDoc)CREATE PRIMARY XML INDEX idx_1 ON docs (xDoc)

Creates secondary XML indexes on values Creates secondary XML indexes on values (VALUE), paths (PATH), properties (PROPERTY)(VALUE), paths (PATH), properties (PROPERTY)CREATE XML INDEX idx_1_Tag ON docs (xDoc) CREATE XML INDEX idx_1_Tag ON docs (xDoc) USING XML INDEX idx_1 FOR PATHUSING XML INDEX idx_1 FOR PATH

Speeds up queriesSpeeds up queriesResults can be served directly from indexResults can be served directly from index

Entire query is optimizedEntire query is optimizedSame award winning cost based optimizerSame award winning cost based optimizer

Indexes are used as availableIndexes are used as available

XQuery: XQuery: qquery XML documents and datauery XML documents and dataStandards-based: W3C working draftStandards-based: W3C working draft

In document 123, return section heading of In document 123, return section heading of section 3 and latersection 3 and laterSELECT id, xDoc.query('SELECT id, xDoc.query('

for $s in for $s in

/doc[@id = 123]//sec[@num >= 3]/doc[@id = 123]//sec[@num >= 3]

return <topic>{data($s/heading)}</topic>return <topic>{data($s/heading)}</topic>

') ') FROM docsFROM docs

For more details see DAT327For more details see DAT327

XML QueryXML QueryXML QueryXML Query

Insert, update, & delete XQuery Insert, update, & delete XQuery extensionsextensionsXML sub-tree modification:XML sub-tree modification:

Add or delete XML sub-treesAdd or delete XML sub-treesUpdate valuesUpdate values

Add a new section after section 1:Add a new section after section 1:UPDATE docs SET xDoc.modify('insertUPDATE docs SET xDoc.modify('insert<section num=''2''><section num=''2''>

<heading>Background</heading><heading>Background</heading> </section> </section>after (/doc/section[@num=1])[1]')after (/doc/section[@num=1])[1]')

XML Data ModificationXML Data ModificationXML Data ModificationXML Data Modification

FOR XMLFOR XMLPublishing relational Data as XMLPublishing relational Data as XML

Query Query ExecutionExecution

FOR XML FOR XML AggregatorAggregator

SELECT …SELECT …FROM …FROM …WHERE …WHERE …

FOR XML mode, directivesFOR XML mode, directives

ROWSETROWSET

Architectural Blueprint: FOR XMLArchitectural Blueprint: FOR XML

XMLXML dtdtnvarcharnvarchar

FOR XML And OpenXMLFOR XML And OpenXMLFOR XML And OpenXMLFOR XML And OpenXML

Fully backward compatible with SS2KFully backward compatible with SS2K

FOR XMLFOR XMLNew directive TYPE returns XML data type New directive TYPE returns XML data type

Nested FOR XMLNested FOR XML

Assignment to XML data typeAssignment to XML data type

Support for new data typesSupport for new data types

element-centric raw modeelement-centric raw mode

inline XSD for raw and auto inline XSD for raw and auto

NULL as xsi:nil or absence for element-centric modesNULL as xsi:nil or absence for element-centric modes

New PATH modeNew PATH mode

OpenXMLOpenXMLXML overflow columnXML overflow column

New types [n]varchar(max), varbinary(max), UDTNew types [n]varchar(max), varbinary(max), UDT

Full-Text IndexingFull-Text IndexingFull-Text IndexingFull-Text Indexing

XML filter XML filter

Index and query XML instancesIndex and query XML instancesMarkup serves as token boundary Markup serves as token boundary

Markup tags (including attributes) removedMarkup tags (including attributes) removed

Syntax — same as for other columnsSyntax — same as for other columnsCREATE FULLTEXT INDEX ON docs (xDoc)CREATE FULLTEXT INDEX ON docs (xDoc)

Can be combined with XQuery: Can be combined with XQuery: Use full-text search as filter, then Use full-text search as filter, then XQuery searchXQuery search

Uses full-text index firstUses full-text index first

Uses XML index on tags, values, pathsUses XML index on tags, values, paths

Full-Text SearchFull-Text SearchQuery examplesQuery examplesFull-Text SearchFull-Text SearchQuery examplesQuery examples

SELECT R.x.query ('//sec[@num=12]')SELECT R.x.query ('//sec[@num=12]')FROMFROM (SELECT * FROM docs (SELECT * FROM docs WHERE contains(xDoc, 'Wrd1 Wrd2')) R(x)WHERE contains(xDoc, 'Wrd1 Wrd2')) R(x)

Supports xml:lang attributeSupports xml:lang attributeUses appropriate language word breakerUses appropriate language word breaker

SELECT * FROM docs SELECT * FROM docs WHERE contains (xDoc, 'Visionen', WHERE contains (xDoc, 'Visionen',

LANGUAGE 'German')LANGUAGE 'German')

Client XML Data AccessClient XML Data AccessClient XML Data AccessClient XML Data Access

Native AccessNative AccessSQLOLEDBSQLOLEDB

As DBTYPE_WSTRAs DBTYPE_WSTRLarge stringLarge string

As ISequentialStreamAs ISequentialStreamSAX or DOM can be built from server character streamSAX or DOM can be built from server character stream

Down Level, ADO and ODBC: large stringDown Level, ADO and ODBC: large stringManaged Access & CLR (XML datatype)Managed Access & CLR (XML datatype)

XML Type exposed as SqlXml XML Type exposed as SqlXml (System.Data.SqlTypes)(System.Data.SqlTypes)

XmlReader can be obtained using XmlReader can be obtained using sqlxml.CreateReader()sqlxml.CreateReader()

HTTP/SOAP native (new) and via ISAPI (as now)HTTP/SOAP native (new) and via ISAPI (as now)XML character streamXML character stream

Support within SqlClientSupport within SqlClientSqlDataReader SqlDataReader

SqlXml GetSqlXml(int i);SqlXml GetSqlXml(int i);

UseUse sqlxml.CreateReader()sqlxml.CreateReader() to obtain an to obtain an XmlReaderXmlReader

DataSetDataSetNew XPathDocument column typeNew XPathDocument column type

Full databinding supportFull databinding support

Support For XML Data Type Support For XML Data Type Inside ADO .NetInside ADO .NetSupport For XML Data Type Support For XML Data Type Inside ADO .NetInside ADO .Net

More information on XQueryMore information on XQueryDAT327 Presentation DAT327 Presentation

Whitepapers and other informationWhitepapers and other informationhttp://msdn.microsoft.com/sql http://msdn.microsoft.com/sql http://msdn.microsoft.com/xml http://msdn.microsoft.com/xml

NewsgroupNewsgroupnews:news:microsoft.public.sqlserver.xml microsoft.public.sqlserver.xml

My contactMy contactEmail: Email: [email protected]@microsoft.com

Weblog: Weblog: http://http://www.sqljunkies.com/weblog/mryswww.sqljunkies.com/weblog/mrys

Visit the SQL Server 2005 website: Visit the SQL Server 2005 website: www.microsoft.com/sql/2005 www.microsoft.com/sql/2005

Learn more about SQL Server 2005 at TechLearn more about SQL Server 2005 at TechEdEd Hands On Labs Hands On Labs

Rooms 6E and 6FRooms 6E and 6F 13 Hands On Labs13 Hands On Labs

Ask the Experts Ask the Experts Track Cabanas located around CommNetTrack Cabanas located around CommNet Experts Available All WeekExperts Available All Week

Next Steps: SQL Server 2005Next Steps: SQL Server 2005

Exclusive TechExclusive TechEd Offer!Ed Offer!Receive Receive Beta 2Beta 2 of of SQL Server 2005SQL Server 2005

Register for SQL Server 2005 Beta 2 at: Register for SQL Server 2005 Beta 2 at: http://www.msteched.com/SqlBetaBits.aspxhttp://www.msteched.com/SqlBetaBits.aspx

Exclusive TechExclusive TechEd Offer!Ed Offer!Receive Receive Beta 2Beta 2 of of SQL Server 2005SQL Server 2005

Register for SQL Server 2005 Beta 2 at: Register for SQL Server 2005 Beta 2 at: http://www.msteched.com/SqlBetaBits.aspxhttp://www.msteched.com/SqlBetaBits.aspx

Q1:Q1: Overall satisfaction with the sessionOverall satisfaction with the session

Q2:Q2: Usefulness of the informationUsefulness of the information

Q3:Q3: Presenter’s knowledge of the subjectPresenter’s knowledge of the subject

Q4:Q4: Presenter’s presentation skillsPresenter’s presentation skills

Q5:Q5: Effectiveness of the presentationEffectiveness of the presentation

Please fill out a session evaluation on CommNetPlease fill out a session evaluation on CommNet

© 2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.