MonetDB/XQuery Technology Preview 1

MonetDB/XQuery

Technology Preview 1

Stefan ManegoldCentrum voor Wiskunde en Informatica

Amsterdam

http://monetdb.cwi.nl/ - http://pathfinder-xquery.org/

European Pathfinder Team

• CWI, Amsterdam (Netherlands)– Peter Boncz, Stefan Manegold, Sjoerd Mullender

• University of Twente (Netherlands)– Maurice van Keulen, Jan Flokstra

• University of Konstanz (Germany)– Torsten Grust, Jens Teubner, Jan Rittinger

Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery

http://www.inf.uni-konstanz.de/dbis/teaching/ws0304/pathfinder/

http://db.cs.utwente.nl/

Results: Performance (1)


XMark benchmark, 110 MB: MonetDB/XQuery vs. X-Hive & Galax



XMark benchmark, 1.1 GB: MonetDB/XQuery vs. X-Hive

Story

• XQuery Example• Relational XQuery

– System Architecture– XML Encoding

• Science & Reseach• Scalability• Outlook

– Conclusions– Roadmaps– Release & References


• For each author, return number of books and receipts

for books published in the past 2 years, ordered by name

let $cat := fn:doc(“www.bn.com/catalog.xml”), (:Documents:) $sales := fn:doc(“www.publishersweekly.com/sales.xml”)

for $author in distinct-values($cat//author) (:Grouping:) let $books := $cat//book[@year >= 2003 and author = $author],(:Sel.:) $receipts := $sales/book[@isbn = $books/@isbn]/receipts (:Join:) order by $author (:Ordering:) return <sales> (:XML Construction:)

{ $author } <count> { fn:count($books) } </count> (:Aggregation:)

<total> { fn:sum($receipts) } </total> </sales>

XQuery Example


• For each author, return number of books and receipts

for books published in the past 2 years, ordered by name

let $cat := fn:doc(“www.bn.com/catalog.xml”), Documents $sales := fn:doc(“www.publishersweekly.com/sales.xml”)

for $author in distinct-values($cat//author) Grouping let $books := $cat//book[@year >= 2003 and author = $author],Sel. $receipts := $sales/book[@isbn = $books/@isbn]/receipts Join order by $author Ordering return <sales> XML Construction

{ $author } <count> { fn:count($books) } </count> Aggregation

<total> { fn:sum($receipts) } </total> </sales>

XQuery Example


XQuery Systems: 2 Approaches

• Existing “native” XML/XQuery systems are built from scratch– Galax, Saxon, …– X-Hive, Tamino, …– (Still have to) re-invent optimization technology

• Our approach:– Build XQuery system on top of an RDBMS– Leverage mature relational technology

to achieve efficient XQuery processing


Architecture


xx

XML in an RDBMS: XPath Accelerator

Node-based relational encoding of XQuery's data model

f/following: SELECT * FROM pre_post WHERE pre > f.pre AND post > f.post f/descendant: SELECT * FROM pre_post WHERE pre > f.pre AND post < f.post f/preceeding: SELECT * FROM pre_post WHERE pre < f.pre AND post < f.post f/ancester: SELECT * FROM pre_post WHERE pre < f.pre AND post > f.post

Similar queries for all 13 XPath axes

<a> <c/> <d/> <e> <f> <g/> <h/> </f> <j/> </e> </a>

0<a> 1 2<c/> 3<d/> 4<e> 5<f> 6<g/> 7<h/> </f> 8 9<j/> </e> </a>

0<a> 1 2<c/>0 1 3<d/>2 4<e> 5<f> 6<g/>3 7<h/>4 </f>5 8 9<j/>6 7 </e>8 </a>9


Science & Research

• More research lead to more optimization– Join Recognition– Embedded XPath processing– Order Awareness

• Various scientific publications


Results: Scalability (3)

Unsurpassed scalability • Standard Opteron PC, 8GB RAM, 64-bit Linux• Can process 11GB documents!

Mostly linear scaling with document size


Conclusions

• Relational approach Works Is fast Is scalable

• Crucial Optimizations– Join recognition– Embedded XPath processing– Order awareness

• Research turned into open-source release


Roadmap

• 30.05.05: MonetDB/XQuery 4.8/0.8 “Mercurius”– Developers Release / Technology Preview 1

• 30.09.05: MonetDB/XQuery 4.10/0.10 “Venus”– Student Release / Technology Preview 2– XUpdate, More Optimization

• 30.12.05: MonetDB/XQuery 4.12/1.12 “Mars”– Final Release– Application Programming Interfaces– End-User Front-Ends


Open Source Release & References

• MonetDB + Pathfinder on SourceForge– Mozilla-like License

• MonetDB homepage– http://monetdb.cwi.nl/

• Pathfinder homepage– http://pathfinder-xquery.org/

• Developers website– http://sf.net/projects/monetdb/

You are welcome to join the MonetDB/XQuery team!


http://monetdb.cwi.nl/Home/

http://sourceforge.net/

XMark 11 GBQ Galax X-Hive MDB/XQ Galax X-Hive MDB/XQ X-Hive MDB/XQ MDB/XQ

1 0.06 0.37 0.05 0.72 1.29 0.41 9.9 1.2 132 0.03 0.45 0.07 0.31 1.75 0.30 33.0 2.4 253 0.14 0.65 0.28 1.76 5.66 1.51 25.1 12.5 1264 0.22 0.10 0.08 2.91 1.00 0.45 18.1 3.8 365 0.05 0.13 0.05 0.63 0.90 0.16 20.7 1.2 116 1.30 1.07 0.02 13.29 10.17 0.05 178.1 0.3 37 2.68 1.57 0.03 30.01 24.84 0.07 278.4 0.4 48 0.16 0.85 0.14 2.12 3.51 0.75 49.1 10.4 2089 113.23 32.25 0.20 DNF 12280.66 0.87 DNF 12.9 289

10 1.74 5.28 0.80 18.61 442.37 5.31 DNF 55.0 188211 2.62 98.91 0.18 DNF 19927.29 3.48 DNF 872.5 DNF12 1.44 23.39 0.14 DNF 5100.19 1.66 DNF 150.7 DNF13 0.03 0.10 0.07 0.66 1.03 0.22 12.9 1.3 1314 1.92 0.72 0.17 99.53 11.16 1.40 110.2 13.7 95915 0.02 0.03 0.09 0.20 0.49 0.28 10.6 1.7 1616 0.03 0.03 0.11 0.46 0.52 0.26 10.9 1.8 1817 0.06 0.09 0.07 0.82 0.85 0.30 11.8 2.8 2618 0.07 0.08 0.04 0.73 0.64 0.13 14.8 0.9 919 1.17 0.67 0.11 14.73 12.15 0.55 254.5 5.3 8820 0.28 0.11 0.24 2.98 1.40 0.62 24.6 4.9 50

11 MB 110 MB 1.1 GB

Stefan Manegold HollandOpen, Amsterdam 31-5-2005MonetDB/XQuery


XMark performance in seconds: MonetDB/XQuery vs. Galax & X-Hive