CPS216: Advanced Database Systems Notes 08:Query Optimization (Plan Space, Query Rewrites) Shivnath Babu
CPS216: Advanced Database Systems Notes 08:Query Optimization (Plan Space, Query Rewrites)
Dec 31, 2015
CPS216: Advanced Database Systems Notes 08:Query Optimization (Plan Space, Query Rewrites). Shivnath Babu. SQL query. parse. parse tree. Query rewriting. statistics. logical query plan. Physical plan generation. physical query plan. execute. result. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CPS216: Advanced Database Systems
Notes 08:Query Optimization (Plan Space, Query Rewrites)
Shivnath Babu
parse
Query rewriting
Physical plan generation
execute
result
SQL query
parse tree
logical query planstatistics
physical query plan
Query Processing - In class order
2; 16.1
3; 16.2,16.3
1; 13, 15
4; 16.4—16.7
Roadmap• Query optimization: problem definition• Space of physical plans
– Counting exercise
• Approaches for query optimization– Heuristic-based (Oracle calls them rule-based)– Cost-based– Hybrid
• Heuristics for query optimization (Query rewrites)
Query Optimization Problem
Pick the best plan from the space of physical plans
The Space of Physical Plans is Very Large
• Algebraic equivalences
• Different physical operators for the same logical operator– nested loop join, hash join, sort-merge join– index-scan, table-scan
• Different plumbing details - pipelining vs. materialization
• Different tree shapes
A Plan Counting Exercise
• Work on blackboard
Approaches for Query Optimization
• Approach 1: Pick some plan– Bad plans can be really bad!
• Approach 2: Heuristics– Ex: maximize use of indexes (MySQL)
• Approach 3: Cost-based– “Enumerate”, find cost, pick best– Be smart about how you iterate through the
plans. Why?
• Hybrid
Query Optimization in Practice
• Hybrid
• Use heuristics, called query rewrite rules– eliminate many of the really bad plans– avoid eliminating good plans
• Cost-based– Be smart about how you iterate through plans– Ex: dynamic programming, genetic search
parse
Query rewriting
Physical plan generation
execute
result
SQL query
parse tree
logical query planstatistics
physical query plan
Initial logical plan
“Best” logical plan
Logical plan
Rewrite rules
Why do we need Query Rewriting?
• Pruning the HUGE space of physical plans– Eliminating redundant conditions/operators– Rules that will improve performance with very
high probability
• Preprocessing– Getting queries into a form that we know how
to handle best
Reduces optimization time drastically without noticeably affecting quality
Query Rewrite Rules
• Transform one logical plan into another– Do not use statistics
• Equivalences in relational algebra• Push-down predicates• Do projects early• Avoid cross-products if possible• Use left-deep trees• Use of constraints, e.g., uniqueness
Example Query
Select B,D
From R,S
Where R.A = “c” R.C=S.C
Example: Parse Tree<Query>
<SFW>
SELECT <SelList> FROM <FromList> WHERE <Cond>
<Attribute> <SelList> <RelName> <FromList> <Cond> AND <Cond>
B <Attribute> R <RelName>
S<Attr> <Op> <Const>
<Attr> <Op> <Attr>
R.A = “c”
R.C S.C=
D
Select B,DFrom R,SWhere R.A = “c” R.C=S.C
Along with Parsing …
• Semantic checks– Do the projected attributes exist in the
relations in the From clause?– Ambiguous attributes?– Type checking, ex: R.A > 17.5
• Expand views
Initial Logical Plan
Relational Algebra: B,D [ R.A=“c” R.C = S.C (RXS)]
Select B,DFrom R,SWhere R.A = “c” R.C=S.C
B,D
R.A = “c” Λ R.C = S.C
X
R S
Apply Rewrite Rule (1)
B,D [ R.C=S.C [R.A=“c”(R X S)]]
B,D
R.A = “c” Λ R.C = S.C
X
R S
B,D
R.A = “c”
X
R S
R.C = S.C
Apply Rewrite Rule (2)
B,D [ R.C=S.C [R.A=“c”(R)] X S]
B,D
R.A = “c”
X
R
S
R.C = S.C
B,D
R.A = “c”
X
R S
R.C = S.C
Apply Rewrite Rule (3)
B,D [[R.A=“c”(R)] S]
B,D
R.A = “c”
R
S
B,D
R.A = “c”
X
R
S
R.C = S.CNatural join
Equivalences in Relational Algebra
R S = S R Commutativity
(R S) T = R (S T) Associativity
Also holds for: Cross Products, Union, Intersection
R x S = S x R
(R x S) x T = R x (S x T)
R U S = S U R
R U (S U T) = (R U S) U T
Rules: Project
Let: X = set of attributes
Y = set of attributes
XY = X U Y
xy (R) = x [y (R)]
Let p = predicate with only R attribs
q = predicate with only S attribs
m = predicate with only R,S attribs
p (R S) =
q (R S) =
Rules: combined
[p (R)] S
R [q (S)]
Rules: combined (continued)
pq (R S) = [p (R)] [q (S)]
pqm (R S) =
m [(p R) (q S)]pvq (R S) =
[(p R) S] U [R (q S)]
p1p2 (R) p1 [p2 (R)]
p (R S) [p (R)] S
R S S R
x [p (R)] x {p [xz (R)]}
Which are “good” transformations?
Conventional wisdom: do projects early
Example: R(A,B,C,D,E) x={E} P: (A=3) (B=“cat”)
x {p (R)} vs. E {p{ABE(R)}}
But: What if we have A, B indexes?
B = “cat” A=3
Intersect pointers to get
pointers to matching tuples
Bottom line:
• No transformation is always good
• Some are usually good: – Push selections down– Avoid cross-products if possible– Subqueries Joins
More Query Rewrite Rules
• Transform one logical plan into another– Do not use statistics
• Equivalences in relational algebra• Push-down predicates• Do projects early• Avoid cross-products if possible• Use left-deep trees• Subqueries Joins• Use of constraints, e.g., uniqueness
Avoid Cross Products (if possible)
• Which join trees avoid cross-products?• If you can't avoid cross products, perform
them as late as possible
Select B,DFrom R,S,T,UWhere R.A = S.B R.C=T.C R.D = U.D
Use Left Deep Plans
• What are some left-deep, right-deep, and bushy plans for this query?
• Why is this heuristic useful?
– Reason #1: We maximize the possibility of using indexes
– Reason #2: Better for nested-loop join
• What about hash joins?
• Homework: Construct examples where (i) right-deep plan is best, (ii) where bushy is best
Select B,DFrom R,S,T,UWhere R.A = S.A R.A=T.A R.A = U.A
More Query Rewrite Rules
• Transform one logical plan into another– Do not use statistics
• Equivalences in relational algebra• Push-down predicates• Do projects early• Avoid cross-products if possible• Use left-deep trees• Subqueries Joins• Use of constraints, e.g., uniqueness
SQL Query with an Uncorrelated SubqueryFind the movies with stars born in 1960
MovieStar(name, address, gender, birthdate) StarsIn(title, year, starName)
SELECT titleFROM StarsInWHERE starName IN (
SELECT nameFROM MovieStarWHERE birthdate LIKE ‘%1960’
);
Parse Tree<Query>
<SFW>
SELECT <SelList> FROM <FromList> WHERE <Condition>
<Attribute> <RelName> <Tuple> IN <Query>
title StarsIn <Attribute> ( <Query> )
starName <SFW>
SELECT <SelList> FROM <FromList> WHERE <Condition>
<Attribute> <RelName> <Attribute> LIKE <Pattern>
name MovieStar birthDate ‘%1960’
Generating Relational Algebra
title
StarsIn <condition>
<tuple> IN name
<attribute> birthdate LIKE ‘%1960’
starName MovieStar
Two-argument selection
Rewrite Rule for Two-argument Selection with Conditions Involving IN
Lexp <condition>
<tuple> IN Rexp
Two-argument selection
Lexp
Rexp
δ
X
<condition>
Applying the Rewrite Rule
title
StarsIn <condition>
<tuple> IN name
<attribute> birthdate LIKE ‘%1960’
starName MovieStar
title
starName=name
StarsIn δ
birthdate LIKE ‘%1960’
MovieStar
name
Improving the Logical Query Plan
title
starName=name
StarsIn name
birthdate LIKE ‘%1960’
MovieStar
title
starName=name
StarsIn δ
birthdate LIKE ‘%1960’
MovieStar
name
SQL Query with an Correlated Subquery
MovieStar(name, address, gender, birthdate) StarsIn(title, year, starName)
SELECT titleFROM StarsInWHERE starName IN (
SELECT nameFROM MovieStarWHERE name LIKE ‘Tom%’ and year = birthdate + 30
);
parse
Query rewriting
Physical plan generation
execute
result
SQL query
parse tree
logical query planstatistics
physical query plan
Query Processing - In class order
2; 16.1
3; 16.2,16.3
1; 13, 15
4; 16.4—16.7
Related Documents
