SQL Workshop

Copyright © 2011 Accenture All Rights Reserved. Accenture, its logo, and High Performance Delivered are trademarks of Accenture.

SQL Workshop

Day 1


Day 1 AgendaSQL data selection

• Introduction– Normalization– Tables– Unique Key / Primary Key– Foreign key

• SQL– Introduction to SQL– SELECT, WHERE, ORDER BY– Selection from multiple tables– Subqueries

• Practice


Normalization

• In the field of relational database design, normalization is a systematic way of ensuring that a database structure is suitable for general-purpose querying and free of certain undesirable characteristics—insertion, update, and deletion anomalies—that could lead to a loss of data integrity

• Informally, a relational database table (the computerized representation of a relation) is often described as "normalized" if it is in the Third Normal Form (3NF).

• A standard piece of database design guidance is that the designer should create a fully normalized design; selective denormalization can subsequently be performed for performance reasons. However, some modeling disciplines, such as the dimensional modeling approach to data warehouse design, explicitly recommend non-normalized designs, i.e. designs that in large part do not adhere to 3NF


Normalization – an example

Name Surname Level Charge

Stefan Anonim Programmer 20

Natalia Kowalska Programmer 20

Janusz Inny Senior Programmer 25

Zofia Miła Programmer 20

Yurij Zdolny Consultant 30

Paweł Szybki Consultant 30

Zbigniew Molenda Analyst 20

Kazimierz Starszy Manager 50

Table before normalisation

Normalized table EMPLOYEE

Name Surname Level

Stefan Anonim Programmer

Natalia Kowalska Programmer

Janusz Inny Senior Programmer

Zofia Miła Programmer

Yurij Zdolny Consultant

Paweł Szybki Consultant

Zbigniew Molenda Analyst

Kazimierz Starszy Manager

Normalized table CHARGE

Level Charge

Programmer 20

Senior Programmer 25

Consultant 30

Analyst 20

Manager 50


Tables


Unique Key

•In relational database design, a unique key (UK) can uniquely identify each row in a table. An unique key comprises a single column or a set of columns.

•A unique key must uniquely identify all possible rows that exist in a table and not only the currently existing rows. Examples of unique keys are telephone numbers. Names, addresses are not a good candidates for an unique key.

•No two distinct rows in a table can have the same value (or combination of values) in those columns if NULL values are not used.

•Depending on its design, a table may have arbitrarily many unique keys.


Primary Key

•A table can have at most one primary key (PK), but more than one unique key.

•A primary key is a combination of columns which uniquely specify a row. It is a special case of unique keys. One difference is that for unique keys the implicit NOT NULL constraint is not automatically enforced, while for primary keys it is enforced. Thus, the values in unique key columns may or may not be NULL.

•Another difference is that primary keys must be defined using another syntax. Thus Primary Key column allows no row having NULL while Unique Key column allows null value.


?Question

What are the differences between UK and PK ?


Unique key – an example

Name Surname Level

Stefan Anonim Programmer

Natalia Kowalska Programmer

Janusz Inny Senior Programmer

Zofia Miła Programmer

Yurij Zdolny Consultant

Paweł Szybki Consultant

Zbigniew Molenda Analyst

Kazimierz Starszy Manager

Bad example


Unique key – an example 2

LvlID Level Charge

PRG Programmer 20

SPR Senior Programmer 25

CNS Consultant 30

ANS Analyst 20

MGR Manager 50

EmpID Name Surname Level

325417 Stefan Anonim PRG

437835 Natalia Kowalska PRG

338765 Janusz Inny SPR

245365 Zofia Miła PRG

264386 Yurij Zdolny CNS

342317 Paweł Szybki ANS

235378 Zbigniew Molenda ANS

97684 Kazimierz Starszy MGR


Foreign Key 1/2

•A foreign key (FK) is a referential constraint between two tables. The foreign key identifies a column or a set of columns in one (referencing) table that refers to a set of columns in another (referenced) table.

•The columns in the referencing table must be the primary key or other candidate key in the referenced table.

•The values in one row of the referencing columns must occur in a single row in the referenced table. Thus, a row in the referencing table cannot contain values that don't exist in the referenced table (except potentially NULL). This way references can be made to link information together and it is an essential part of database normalization.

•Multiple rows in the referencing table may refer to the same row in the referenced table. Most of the time, it reflects the one (master table, or referenced table) to many (child table, or referencing table) relationship.


Foreign Key 2/2

•A table may have multiple foreign keys, and each foreign key can have a different referenced table. Each foreign key is enforced independently by the database system.

•Cascading relationships between tables can be established using foreign keys.

•The referencing and referenced table may be the same table, i.e. the foreign key refers back


Foreign key – an example

EmpID Name Surname Level

325417 Stefan Anonim PRG

437835 Natalia Kowalska PRG

338765 Janusz Inny SPR

245365 Zofia Miła PRG

264386 Yurij Zdolny CNS

342317 Paweł Szybki ANS

235378 Zbigniew Molenda ANS

97684 Kazimierz Starszy MGR

LvlID Level Charge DptIDPRG Programmer 20 ATSSPR Senior Programmer 25 ATSCNS Consultant 30 CONSANS Analyst 20 CONSMGR Manager 50 CONS

ID DepartmentATS Accenture Technology SolutionsCONS Accenture Consulting


Introduction to SQL

SQL (Structured Query Language) allows the users to access data in relational database management systems, such as Oracle, Sybase, Informix, Microsoft SQL Server, Access, and others using English-like statements giving the users the ability to describe the data they wish to see.

SQL also allows users to define, view and manipulate the data in a database.


SQL Statements

• Data retrievalSELECT

• Data manipulation language (DML)INSERT, UPDATE, DELETE, MERGE

• Data definition language (DDL)CREATE, ALTER, DROP, RENAME, TRUNCATE

• Transaction controlCOMMIT, ROLLBACK, SAVEPOINT

• Data control languageGRANT, REVOKE


SQL statements characteristics

• Are not case sensitive• Can be one or more lines• Keywords can not be abbreviated or split

across lines• Clauses are usually places in separated lines • Indents are used to enhance readability

• Arithmetical expressions can be used (+ - * / )


SQL standards

• Use a single case for all SQL verbs• Begin all SQL verbs on a new line • Right or left align verbs within the initial

SQL verb • Separate all words with a single space


SELECT statement

• SELECT identifies what columns• FROM identified which tables

SELECT * | {DISTINCT column, expression [alias],… } FROM table;

SELECT * FROM employees;

SELECT job_id, first_name FROM employees;


SELECTData projection


SELECT statement -duplicated rows

• By default SELECT returns all rows• Duplicates can be eliminated by using the DISTINCT keyword

SELECT job_id FROM employees;

SELECT DISTINCT job_id FROM employees;


WHERE clause

• Restrict rows selected by using the WHERE clause

• The WHERE clause follows FROM clause

SELECT * | {distinct column, expression [alias],… } FROM table[WHERE condition(s)];

SELECT * FROM emp;

SELECT * FROM emp WHERE deptno = 30;


WHEREdata selection


Comparison conditions

=, >, <, >=, <=, <>BETWEEN … AND …

IN (set)LIKE

IS NULL


Examples

SELECT e1.first_name FROM employees e1 WHERE e1.salary > (SELECT MAX(e2.salary) FROM employees e2 WHERE e2.job = ’SALESMAN’);

SELECT first_name FROM employees WHERE substr(first_name, 1, 1) IN (’A’,’W’,’J’);


?Question

How many rows will return following query ?

SELECT * FROM employees WHERE commission_pct = NULL;


Logical conditions

• AND

• OR

• NOT

WHERE salary >= 1000 AND last_name like ‘%MAN%’

WHERE salary > 1000 OR last_name like ‘%MAN%’

WHERE last_name NOT in (‘DOE’,SMITH’)


ORDER BY clause

• Sort rows with ORDER BY clause– ASC ascending (default)

• Numeric columns with the lowest values first• Character columns in alphabetical order• Date columns with earlier dates first

– DESC descending

SELECT * | {distinct column, expression [alias],… } FROM table[ORDER BY {column, expr} [ASC|DESC]];

SELECT * FROM employees ORDER BY department_id, salary DESC;


?Question

What will be returned by following query ?

SELECT commission_pct FROM employees ORDER BY commission_pct DESC;


ORDER BY clause

• Sorting null valuesNULLS FIRST / NULLS LAST (default)

• Column numbers aliases

• Expressions in ORDER BY clause

ORDER BY commission_pct NULLS FIRST;

ORDER BY 2, 1 DESC;

ORDER BY DECODE(last_name, ‘DOE', 'A', ‘SMITH', 'B', ‘KING', 'C', 'Z');


Selecting data from multiple tables


?Question

How many rows will return following query ?

SELECT * FROM employees, departments;


Selecting data from multiple tablesCartesian Product

• emp (count = 13), dept (count = 4)• Result (13 x 4 = 52 rows )

• Cartesian product is generated when WHERE clause is omitted

SELECT first_name, department_name FROM employees, departments;


Joining tablesOracle syntax

• Write JOIN condition in WHERE clause• Prefix column name with table name when the same column

appears in both tables.

SELECT table1.column, table2.column FROM table1, table2 WHERE table1.column1 = table2.column2

SELECT first_name, department_name FROM employees, departments WHERE employees. department_id = departments. department_id;


Joining tablesWhat is in WHERE clause?

SELECT ename, dname FROM employees, departments WHERE employees.department_id=departments.department_id;

FK PK


Joining more than two tables

SELECT table1.column, table2.column, table3.column FROM table1, table2, table3 WHERE table1.column1 = table2.column2 AND table2.column3 = table3.column4


Outer joinOracle syntax

• Use outer join to see rows that do not meet the join condition• Outer join is (+) operator

SELECT table1.column, table2.column FROM table1, table2 WHERE table1.column1 (+) = table2.column2;

SELECT first_name, department_name FROM employees, departments WHERE employees.department_id(+)= departments.department_id;

SELECT table1.column, table2.column FROM table1, table2 WHERE table1.column1 = table2.column2 (+);


?Question

You want to create a report displaying employee, department names and locations.Which query should you use to create an equi-join.

1. SELECT first_name, department_id, location_id FROM employees, departments;

2. SELECT emp.first_name, dept.department_id, dept.location_id FROM employees e, departments d WHERE e. department_id =d.department_id

3. SELECT e. first_name, d.department_id, d.location_id FROM employees e, departments d WHERE manager_id = manager_id

4. SELECT e.first_name, d.department_id, d. location_id FROM employees e, departments dWHERE e.department_id =d.department_id


Subqueries

Subqueries allow the results from one query to be passed directly into another query.

A subquery is sometimes called an ‘inner query’, the parent query being called the ‘outer query’.

Subqueries can return none, one or more than one row to the parent query in which it has been embedded.

Subqueries can be correlated or non correlated.


Subqueriessyntax

SELECT column1, … FROM table1 WHERE condition expr

(SELECT column2 FROM table 2

WHERE condition);

SELECT column1, (SELECT column2 FROM table 2

WHERE condition) FROM table1 WHERE condition;


Subqueriesexamples

SELECT first_name, department_id, (SELECT department_name FROM departments WHERE departments.department_id = employees.department_id) FROM employees

SELECT first_name, department_id FROM employees WHERE department_id IN (SELECT department_id FROM departments WHERE department_name LIKE '%A%')


SELECT from SELECT

SELECT table1.column1, table2.column2, … FROM table1,

(SELECT column1, column2 FROM table 2

WHERE condition) vtable2 WHERE table1.column = vtable2.column2


IN / EXISTS condition

SELECT department_id FROM departments WHERE NOT EXISTS (SELECT null FROM employees WHERE employees.department_id = departments.department_id)

SELECT department_id FROM departments WHERE department_id NOT IN (SELECT emp.department_id FROM employees emp)


?Question

Evalute the SQL statement:

What is the result when the query is executed ?

SELECT * FROM departments WHERE department_id = (SELECT t.department_id

FROM employees t WHERE t.job_id = ‘IT_PROG’)


Practice SQL_1

Please write statement which lists the name, salary and commision for all the employees who have a manager and earn more than 1500.


Practice SQL_2

You would like to display the name of employee with his salary and name of his supervisor. If the supervisor isn’t specified for the employee, the value should be NULL. Please write this SQL statement.


SQL Performance tuning

• What shall be checked before we start optimalization

• Understand how our query is executed

• Understand if it can run faster

• Chose best option (execution plan)


Practice SQL query optimalization

Try to optimize the query:

SELECT * FROM employees_opt eo WHERE eo.last_name = 'Mavris';

00 sql for optimalization.sql



As the fisrt step of optimalization you shall check if statistics calculated for tables used in the query are in place.

TIP: use user_tables view

Useful columns of user_tables view:table_name – the name of the tablenum_rows – number of rows in the table blocks – number of blocks in the table avg_row_len – average row length of the table last_analyzed – the last date of gather statistics for table



Answer:

SELECT table_name, num_rows, blocks, avg_row_len, to_char(last_analyzed, 'YYYY-MM-DD HH24:MI:SS') FROM user_tables WHERE table_name = 'EMPLOYEES_OPT';

01 user_tables.sql



As the second step of optimalization you shall check if statistics calculated for indexes of tables used in the query are in place

TIP use user_indexes view.

Useful columns of user_indexes view:table_name – the name of the tableindex_name – the name of the indexnum_rows – number of rows in the index distinct_keys – number of distinct values in the index last_analyzed – the last date of gather statistics for index



Answer:

SELECT index_name, to_char(last_analyzed, 'YYYY-MM-DD HH24:MI:SS'), num_rows, distinct_keys FROM user_indexes WHERE table_name = 'EMPLOYEES_OPT' ORDER BY index_name;

02 user_indexes.sql



Check also date of histograms for tables used in the query.

TIP: use user_tab_histograms view.

Useful columns of user_tab_histograms view:table_name – the name of the tablecolumn_name – the name of the column on which histograms was calculatedendpoint_number – histogram bucket number endpoint_value – normalized endpoint value for this bucket



Answer:

03 user_tab_histograms.sql

SELECT table_name, column_name, endpoint_number, endpoint_value FROM user_tab_histograms WHERE table_name = 'EMPLOYEES_OPT';



If it is necessary, gather the statistics use procedure listed below:

EXEC dbms_stats.gather_table_stats(ownname => 'HR',tabname => 'EMPLOYEES_OPT',

method_opt => 'for all columns size auto',

estimate_percent => 10,cascade => true);

gather statistics for all table columns, size auto means that Oracle determines the columns to collect histograms based on

data distribution and the workload of the columns

gather statistics on the indexes for this table also

% of rows to scan

04 gather_stats.sql



Execute the query which you want to optimize:





Check total of memory blocks read and total of disk blocks read per execution for SQL statement.

TIP: use v$sql view.

Useful columns of v$sql view:hash_value – a hash value for the statement which is guaranteed to be unique

disk_reads – cumulative total of disk blocks read for this statementbuffer_gets – cumulative total of memory blocks read for this statementexecutions – how many times the statement has been executed

sql_text – the first 1000 characters of the statement

rows_processed – cumulative total of rows processed by this statement



Answer:

SELECT hash_value, disk_reads, buffer_gets, executions, round(disk_reads / executions, 2), round(buffer_gets / executions, 2), sql_text, rows_processed FROM v$sql WHERE executions != 0 AND lower(sql_text) LIKE '%select *%from employees_opt eo%where eo.last_name = ''mavris''%' ORDER BY 6 DESC;

05 v$sql.sql



Receive explain plan for the query

TIP: use v$sql_plan, v$sql view.

Useful columns of v$sql_plan view:hash_value – hash value of the parent statement in the library cacheid – a number assigned to each step in the execution planparent_id – ID of the next execution step that operates on the output of the

current stepdepth – depth (or level) of the operation in the treecost – cost of the operation as estimated by the optimizer's cost-

based approachoperation – name of the internal operation performed in this step

(for example, TABLE ACCESS)options – a variation on the operation described in the OPERATION

column (for example, FULL)object_name - name of the table or index



Answer:SELECT DISTINCT id, parent_id, depth, cost, lpad(' ', LEVEL - 1) || operation || ' ' || options operation, object_name FROM (SELECT id, parent_id, depth, cost, operation, options, object_name FROM v$sql_plan WHERE hash_value = (SELECT MAX(hash_value) FROM v$sql WHERE lower(sql_text) LIKE '%select *%from employees_opt eo%where eo.last_name = ''mavris''%')) START WITH id = 0CONNECT BY PRIOR id = parent_id;

06 execution plan.sql



HASH_VALUE

DISK_READS

BUFFER_GETS

EXECUTIONS

ROUND(DISK_READS/EXECUTIONS,2)

ROUND(BUFFER_GETS/EXECUTIONS,2

SQL_TEXT

ROWS_PROCESSED

1441343695 0

25 322 1 0

12 661

select * from employees_opt eo where eo.last_name = 'Mavris' 1

IDPARENT_ID DEPTH COST OPERATION OBJECT_NAME

1 0 1 3 498

TABLE ACCESS FULL EMPLOYEES_OPT

0 0 3 498

SELECT STATEMENT



?Share your ideas what shall be done



Create index on last_name and first_name column of employees_opt table.

Gather statistics for new index.

create index EMP_OPT_NAME_IX on employees_opt (LAST_NAME, FIRST_NAME);

EXEC dbms_stats.gather_index_stats('HR', 'EMP_OPT_NAME_IX');



Check statistics for indexes for EMPLOYEES_OPT table.

TIP:use user_indexes view.

02 user_indexes.sql








Check total of memory blocks read and total of disk blocks read per execution and execution plan.



05 v$sql.sql




HASH_VALUE

DISK_READS

BUFFER_GETS

EXECUTIONS



SQL_TEXT

ROWS_PROCESSED

3080746679 0

14 2 0

14

select * from employees_opt eo1 where eo1.last_name = 'Mavris' 2

ID

PARENT_ID DEPTH COST OPERATION OBJECT_NAME

1 0 1

4

TABLE ACCESS BY INDEX ROWID EMPLOYEES_OPT

2 1 2

3

INDEX RANGE SCAN EMP_OPT_NAME_IX

0 0

4

SELECT STATEMENT



Conclusion:

In this case index range access is better

than full access.





SELECT COUNT(DISTINCT eo.employee_id) FROM employees_opt eo WHERE eo.last_name LIKE 'King%';






11 v$sql.sql




HASH_VALUE

DISK_READS

BUFFER_GETS

EXECUTIONS



SQL_TEXT

ROWS_PROCESSED

2342550598 461

205 112 1 461

205 112

select count(distinct eo.employee_id) from employees_opt eo where eo.last_name like 'King%' 1


1 0 1 SORT GROUP BY

2 1 2

4

TABLE ACCESS BY INDEX ROWID EMPLOYEES_OPT

0 0

4

SELECT STATEMENT

3 2 3

3

INDEX RANGE SCAN EMP_OPT_NAME_IX



?Share your ideas what shall be done





SELECT /*+ FULL(eo) */ COUNT(DISTINCT eo.employee_id) FROM employees_opt eo WHERE eo.last_name LIKE 'King%';






11 v$sql.sql




HASH_VALUE

DISK_READS

BUFFER_GETS

EXECUTIONS



SQL_TEXT

ROWS_PROCESSED

276556636 377

12 666 1 377

12 666

select /*+ FULL(eo) */ count(distinct eo.employee_id) from employees_opt eo where eo.last_name like 'King%' 1


1 0 1 SORT GROUP BY

2 1 2 3 509

TABLE ACCESS FULL EMPLOYEES_OPT

0 0 3 509

SELECT STATEMENT



Conclusion:

In this case table full scanis better

than index range access.

SQL Workshop

Documents