Databse & Technology 2 | Connor McDonald | Managing Optimiser Statistics - A better way.pdf

NOTE

itty bitty fonts in this

presentation

SQL> exec sample_font

Can you read this ?

1

Connor McDonald

OracleDBA

co

.uk

2

3

bio slide

4

Connor McDonald

a funny story

6

7

8

9

why ?

10

12

life was simple

14

15

(good) cost optimizer

16

17

version 7.0

18

19

optimizer_mode = CHOOSE

"we choose RULE"

20

21

SQL> analyze table SALES estimate statistics;

22

eventually....

23

it got better

24

added more functionality

25

26

SQL> analyze table SALES estimate statistics

2 for table

3 for columns size 10

4 for ....;

SQL syntax engine

27

oracle 8i

28

DBMS_STATS

29

problem....

30

31

analyze table SALES estimate statistics;SQL>

41 characters

32

SQL>

2

3

4

5

6

7

8

128 characters

begin

dbms_stats.gather_table_stats(

ownname=>'HR',

tabname=>'SALES',

cascade=>true,

estimate_percent=>20

);

end;

don't get chaining

33

s l o w e r

34

"I don't think so....."

35

times have changed ...

36

37

SQL> desc DBMS_STATS

PROCEDURE ALTER_DATABASE_TAB_MONITORING

PROCEDURE ALTER_SCHEMA_TAB_MONITORING

PROCEDURE ALTER_STATS_HISTORY_RETENTION

PROCEDURE CLEANUP_STATS_JOB_PROC

PROCEDURE CONVERT_RAW_VALUE





PROCEDURE CONVERT_RAW_VALUE_NVARCHAR

PROCEDURE CONVERT_RAW_VALUE_ROWID

PROCEDURE COPY_TABLE_STATS

FUNCTION CREATE_EXTENDED_STATS RETURNS VARCHAR2

PROCEDURE CREATE_STAT_TABLE

PROCEDURE DELETE_COLUMN_STATS

PROCEDURE DELETE_DATABASE_PREFS

126 routines !

38

39

features

40

profiles

baselines

tuning sets

adaptive cursor sharing

bind peeking

better stats needed ...

41

... for the optimizer

42

hard to convince

43

44

stop using analyze

45

ego

46

"good ol' days"

47

48

49

today....

50

optimizer is probably...

51

... smarter than you

52

53

SQL> select count(e.hiredate)

2 from DEPT d, EMP e

3 where e.deptno = d.deptno(+)

4 and e.sal > 10;

nested loop outer

sort merge outer

hash hash anti

nested loop anti

54

-------------------------------------------

| Id | Operation | Name | Rows |

-------------------------------------------

| 0 | SELECT STATEMENT | | 1 |

| 1 | SORT AGGREGATE | | 1 |

|* 2 | TABLE ACCESS FULL| EMP | 14 |

-------------------------------------------

no DEPT ?

55

SQL> select count(e.hiredate)

2 from DEPT d, EMP e

3 where e.deptno = d.deptno(+)

4 and e.sal > 10;

not null

foreign key

key preserved

Oracle's solution....

56

statistics by stealth...

57

10g

58

59

SQL> select owner, job_name, enabled

2 from dba_scheduler_jobs

3 where owner = 'SYS';

OWNER JOB_NAME ENABLED

--------------- ------------------------------ -------

SYS PURGE_LOG TRUE

SYS FGR$AUTOPURGE_JOB TRUE

SYS GATHER_STATS_JOB TRUE

SYS AUTO_SPACE_ADVISOR_JOB TRUE

FALSE

11g

60

61

SQL> select owner, job_name, enabled

2 from dba_scheduler_jobs

3 where owner = 'SYS';

OWNER JOB_NAME ENABLED

--------------- ------------------------------ -------

SYS AUTO_SPACE_ADVISOR_JOB FALSE

SYS BSLN_MAINTAIN_STATS_JOB TRUE

SYS DRA_REEVALUATE_OPEN_FAILURES TRUE

SYS FGR$AUTOPURGE_JOB FALSE

SYS GATHER_STATS_JOB FALSE

SYS HM_CREATE_OFFLINE_DICTIONARY FALSE

SYS ORA$AUTOTASK_CLEAN TRUE

SYS PURGE_LOG TRUE

SYS XMLDB_NFS_CLEANUP_JOB FALSE

stats STILL being collected

62

automatic "tasks"

63

"super stealth mode"

64

SQL> select client_name, status

2 from DBA_AUTOTASK_CLIENT;

CLIENT_NAME STATUS

------------------------------------ --------

auto optimizer stats collection ENABLED

auto space advisor ENABLED

sql tuning advisor ENABLED

most sites

65

stats every night

default options

in this session...

66

default behaviour

67

BAD

IDEA68

collecting statistics

69

BAD

IDEA70

but....

71

some default behaviour

72

GOOD

IDEA73

collecting some statistics

74

GOOD

IDEA75

76

77

Inflammatory

statements

which will

alienate the

audience

Presentation Duration

1

we're all hypocrites

78

statistical hypocrisy

79

80

"I need to change somereference data in my system"

wrong case

82

"nope...."

83

ITIL

84

Information

Technology

Infrastructure

Library

servicecall

helpdesk

problemrecordDONE !

86

phew....

corrected

88

same site....

89

"Every night, I would like to potentially change the

performance characteristics of every single SQL statement in

the database"

90

"no problem...."

91

collecting stats = risk

92

Ensor's paradox

93

1987

performance group

bstat/estat

tkprof

BMC patrol

afiedt.buf

94

"It is only safe to gather statistics ..."

"...when to do so will make no difference"

95

recommendation #1

96

97

unless things are bad...

do not change statistics

98

"surely it can't hurt?"

99

10g

100

options=>'GATHER STALE'

101

options=>'GATHER EMPTY'

102

options=>'GATHER AUTO'

103

"no plans changed"

104

"no queries ran slower"

still problems

105

problem # 1

106

107

dbms_stats

108

the goal of statistics

minimise expensive SQL....

109

added more expensive SQL !

110

111

SQL> alter session set sql_true = true;

Session altered.

SQL> begin

2 dbms_stats.gather_table_stats(

3 'DEMO',

4 'PEOPLE);

5 end;

6 /

112

SQL> select

2 count(distinct GENDER),

3 min(GENDER),

4 max(GENDER),

5 count(distinct NAME),

6 min(NAME),

7 max(NAME)

...

...

21 from PEOPLE;

hard !

113

problem #2

115

116

lingering pain

invalidation

117

library cache

119

120

select *

from PEOPLE

insert into PEOPLE

select * from ...select ...

from PEOPLE,

DEPT

where ...

delete from T

where X in

( select PID

from PEOPLE )

declare

v people.name%type;

begin

...

SQL> begin


3 'DEMO',

4 'PEOPLE);

5 end;

6 /

121

recap

122

gathering statistics

123

really hard core SQL

belted CPU with hard parsing

everything ran the same

oracle 9

124

125

SQL> desc DBMS_STATS

PROCEDURE GATHER_TABLE_STATS

Argument Name Type In/Out Default?

----------------------- -------------- ------ --------

OWNNAME VARCHAR2 IN

TABNAME VARCHAR2 IN

PARTNAME VARCHAR2 IN DEFAULT

ESTIMATE_PERCENT NUMBER IN DEFAULT

BLOCK_SAMPLE BOOLEAN IN DEFAULT

METHOD_OPT VARCHAR2 IN DEFAULT

DEGREE NUMBER IN DEFAULT

GRANULARITY VARCHAR2 IN DEFAULT

CASCADE BOOLEAN IN DEFAULT

STATTAB VARCHAR2 IN DEFAULT

STATID VARCHAR2 IN DEFAULT

STATOWN VARCHAR2 IN DEFAULT

NO_INVALIDATE BOOLEAN IN DEFAULT

STATTYPE VARCHAR2 IN DEFAULT

FORCE BOOLEAN IN DEFAULT

NO_INVALIDATE BOOLEAN IN DEFAULT

126

no_invalidate => false | true

?

?

127

oracle 10

128

no_invalidate => auto

129

SQL> select

2 x.ksppinm name,

3 y.kspftctxvl value

4 from

5 sys.x$ksppi x,

6 sys.x$ksppcv2 y

7 where

8 x.indx+1 = y.kspftctxpn and

9 x.ksppinm = '_optimizer_invalidation_period'

NAME VALUE

------------------------------ ------------

_optimizer_invalidation_period 18000

130

SQL> exec dbms_stats.gather_table_stats(

user,'PEOPLE');

select *

from PEOPLEinsert into PEOPLE

select * from ...

select ...

from PEOPLE,

DEPT

where ...

delete from T

where X in

( select PID

from PEOPLE )

declare

v people.name%type;

begin

...

131

better ?

132

hard parse storm avoided

133

5 hours before problems

134

135

unless things are bad...

DO NOT CHANGE STATISTICS

136

Inflammatory

statements

which will

alienate the

audience


1

2

137

we don't know if things are bad

138

until its too late

139

SQL's are slow

users fed up

stats out of date

140

141

142

2 hours to gather stats

143

144

collect stats

145

in readiness

146

147

pending statistics

148

"pending" - private

"published" – optimizer uses

149

SQL> begin

2 dbms_stats.set_schema_prefs(

3 ownname=>'DEMO',

4 pname=>'PUBLISH',

5 pvalue=>'FALSE');

6 end;

database,

table

150

SQL> alter session set

2 optimizer_use_pending_statistics = true;

151

SQL> begin

2 dbms_stats.publish_pending_stats(

3 ownname=>'DEMO',

4 tabname=>null);

5 end;

152

recommendation #2

153

when collecting statistics

154

PUBLISH = false

always....

155

even if you immediately publish

156

atomic publication

157

and don't wait 5 hours

158

_optimizer_invalidation_period

159

for most tables

160

dbms_stats.lock_table_stats

161

collecting system stats

162

different story

163

absolutely vital…

…to have them

164

maximize I/O throughput

165

SQL> set autotrace traceonly explain

SQL> select * from T;

---------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|

---------------------------------------------------------------

| 0 | SELECT STATEMENT | | 53459 | 4489K| 113 (1)|

| 1 | TABLE ACCESS FULL| T | 53459 | 4489K| 113 (1)|

---------------------------------------------------------------

SQL> select value

2 from v$spparameter

3 where name = 'db_file_multiblock_read_count';

VALUE

-----------

16

166

SQL> alter session set events

2 = '10046 trace name context forever, level 8';

SQL> select * from T;

PARSING IN CURSOR #22 len=15 dep=0 uid=124 oct=3 lid=124 ...

select * from T

END OF STMT

PARSE #22:c=10000,e=108,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=4

EXEC #22:c=0,e=65,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=4

WAIT #22: nam='db file scattered read' ela= 3387

file#=5 block#=13122 blocks=16 obj#=199963









167

SSTIOMAX

168

1 MB

169

db_block_size = 8192

170

db_file_multiblock_read_count = 128

171

?

172

173

the solution ?

174

system stats

175

SQL> select pname, pval1

2 from sys.aux_stats$

3 /

PNAME PVAL1

------------------------------ ----------

SREADTIM 4.065

MREADTIM 6.173

CPUSPEED 567

MBRC 12

MAXTHR 48203776

SLAVETHR -1

176

use MBRC to cost

177

use db_file_multiblock_read_count to read

178

Inflammatory

statements

which will

alienate the

audience


1

2

3

179

forget about system statistics

180

do not use them

181

182

183

absolutely vital…

…to have them

184

do not “use”* them

* = collect, change, set

185

problem #1

186

187

problem #2

188

monitoring for bad....

189

optimizing for bad

190

recommendation #3

191

gather

into STAT table

192

monitor

193

set once

194

the defaults are probably fine

195

SQL> select pname, pval1

2 from sys.aux_stats$

3 /

PNAME PVAL1

---------------------- ----------

IOSEEKTIM 10 (ms)

IOTFRSPEED 4096 (bytes/ms)

SREADTIM = IOSEEKTIM + db_block_size / IOTFRSPEED = 10

MREADTIM = IOSEEKTIM + mbrc * db_block_size / IOTFRSPEED = 26

196

10.2 and above

197

db_file_multiblock_read_count

_db_file_exec_read_count

_db_file_optimizer_read_count

198

dbms_resource_manager

to calibrate

199

so far...

200

not changing statistics

201

eventually....

202

the time will come...

203

one possible reason

204

even if stats unchanged …

205

… your plans might !

SQL> create table T as

2 select to_date('01-AUG-2011')+

3 trunc(dbms_random.value(0,7)) dte,

4 rpad('padding',20) padding

5 from dual

6 connect by level <= 100000;

Table created.

SQL> select dte, count(*)

2 from t

3 group by dte

4 order by 1;

DTE COUNT(*)

--------- ----------

01-AUG-11 14334

02-AUG-11 14222

03-AUG-11 14167

04-AUG-11 14510

05-AUG-11 14346

06-AUG-11 14349

07-AUG-11 14072

208

15,000 rows per day always

SQL> exec dbms_stats.gather_table_stats(user,'T')

PL/SQL procedure successfully completed.

SQL> create index IX on T ( dte ) ;

Index created.

SQL> select * from t where dte = '03-AUG-2011';

--------------------------------------------------

| Id | Operation | Name | Rows | Bytes |

--------------------------------------------------

| 0 | SELECT STATEMENT | | 14286 | 404K|

|* 1 | TABLE ACCESS FULL| T | 14286 | 404K|

--------------------------------------------------

211

a week later…

SQL> insert into T

2 select to_date('08-AUG-2011')+

3 trunc(dbms_random.value(0,7)) dte,

4 rpad('padding',20) padding

5 from dual

6 connect by level <= 100000;

100000 rows created.

SQL> select dte, count(*)

2 from t

3 group by dte

4 order by 1;

DTE COUNT(*)

--------- ----------

01-AUG-11 14334

...

07-AUG-11 14072

08-AUG-11 14261

09-AUG-11 14106

10-AUG-11 14410

11-AUG-11 14289

12-AUG-11 14358

13-AUG-11 14252

14-AUG-11 14324

214

statistics unchanged


----------------------------------------------------


----------------------------------------------------


| 1 | TABLE ACCESS BY INDEX ROWID| T | 2381 |

|* 2 | INDEX RANGE SCAN | IX | 2381 |

----------------------------------------------------


----------------------------------------------------


----------------------------------------------------


| 1 | TABLE ACCESS BY INDEX ROWID| T | 1 |

|* 2 | INDEX RANGE SCAN | IX | 1 |

----------------------------------------------------

217

why ?

SQL> select low_value, high_value

2 from user_tab_columns

3 where table_name = 'T'

4 and column_name = 'DTE';

LOW_VALUE HIGH_VALUE

-------------------- -----------------

786F0801010101 786F0807010101

SQL> set serverout on

SQL> declare

2 res date;

3 begin

4 dbms_stats.convert_raw_value(

5 '786F0801010101',res);

6 dbms_output.put_line(result);

7 dbms_stats.convert_raw_value(

8 '786F0806010101',res);

9 dbms_output.put_line(result);

10 end;

11 /

01-AUG-11

06-AUG-11

01-AUG-11 06-AUG-11

221

when the time comes…

222

as accurate as possible

223

as cheaply as possible

224

statistics accurately

225

extended statistics

226

227

228

cardinality is everything

228

229

same with Oracle

229

230

some real(ish) data

230

231

SQL> desc VEHICLE

Name Null? Type

-------------------------- -------- -------------

ID NUMBER

MAKE VARCHAR2(6)

MODEL VARCHAR2(6)

SQL> select count(*)

2 from VEHICLE;

COUNT(*)

------------

2,157,079

231

232

default stats not enough

232

233


2 from VEHICLE

3 where MAKE = 'HOLDEN';

COUNT(*)

----------

415387

------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost |

------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | 7 | 138|

| 1 | SORT AGGREGATE | | 1 | 7 | |

|* 2 | INDEX RANGE SCAN| MAKE_IX | 55310 | 378K| 138|

------------------------------------------------------------

233

234

histogram

234

235

SQL> begin

2 dbms_stats.gather_table_stats(user,'VEHICLE',

3 method_opt=>'for all columns size 1,'||

4 'for columns MAKE size 254,'||

5 'for columns MODEL size 254');

6 end;

7 /


235

236


2 from VEHICLE

3 where MAKE = 'HOLDEN';

COUNT(*)

----------

415387

-----------------------------------------------------------


-----------------------------------------------------------


| 1 | SORT AGGREGATE | | 1 | 7 | |

|* 2 | INDEX RANGE SCAN| MAKE_IX | 418K| 2859K| 1024|

-----------------------------------------------------------

236

237

make AND model

237

238


2 from VEHICLE

3 where MAKE = 'HOLDEN'

4 and MODEL = 'COMMODORE';

COUNT(*)

----------

214468

--------------------------------------------------------------------


---------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | 14 |

| 1 | SORT AGGREGATE | | 1 | 14 |

| 2 | BITMAP CONVERSION COUNT | | 39527 | 540K|

| 3 | BITMAP AND | | | |

| 4 | BITMAP CONVERSION FROM ROWIDS| | | |

|* 5 | INDEX RANGE SCAN | MODEL_IX | | |

| 6 | BITMAP CONVERSION FROM ROWIDS| | | |

|* 7 | INDEX RANGE SCAN | MAKE_IX | | |

---------------------------------------------------------------------

238

50% holdens are commodores

239

two things

239

240

241241

242

no correlation

10g and before

242

243

11g

243

244

SQL> select

2 DBMS_STATS.CREATE_EXTENDED_STATS(

3 user, 'VEHICLE','(MAKE,MODEL)') tag

4 from dual;

TAG

----------------------------------

SYS_STU8QPK2S$PEWHARK2CP3#1F#G

244

245

SQL> begin

2 dbms_stats.gather_table_stats(user,'VEHICLE',

3 method_opt=>

4 'for columns (make,model) size 254');

5 end;

6 /


245

246


2 from VEHICLE



COUNT(*)

----------

214468

-------------------------------------------------------------


------------------------------------------------------------


| 1 | SORT AGGREGATE | | 1 | 14 | |

|* 2 | TABLE ACCESS FULL| VEHICLE | 220K| 3018K| 1956|

-------------------------------------------------------------

246

247

composite indexes

247

248

11.2 column groups

248

249

SQL> exec DBMS_STATS.SEED_COL_USAGE(NULL,NULL,60);

249

250

implemented via virtual columns

250

251

various implications

251

252

accurate stats not always possible...

253

you can't have stats

254

on everything !

255

when stats wont do...

256

... use the real data

257

dynamic sampling

258

SQL> drop table TOUGH_DATA purge;

Table dropped.

SQL> create table TOUGH_DATA nologging as

2 select

3 rownum pk,

4 dbms_random.string('U',10) str

5 from dual

6 connect by level < 1000000

7 /

Table created.


user,'TOUGH_DATA')

259


2 from TOUGH_DATA

3 where str like '%XX'

4 /

COUNT(*)

----------

1452

hard

260


2 from TOUGH_DATA


4 /

-------------------------------------------------


-------------------------------------------------



|* 2 | TABLE ACCESS FULL| TOUGH_DATA | 50000 |

-------------------------------------------------

5% assumption

261

SQL> select /*+ dynamic_sampling(t 2) */ count(*)

2 from TOUGH_DATA t


4 /

-------------------------------------------------


-------------------------------------------------



|* 2 | TABLE ACCESS FULL| TOUGH_DATA | 1252 |

-------------------------------------------------

262

simple queries as well

263

SQL> create table EASY_DATA nologging as

2 select

3 rownum pk,

4 chr(65+trunc(rownum/40000)) str

5 from dual

6 connect by level < 1000000

7 /

Table created.

SQL> exec dbms_stats.gather_table_stats(user,'EASY_DATA')


A,A,A,A,A,B,B,B,B,.....1,2,3,4,5,6......

264


2 from EASY_DATA

3 where str = 'F'

4 and pk > 900000;

COUNT(*)

----------

0

265



2 from EASY_DATA

3 where str = 'F'

4 and pk > 900000;

------------------------------------------------


------------------------------------------------



|* 2 | TABLE ACCESS FULL| EASY_DATA | 4000 |

------------------------------------------------

266


SQL> select /*+ dynamic_sampling(t 2) */ count(*)

2 from EASY_DATA t

3 where str = 'F'

4 and pk > 900000;

------------------------------------------------


------------------------------------------------




------------------------------------------------

267

11.2

268

parallel queries (maybe) sampled

269


2 from EASY_DATA t

3 where str = 'F'

4 and pk > 900000;

----------------------------------------------------


----------------------------------------------------



| 2 | PX COORDINATOR | | |

| 3 | PX SEND QC (RANDOM) | :TQ10000 | 1 |


| 5 | PX BLOCK ITERATOR | | 4 |


----------------------------------------------------

Note

-----

- dynamic sampling used for this statement (level=4)

270

low frequency

271

high cost

272

statistics cheaply

273

SQL> desc DBA_TABLES

Name Null? Type

----------------------------- -------- -------------

OWNER NOT NULL VARCHAR2(30)

TABLE_NAME NOT NULL VARCHAR2(30)

COLUMN_NAME NOT NULL VARCHAR2(30)

...

NUM_ROWS NUMBER

274

SQL> desc DBA_TAB_COLS

Name Null? Type

----------------------------- -------- -------------

OWNER NOT NULL VARCHAR2(30)

TABLE_NAME NOT NULL VARCHAR2(30)

COLUMN_NAME NOT NULL VARCHAR2(30)

...

NUM_DISTINCT NUMBER

LOW_VALUE RAW(32)

HIGH_VALUE RAW(32)

...

275

SQL> desc PEOPLE

Name Null? Type

----------------------------- -------- -------------

PID NUMBER

GENDER CHAR(1)

NAME VARCHAR2(47)

AGE NUMBER

DEATH_RATE NUMBER

276

SQL> alter session set sql_true = true;

Session altered.

SQL> begin


3 'DEMO',

4 'PEOPLE);

5 end;

6 /

277

select

/*+ no_parallel(t)

no_parallel_index(t)

dbms_stats

cursor_sharing_exact

use_weak_name_resl

dynamic_sampling(0)

no_monitoring

no_substrb_pad */

count(*),

...

count("GENDER"),

count(distinct "GENDER"),

substrb(dump(min("GENDER"),16,0,32),1,120),

substrb(dump(max("GENDER"),16,0,32),1,120),

...

from "MCDONAC"."PEOPLE" t

count("GENDER")

count(distinct "GENDER")

min("GENDER")

max("GENDER")

278

count("GENDER")

count(distinct "GENDER")

min("GENDER")

max("GENDER")

one pass

one pass

one pass

hard

279

SQL> select count(*) from PEOPLE;

COUNT(*)

------------

500000000

Elapsed: 00:04:43.73

280

SQL> begin


3 user,

4 'PEOPLE',

5 estimate_percent=>25);

6 end;

7 /

ERROR at line 1:

ORA-01652: unable to extend temp

segment by 128 in tablespace TEMP

Elapsed: 00:16:37.12

281

11g

282

one pass NDV

283

HASH

12 6 3 12 3 7 11 12 33 6 11 12 6 45 15 7 15 17 45 6 17

12 6 3 7 11 33 45 15 17

284

NDV = 9

12 6 3 7 11 33 45 15 17

285

what about large NDV ?

12

6

3

7

11

33

45

15

17

21

92

71

34

56

615

2

41

64

91

73

286

"Magic" HASH

287

NDV =

remaining hashes x

2^number of splits

288

demo

289

SQL> create table ONE_PASS nologging

2 as select substr(text,1,1) single_char

3 from DBA_SOURCE;

Table created.

SQL> select count(distinct single_char) ndv

2 from one_pass;

NDV

----------

83

290

0 127

16 buckets...

291

64 127

292

96 127

293

SQL> set serverout on

SQL> declare

2 type t_bucket is table of varchar2(1);

3 l_synopsis t_bucket;

4 l_splits number := 0;

5 l_hash int;

6 l_min_val int := 0;

7 l_synopsis_size int := 16;

8 begin

9 for i in ( select single_char from one_pass ) loop

10 l_hash := ascii(i.single_char);

11

12 if l_synopsis.count = l_synopsis_size then

13 l_min_val :=

14 case

15 when l_min_val = 0 then 64




19 end;

20 l_splits := l_splits + 1;

294

22

23 for j in 1 .. l_min_val loop

24 if l_synopsis.exists(j) then

25 l_synopsis.delete(j);

26 end if;

27 end loop;

28 end if;

29

30 if l_hash > l_min_val then

31 l_synopsis(l_hash) := 'Y';

32 end if;

33 end loop;

34 dbms_output.put_line(l_synopsis.count *

35 power(2,l_splits));

36 end;

37 /

Splitting, keeping entries above 64



88

295

the reality

16384 bucket limit

18,446,744,073,709,551,616 hash range

296

SQL> begin


3 user,

4 'PEOPLE',

5 estimate_percent=>25);

6 end;

7 /


Elapsed: 00:05:39.82

297

recommendation #4

298

estimate_percent

use DEFAULT size (AUTO)

299

only for columns

without histograms

300

Inflammatory

statements

which will

alienate the

audience


1

2

3

4

301

histograms...

302

...SUCK

303

not really but...

304

9i

306


SQL> select * from MY_TABLE_100K_ROWS

2 where r = :b1

3 /

------------------------------------------


------------------------------------------


| 1 | INDEX UNIQUE SCAN| PK | 1 |

------------------------------------------

~ num rows / num distinct

307


SQL> select * from MY_TABLE_100K_ROWS

2 where r < :b1

3 /

-----------------------------------------


-----------------------------------------


| 1 | INDEX RANGE SCAN| PK | 5000 |

-----------------------------------------

5%

308

309

cool ?

310

10g

311

method_opt

for all columns size auto

312

sys.col_usage$

313

SQL> desc SYS.COL_USAGE$

Name Null? Type

----------------------- -------- ---------

OBJ# NUMBER

INTCOL# NUMBER

EQUALITY_PREDS NUMBER

EQUIJOIN_PREDS NUMBER

NONEQUIJOIN_PREDS NUMBER

RANGE_PREDS NUMBER

LIKE_PREDS NUMBER

NULL_PREDS NUMBER

TIMESTAMP DATE

314

SQL> create table T (

2 skew varchar2(10),

3 even number);

Table created.

SQL> insert into T

2 select

3 case

4 when rownum > 99995 then 'SPECIAL'

5 else dbms_random.string('U',8)

6 end,

7 mod(rownum,200)

8 from dual

9 connect by level <= 100000

10 /

100000 rows created.

5 special

values

even

distribution

315


user,'T', estimate_percent=>null);


SQL> select COLUMN_NAME,NUM_DISTINCT,DENSITY,

2 ( select count(*)

3 from user_tab_histograms


5 and column_name = c.column_name ) hist_cnt

6 from user_tab_cols c


8 order by table_name,COLUMN_ID

9 /

COLUMN_NAME NUM_DISTINCT DENSITY HIST_CNT

------------------- ------------ ---------- ----------

SKEW 99996 .00001 2

EVEN 200 .005 2

316

SQL> select * from T where skew = 'SPECIAL';

SKEW VAL

---------- ----------

SPECIAL 196

SPECIAL 197

SPECIAL 198

SPECIAL 199

SPECIAL 0

5 rows selected.

SQL> select * from T where even = 5;

TAG VAL

---------- ----------

IBRXGVIE 5

[snip]

500 rows selected.

317


user,'T', estimate_percent=>null);


SQL> select COLUMN_NAME,NUM_DISTINCT,DENSITY,

2 ( select count(*)

3 from user_tab_histograms


5 and column_name = c.column_name ) hist_cnt

6 from user_tab_cols c


8 order by table_name,COLUMN_ID

9 /

COLUMN_NAME NUM_DISTINCT DENSITY HIST_CNT

------------------- ------------ ---------- ----------

SKEW 99996 .00001 2

EVEN 200 .000005 200

318

recommendation #5

319

set_schema_prefs

for all columns size 1

320

explicit control for each table

321

set_table_prefs

for column CCC size nnn

322

one more useful tool

323

324

SQL> desc CONFERENCE_ATTENDEES

Name Null? Type

----------------------------- -------- --------------

PID NUMBER

GENDER CHAR(1)

NAME VARCHAR2(40)

AGE NUMBER

DEATH_RATE NUMBER

325

326

SQL> select

2 count(distinct GENDER) NDV

3 from

4 CONFERENCE_ATTENDEES

NDV

----------------------

???

327

SQL> select

2 ntile(GENDER) over ( ... )

3 from

4 CONFERENCE_ATTENDEES

328

set_table_stats

num_rows

avg_row_len

329

set_column_stats

high_value

num_distinct

330

when all else fails ...

331

... lie and cheat

332

333

no

334

analyze ... validate structure

335

SQL> desc INDEX_STATS

Name Null? Type

----------------------------- -------- --------------

HEIGHT NUMBER

BLOCKS NUMBER

NAME VARCHAR2(30)

PARTITION_NAME VARCHAR2(30)

...

OPT_CMPR_COUNT NUMBER

OPT_CMPR_PCTSAVE NUMBER

336

wrap up

337

don't collect stats .... unless

don't collect system stats ... unless

don't collect histograms ... unless

default estimate size (NDV)

lie and cheat

Connor McDonald

OracleDBA

co

.uk

338

339

ORA-00041

www.oracledba.co.uk

“active time limit exceeded - session terminated”

340340

11.2

341

actual versus estimate

341

342

SQL> select /*+ GATHER_PLAN_STATISTICS */ count(*)

2 from VEHICLE



COUNT(*)

----------

214468

SQL> SELECT *

2 FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(

3 NULL, NULL, 'ALLSTATS LAST'));

----------------------------------------------------------------

| Id | Operation | Name | Starts | E-Rows | A-Rows |

----------------------------------------------------------------

| 1 | SORT AGGREGATE | | 1 | 1 | 1 |

|* 2 | TABLE ACCESS FULL| VEHICLE | 1 | 220K| 214K|

-----------------------------------------------------------------

342

343343

employ someone....

344344

check every query...

345

SQL> select /*+ GATHER_PLAN_STATISTICS */ count(*)

2 from VEHICLE



COUNT(*)

----------

214468

SQL> SELECT *

2 FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(

3 NULL, NULL, 'ALLSTATS LAST'));

-----------------------------------------------------------------

| Id | Operation | Name | Starts | E-Rows | A-Rows |

-----------------------------------------------------------------

| 1 | SORT AGGREGATE | | 1 | 1 | 1 |

|* 2 | TABLE ACCESS FULL| VEHICLE | 1 | 220K| 214K|

-----------------------------------------------------------------

345

"ok"

346346

347347

but just maybe ....

348348

... someone already is

very

349

very

cool

SQL> create table EMP as

2 select rownum empno,

3 mod(rownum,10) jobid,

4 mod(rownum,10)*1000 salary,

5 mod(rownum,50)+1 deptno

6 from dual

7 connect by rownum < 100000;

SQL> create table DEPT as

2 select rownum deptno,

3 'dept'||rownum dname

4 from dual

5 connect by rownum <= 100;

350

100,000 rows

100 rows

SQL> exec dbms_stats.gather_table_stats(user,'EMP');

SQL> exec dbms_stats.gather_table_stats(user,'DEPT');

SQL> create index EMP_IX on EMP ( deptno );

SQL> create index DEPT_IX on DEPT ( deptno );

351

SQL> select e.empno, d.dname

2 from emp e, dept d

3 where d.deptno = e.deptno

4 and e.jobid = 1

5 and e.salary > 5000;

no rows selected

----------------------------------------------------------------


----------------------------------------------------------------

| 0 | SELECT STATEMENT | | | |

| 1 | MERGE JOIN | | 4444 | 104K |

| 2 | TABLE ACCESS BY INDEX ROWID| DEPT | 100 | 1000 |

| 3 | INDEX FULL SCAN | DEPT_IX | 100 | |

|* 4 | SORT JOIN | | 4444 | 62216 |

|* 5 | TABLE ACCESS FULL | EMP | 4444 | 62216 |

----------------------------------------------------------------

352

4 and e.jobid = 1


hard to

optimize

re-run the query

353

354

no anythingchanges to

SQL> select e.empno, d.dname

2 from emp e, dept d

3 where d.deptno = e.deptno

4 and e.jobid = 1


no rows selected

---------------------------------------------------


---------------------------------------------------

| 0 | SELECT STATEMENT | | | |

|* 1 | HASH JOIN | | 89 | 2136 |

| 2 | TABLE ACCESS FULL| DEPT | 1 | 10 |

|* 3 | TABLE ACCESS FULL| EMP | 4444 | 62216 |

---------------------------------------------------

355

11.2

356

the optimizer knows what "hard" is

357

cardinality feedback loop

358

its not continuous

359

several restrictions

360

but still very cool

Databse & Technology 2 | Connor McDonald | Managing Optimiser Statistics - A better way.pdf

Documents