7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 1/20
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 2/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 2 S i m
p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Opening Remarks…
Storage Areas
Dumping & Loading
Surprising Results
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 3/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 3 S i m
p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Storage Areas
Structure for Grouping Tables and Indexes Administrative Benefits
– Index Rebuilds
– Dump & Load
– Dropping Areas
– DB Analysis
Drawbacks
–More complex structure
– No single “high water” mark
– More “overhead” (free space in areas)
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 4/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 4 S i m
p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Dump & Load Choices
“Classic” Dictionary Dump & Load Bulk Loader
Binary Dump & Load
Parallelization
– Many Benefits
– Difficult to Effectively Balance
Automating the process Index Rebuild – inline vs single thread
Exotic techniques – buffer copy…, -RO, TableFilters
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 5/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 5 S i m
p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Classic Dictionary D&L
Order-lineCust order hist
cust Order-line order
1st the dump…
… and then the load.
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 6/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 6 S i m
p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Multi-threaded D&L
Order-line
cust
order hist
cust
Order-line order
1st the dump…
… and then the load.
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 7/20© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 7 S i m
p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Load Balanced D&L
Order-line
cust
order hist
cust
Order-line
order
Both the dump…
… and the load – in
parallel, multi-
threaded and load-
balanced.
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 8/20© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 8
S i m
p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Drivers
What is the window?
What are the goals?
What resources are available?
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 9/20
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 10/20© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 10
S i m
p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
The Goals
Reduce Downtime for Maintenance
– Index Rebuilds
– Dump & Load
–Application Maintenance
Reduce Disk Footprint
Improve Reporting Performance
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 11/20© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 11
S i m
p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Resources
Production Server (HP rp7410) – 8 x 750mhz
– 16GB RAM
– HPUX 11i
Failover Server (HP V2250)
–16 x 240mhz
– 8GB RAM
– HPUX 11
Old EMC Disk Array Symmetrix 8100 –
Database striped (and mirrored) across 16 disks New EMC Disk Array Symmetrix 8830
– 4 Meta volumes of 4 LUNs
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 12/20© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 12
S i m
p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
The SA Design
Separate Tables from Indexes
Isolate “large” Tables (80/20 rule)
– Any table over 1 million rows
–Any table over 1GB
Isolate Transaction Data from Overhead Data
– Easily identifiable in dictionary
256 Records per Block
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 13/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 13 S i m
p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
The SA Design -- continued
80 Storage Areas 40 Data, 40 Index
– 1 “miscellaneous” transaction data
– 1 “miscellaneous” transaction indexes
– 1 “miscellaneous” overhead data
– 1 “miscellaneous” overhead indexes
– 38 large table data areas
–38 index areas for large tables
– Plus schema, bi & ai areas
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 14/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 14 S i m p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Load Balanced Binary D&L
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 15/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 15 S i m p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Parallel Binary Dump & Load
24 Dump Threads – Effect of adding threads
47 Load Threads – Resource contention issues
Tabanalys
Expected D&L Time is Longest Combined D&L of asingle given table – Longest dump may not be longest load
– And vice versa
– Some variability is possible
– Other d&l threads can overlap – reducing contention
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 16/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 16 S i m p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
(Expected) Results
Old New
DB Size 110GB 64GB
Tabanalys 10.0h 1.0h
Warehouse Extract 90m 15m
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 17/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 17 S i m p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Application Load
Graphs show before &after application load
(unchanged)
– Logical Reads
–Commits/sec
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 18/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 18 S i m p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Surprise!
Dramatic Reduction in: – OS Reads
– Latch Contention
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 19/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 19 S i m p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Why?
Homogenous Data Blocks Take Better Advantage of “Locality of Reference”
– LOR says that most data access is “close” toprevious access
–Mixed data blocks scatter data widely Prior to d&l a block with 64 slots might have 3
records from a table; best case hit ratio for atable scan is about 85%
Post d&l it has 256; best case hit ratio becomes99.98%
Large number of SAs enables more tables toshare in benefit
7/28/2019 Progress surprises
http://slidepdf.com/reader/full/progress-surprises 20/20
© 2003 Progress Software CorporationBP0430: Some Surprising Bebefits Of Using Storage Areas - 20 S i m p l i f y y o u r b u s i n e s s
2003 Exchange
PROGRESS
Questions
http://www.greenfieldtech.com/downloads.shtml