SAS-BLKSIZE-And-BUFSIZE-Options-Ted-Keller-2010-Q2.ppt
Post on 03-Dec-2015
213 Views
Preview:
Transcript
An Exercise in Improving SAS Performance on Mainframe
Processors
SAS BLKSIZE and BUFSIZE Options
Forward
• At the last KCASUG meeting, George Hurley presented “Customizing Your SAS Initialization II.”
• In this presentation, George suggested that it is possible to save CPU in SAS jobs by tuning the BUFSIZE parameter.
• With our current interest in saving CPU and stretching the life of mainframe equipment, I decided to investigate what kind of savings were possible in our environment.
Background
• In the 1990s and earlier, disk storage for mainframes consisted of a stack of 14” platters arranged in what was called a disk drive.
• There was a separate read/write head for each surface– All read/write heads were aligned at the same relative position
and moved together
• Disk drives were organized into tracks and cylinders.– A track represented the data that could be accessed from one
surface with one revolution of the disk– A cylinder was all the tracks that could be accessed from the
same relative location of the read/writes heads.
• Data was stored with gaps between records in a CKD format
Background
• 3390s were the final generation of IBM classical disk drives– Each track could hold up to 56,664 bytes– The largest size record was 32,767 bytes– While records could be larger, records were rarely
larger than 27,998 bytes• This is the largest record size that allowed 2 records per
track• Record sizes approaching 27,998 bytes provided optimal use
of disk storage on 3390 devices– This is commonly referred to as a “half track” record size
Background
• When modern storage controllers started replacing classical mainframe storage, the storage controllers emulated classical storage devices, particularly the 3390
• While data is actually stored in stripes with multiple layers of virtualization, access to the data still follows the protocol of classical mainframe storage
Mainframe SAS Files
• Two factors have the most influence on the performance of I/Os for SAS datasets– BLKSIZE – the size of the block (physical
record)– BUFSIZE – the size of the storage buffer
• Should be a multiple of BLKSIZE
SAS BLKSIZE
• BLKSIZE– Larger block sizes are more efficient
• With smaller block sizes, there is additional overhead in SAS to manage each block
– SAS files can have any BLKSIZE up to 32,760– The optimal BLKSIZE for SAS files is 27,648
• Largest “half-track” size for SAS files• Provides optimal balance of performance and disk
storage utilization
SAS BUFSIZE
• BUFSIZE– When SAS schedules an I/O for a SAS dataset, it
builds the I/O command to transfer as much data as will fit in the buffer as a single I/O command
• This saves the operating system overhead related to managing multiple I/Os
• SAS uses its own channel programming (EXCPs) for SAS files, not normal operating system access methods
– For example, with a BLKSIZE of 27,648 and a BUFSIZE of 110,592, SAS would build I/O commands to transfer 4 blocks with each I/O command
SAS BUFSIZE
• BUFSIZE– Buffer sizes of between 110,592 and 221,184
tend to be fairly efficient– MEMSIZE may need to be increased when
BUFSIZE is increased
Controlled Tests
• Performed some controlled tests
• One controlled test– Wrote 250,000 records to a SAS file (each
about 1.6K of data)– In separate step, read the records (in a
_NULL_ data step, SET the input to the file just created)
– Varied BLKSIZE and BUFSIZE in each run
Controlled Tests
• Tests showed that a BLKSIZE of 27,648 performed better than a BLKSIZE of 6,144 for similar buffer sizes– A BLKSIZE of 6,144 was the old standard in
our shop
• Tests also suggested limited improvements in CPU and run times with buffer sizes above 110,592 to 221,184– In fact, sometimes performance appeared to
deteriorate with larger buffer sizes
Test - Write 250,000 RecordsCPU per Run
0.00
0.50
1.00
1.50
2.00
2.50
6,144 /6,144
6,144 /12,288
6,144 /24,576
6,144 /49,152
6,144 /110,592
6,144 /221,184
27,648 /27,648
27,648 /55,296
27,648 /110,592
27,648 /221,184
27,648 /331,776
27,648 /442,368
BLKSIZE / BUFSIZE
CP
U (
seco
nd
s)
Test - Read 250,000 RecordsCPU per Run
0.00
0.50
1.00
1.50
2.00
2.50
6,144 /6,144
6,144 /12,288
6,144 /24,576
6,144 /49,152
6,144 /110,592
6,144 /221,184
27,648 /27,648
27,648 /55,296
27,648 /110,592
27,648 /221,184
27,648 /331,776
27,648 /442,368
BLKSIZE / BUFSIZE
CP
U (
seco
nd
s)
Test - Write 250,000 RecordsRun Time
0
5
10
15
20
25
30
35
40
45
50
6,144 /6,144
6,144 /12,288
6,144 /24,576
6,144 /49,152
6,144 /110,592
6,144 /221,184
27,648 /27,648
27,648 /55,296
27,648 /110,592
27,648 /221,184
27,648 /331,776
27,648 /442,368
BLKSIZE / BUFSIZE
Ru
n T
ime
(sec
on
ds)
Test - Read 250,000 RecordsRun Time
0
5
10
15
20
25
30
35
40
45
50
6,144 /6,144
6,144 /12,288
6,144 /24,576
6,144 /49,152
6,144 /110,592
6,144 /221,184
27,648 /27,648
27,648 /55,296
27,648 /110,592
27,648 /221,184
27,648 /331,776
27,648 /442,368
BLKSIZE / BUFSIZE
Ru
n T
ime
(sec
on
ds)
Production Pilots
• Identified the jobs that were using the largest total amount of CPU
• Ran pilots on 2 of the top 5 jobs to explore potential benefits with real jobs– Changed BKLSIZE from 6,144 to 27,648– Increased BUFSIZE to 221,184
• Ran several parallel runs of the MXG job with various BLKSIZE and BUFSIZE (MXG is a common SAS-based mainframe tool to capture and manage mainframe performance data)– Experimented with various block sizes – Have not placed changes to MXG in production yet
• Rewrote one job in another language
Pilot Results
• Pilot results were quite favorable– Job using largest amount of CPU (runs many times
each day) – see charts for Job 1• 6% reduction in CPU• 25% improvement in run time
– Job using 5th largest amount of CPU (runs many times each day) – see charts for Job 2
• 9% reduction in CPU• 43% improvement in run time
– MXG (2nd largest user of CPU – runs once daily)• 5% reduction in CPU• ~ 10% improvement in run time
Changes in CPU TimeJob 1
0
20
40
60
80
100
120
Ave
rag
e C
PU
Tim
e
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
CPU Time (Seconds) Total I/Os
6% reduction in CPU
Changes in Run TimeJob 1
0
50
100
150
200
250
300
350
400
450
Ave
rag
e R
un
Tim
e
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
Run Time (Seconds) Total I/Os
25% improvement in average run time
Changes in CPU TimeJob 2
0
5
10
15
20
25
30
3/28
/201
0
3/29
/201
0
3/30
/201
0
3/31
/201
0
4/1/
2010
4/2/
2010
4/3/
2010
4/4/
2010
4/5/
2010
4/6/
2010
4/7/
2010
4/8/
2010
4/9/
2010
4/10
/201
0
4/11
/201
0
4/12
/201
0
4/13
/201
0
4/14
/201
0
4/15
/201
0
4/16
/201
0
4/17
/201
0
4/18
/201
0
4/19
/201
0
4/20
/201
0
4/21
/201
0
4/22
/201
0
4/23
/201
0
4/24
/201
0
4/25
/201
0
4/26
/201
0
4/27
/201
0
Ave
rag
e C
PU
Tim
e
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
CPU Time (Seconds) Total I/Os
9% reduction in CPU
Changes in Run TimeJob 2
0
20
40
60
80
100
120
3/28
/201
0
3/29
/201
0
3/30
/201
0
3/31
/201
0
4/1/
2010
4/2/
2010
4/3/
2010
4/4/
2010
4/5/
2010
4/6/
2010
4/7/
2010
4/8/
2010
4/9/
2010
4/10
/201
0
4/11
/201
0
4/12
/201
0
4/13
/201
0
4/14
/201
0
4/15
/201
0
4/16
/201
0
4/17
/201
0
4/18
/201
0
4/19
/201
0
4/20
/201
0
4/21
/201
0
4/22
/201
0
4/23
/201
0
4/24
/201
0
4/25
/201
0
4/26
/201
0
4/27
/201
0
Ave
rag
e R
un
Tim
e
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
Run Time (Seconds) Total I/Os
43% improvement in average run time
Production Implementation
• Changed BLKSIZE to 27,648– Changed both CONFIG member and SAS PROC
• Changed BUFSIZE to 221,184– Changed CONFIG member
• Made changes to ensure jobs would not fail with memory issues– MEMSIZE parameter removed from CONFIG
• Defaults to 0 (no limitation on memory)
– Changed REGION to 0M in SAS PROC– Made mass change to production SAS jobs to remove
REGION parameter overrides
Implementation Results
• Measured results based on production jobs that ran daily– Compared results on job / weekday basis
• For jobs that ran during the day:– 10% average reduction in CPU
• Varied from no gain to 15-20% improvement
– 30% average improvement in run times• Varied considerably from job to job
• For jobs that ran at night– 3% reduction in CPU– 10% improvement in run times
Issues and Opportunities
• Many production jobs reuse same SAS files without ever deleting and recreating them– BLKSIZE remains smaller size
• Many production jobs use their own customized SAS PROCs or CONFIG members– Cannot easily take advantage of changes
• Will need to look for opportunities to tune these jobs later
Thinking Outside the Box
• One very large SAS job runs daily– Job would read 10-12 million rows– Sort data on 4 keys– Summarize 32 columns using PROC UNIVARIATE
• Rewrote job in another language– Took advantage of partial natural order of data and
used hashing algorithm to organize data• Initial level summary done in summary program
– Summarized data was then input to SAS
Changes in Rewritten Job
• Reduced CPU 95%
• Improved run time 97%
It is worth noting that I could find only two large SAS jobs that could take advantage of this technique. All other SAS jobs that I looked at were far too complex to consider
doing this.
Changes in CPU TimeRewritten Job
0
200
400
600
800
1000
1200
1400
1600
1800
CP
U T
ime
(sec
on
ds)
0
200,000
400,000
600,000
800,000
1,000,000
1,200,000
CPU Time (Seconds) Total I/Os
95% reduction in CPU
top related