Oracle Database 12c Release 2: Revisiting PGA Memory LimitsOracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production SQL> alter system set pga_aggregate_target=2048g;

Oracle Database 12c Release 2:

Revisiting PGA Memory Limits

February 2018

• Oracle Database Consultant at Accenture Enkitec Group

• Oracle DBA / Developer

• Oracle GoldenGate / Exadata / RAC / ASM

• Technical Reviewer

• OakTable Member

Who am I?

Copyright © 2018 Accenture All rights reserved. 2

Alex Fatkulin

Hardware Then and Now


Large Memory Systems I’m Working With

• 2012 Exadata X3-8

– 2TB of RAM

• 2017 Exadata X6-8

– 6TB of RAM

Hardware Then and Now

4Copyright © 2018 Accenture All rights reserved.

Data Volume

Processing Power

Application Design

IncreasingDecreasing

How many people have systems

with more than 128G of RAM?


• Processing patterns change

• Things that didn’t matter before matter now

• Legacy RDBMS code vs 64-bit systems

• KIWI (Kill It With Iron)

“640K Problems”


PGA Memory


PGA Memory


Sort AreaHash Area Bitmap Merge Area

SQL Work Areas

Session Memory Private SQL Area

Process Memory

PGA Memory (Dedicated Server)

PGA Memory


Work Area Size

Manual Management Auto Management

hash_area_size

sort_area_size

bitmap_merge_area_size

create_bitmap_area_size

pga_aggregate_target

Query Execution Work Areas

PGA Memory Limits


PGA Memory Limits


SQL> select banner from v$version where rownum=1;

BANNER

--------------------------------------------------------------------------------

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> alter session set sort_area_size=4294967296; -- 4GB

alter session set sort_area_size=4294967296

ORA-02017: integer value required

SQL> alter session set sort_area_size=2147483647; -- 2GB-1

Session altered

• 32-bit Signed Integer Limit

• Exadata X6-8 (Linux x64, 6TB RAM)

Manual Work Area Management

PGA Memory Limits


SQL> select banner from v$version where rownum=1;

BANNER

--------------------------------------------------------------------------------

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> alter system set pga_aggregate_target=2048g;

System altered

SQL> alter system set pga_aggregate_target=4095g;

alter system set pga_aggregate_target=4095g

ORA-02097: parameter cannot be modified because specified value is invalid

ORA-00855: PGA_AGGREGATE_TARGET cannot be set because of insufficient physical memory.

oracle@exa1db01 ~> cat /proc/meminfo | grep MemFree

MemFree: 5861348908 kB

• Host has ~5.5TB free RAM (2705G max P.A.T. I was able to set)

Auto Work Area Management

• Exadata X6-8 (Linux x64, 6TB RAM)

PGA Memory Limits


• pga_aggregate_limit

– limit on the aggregate PGA memory consumed by the instance

• _smm_max_size/_smm_max_size_static

– maximum work area size per process (px/serial)

• _smm_px_max_size/_smm_px_max_size_static

– maximum work area size per query (px)

• _pga_max_size

– maximum PGA size per process (px/serial)

PGA_AGGREGATE_TARGET Derived Parameters

PGA Memory Limits


PGA_AGGREGATE_TARGET 1GB – 10GB

1048576

2097152

4194304

6291456

8388608

10485760

0

2000000

4000000

6000000

8000000

10000000

12000000

1048576 2097152 4194304 6291456 8388608 10485760

pga_aggregate_target _pga_max_size _smm_max_size _smm_px_max_size

• _pga_max_size = 20% / _smm_max_size = 10% / _smm_px_max_size = 50%

PGA Memory Limits


PGA_AGGREGATE_TARGET 10GB – 20GB

• _pga_max_size = 2GB / _smm_max_size = 1GB / _smm_px_max_size = 50%

10485760

12582912

14680064

16777216

18874368

20971520

0

5000000

10000000

15000000

20000000

25000000

10485760 12582912 14680064 16777216 18874368 20971520

pga_aggregate_target _pga_max_size _smm_max_size _smm_px_max_size

• _SMM_MAX_SIZE = 1GB (max default value)

• _PGA_MAX_SIZE = 2GB (max default value)

• If you’re bumping into per-process limit further increasing

PGA_AGGREGATE_TARGET will not help

PGA Memory Limits


PGA_AGGREGATE_TARGET >= 10GB

PGA Memory My Oracle Support Notes


PGA Memory Limits


Is There A Way To Allocate More Than 2Gb For Large Hash Joins Per Process? (Doc ID 2070261.1)

PGA Memory Limits


How To Super-Size Work Area Memory Size Used By Sessions? (Doc ID 453540.1)

PGA Memory Limits


Sort Area Does Not Go Beyond 3.7G Even If The PGA_AGGREGATE_TARGET Is High Enough

(Doc ID 2207390.1)

PGA Memory Real World Tests


• 10.1B rows

• 347.5GB HCC compressed (QUERY HIGH)



TRANS Table

select /*+ parallel(trans 8) */ count (distinct basket_id)

from trans;

• Exadata X6-8 (6TB RAM)

• 12.2.1.1.2.170714

Hardware / Software



PGA_AGGREGATE_TARGET=256G / DEFAULTS



PGA_AGGREGATE_TARGET=512G / DEFAULTS



PGA_AGGREGATE_TARGET=512G / _PGA_MAX_SIZE = 4G / _SMM_MAX_SIZE = 2G









_PX_MAX_SIZE / _SMM_MAX_SIZE Results

_PX_MAX_SIZE /

_SMM_MAX_SIZE

PGA TEMP

DEFAULT (2GB / 1GB) 13.14GB 88.53GB

4GB / 2GB 40.98GB 64.02GB

8GB / 4GB 62.17GB 34.57GB

12GB / 6GB 61.21GB 34.84GB

• _PGA_MAX_SIZE = 8G / _SMM_MAX_SIZE = 4G results in maximum PGA

• No errors or issues were observed with higher values

Performance Study


• 88.53GB TEMP

Performance Study


TEMP Consumption vs Runtime

• 34.57GB TEMP

Performance Study


TEMP I/O Flash Cache

Top 10 Foreground Events by Total Wait Time

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Total Wait Avg % DB Wait

Event Waits Time (sec) Wait time Class

------------------------------ ----------- ---------- --------- ------ --------

DB CPU 16K 79.4

resmgr:cpu quantum 62,798 261.8 4.17ms 1.3 Schedule

cell single block physical rea 51,123 22.3 435.74us .1 User I/O

buffer busy waits 2,192 20.4 9.31ms .1 Concurre

cell smart table scan 65,755 18.4 279.90us .1 User I/O

direct path write temp 57,347 11.5 200.18us .1 User I/O

local write wait 21,255 10.9 513.64us .1 User I/O

cell multiblock physical read 11,346 6.3 559.35us .0 User I/O

KSV master wait 1,080 4.9 4.49ms .0 Other

PGA memory operation 63,749 4.1 65.05us .0 Other

• Flash Cache

Performance Study


TEMP I/O Flash Cache vs Linux TMPFS

• Linux TMPFS

• Flash Cache

Performance Study


TEMP I/O Flash Cache vs In-Memory

• In-Memory

Performance Study



• In-Memory Actual Rows: 3.2B vs 10.1B

Performance Study



• In-Memory Temp Usage: 38.69GB vs 90.41GB

Performance Study


Distinct vs Approximate Count Distinct

• Distinct

• Approximate Count Distinct

Performance Study


Distinct vs Approximate Count Distinct

• Approximate Count Distinct PGA Usage

Performance Study


Approximate Count Distinct (In-Memory)

• Approximate Count Distinct

• Approximate Count Distinct (In-Memory)

Questions?


Oracle Database 12c Release 2: Revisiting PGA Memory LimitsOracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production SQL> alter system set pga_aggregate_target=2048g;

Documents