Oracle GoldenGate Presentation from OTN Virtual Technology Summit - 7/9/14 (PDF)

Copyright  ©  2014  Oracle  and/or  its  affiliates.  All  rights  reserved.    |  

Understanding  Oracle  GoldenGate  Architecture  

Bobby  CurCs  Senior  Technical  Consultant  Accenture  Enkitec  Group  July  9,  2014  

Oracle  ConfidenCal  –  Internal/Restricted/Highly  Restricted   1  

Performance  Tuning  to  the  OS  Level  

Speaker  Info  •  Douglasville,  Georgia  •  Senior  Technical  Consultant  •  Expert  OEM12c  (2013)/PracCcal  ODA  

(2014)  •  TwiYer:  @dbasolved  •  Blog:  hYp://dbasolved.com  •  Email:  [email protected]                                    [email protected]  

Accenture  Enkitec  Group  

Enkitec  has  joined  Accenture’s  Infrastructure  Services  as  Accenture  Enkitec  Group  

•  17,000  Infrastructure  Services  professionals  •  52,000  Oracle  professionals  

Focus  on  Oracle  Engineered  Systems  SoluCons  •  Database  MigraCons  &  TransformaCons  •  Database-­‐as-­‐a-­‐Service  •  Oracle  ApplicaCons  on  Engineered  Systems  •  Cloud-­‐based  soluCons  leveraging  Engineered  

Systems  

Agenda  

•  General  InformaCon  •  What  is  GoldenGate  •  Benefits  •  Basic  Architecture  •  Use  Cases  

•  Tuning  InformaCon  •  GoldenGate  Tuning  •  OperaCng  System  Tuning  

•  Processes  •  Memory  •  Disk  

General  InformaCon  

What  is  Oracle  GoldenGate  •  Comprehensive  soeware  for  real-­‐Cme  data  

integraCon  in  heterogeneous  environments  •  Benefits  

•  High  Availability  SoluCons  •  Real-­‐Time  Data  IntegraCon  

•  TransformaCon  of  Data  •  TransacConal  Change  Data  Capture  

•  Data  ReplicaCon  •  VerificaCon  of  Data  (Veridata)  

•   Primary  ReplicaCon  Tool  in  Oracle  Database  12c  (12.1.0.1)  •  Oracle  Upgrade  Guide,  SecCons  8.1.6  &  8.1.7  

•  MulCple  ways  to  successfully  monitor  Oracle  GoldenGate  

Benefits    

•  Real-­‐Time  Data  Capture  •  Near-­‐Zero  DownCme  migraCons  

•  Quickly  upgrade  databases  •  Open  Architecture  (Heterogeneous)  •  Disaster  Recovery  •  Data  DistribuCon  

•  Distributed  Offices  •  Query  Off-­‐loading  

•  ReporCng  Instances  

Basic  Architecture  

Capture:  commiYed  transacCons  are  captured  (and  can  be  filtered)  as  they  occur  by  reading  the  transacCon  logs.  

Trail:  stages  and  queues  data  for  rouCng.  

Pump:  distributes  data  for  rouCng  to  target(s).    

Route:  data  is  compressed,  encrypted  for  rouCng  to  target(s).  

Delivery:  applies  data  with  transacCon  integrity,  transforming  the  data  as  required.  

Source Oracle / Non-Oracle Database

Target Oracle / Non-Oracle

Database

Capture

Delivery

Trail Files

Pump

Trail Files

Pump

Delivery

Capture

Bi-directional

LAN / WAN / Internet Over TCP/IP

Trail Files

Trail Files

Modular  Architecture  

Use  Cases    Unidirec8onal  Query  Offloading  Zero-­‐DownCme  MigraCon  Data  IntegraCon  Cloud  or  On-­‐Premise    

Bi-­‐Direc8onal  AcCve-­‐AcCve  for  MulC-­‐Master/HA  Cloud  or  On-­‐Premises  

Data  Distribu8on  via  Messaging  

Cloud  Apps  Integra8on  

Big  Data  Delivery  

Real/Time  and  Batch  Delivery  Structured  Data  to  Data  Reservoir  

Tuning  InformaCon  

Performance  Tuning    

•  Per  Oracle  •  Baselining  

•  GoldenGate  Tuning  •  Lag  Times  •  AWR/ASH  Reports  

•  CPU  •  Memory  •  I/O  

•  OperaCng  System  Tuning  

Per  Oracle  

•  MulCple  Process  Groups  •  Data  Integrity  •  Number  of  Groups  

•  Dependent  on  amount  of  memory  •  Use  Network  Efficiently  

•  Benchmark  Extract/Replicat  (checkpoints)  

•   Eliminate  disk  I/O  •  Writes  data  sequenCally,  RAID  0+1  

•   Manage  VM  and  Paging  •  Cachemgr  

Baselining  

•  Should  be  first  thing  you  gather  •  What  should  be  gathered?  

•  GoldenGate  •  Lag  •  Checkpoint  Info  •  General  AWR  InformaCon  (DB)  

•  OperaCng  System  •  CPU  performance  (mpstat)  •  Memory  performance  (vmstat)  •  I/O  performance  (iostat)  

GoldenGate  Lag  Info  

•  ReporCng  Lag  •  Gives  an  idea  of  how  long  it  takes  data  to  transfer  on  network  

•  GGSCI  commands  •  lag  extract  <group  name>  •  lag  replicat  <group  name>  

GoldenGate  Checkpoint  Info  

•  Checkpoint  •  Helps  detect  network  boYlenecks  •  GGSCI  commands  

•  info  extract  <group  name>,  showch  <number>  

Database  Tuning  

•  AWR  Reports  •  Wait  Event:  Streams  Miscellaneous  Event  

CPU  InformaCon  

•  Use  mpstat  •  mpstat  –P  ALL  [count  [interval]]  

•  sar  can  be  used  as  well  

Memory  Performance  

•  Use  vmstat  •  Reports  on  virtual  memory  staCsCcs  •  vmstat  –S  M  [count  [interval]]  

Disk  I/O  Tuning  

•  Use  iostat  •  Reports  staCsCcs  transfered  between  devices  •  iostat  –m  [count  [interval]]  

Baseline  Completed  

Metric   Baseline   Running  

Lag   5  sec  (ext)/12  sec  (rep)   1  sec  (ext)/4  sec  (rep)  

CPU   4.64%  (iowait  (all))   7.73%  (iowait(all))  

Memory   2214M  (free)   2109M  (free)  

Disk  IO   29.16  tps   29.06  tps  

•  ObservaCon:  •  Serial  processing  (no  parallel)  •  Baseline  was  taken  with  higher  transacCon  count  •  Disk  I/O  was  slightly  lower  with  high  CPU  waits  •  More  memory  was  used  during  the  “running”  test  

Tune  GoldenGate  

Tune  GoldenGate  •  MulCple  Process  Groups  (Classic)  

•  Parallel  Replicat  Groups  •  3  to  1  raCo  (replicat  to  trail  (classic))  

•  Reduced  with  Integrated  Replicat  •  DB  (11.2.0.4)  and  OGG  12c  (12.1.2)  

Source  Oracle  &  Non-­‐Oracle  Database(s)  

Target  Oracle  &  Non-­‐Oracle  

Database(s)  

Capture Trail Files

Pump Delivery Trail Files

Delivery

Delivery LAN / WAN / Internet Over TCP/IP

Tune  GoldenGate  •  MulCple  Process  Groups  (Classic)  

•  Parallel  Extract  Groups  •  1  to  1  raCo  (extract  to  trail  to  replicat)  

•  Reduced  with  Integrated  processes  

Source  Oracle  &  Non-­‐Oracle  Database(s)  

Target  Oracle  &  Non-­‐Oracle  

Database(s)  

Trail Files

Pump Trail Files

Delivery

Delivery Delivery

LAN / WAN / Internet Over TCP/IP

Capture

Capture

Capture

Tune  GoldenGate  

•  Use  Network  Efficiently  •  Lag  Cmes  increase  

•  Check  target  database  performance  •  Consult  Network  Team  for  any  performance  issue  

•  Checkpoint  Extract/Replicat  •  send replicat rep, status

•  Indicates  any  delay  with  replicat  

Tune  GoldenGate  

•  Eliminate  disk  I/O  •  Try  to  eliminate  this  first  •  System  ConfiguraCons  

•  Most  shops,  disk  are  controlled  by  SAN  Admin  or  internal  devices  

•  Choose  RAID  0+1  over  RAID  5  •  GoldenGate  ConfiguraCon  

•  Increase  values  of  parameters:  •  CHECKPOINTSECS •  GROUPTRANSOPS •  EOFDELAY or EOFDELAYSECS

Tune  GoldenGate  

•  Manage  VM  and  Paging  •  Default:  use  as  much  memory  allowed  •  Default:  ./dirtmp  and  as  much  space  as  allowed  

•  Can  be  controlled  with  cachemgr parameter  •  Valid  for  Extract/Replicat  •  Current  seyngs  can  be  viewed  in  report  file  

Tune  GoldenGate  (cachemgr  con’t)  

•  EsCmate  swap  space  needed  •  Extract_Swap_Needed =(PROCESS_VM *

num_extracts) + (swap_other_processes) •  Replicat_Swap_Needed =(PROCESS_VM *

num_replicats) + (swap_other_processes)  •   Review  in  report  file  

•  send [extract/replicat] <group name>,

cachemanager    

Profiling  GoldenGate  

Profiling  the  GoldenGate  Processes  

•  perf    •  Used  to  profile  OS  processes  and  hardware  performance  

•  strace  •  Used  to  see  what  processes  are  calling  

•  backtrace  file  

Profiling  the  GoldenGate  Extract  

•  backtrace  file  output  •  InteresCng  find  with  extract  process    (6  Threads  =  1  extract)  

Profiling  the  GoldenGate  Extract  

•  Threads  IdenCfied  for  extract  •  Number  of  threads  increase/add  extracts  •  1  extract  =  6  threads  •  2  extracts  =  12  threads,  etc..  

•  Each  thread  takes/uses  memory  •  Average  Memory  22-­‐50  MB  

•  3.6-­‐8.3  MB  per  thread  

Conclusion  

•  Reviewed  Oracle  GoldenGate  Architecture  

•  Discussed  tuning  aspects  of  Oracle  GoldenGate  for  applicaCon  and  operaCng  system  

•  Quickly  looked  at  trace  of  an  extract  process  

Speaker  Info  •  Douglasville,  Georgia  •  Senior  Technical  Consultant  •  Expert  OEM12c  (2013)/PracCcal  ODA  

(2014)  •  TwiYer:  @dbasolved  •  Blog:  hYp://dbasolved.com  •  Email:  [email protected]                                    [email protected]  

Copyright  ©  2014  Oracle  and/or  its  affiliates.  All  rights  reserved.    |  

Understanding  Oracle  GoldenGate  Architecture  

Bobby  CurCs  Senior  Technical  Consultant  Accenture  Enkitec  Group  July  9,  2014  

Oracle  ConfidenCal  –  Internal/Restricted/Highly  Restricted   35  

Performance  Tuning  to  the  OS  Level