Top Banner
PRODUCT DOCUMENTATION Pivotal Greenplum Database ® Version 5.28 Pivotal Greenplum Database Documentation Rev: A07 © 2021 VMware, Inc.
1972

Pivotal Greenplum Database · Greenplum Database Concepts.....232 About the Greenplum Architecture.....232 About Management and Monitoring Utilities..... 234 About Concurrency Control

Oct 24, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
  • PRODUCT DOCUMENTATION

    Pivotal™ GreenplumDatabase®Version 5.28

    Pivotal Greenplum DatabaseDocumentationRev: A07

    © 2021 VMware, Inc.

  • Copyright OpenTopic

    2

    Notice

    Copyright

    Privacy Policy | Terms of Use

    Copyright © 2020 VMware, Inc. or its affiliates. All Rights Reserved.

    Revised March 2021 (5.28.6)

    http://pivotal.io/privacy-policyhttp://pivotal.io/terms-of-use

  • Contents OpenTopic

    3

    Contents

    Chapter 1: Pivotal Greenplum 5.28.6 Release Notes............................. 17Welcome to Pivotal Greenplum 5.28.6............................................................................................. 18Changed Features.............................................................................................................................19Resolved Issues................................................................................................................................ 20Beta Features.................................................................................................................................... 21Deprecated Features.........................................................................................................................22Known Issues and Limitations.......................................................................................................... 23Differences Compared to Open Source Greenplum Database.........................................................46Supported Platforms..........................................................................................................................47

    Veritas NetBackup.................................................................................................................. 49Supported Platform Notes...................................................................................................... 50

    Pivotal Greenplum Tools and Extensions Compatibility................................................................... 51Client Tools.............................................................................................................................51Extensions...............................................................................................................................52Pivotal Greenplum Data Connectors......................................................................................53Pivotal GPText Compatibility.................................................................................................. 53Pivotal Greenplum Command Center.................................................................................... 53

    Hadoop Distribution Compatibility..................................................................................................... 54Upgrading to Greenplum Database 5.28.6....................................................................................... 55

    Upgrading from 5.x to 5.28.6................................................................................................. 56Troubleshooting a Failed Upgrade.........................................................................................59

    Migrating Data to Pivotal Greenplum 5.x..........................................................................................60Pivotal Greenplum on DCA Systems................................................................................................62

    Installing the Pivotal Greenplum 5.28.6 Software Binaries on DCA Systems........................ 62Upgrading from 5.x to 5.28.6 on DCA Systems.....................................................................62

    Update for gp_toolkit.gp_bloat_expected_pages Issue.................................................................... 65Update for gp_toolkit.gp_bloat_diag Issue........................................................................................67

    Chapter 2: Pivotal Greenplum 5.28.5 Release Notes............................. 69Welcome to Pivotal Greenplum 5.28.5............................................................................................. 70Changed Features.............................................................................................................................71Resolved Issues................................................................................................................................ 72Beta Features.................................................................................................................................... 73Deprecated Features.........................................................................................................................74Known Issues and Limitations.......................................................................................................... 75Differences Compared to Open Source Greenplum Database.........................................................98Supported Platforms..........................................................................................................................99

    Veritas NetBackup................................................................................................................ 101Supported Platform Notes.................................................................................................... 102

    Pivotal Greenplum Tools and Extensions Compatibility................................................................. 103Client Tools...........................................................................................................................103Extensions.............................................................................................................................104Pivotal Greenplum Data Connectors....................................................................................105Pivotal GPText Compatibility................................................................................................ 105Pivotal Greenplum Command Center.................................................................................. 105

    Hadoop Distribution Compatibility................................................................................................... 106Upgrading to Greenplum Database 5.28.5..................................................................................... 107

    Upgrading from 5.x to 5.28.5............................................................................................... 108

  • Contents OpenTopic

    4

    Troubleshooting a Failed Upgrade....................................................................................... 111Migrating Data to Pivotal Greenplum 5.x........................................................................................112Pivotal Greenplum on DCA Systems..............................................................................................114

    Installing the Pivotal Greenplum 5.28.5 Software Binaries on DCA Systems...................... 114Upgrading from 5.x to 5.28.5 on DCA Systems...................................................................114

    Update for gp_toolkit.gp_bloat_expected_pages Issue.................................................................. 117Update for gp_toolkit.gp_bloat_diag Issue......................................................................................119

    Chapter 3: Pivotal Greenplum 5.28.4 Release Notes........................... 121Welcome to Pivotal Greenplum 5.28.4........................................................................................... 122Resolved Issues.............................................................................................................................. 123Beta Features.................................................................................................................................. 124Deprecated Features.......................................................................................................................125Known Issues and Limitations........................................................................................................ 126Differences Compared to Open Source Greenplum Database.......................................................149Supported Platforms........................................................................................................................150

    Veritas NetBackup................................................................................................................ 152Supported Platform Notes.................................................................................................... 153

    Pivotal Greenplum Tools and Extensions Compatibility................................................................. 154Client Tools...........................................................................................................................154Extensions.............................................................................................................................155Pivotal Greenplum Data Connectors....................................................................................156Pivotal GPText Compatibility................................................................................................ 156Pivotal Greenplum Command Center.................................................................................. 156

    Hadoop Distribution Compatibility................................................................................................... 157Upgrading to Greenplum Database 5.28.4..................................................................................... 158

    Upgrading from 5.x to 5.28.4............................................................................................... 159Troubleshooting a Failed Upgrade....................................................................................... 162

    Migrating Data to Pivotal Greenplum 5.x........................................................................................163Pivotal Greenplum on DCA Systems..............................................................................................165

    Installing the Pivotal Greenplum 5.28.4 Software Binaries on DCA Systems...................... 165Upgrading from 5.x to 5.28.4 on DCA Systems...................................................................165

    Update for gp_toolkit.gp_bloat_expected_pages Issue.................................................................. 168Update for gp_toolkit.gp_bloat_diag Issue......................................................................................170

    Chapter 4: Pivotal Greenplum 5.28.3 Release Notes........................... 172Welcome to Pivotal Greenplum 5.28.3........................................................................................... 173Changed Features...........................................................................................................................174Resolved Issues.............................................................................................................................. 175Beta Features.................................................................................................................................. 176Deprecated Features.......................................................................................................................177Known Issues and Limitations........................................................................................................ 178Differences Compared to Open Source Greenplum Database.......................................................201Supported Platforms........................................................................................................................202

    Veritas NetBackup................................................................................................................ 204Supported Platform Notes.................................................................................................... 205

    Pivotal Greenplum Tools and Extensions Compatibility................................................................. 206Client Tools...........................................................................................................................206Extensions.............................................................................................................................207Pivotal Greenplum Data Connectors....................................................................................208Pivotal GPText Compatibility................................................................................................ 208Pivotal Greenplum Command Center.................................................................................. 208

    Hadoop Distribution Compatibility................................................................................................... 209Upgrading to Greenplum Database 5.28.3..................................................................................... 210

  • Contents OpenTopic

    5

    Upgrading from 5.x to 5.28.3............................................................................................... 211Troubleshooting a Failed Upgrade....................................................................................... 214

    Migrating Data to Pivotal Greenplum 5.x........................................................................................215Pivotal Greenplum on DCA Systems..............................................................................................217

    Installing the Pivotal Greenplum 5.28.3 Software Binaries on DCA Systems...................... 217Upgrading from 5.x to 5.28.3 on DCA Systems...................................................................217

    Update for gp_toolkit.gp_bloat_expected_pages Issue.................................................................. 220Update for gp_toolkit.gp_bloat_diag Issue......................................................................................222

    Chapter 5: Pivotal Greenplum 5.28.2 Release Notes........................... 224Welcome to Pivotal Greenplum 5.28.2........................................................................................... 225Changed Features...........................................................................................................................226Resolved Issues.............................................................................................................................. 227Beta Features.................................................................................................................................. 229Deprecated Features.......................................................................................................................230Known Issues and Limitations........................................................................................................ 231Differences Compared to Open Source Greenplum Database.......................................................254Supported Platforms........................................................................................................................255

    Veritas NetBackup................................................................................................................ 257Supported Platform Notes.................................................................................................... 258

    Pivotal Greenplum Tools and Extensions Compatibility................................................................. 259Client Tools...........................................................................................................................259Extensions.............................................................................................................................260Pivotal Greenplum Data Connectors....................................................................................261Pivotal GPText Compatibility................................................................................................ 261Pivotal Greenplum Command Center.................................................................................. 261

    Hadoop Distribution Compatibility................................................................................................... 262Upgrading to Greenplum Database 5.28.2..................................................................................... 263

    Upgrading from 5.x to 5.28.2............................................................................................... 264Troubleshooting a Failed Upgrade....................................................................................... 267

    Migrating Data to Pivotal Greenplum 5.x........................................................................................268Pivotal Greenplum on DCA Systems..............................................................................................270

    Installing the Pivotal Greenplum 5.28.2 Software Binaries on DCA Systems...................... 270Upgrading from 5.x to 5.28.2 on DCA Systems...................................................................270

    Update for gp_toolkit.gp_bloat_expected_pages Issue.................................................................. 273Update for gp_toolkit.gp_bloat_diag Issue......................................................................................275

    Chapter 6: Pivotal Greenplum 5.28.1 Release Notes........................... 277Welcome to Pivotal Greenplum 5.28.1........................................................................................... 278Changed Features...........................................................................................................................279Resolved Issues.............................................................................................................................. 280Beta Features.................................................................................................................................. 282Deprecated Features.......................................................................................................................283Known Issues and Limitations........................................................................................................ 284Differences Compared to Open Source Greenplum Database.......................................................307Supported Platforms........................................................................................................................308

    Veritas NetBackup................................................................................................................ 310Supported Platform Notes.................................................................................................... 311

    Pivotal Greenplum Tools and Extensions Compatibility................................................................. 312Client Tools...........................................................................................................................312Extensions.............................................................................................................................313Pivotal Greenplum Data Connectors....................................................................................314Pivotal GPText Compatibility................................................................................................ 314Pivotal Greenplum Command Center.................................................................................. 314

  • Contents OpenTopic

    6

    Hadoop Distribution Compatibility................................................................................................... 315Upgrading to Greenplum Database 5.28.1..................................................................................... 316

    Upgrading from 5.x to 5.28.1............................................................................................... 317Troubleshooting a Failed Upgrade....................................................................................... 320

    Migrating Data to Pivotal Greenplum 5.x........................................................................................321Pivotal Greenplum on DCA Systems..............................................................................................323

    Installing the Pivotal Greenplum 5.28.1 Software Binaries on DCA Systems...................... 323Upgrading from 5.x to 5.28.1 on DCA Systems...................................................................323

    Update for gp_toolkit.gp_bloat_expected_pages Issue.................................................................. 326Update for gp_toolkit.gp_bloat_diag Issue......................................................................................328

    Chapter 7: Pivotal Greenplum 5.28.0 Release Notes........................... 330Welcome to Pivotal Greenplum 5.28.0........................................................................................... 331New Features.................................................................................................................................. 332Resolved Issues.............................................................................................................................. 333Beta Features.................................................................................................................................. 334Deprecated Features.......................................................................................................................335Known Issues and Limitations........................................................................................................ 336Differences Compared to Open Source Greenplum Database.......................................................359Supported Platforms........................................................................................................................360

    Veritas NetBackup................................................................................................................ 362Supported Platform Notes.................................................................................................... 363

    Pivotal Greenplum Tools and Extensions Compatibility................................................................. 364Client Tools...........................................................................................................................364Extensions.............................................................................................................................365Pivotal Greenplum Data Connectors....................................................................................366Pivotal GPText Compatibility................................................................................................ 366Pivotal Greenplum Command Center.................................................................................. 366

    Hadoop Distribution Compatibility................................................................................................... 367Upgrading to Greenplum Database 5.28.0..................................................................................... 368

    Upgrading from 5.x to 5.28.0............................................................................................... 369Troubleshooting a Failed Upgrade....................................................................................... 372

    Migrating Data to Pivotal Greenplum 5.x........................................................................................373Pivotal Greenplum on DCA Systems..............................................................................................375

    Installing the Pivotal Greenplum 5.28.0 Software Binaries on DCA Systems...................... 375Upgrading from 5.x to 5.28.0 on DCA Systems...................................................................375

    Update for gp_toolkit.gp_bloat_expected_pages Issue.................................................................. 378Update for gp_toolkit.gp_bloat_diag Issue......................................................................................380

    Chapter 8: Greenplum Database Installation Guide.............................382Introduction to Greenplum...............................................................................................................383

    The Greenplum Master.........................................................................................................383The Segments...................................................................................................................... 384The Interconnect...................................................................................................................385ETL Hosts for Data Loading................................................................................................ 386Greenplum Performance Monitoring.................................................................................... 387

    Estimating Storage Capacity........................................................................................................... 388Calculating Usable Disk Capacity........................................................................................ 388Calculating User Data Size.................................................................................................. 388Calculating Space Requirements for Metadata and Logs....................................................389

    Configuring Your Systems and Installing Greenplum..................................................................... 390System Requirements...........................................................................................................390Disabling SELinux and Firewall Software............................................................................ 391Setting the Greenplum Recommended OS Parameters...................................................... 392

  • Contents OpenTopic

    7

    Creating the Greenplum Database Administrative User Account........................................ 398Installing the Greenplum Database Software.......................................................................399Installing and Configuring Greenplum on all Hosts..............................................................402

    Validating Your Systems................................................................................................................. 409Validating OS Settings..........................................................................................................409Validating Hardware Performance........................................................................................409Validating Disk I/O and Memory Bandwidth.........................................................................410

    Initializing a Greenplum Database System..................................................................................... 412Overview............................................................................................................................... 412Initializing Greenplum Database...........................................................................................412Setting Greenplum Environment Variables.......................................................................... 416Next Steps............................................................................................................................ 416

    Installing Optional Extensions......................................................................................................... 418Procedural Language, Machine Learning, and Geospatial Extensions................................418Python Data Science Module Package................................................................................418R Data Science Library Package......................................................................................... 420Greenplum Platform Extension Framework (PXF)............................................................... 422Oracle Compatibility Functions.............................................................................................422dblink Connectivity Functions............................................................................................... 423pgcrypto Cryptographic Functions........................................................................................423

    Configuring Timezone and Localization Settings............................................................................425Configuring the Timezone.................................................................................................... 425About Locale Support in Greenplum Database................................................................... 425Character Set Support..........................................................................................................427Setting the Character Set.....................................................................................................429Character Set Conversion Between Server and Client........................................................430

    About Implicit Text Casting in Greenplum Database......................................................................433Workaround: Manually Creating Missing Operators.............................................................434

    Installation Management Utilities.....................................................................................................437Greenplum Environment Variables................................................................................................. 438

    Required Environment Variables..........................................................................................438Optional Environment Variables........................................................................................... 438

    Chapter 9: Greenplum Database Administrator Guide........................ 440Greenplum Database Concepts...................................................................................................... 441

    About the Greenplum Architecture....................................................................................... 441About Management and Monitoring Utilities........................................................................ 443About Concurrency Control in Greenplum Database...........................................................444About Parallel Data Loading................................................................................................ 452About Redundancy and Failover in Greenplum Database...................................................453About Database Statistics in Greenplum Database............................................................. 455

    Managing a Greenplum System..................................................................................................... 463About the Greenplum Database Release Version Number................................................. 463Starting and Stopping Greenplum Database....................................................................... 463Accessing the Database.......................................................................................................466Configuring the Greenplum Database System.....................................................................474Enabling High Availability and Data Consistency Features................................................. 485Backing Up and Restoring Databases................................................................................. 503Expanding a Greenplum System..........................................................................................563Migrating Data...................................................................................................................... 578Monitoring a Greenplum System..........................................................................................585Routine System Maintenance Tasks....................................................................................605Recommended Monitoring and Maintenance Tasks............................................................ 609

    Managing Greenplum Database Access.........................................................................................617Configuring Client Authentication......................................................................................... 617

  • Contents OpenTopic

    8

    Managing Roles and Privileges............................................................................................645Defining Database Objects..............................................................................................................652

    Creating and Managing Databases......................................................................................652Creating and Managing Tablespaces...................................................................................654Creating and Managing Schemas........................................................................................656Creating and Managing Tables............................................................................................ 658Choosing the Table Storage Model..................................................................................... 661Partitioning Large Tables......................................................................................................672Creating and Using Sequences........................................................................................... 685Using Indexes in Greenplum Database............................................................................... 687Creating and Managing Views............................................................................................. 690

    Distribution and Skew..................................................................................................................... 698Local (Co-located) Joins.......................................................................................................698Data Skew............................................................................................................................ 698Processing Skew.................................................................................................................. 699

    Inserting, Updating, and Deleting Data...........................................................................................702About Concurrency Control in Greenplum Database...........................................................702Inserting Rows...................................................................................................................... 703Updating Existing Rows........................................................................................................704Deleting Rows.......................................................................................................................704Working With Transactions...................................................................................................704Vacuuming the Database..................................................................................................... 706

    Querying Data................................................................................................................................. 707About Greenplum Query Processing....................................................................................707About GPORCA....................................................................................................................710Defining Queries................................................................................................................... 723WITH Queries (Common Table Expressions)......................................................................733Using Functions and Operators............................................................................................735Working with JSON Data..................................................................................................... 745Working with XML Data........................................................................................................749Query Performance.............................................................................................................. 761Managing Spill Files Generated by Queries........................................................................ 761Query Profiling...................................................................................................................... 762

    Working with External Data.............................................................................................................767Defining External Tables...................................................................................................... 767Accessing External Data with PXF...................................................................................... 786Accessing HDFS Data with gphdfs (Deprecated)................................................................ 786Using the Greenplum Parallel File Server (gpfdist)..............................................................809

    Loading and Unloading Data.......................................................................................................... 814Loading Data Using an External Table................................................................................ 815Loading and Writing Non-HDFS Custom Data.................................................................... 815Handling Load Errors............................................................................................................818Loading Data with gpload.....................................................................................................820Accessing External Data with PXF...................................................................................... 821Transforming External Data with gpfdist and gpload........................................................... 822Loading Data with COPY..................................................................................................... 832Running COPY in Single Row Error Isolation Mode............................................................832Optimizing Data Load and Query Performance................................................................... 832Unloading Data from Greenplum Database......................................................................... 833Formatting Data Files........................................................................................................... 835Example Custom Data Access Protocol.............................................................................. 838

    Managing Performance................................................................................................................... 845Defining Database Performance.......................................................................................... 845Common Causes of Performance Issues............................................................................ 846Greenplum Database Memory Overview............................................................................. 849Managing Resources............................................................................................................853

  • Contents OpenTopic

    9

    Investigating a Performance Problem.................................................................................. 883

    Chapter 10: Greenplum Database Security Configuration Guide....... 886Securing the Database....................................................................................................................887Greenplum Database Ports and Protocols..................................................................................... 888Configuring Client Authentication.................................................................................................... 893

    Allowing Connections to Greenplum Database....................................................................893Editing the pg_hba.conf File.................................................................................................895Authentication Methods........................................................................................................ 895SSL Client Authentication.....................................................................................................898PAM-Based Authentication...................................................................................................900Radius Authentication...........................................................................................................901Limiting Concurrent Connections......................................................................................... 901Encrypting Client/Server Connections..................................................................................902

    Configuring Database Authorization................................................................................................904Access Permissions and Roles............................................................................................904Managing Object Privileges..................................................................................................904Using SSH-256 Encryption...................................................................................................905Restricting Access by Time..................................................................................................907Dropping a Time-based Restriction.................................................................................... 909

    Greenplum Command Center Security........................................................................................... 910Auditing............................................................................................................................................ 913Encrypting Data and Database Connections.................................................................................. 918

    Encrypting gpfdist Connections............................................................................................ 918Encrypting Data at Rest with pgcrypto.................................................................................919

    Enabling gphdfs Authentication with a Kerberos-secured Hadoop Cluster (Deprecated)............... 927Prerequisites......................................................................................................................... 927Configuring the Greenplum Cluster......................................................................................927Creating and Installing Keytab Files.................................................................................... 928Configuring gphdfs for Kerberos.......................................................................................... 930Testing Greenplum Database Access to HDFS...................................................................931Troubleshooting HDFS with Kerberos..................................................................................932

    Security Best Practices................................................................................................................... 934

    Chapter 11: Greenplum Database Best Practices................................ 938Best Practices Summary.................................................................................................................939System Configuration...................................................................................................................... 945Schema Design............................................................................................................................... 950

    Data Types........................................................................................................................... 950Storage Model...................................................................................................................... 950Compression......................................................................................................................... 951Distributions.......................................................................................................................... 952Partitioning............................................................................................................................ 955Indexes..................................................................................................................................957Column Sequence and Byte Alignment............................................................................... 957

    Memory and Resource Management with Resource Groups......................................................... 959Memory and Resource Management with Resource Queues........................................................ 962System Monitoring and Maintenance..............................................................................................966

    Monitoring............................................................................................................................. 966Updating Statistics with ANALYZE.......................................................................................967Managing Bloat in the Database..........................................................................................968Monitoring Greenplum Database Log Files..........................................................................972

    Loading Data................................................................................................................................... 974INSERT Statement with Column Values..............................................................................974

  • Contents OpenTopic

    10

    COPY Statement.................................................................................................................. 974External Tables.....................................................................................................................974External Tables with Gpfdist................................................................................................ 974Gpload...................................................................................................................................975Best Practices.......................................................................................................................976

    Migrating Data with gptransfer........................................................................................................ 977Security............................................................................................................................................ 983Encrypting Data and Database Connections.................................................................................. 986Tuning SQL Queries....................................................................................................................... 995

    How to Generate Explain Plans........................................................................................... 995How to Read Explain Plans................................................................................................. 995Optimizing Greenplum Queries............................................................................................ 997

    High Availability............................................................................................................................... 999Disk Storage......................................................................................................................... 999Master Mirroring....................................................................................................................999Segment Mirroring.............................................................................................................. 1000Dual Clusters...................................................................................................................... 1001Backup and Restore...........................................................................................................1001Detecting Failed Master and Segment Instances.............................................................. 1002Segment Mirroring Configurations......................................................................................1003

    Chapter 12: Greenplum Database Utility Guide..................................1008Management Utility Reference...................................................................................................... 1009

    Backend Server Programs................................................................................................. 1010analyzedb............................................................................................................................1011gpactivatestandby............................................................................................................... 1015gpaddmirrors....................................................................................................................... 1017gpbackup.............................................................................................................................1020gpcheck...............................................................................................................................1026gpcheckcat.......................................................................................................................... 1028gpcheckperf.........................................................................................................................1031gpconfig...............................................................................................................................1034gpcrondump........................................................................................................................ 1037gpdbrestore......................................................................................................................... 1051gpcopy.................................................................................................................................1058gpdeletesystem................................................................................................................... 1058gpexpand............................................................................................................................ 1059gpfdist..................................................................................................................................1063gpfilespace.......................................................................................................................... 1066gpinitstandby....................................................................................................................... 1069gpinitsystem........................................................................................................................ 1072gpload................................................................................................................................. 1079gplogfilter.............................................................................................................................1089gpmapreduce...................................................................................................................... 1092gpmfr................................................................................................................................... 1094gpmovemirrors.................................................................................................................... 1097gpperfmon_install................................................................................................................1098gppkg.................................................................................................................................. 1102gprecoverseg...................................................................................................................... 1104gpreload.............................................................................................................................. 1109gprestore............................................................................................................................. 1111gpscp...................................................................................................................................1116gpseginstall......................................................................................................................... 1118gpssh...................................................................................................................................1120gpssh-exkeys...................................................................................................................... 1123

  • Contents OpenTopic

    11

    gpstart................................................................................................................................. 1125gpstate................................................................................................................................ 1127gpstop................................................................................................................................. 1131gpsys1.................................................................................................................................1133gptransfer............................................................................................................................ 1134pgbouncer........................................................................................................................... 1146pgbouncer.ini.......................................................................................................................1148pgbouncer-admin................................................................................................................ 1160plcontainer...........................................................................................................................1169plcontainer Configuration File.............................................................................................1173

    Client Utility Reference..................................................................................................................1178clusterdb..............................................................................................................................1178createdb.............................................................................................................................. 1179createlang........................................................................................................................... 1181createuser........................................................................................................................... 1182dropdb................................................................................................................................. 1184droplang.............................................................................................................................. 1186dropuser.............................................................................................................................. 1187pg_config.............................................................................................................................1188pg_dump............................................................................................................................. 1190pg_dumpall..........................................................................................................................1195pg_restore........................................................................................................................... 1199psql......................................................................................................................................1203reindexdb............................................................................................................................ 1221vacuumdb............................................................................................................................1222

    Additional Supplied Modules......................................................................................................... 1225citext Data Type................................................................................................................. 1225dblink Functions..................................................................................................................1227hstore Functions................................................................................................................. 1230Oracle Compatibility Functions...........................................................................................1233

    Chapter 13: Greenplum Database Reference Guide.......................... 1255SQL Command Reference............................................................................................................ 1256

    SQL Syntax Summary........................................................................................................1258ABORT................................................................................................................................1286ALTER AGGREGATE.........................................................................................................1286ALTER CONVERSION....................................................................................................... 1287ALTER DATABASE............................................................................................................ 1288ALTER DOMAIN.................................................................................................................1290ALTER EXTENSION...........................................................................................................1291ALTER EXTERNAL TABLE................................................................................................1294ALTER FILESPACE............................................................................................................1296ALTER FUNCTION.............................................................................................................1296ALTER GROUP.................................................................................................................. 1299ALTER INDEX.................................................................................................................... 1300ALTER LANGUAGE........................................................................................................... 1301ALTER OPERATOR........................................................................................................... 1302ALTER OPERATOR CLASS.............................................................................................. 1302ALTER OPERATOR FAMILY.............................................................................................1303ALTER PROTOCOL........................................................................................................... 1306ALTER RESOURCE GROUP............................................................................................ 1307ALTER RESOURCE QUEUE.............................................................................................1309ALTER ROLE..................................................................................................................... 1311ALTER SCHEMA................................................................................................................1315ALTER SEQUENCE........................................................................................................... 1316

  • Contents OpenTopic

    12

    ALTER TABLE....................................................................................................................1318ALTER TABLESPACE........................................................................................................1328ALTER TYPE......................................................................................................................1329ALTER USER..................................................................................................................... 1330ALTER VIEW...................................................................................................................... 1331ANALYZE............................................................................................................................1331BEGIN................................................................................................................................. 1335CHECKPOINT.....................................................................................................................1337CLOSE................................................................................................................................ 1337CLUSTER........................................................................................................................... 1338COMMENT..........................................................................................................................1339COMMIT..............................................................................................................................1341COPY.................................................................................................................................. 1342CREATE AGGREGATE......................................................................................................1352CREATE CAST...................................................................................................................1356CREATE CONVERSION.................................................................................................... 1359CREATE DATABASE......................................................................................................... 1360CREATE DOMAIN..............................................................................................................1361CREATE EXTENSION........................................................................................................1363CREATE EXTERNAL TABLE.............................................................................................1364CREATE FUNCTION..........................................................................................................1374CREATE GROUP............................................................................................................... 1380CREATE INDEX................................................................................................................. 1381CREATE LANGUAGE........................................................................................................ 1384CREATE OPERATOR........................................................................................................ 1386CREATE OPERATOR CLASS........................................................................................... 1390CREATE OPERATOR FAMILY..........................................................................................1393CREATE PROTOCOL........................................................................................................ 1394CREATE RESOURCE GROUP......................................................................................... 1395CREATE RESOURCE QUEUE..........................................................................................1398CREATE ROLE.................................................................................................................. 1402CREATE RULE...................................................................................................................1406CREATE SCHEMA.............................................................................................................1408CREATE SEQUENCE........................................................................................................ 1409CREATE TABLE.................................................................................................................1412CREATE TABLE AS...........................................................................................................1423CREATE TABLESPACE.....................................................................................................1427CREATE TYPE...................................................................................................................1428CREATE USER.................................................................................................................. 1433CREATE VIEW................................................................................................................... 1434DEALLOCATE.................................................................................................................... 1436DECLARE........................................................................................................................... 1436DELETE.............................................................................................................................. 1439DISCARD............................................................................................................................ 1441DO.......................................................................................................................................1442DROP AGGREGATE..........................................................................................................1443DROP CAST.......................................................................................................................1444DROP CONVERSION........................................................................................................ 1445DROP DATABASE............................................................................................................. 1445DROP DOMAIN.................................................................................................................. 1446DROP EXTENSION............................................................................................................1447DROP EXTERNAL TABLE.................................................................................................1448DROP FILESPACE.............................................................................................................1449DROP FUNCTION..............................................................................................................1449DROP GROUP................................................................................................................... 1450DROP INDEX..................................................................................................................... 1451

  • Contents OpenTopic

    13

    DROP LANGUAGE.............................................................................................................1451DROP OPERATOR............................................................................................................ 1452DROP OPERATOR CLASS............................................................................................... 1453DROP OPERATOR FAMILY.............................................................................................. 1454DROP OWNED...................................................................................................................1455DROP PROTOCOL............................................................................................................ 1456DROP RESOURCE GROUP..............................................................................................1456DROP RESOURCE QUEUE.............................................................................................. 1457DROP ROLE.......................................................................................................................1458DROP RULE.......................................................................................................................1459DROP SCHEMA................................................................................................................. 1460DROP SEQUENCE............................................................................................................ 1461DROP TABLE..................................................................................................................... 1461DROP TABLESPACE.........................................................................................................1462DROP TYPE....................................................................................................................... 1463DROP USER...................................................................................................................... 1464DROP VIEW....................................................................................................................... 1464END.....................................................................................................................................1465EXECUTE........................................................................................................................... 1465EXPLAIN............................................................................................................................. 1466FETCH................................................................................................................................ 1469GRANT................................................................................................................................1471INSERT............................................................................................................................... 1476LOAD.................................................................................................................................. 1478LOCK.................................................................................................................................. 1478MOVE..................................................................................................................................1481PREPARE........................................................................................................................... 1483REASSIGN OWNED...........................................................................................................1484REINDEX............................................................................................................................ 1485RELEASE SAVEPOINT......................................................................................................1486RESET................................................................................................................................ 1487REVOKE............................................................................................................................. 1488ROLLBACK......................................................................................................................... 1490ROLLBACK TO SAVEPOINT.............................................................................................1490SAVEPOINT........................................................................................................................1492SELECT.............................................................................................................................. 1493SELECT INTO.................................................................................................................... 1507SET..................................................................................................................................... 1508SET ROLE.......................................................................................................................... 1510SET SESSION AUTHORIZATION..................................................................................... 1511SET TRANSACTION.......................................................................................................... 1513SHOW................................................................................................................................. 1515START TRANSACTION..................................................................................................... 1515TRUNCATE.........................................................................................................................1516UPDATE..............................................................................................................................1517VACUUM.............................................................................................................................1520VALUES.............................................................................................................................. 1523

    SQL 2008 Optional Feature Compliance......................................................................................1526Greenplum Environment Variables............................................................................................... 1555

    Required Environment Variables........................................................................................1555Optional Environment Variables......................................................................................... 1555

    System Catalog Reference........................................................................................................... 1557System Tables.................................................................................................................... 1557System Views..................................................................................................................... 1558System Catalogs Definitions...............................................................................................1559

    The gp_toolkit Administrative Schema..........................................................................................1648

  • Contents OpenTopic

    14

    Checking for Tables that Need Routine Maintenance........................................................1648Checking for Locks.............................................................................................................1649Checking Append-Optimized Tables.................................................................................. 1651Viewing Greenplum Database Server Log Files................................................................ 1655Checking Server Configuration Files..................................................................................1658Checking for Failed Segments........................................................................................... 1659Checking Resource Group Activity and Status.................................................................. 1660Checking Resource Queue Activity and Status................................................................. 1662Checking Query Disk Spill Space Usage...........................................................................1664Viewing Users and Groups (Roles)....................................................................................1666Checking Database Object Sizes and Disk Space............................................................ 1667Checking for Uneven Data Distribution.............................................................................. 1671

    The gpperfmon Database..............................................................................................................1673database_*.........................................................................................................................1675diskspace_*....................................................................................................................... 1676interface_stats_*................................................................................................................ 1676log_alert_*..........................................................................................................................1678queries_*............................................................................................................................. 1679segment_*..........................................................................................................................1681socket_stats_*.....................................................................................................................1682system_*............................................................................................................................. 1683dynamic_memory_info........................................................................................................ 1685memory_info...................................................................................................................... 1685

    Greenplum Database Data Types.................................................................................................1687Character Set Support...................................................................................................................1692

    Setting the Character Set...................................................................................................1694Character Set Conversion Between Server and Client...................................................... 1694

    Server Configuration Parameters..................................................................................................1697Parameter Types and Values.............................................................................................1697Setting Parameters............................................................................................................. 1697Parameter Categories.........................................................................................................1698Configuration Parameters...................................................................................................1708

    Summary of Built-in Functions...................................................................................................... 1795Greenplum Database Function Types................................................................................1795Built-in Functions and Operators........................................................................................1796JSON Functions and Operators......................................................................................... 1799Window Functions.............................................................................................................. 1802Advanced Aggregate Functions......................................................................................... 1804

    Greenplum MapReduce Specification...........................................................................................1806Greenplum MapReduce Document Format........................................................................1806Greenplum MapReduce Document Schema......................................................................1807Example Greenplum MapReduce Document..................................................................... 1814

    Greenplum PL/pgSQL Procedural Language............................................................................... 1820About Greenplum Database PL/pgSQL............................................................................. 1820PL/pgSQL Plan Caching.....................................................................................................1822PL/pgSQL Examples...........................................................................................................1822References..........................................................................................................................1826

    Greenplum PostGIS Extension..................................................................................................... 1827About PostGIS.................................................................................................................... 1827Enabling and Removing PostGIS Support......................................................................... 1828Usage..................................................................................................................................1829PostGIS Extension Support and Limitations...................................................................... 1830PostGIS Support Scripts.....................................................................................................1831

    Greenplum PL/R Language Extension..........................................................................................1834About Greenplum Database PL/R......................................................................................1834

    Greenplum PL/Python Language Extension................................................................................. 1840

  • Contents OpenTopic

    15

    About Greenplum PL/Python..............................................................................................1840Enabling and Removing PL/Python support...................................................................... 1840Developing Functions with PL/Python................................................................................1841Installing Python Modules...................................................................................................1845Examples............................................................................................................................ 1849References..........................................................................................................................1850

    PL/Container Language.................................................................................................................1851About the PL/Container Language Extension.................................................................... 1851Install PL/Container............................................................................................................ 1853Upgrade PL/Container........................................................................................................ 1858Uninstall PL/Container........................................................................................................ 1860Using PL/Container.............................................................................................................1861PL/Container Resource Management................................................................................ 1861PL/Container Functions...................................................................................................... 1864Docker References............................................................................................................. 1868

    Greenplum PL/Java Language Extension.....................................................................................1870About PL/Java.................................................................................................................... 1870About Greenplum Database PL/Java.................................................................................1871Installing PL/Java................................................................................................................1872Enabling PL/Java and Installing JAR Files........................................................................ 1873Uninstalling PL/Java........................................................................................................... 1874Writing PL/Java functions................................................................................................... 1874Using JDBC........................................................................................................................ 1880Exception Handling.............................................................................................................1880Savepoints.......................................................................................................................... 1881Logging............................................................................................................................... 1881Security............................................................................................................................... 1881Some PL/Java Issues and Solutions..................................................................................1882Example.............................................................................................................................. 1883References..........................................................................................................................1884

    Greenplum PL/Perl Language Extension......................................................................................1885About Greenplum PL/Perl...................................................................................................1885Greenplum Database PL/Perl Limitations.......................................................................... 1885Trusted/Untrusted Language.............................................................................................. 1885Enabling and Removing PL/Perl Support...........................................................................1886Developing Functions with PL/Perl.....................................................................................1886

    Greenplum MADlib Extension for Analytics.................................................................................. 1890About MADlib......................................................................................................................1890About Deep Learning..........................................................................................................1890Installing MADlib................................................................................................................. 1890Upgrading MADlib...............................................................................................................1891Uninstalling MADlib.............................................................................................................1892Examples............................................................................................................................ 1893References..........................................................................................................................1899

    Greenplum Partner Connector API............................................................................................... 1900Using the GPPC API..........................................................................................................1900Building a GPPC Shared Library with PGXS.....................................................................1912Registering a GPPC Function with Greenplum Database................................................. 1912Packaging and Deployment Considerations.......................................................................1913GPPC Text Function Exam