Greenplum 6 OLTP (TPC-B) 性能提升60倍

Greenplum 6针对OLTP场景进行了优化,TPC-B性能提升60倍,单条更新操作提升70倍,单条插入和查询性能也显著增强。通过合并Postgres内核至9.4、全局死锁检测和优化全局事务,减少了锁竞争,提升了多并发操作的性能。测试显示,调整特定配置如wal_segment_size可进一步改善性能稳定性。
摘要由CSDN通过智能技术生成

了解更多Greenplum技术干货,欢迎访问Greenplum中文社区网站

Greenplum 6针对OLTP的使用场景完成了多项优化,极大的改进了多并发情况下简单查询、删除和更新操作的性能。这些改进包括:

  1. 合并Postgres内核版本至9.4,这些合并在带来一系列新功能的同时,也提升了系统的整体性能。例如,引入fastpath等锁优化,可以减少多并发情况下的锁竞争开销。
  2. 提供全局死锁检测,从而支持针对同一张HEAP表的并发更新/删除操作。
  3. 优化全局事务,从而减少开始事务和结束事务时的延迟。

基于这些优化,在我们的测试环境中,Greenplum 6的TPC-B性能相比最新的Greenplum 5提升了60倍,单条更新操作性能提升了70倍,单条插入峰值性能提升到3.6倍,单条查询性能也提升到3.5倍。尤其是对于单条查询的场景,我们在Greenplum 6中消除了大部分锁竞争,使得master的CPU使用率可以超过90%,这样通过提升master硬件性能可以进一步提升查询的TPS性能,在我们的一个单机192核的测试环境中(1个master+18个segment),单条查询TPS可以达到22万

1 测试环境与方法

1.1 测试环境

我们的测试环境基于谷歌云平台(Google Cloud Platform,简称GCP),为5个虚拟主机的集群,包含一个master主机和四个segment主机,master和segment虚拟主机的配置信息如下:

在这里插入图片描述
每一台segment主机运行一个segment节点,整个集群没有配置mirror与standby节点。除此之外还需要一台虚拟主机来运行测试工具pgbench,它的配置不需要很高,在我们的测试中采

greenplum-db-6.2.1-rhel7-x86_64.rpm Pivotal Greenplum 6.2 Release Notes This document contains pertinent release information about Pivotal Greenplum Database 6.2 releases. For previous versions of the release notes for Greenplum Database, go to Pivotal Greenplum Database Documentation. For information about Greenplum Database end of life, see Pivotal Greenplum Database end of life policy. Pivotal Greenplum 6 software is available for download from the Pivotal Greenplum page on Pivotal Network. Pivotal Greenplum 6 is based on the open source Greenplum Database project code. Important: Pivotal Support does not provide support for open source versions of Greenplum Database. Only Pivotal Greenplum Database is supported by Pivotal Support. Release 6.2.1 Release Date: 2019-12-12 Pivotal Greenplum 6.2.1 is a minor release that includes new features and resolves several issues. New Features Greenplum Database 6.2.1 includes these new features: Greenplum Database supports materialized views. Materialized views are similar to views. A materialized view enables you to save a frequently used or complex query, then access the query results in a SELECT statement as if they were a table. Materialized views persist the query results in a table-like form. Materialized view data cannot be directly updated. To refresh the materialized view data, use the REFRESH MATERIALIZED VIEW command. See Creating and Managing Materialized Views. Note: Known Issues and Limitations describes a limitation of materialized view support in Greenplum 6.2.1. The gpinitsystem utility supports the --ignore-warnings option. The option controls the value returned by gpinitsystem when warnings or an error occurs. If you specify this option, gpinitsystem returns 0 if warnings occurred during system initialization, and returns a non-zero value if a fatal error occurs. If this option is not specified, gpinitsystem returns 1 if initialization completes with warnings, and returns value of 2 or greater if a fatal error occurs. PXF version 5.10.0 is included, which introduces several new and changed features and bug fixes. See PXF Version 5.10.0 below. PXF Version 5.10.0 PXF 5.10.0 includes the following new and changed features: PXF has improved its performance when reading a large number of files from HDFS or an object store. PXF bundles newer tomcat and jackson libraries. The PXF JDBC Connector now supports pushdown of OR and NOT logical filter operators when specified in a JDBC named query or in an external table query filter condition. PXF supports writing Avro-format data to Hadoop and object stores. Refer to Reading and Writing HDFS Avro Data for more information about this feature. PXF is now certified with Hadoop 2.x and 3.1.x and Hive Server 2.x and 3.1, and bundles new and upgraded Hadoop libraries to support these versions. PXF supports Kerberos authentication to Hive Server 2.x and 3.1.x. PXF supports per-server user impersonation configuration. PXF supports concurrent access to multiple Kerberized Hadoop clusters. In previous releases of Greenplum Database, PXF supported accessing a single Hadoop cluster secured with Kerberos, and this Hadoop cluster must have been configured as the default PXF server. PXF introduces a new template file, pxf-site.xml, to specify the Kerberos and impersonation property settings for a Hadoop or JDBC server configuration. Refer to About Kerberos and User Impersonation Configuration (pxf-site.xml) for more information about this file. PXF now supports connecting to Hadoop with a configurable Hadoop user identity. PXF previously supported only proxy access to Hadoop via the gpadmin Greenplum user. PXF version 5.10.0 deprecates the following configuration properties. Note: These property settings continue to work. The PXF_USER_IMPERSONATION, PXF_PRINCIPAL, and PXF_KEYTAB settings in the pxf-env.sh file. You can use the pxf-site.xml file to configure Kerberos and impersonation settings for your new Hadoop server configurations. The pxf.impersonation.jdbc property setting in the jdbc-site.xml file. You can use the pxf.service.user.impersonation property to configure user impersonation for a new JDBC server configuration. Note: If you have previously configured a PXF JDBC server to access Kerberos-secured Hive, you must upgrade the server definition. See Upgrading PXF in Greenplum 6.x for more information. Changed Features Greenplum Database 6.2.1 includes these changed features: Greenplum Stream Server version 1.3.1 is included in the Greenplum distribution. Resolved Issues Pivotal Greenplum 6.2.1 is a minor release that resolves these issues: 29454 - gpstart During Greenplum Database start up, the gpstart utility did not report when a segment instance failed to start. The utility always displayed 0 skipped segment starts. This issue has been resolved. gpstart output was also enhanced to provide additional warnings and summary information about the number of skipped segments. For example: [WARNING]:-********
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值