Greenplum gpcopy 安装使用介绍

最新推荐文章于 2024-04-20 10:33:09 发布
最新推荐文章于 2024-04-20 10:33:09 发布
阅读量2k 收藏 4
点赞数
分类专栏： # Greenplum
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/dazuiba008/article/details/107068259
版权

首先这款工具是两个greenplum集群之间迁移数据的时候使用的，可以迁移完整的数据库，或者是表，可以跨版本迁移，兼容Greenplum Database 4.3.26 以上版本, 以及5.9 以上, 6.x版本。

效率上面要比之前的gptransfer要高，因为是segment之间可以实现并行对拷，但是我这里不会做比对。目前最新版本为2.2版本。

下载地址：https://network.pivotal.io/products/gpdb-data-copy

安装步骤：

1.解压安装包
tar xzvf gpcopy-2.2.0.tar.gz 

2.在master节点，复制gpcopy和gpcopy_helper到相应的bin目录 
$ cp gpcopy $GPHOME/bin  
$cp gpcopy_helper $GPHOME/bin

3.赋予权限
$ chmod 755 $GPHOME/bin/gpcopy
$ chmod 755 $GPHOME/bin/gpcopy_helper

4.在segment节点的操作如下，只拷贝gpcopy_helper到相应目录下并赋予权限即可。
$ gpscp -h sdw1 -h sdw2 gpcopy_helper =:/usr/local/greenplum-db-6.0.0/bin
$ gpssh -h sdw1 -h sdw2 -e 'chmod 755 /usr/local/greenplum-db-6.0.0/bin/gpcopy_helper'

迁移数据实例，这里我的版本为从4.3.30.0迁移数据至6.0.1：

这里只迁移一张表测试，在源端发起gpcopy命令迁移一张名为book_info的表,这里打开debug选项，可以看到具体的执行过程，也方便排错。
gpcopy --source-host 127.0.0.1 --source-user gpadmin  --source-port 1921 --dest-host 192.168.xxx.xxx  --dest-user gpadmin --dest-port 5432 --include-table warehouse.read30.book_info --drop  --dest-table warehouse.read30.book_info --debug

日志如下：

20200701:11:36:20 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-"--drop" will work as "--truncate" on partition tables
20200701:11:36:20 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-Starting copy...
20200701:11:36:20 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-Copy Timestamp = 1593574580239008343
20200701:11:36:20 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-'gpcopy' --debug --dest-host '192.168.102.44' --dest-port '5432' --dest-table 'warehouse.read30.book_info' --dest-user 'gpadmin' --drop --include-table 'warehouse.read30.book_info' --source-host '127.0.0.1' --source-port '1921' --source-user 'gpadmin'
20200701:11:36:20 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-pg_dump (PostgreSQL) 8.2.15
20200701:11:36:20 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-pg_dumpall (PostgreSQL) 8.2.15
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node186
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node186
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node186
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node186
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node187
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node187
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node187
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node187
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node188
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node188
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node188
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node188
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node189
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node189
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node189
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node189
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node190
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node190
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node190
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node190
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node191
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node191
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node191
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node191
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node192
20200701:11:36:21 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node192
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node192
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node192
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node193
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node193
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node193
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node193
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node194
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node194
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node194
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node194
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node195
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node195
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node195
20200701:11:36:22 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Resolving IP address of node195
20200701:12:38:30 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-Copying source database "warehouse"
20200701:12:38:30 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-Copying selected tables from database "warehouse" => "warehouse"
20200701:12:38:30 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-Copying source database "warehouse"
20200701:12:38:30 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-Copying selected tables from database "warehouse" => "warehouse"
20200701:13:41:35 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] Generated Dump Command: pg_dump --gp-syntax -h 127.0.0.1 -p 1921 -U gpadmin -s -x -t '"read30"."book_info"' 'warehouse'
20200701:13:41:50 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-[Worker 0] Creating table "warehouse"."read30"."book_info"
20200701:13:41:52 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-[Worker 0] Transferring table "warehouse"."read30"."book_info" => "warehouse"."read30"."book_info"
20200701:13:41:52 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] There are 210750 rows in the source table "warehouse"."read30"."book_info"
20200701:13:41:52 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] EXTERNAL WEB TABLE command of receiving data: CREATE EXTERNAL WEB TEMP TABLE gpcopy_ext_5811f956934f44d09a3266c2804c73b0 (like "read30"."book_info") EXECUTE '[ "$GP_SEGMENT_ID" -ge 32 ] && exit 0 || gpcopy_helper   --listen --seg-id $GP_SEGMENT_ID --cmd-id 06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-1 ' FORMAT 'csv'
20200701:13:41:52 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] Querying ports of helper programs: COPY gpcopy_ports_temp_1593574580239008343 FROM PROGRAM '[ ! -f /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-1-<SEGID>.txt ] && exit 0 || cat /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-1-<SEGID>.txt' ON SEGMENT
20200701:13:41:52 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] sendData() retried 0 times to get notification from receiveData()
20200701:13:41:52 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] Querying ports of helper programs: COPY gpcopy_ports_temp_1593574580239008343 FROM PROGRAM '[ ! -f /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-1-<SEGID>.txt ] && exit 0 || cat /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-1-<SEGID>.txt' ON SEGMENT
20200701:13:41:52 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] sendData() retried 1 times to get notification from receiveData()
20200701:13:41:52 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] Querying ports of helper programs: COPY gpcopy_ports_temp_1593574580239008343 FROM PROGRAM '[ ! -f /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-1-<SEGID>.txt ] && exit 0 || cat /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-1-<SEGID>.txt' ON SEGMENT
20200701:13:41:52 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] Retried 2 times to get segment helpers' ports, got 32 items
20200701:13:41:52 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] COPY command of sending data: COPY "read30"."book_info" TO PROGRAM 'MATCHED=0; SEGMENTS=(0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62); for i in "${SEGMENTS[@]}"; do [ $i = <SEGID> ] && MATCHED=1; done; [ $MATCHED = 1 ] && gpcopy_helper  --seg-id <SEGID> --host 192.168.102.186,99999,192.168.102.186,99999,192.168.102.186,99999,192.168.102.186,99999,192.168.102.187,99999,192.168.102.187,99999,192.168.102.187,99999,192.168.102.187,99999,192.168.102.188,99999,192.168.102.188,99999,192.168.102.188,99999,192.168.102.188,99999,192.168.102.189,99999,192.168.102.189,99999,192.168.102.189,99999,192.168.102.189,99999,192.168.102.190,99999,192.168.102.190,99999,192.168.102.190,99999,192.168.102.190,99999,192.168.102.191,99999,192.168.102.191,99999,192.168.102.191,99999,192.168.102.191,99999,192.168.102.192,99999,192.168.102.192,99999,192.168.102.192,99999,192.168.102.192,99999,192.168.102.193,99999,192.168.102.193,99999,192.168.102.193,99999,192.168.102.193,99999 --port 2957,99999,20070,99999,13858,99999,29752,99999,16716,99999,22531,99999,12541,99999,22943,99999,22064,99999,23146,99999,14673,99999,9919,99999,18608,99999,19244,99999,31550,99999,26638,99999,25206,99999,18879,99999,27699,99999,11510,99999,1722,99999,26931,99999,9677,99999,9313,99999,17531,99999,25182,99999,2143,99999,19173,99999,12249,99999,22772,99999,22354,99999,8994,99999 || cat > /dev/null' ON SEGMENT CSV IGNORE EXTERNAL PARTITIONS
20200701:13:41:53 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] Querying ports of helper programs: COPY gpcopy_ports_temp_1593574580239008343 FROM PROGRAM '[ ! -f /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-2-<SEGID>.txt ] && exit 0 || cat /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-2-<SEGID>.txt' ON SEGMENT
20200701:13:41:54 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] sendData() retried 0 times to get notification from receiveData()
20200701:13:41:54 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] Querying ports of helper programs: COPY gpcopy_ports_temp_1593574580239008343 FROM PROGRAM '[ ! -f /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-2-<SEGID>.txt ] && exit 0 || cat /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-2-<SEGID>.txt' ON SEGMENT
20200701:13:41:54 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] sendData() retried 1 times to get notification from receiveData()
20200701:13:41:54 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] EXTERNAL WEB TABLE command of receiving data: CREATE EXTERNAL WEB TEMP TABLE gpcopy_ext_e5fe6d3080974adfbbd5f52092ac9299 (like "read30"."book_info") EXECUTE '[ "$GP_SEGMENT_ID" -ge 32 ] && exit 0 || gpcopy_helper   --listen --seg-id $GP_SEGMENT_ID --cmd-id 06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-2 ' FORMAT 'csv'
20200701:13:41:54 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] Querying ports of helper programs: COPY gpcopy_ports_temp_1593574580239008343 FROM PROGRAM '[ ! -f /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-2-<SEGID>.txt ] && exit 0 || cat /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-2-<SEGID>.txt' ON SEGMENT
20200701:13:41:54 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] sendData() retried 2 times to get notification from receiveData()
20200701:13:41:54 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] Querying ports of helper programs: COPY gpcopy_ports_temp_1593574580239008343 FROM PROGRAM '[ ! -f /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-2-<SEGID>.txt ] && exit 0 || cat /tmp/gpcopy-06ffde17-f435-4f1d-bcb6-82ae0c3bdf37-2-<SEGID>.txt' ON SEGMENT
20200701:13:41:54 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] Retried 3 times to get segment helpers' ports, got 32 items
20200701:13:41:54 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-[Worker 0] COPY command of sending data: COPY "read30"."book_info" TO PROGRAM 'MATCHED=0; SEGMENTS=(1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63); for i in "${SEGMENTS[@]}"; do [ $i = <SEGID> ] && MATCHED=1; done; [ $MATCHED = 1 ] && gpcopy_helper  --seg-id <SEGID> --host 99999,192.168.102.186,99999,192.168.102.186,99999,192.168.102.186,99999,192.168.102.186,99999,192.168.102.187,99999,192.168.102.187,99999,192.168.102.187,99999,192.168.102.187,99999,192.168.102.188,99999,192.168.102.188,99999,192.168.102.188,99999,192.168.102.188,99999,192.168.102.189,99999,192.168.102.189,99999,192.168.102.189,99999,192.168.102.189,99999,192.168.102.190,99999,192.168.102.190,99999,192.168.102.190,99999,192.168.102.190,99999,192.168.102.191,99999,192.168.102.191,99999,192.168.102.191,99999,192.168.102.191,99999,192.168.102.192,99999,192.168.102.192,99999,192.168.102.192,99999,192.168.102.192,99999,192.168.102.193,99999,192.168.102.193,99999,192.168.102.193,99999,192.168.102.193 --port 99999,17797,99999,15665,99999,4329,99999,15049,99999,20824,99999,6720,99999,8586,99999,23038,99999,10541,99999,30364,99999,9500,99999,7701,99999,23928,99999,3665,99999,16433,99999,2692,99999,13524,99999,13027,99999,15874,99999,23137,99999,27277,99999,21208,99999,6724,99999,11457,99999,4168,99999,22219,99999,5697,99999,25952,99999,33248,99999,30903,99999,18538,99999,3264 || cat > /dev/null' ON SEGMENT CSV IGNORE EXTERNAL PARTITIONS
20200701:13:41:55 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-[Worker 0] Committing changes of table "warehouse"."read30"."book_info" => "warehouse"."read30"."book_info"
20200701:13:41:55 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-[Worker 0] [Progress: (0/1) DBs, (1/1) tables done] Finished copying table "warehouse"."read30"."book_info" => "warehouse"."read30"."book_info"
20200701:13:41:55 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-[Progress: (1/1) DBs, (1/1) tables done] Finished copying database "warehouse"
20200701:13:41:55 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Beginning cleanup
20200701:13:41:55 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-------------------------------------------------
20200701:13:41:55 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-Total elapsed time: 2h5m35.087123637s
20200701:13:41:55 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-Copied 1 databases
20200701:13:41:55 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:- Database warehouse: successfully copied 1 tables, skipped 0 tables, failed 0 tables
20200701:13:41:55 gpcopy:gpadmin:192-168-102-67:024611-[DEBUG]:-Cleanup complete
20200701:13:41:55 gpcopy:gpadmin:192-168-102-67:024611-[INFO]:-Copy completed successfully

迁移先决条件，迁移命令参数，等等可以参考官方文档，如下：

参考：
https://gpdb.docs.pivotal.io/data-copy/2-2/install.html
https://gpdb.docs.pivotal.io/data-copy/2-2/gpcopy-migrate.html
https://gpdb.docs.pivotal.io/data-copy/2-2/gpcopy.html

魂醉
关注
专栏目录
gpcopy实战
MyySophia的博客
09-17 526
目录 一、GPCOPY 简介 二、gpcopy 例子 2.1 GP4 2 GP6 2.1 迁移效能 一、GPCOPY 简介 参考一 参考二 二、gpcopy 例子 2.1 GP4 2 GP6 1、表结构准备 pg_dump -s --table=sor.wpp_adefect_f_n qmstst |pg_dump -h 10.50.* .* -d qmstst 2、清空表 qmstst=# truncate table sor.wpp_adefect_f_n; TR...
Greenplum数据迁移工具——gpcopy
weixin_51052054的博客
04-08 3338
gpcopy是一个数据迁移实用程序，可以在不同集群之间进行传输数据，可以将一个集群中的Greenplum数据库的元数据和数据复制到另一个集群的Greenplum数据库中。gpcopy可以迁移数据库的全部内容，包括数据库架构、表数据、索引、视图、角色、用户自定义函数、资源队列、资源组。 gpcopy-2.3.0.tar.gz 使用说明： 前提： 1、gpcopy压缩包，可到官网下载，建议先确定源库以及目标库的GP版本，对照gpcopy说明下载相应的版本。 2、GPCOPY需要两个GP（FD）集群做数据迁移，不
gpcopy-2.2.0.zip
04-11
gpcopy 工具是MPP 数据库Greenplum 数据迁移利器 兼容 4.3 到 5.x/6.x 目前最新版本 2.2(截止 20200410)
使用gpcopy从Greenplum4迁移数据到Greenplum5
xiadan_xiadan的博客
12-22 867
目录1.环境说明2.GP4.3.12升级到GP4.3.333.在GP4.3.33上安装gpcopy4.打通GP4 master机器到其他机器的SSH免密登录5.迁移数据案例：全库复制以append的方式复制一张表 现有环境要从GP4.3.12迁移到GP5.28.3上。在本地做了个实验成功，记录一下。 1.环境说明 GP4.3.12环境参考：https://blog.csdn.net/xiadan_xiadan/article/details/110953238 192.168.122.150 mdw mas
GP中copy命令
learning_oracle的博客
01-08 8230
--把外部文件（cvs格式）copy至数据库内部表mytable中 COPY mytable FROM '/data/myfile.csv' WITH CSV HEADER; （文件生成在Master） \COPY mytable FROM ‘/data/myfile.csv’ WITH CSV HEADER;（文件生成在本地） --把 '|' 为分隔符的外部文件copy至数据库内部
【晟数学院】PostgreSQL—greenplum使用gpcopy数据传输方案
weixin_45694422的博客
04-20 420
更多精彩内容，请登录：ke.sandata.com.cn 一 架构基础环境 主机mdw和备机mdwbak节点对等，都采用2+4架构。 1.1 主机列表 1.2 CPU及内存 所有节点都采用华为云服务器，CPU核心数为16核心，内存为32G，节点总数为12。主机备机都采用内部万兆网络。磁盘为单独挂载磁盘，采用高性能IO磁盘，实际最高读写性能为350M/s。 1.3 计算节点和镜像节点 计算节点primary数量为主备机都为24，镜像节点mirror数量为主备机都为24，总节点数量为48。 1.4 数据库版本
一篇文章帮你详细了解Greenplum迁移工具—GPCopy
Greenplum中文社区
03-12 665
以下资料是根据Pivotal Greenplum官网翻译、Grenplum中文社区博客以及个人测试所得，如有部分描述错误，欢迎下方评论指出，共同进步。 ...
「揭秘GP」Greenplum新一代数据迁移工具gpcopy，更快更稳更易用
Greenplum中文社区
06-24 880
gpcopy 是新一代的 Greenplum 数据迁移工具，可以帮助客户在不同集群间，不同版本间，快速稳定地迁移数据。同上一代迁移工具 gptransfer 相比，gpcopy 具有巨大的优势：更快，更稳定，更易用，功能更丰富。另外，gpcopy 只包含在商业版本中。 gpcopy 可以干什么 gpcopy 可以迁移整个集群，也可以具体传输某些数据库、某些命名空间和某些表；可以从文件读取传输或者略过的表，支持正则表达式；可以略过、追加或者替换目标集群的数据；可以并行传输；可以只迁移结构信息；可以静默自动化执
Greenplum数据迁移--gpcopy的知识点整理
非洲的小白的博客
11-25 8199
以下资料是根据Pivotal Greenplum官网翻译、Grenplum中文社区博客以及个人测试所得，如有部分描述错误，欢迎下方评论指出，共同进步。 目录 一：gpcopy介绍 二：gpcopy相较于gptransfer 三：gpcopy版本发展史 四：gpcopy命令（基于gpcopy1.5.0） 五：gpcopy官方提示注意点 六：gpcopy使用测试注意点 七：gpcopy...
gpcopy命令详解
数据源的港湾
09-12 769
注：本文部分翻译自gpcopy实用程序将对象从源Greenplum数据库系统中的数据库复制到目标Greenplum数据库系统中的数据库。
gpcopy-2.3.0.tar.gz
02-25
gpcopy 202007版本
【揭秘GP】全新 Greenplum 集群传输工具—GPCOPY 2.1.0 正式发布
MyySophia的博客
06-24 788
导读：GPCOPY 是新一代的支持 Greenplum 集群之间快速高效传输数据的工具。作为 Greenplum 集群数据传输的官方首选配套工具，GPCOPY 除了具有高速稳定易用的特点外，还支持不同版本 Greenplum 集群之间的传输（当然支持同版本之间的传输）。GPCOPY 支持从 GP4.3.x 到 GP 5.x、GP5.x 到 GP6.x、甚至 GP4.3.x 到 GP6.x 的数据传输。它也同时支持同等规模集群和不等规模集群之间的传输。另外 GPCOPY 还支持数据的校验，支持事务，增加了数.
全新Greenplum集群传输工具—GPCOPY 2.1.0正式发布
Greenplum中文社区
04-13 555
GPCOPY是新一代的支持Greenplum集群之间快速高效传输数据的工具。作为Greenplum集群数据传输的官方首选配套工具，GPCOPY除了具有高速稳定易用的特点外，还支持不同版本Greenplum集群之间的传输（当然支持同版本之间的传输）。GPCOPY支持从GP4.3.x到GP 5.x、GP5.x到GP6.x、甚至GP4.3.x到GP6.x的数据传输。它也同时支持同等规模集群和不等规模集群...
新一代Greenplum集群迁移工具数据 gpcopy 升级到 1.1.0
Greenplum中文社区
11-07 191
...
greenplum在使用gpcopy数据迁移记录
最新发布
weixin_41096226的博客
04-20 458
greenplum在使用gpcopy数据迁移时的记录，100T数据的迁移过程，双库提供服务不中断，迁移速度显示一开始大概在每小时5T左右，越往后越慢，最后在每小时1.8T，实际迁移花销大概在150小时。
GP数据库(Greenplum)单机版安装
weixin_54501932的博客
01-23 2318
2、安装完后会在/usr/local/下新建greenplum-db的文件夹，里面就是我们的安装的greenplum客户端。1、拷贝greenplum安装包带的配置文件到/home/gpadmin/gpconfigs/下。（2）主机填写自己的虚拟机的ip,一般都是静态IP，不要写localhost。1、首先切换到rpm安装包所在的路径，执行如下命令。（5）输入用户的登陆密码(如果上文配置无密码则不需要输入密码)（4）用户名写自己设置的用户如我的是gpadmin。（6）点击测试连接，如果连接成功就好了。
GP copy 数据迁移
MyySophia的博客
12-13 1524
实际业务需求：Oracle - >GP 要求1、栏位不一致 2、栏位的类型需要改变 3、源表新旧数据整合到目标表 array_glass_his_t/array_glass_his_old_t - > array_glass_his_t 4、.........(可以对数据做一些简单清洗) 在Oracle建立与目标表结构一致的表，使用 ...
GreenPlum数据的装载与卸载之copy命令的使用
weixin_34107955的博客
06-19 708
Copy是PostgreSql数据库自带的数据工具，它不仅支持表于表之间的数据加载，也支持文件于表之间的数据加载和表对文件的数据卸载。对于Greenplum数据库，使用copy命令进行数据加载，数据需要经过Master节点分发到Segment节点;同样使用copy命令进行数据卸载，数据也需要由Segment发送到Master节点，由Master节点汇总后再写入外部文件，这样就限制了数据加载与卸载的...
GreenPlum通过copy 和 gploader载入数据
MyySophia的博客
11-15 1382
GreenPlum数据加载 目录 GreenPlum数据加载 1. copy命令 1.1 创建测试表 1.2 准备测试数据 1.3copy命令语法 1.4数据加载 1.5数据卸载 1.6其他参数解释 2.使用gpfdist的外部表 2.1 创建实验环境 2.2gpfdist加载数据 2.3卸载数据 3.GreenPlum数据加载工具gpload 3.1...
GreenPlum数据库安装与使用指南
GreenPlum使用手册是一份详尽的指南，涵盖了GreenPlum数据库集群的安装、配置、管理和优化等内容。该手册主要分为以下几个部分： 1. **体系结构介绍**：Greenplum数据库是一款企业级数据仓库解决方案，由Greenplum...
评论 4
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值