PostgreSQL 在线升级

PostgreSQL 升级

安装 9.6 数据库

• 解压，编译

clw-db1:/home/postgres/pgsoft/postgresql-9.6.8 # ./configure --prefix=/opt/pgsql-9.6

• 遇到的错误

checking syslog.h presence... yes
checking for syslog.h... yes
checking for opterr... yes
checking for optreset... no
checking for strtoll... yes
checking for strtoull... yes
checking for rl_completion_append_character... yes
checking for rl_completion_matches... yes
checking for rl_filename_completion_function... yes
checking for rl_reset_screen_size... yes
checking for append_history... yes
checking for history_truncate_file... yes
checking test program... failed
configure: error:
Could not execute a simple test program.  This may be a problem
related to locating shared libraries.  Check the file 'config.log'
for the exact reason.

• 无法加载共享库，解决方法：加入环境变量后，重新编译即可

clw-db1:/home/postgres/pgsoft/postgresql-9.6.6 # export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

• 重新编译

./configure --prefix=/opt/pgsql-9.6

configure: using compiler=gcc (SUSE Linux) 4.3.4 [gcc-4_3-branch revision 152973]
configure: using CFLAGS=-Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -O2
configure: using CPPFLAGS= -D_GNU_SOURCE
configure: using LDFLAGS=  -Wl,--as-needed
configure: creating ./config.status
config.status: creating GNUmakefile
config.status: creating src/Makefile.global
config.status: creating src/include/pg_config.h
config.status: creating src/include/pg_config_ext.h
config.status: creating src/interfaces/ecpg/include/ecpg_config.h
config.status: linking src/backend/port/tas/dummy.s to src/backend/port/tas.s
config.status: linking src/backend/port/dynloader/linux.c to src/backend/port/dynloader.c
config.status: linking src/backend/port/sysv_sema.c to src/backend/port/pg_sema.c
config.status: linking src/backend/port/sysv_shmem.c to src/backend/port/pg_shmem.c
config.status: linking src/backend/port/dynloader/linux.h to src/include/dynloader.h
config.status: linking src/include/port/linux.h to src/include/pg_config_os.h
config.status: linking src/makefiles/Makefile.linux to src/Makefile.port
clw-db1:/home/postgres/pgsoft/postgresql-9.6.6 #

编译检查成功

• make 编译安装所有

make world -j 8

看到 PostgreSQL, contrib, and documentation successfully made. Ready to install ，表示编译成功
- 安装所有

clw-db1:/home/postgres/pgsoft/postgresql-9.6.6 # make install-world

看到 PostgreSQL, contrib, and documentation installation complete. 表示安装成功

• 创建数据库存储目录

clw-db1:/opt/pgsql-9.6 #  mkdir -p /pgdata/9.6/data
clw-db1:/opt/pgsql-9.6 # chown postgres.postgres /pgdata/ -R
clw-db1:/opt/pgsql-9.6 #

• 切换用户，初始化数据库

postgres@clw-db1:~> initdb -D /pgdata/9.6/data/

看到 Success. You can now start the database server using: 表示成功

• 启动服务

postgres@clw-db2:/pgdata/9.6/poc/data> /opt/pgsql-9.6/bin/pg_ctl -D /pgdata/9.6/poc/data/ start
server starting
postgres@clw-db2:/pgdata/9.6/poc/data> LOG:  redirecting log output to logging collector process
HINT:  Future log output will appear in directory "pg_log".

• 创建测试数据库和表

postgres=# create database pgupgradedb;
CREATE DATABASE
postgres=# \c pgupgradedb
You are now connected to database "pgupgradedb" as user "postgres".
pgupgradedb=# create table pgupgrade  (id serial,age bigint ,insertTime timestamp default now());
CREATE TABLE
pgupgradedb=#

• 添加测试数据

pgupgradedb=# insert into pgupgrade (age) values (11);
INSERT 0 1
pgupgradedb=# insert into pgupgrade (age) values (22);
INSERT 0 1
pgupgradedb=# select * from pgupgrade;
 id | age |         inserttime
----+-----+----------------------------
  1 |  11 | 2018-06-07 14:34:27.237998
  2 |  22 | 2018-06-07 14:34:39.871966
(2 rows)

pgupgradedb=#

PostgreSQL 9.6 –》10.2 pg_dumpall 大版本升级

• 登陆数据库，查看数据库当前版本

pgupgradedb=# select version();
                                                       version
----------------------------------------------------------------------------------------------------------------------
 PostgreSQL 9.6.8 on x86_64-pc-linux-gnu, compiled by gcc (SUSE Linux) 4.3.4 [gcc-4_3-branch revision 152973], 64-bit
(1 row)
pgupgradedb=#

• 下载需要升级的版本，此处为 postgresql-10.2.tar.gz，为了减少数据库的停机时间，我们先安装需要升级的数据库

• 解压 postgresql-10.2.tar.gz

postgres@clw-db2:/home/postgres/pgsoft> tar xf postgresql-10.2.tar.gz
postgres@clw-db2:/home/postgres/pgsoft>

• 编译检查，指定新的编译目录

postgres@clw-db2:/home/postgres/pgsoft> tar xf postgresql-10.2.tar.gz
postgres@clw-db2:/home/postgres/pgsoft> cd postgresql-10.2/
postgres@clw-db2:/home/postgres/pgsoft/postgresql-10.2> ./configure --prefix=/opt/pgsql-10.2

• 编译所有软件包

make world 

• 安装所有

make install -world

• 初始化新的数据库目录,初始化时 参数除prefix外，其余参数必须与原有数据库初始化的参数一致 ，防止不必要的错误

postgres@clw-db2:/pgdata/10.2/data> /opt/pgsql-10.2/bin/initdb -D /pgdata/10.2/data/ -E UTF-8 --locale=zh_CN.UTF-8
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "zh_CN.UTF-8".
initdb: could not find suitable text search configuration for locale "zh_CN.UTF-8"
The default text search configuration will be set to "simple".

Data page checksums are disabled.

fixing permissions on existing directory /pgdata/10.2/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    /opt/pgsql-10.2/bin/pg_ctl -D /pgdata/10.2/data/ -l logfile start

postgres@clw-db2:/pgdata/10.2/data>

• copy 原9.6版本的配置文件和认证文件

postgres@clw-db2:/pgdata/10.2/data> cp /pgdata/9.6/poc/data/pg_hba.conf /pgdata/9.6/poc/data/postgresql.conf /pgdata/10.2/data/
postgres@clw-db2:/pgdata/10.2/data> ls
pg_hba.conf  postgresql.conf
postgres@clw-db2:/pgdata/10.2/data>

• 修改 10.2 data的postgresql.conf 的端口号为 5439

• 使用 10.2命令 启动新的服务

postgres@clw-db2:/pgdata/10.2/data> /opt/pgsql-10.2/bin/pg_ctl -D /pgdata/
10/       10.2/     9.6/      estest10/ master10/
postgres@clw-db2:/pgdata/10.2/data> /opt/pgsql-10.2/bin/pg_ctl -D /pgdata/10.2/data/ start
waiting for server to start....2018-06-07 15:04:35.351 CST [2386] LOG:  listening on IPv4 address "0.0.0.0", port 5439
2018-06-07 15:04:35.352 CST [2386] LOG:  listening on IPv6 address "::", port 5439
2018-06-07 15:04:35.363 CST [2386] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5439"
2018-06-07 15:04:35.388 CST [2386] LOG:  ending log output to stderr
2018-06-07 15:04:35.388 CST [2386] HINT:  Future log output will go to log destination "csvlog".
2018-06-07 15:04:35.398 CST [2415] LOG:  database system was shut down at 2018-06-07 14:57:37 CST
2018-06-07 15:04:35.426 CST [2386] LOG:  database system is ready to accept connections
 done
server started
postgres@clw-db2:/pgdata/10.2/data>

• 此时需要停止用户对数据库的变更操作

• 执行命令进行在线备份和传输，以减少数据库服务停止时间,5438为9.6数据库端口，5439为10.2数据库端口

postgres@clw-db2:/pgdata/10.2/data> /opt/pgsql-10.2/bin/pg_dumpall -p 5438 | /opt/pgsql-10.2/bin/psql -d postgres -p 5439

• 执行命令时确保数据库不对外提供修改和删除数据库的服务 ，执行命令后看到 包含如下表示成功

...
You are now connected to database "pgupgradedb" as user "postgres".
SET
SET
SET
SET
SET
SET
SET
SET
SET
CREATE EXTENSION
COMMENT
SET
SET
SET
CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
ALTER TABLE
ALTER SEQUENCE
ALTER TABLE
COPY 2
 setval
--------
      2
. . .

• 连接新的 10.2数据库查询表和数据是否同步

postgres@clw-db2:~> /opt/pgsql-10.2/bin/psql -p 5439
psql (10.2)
Type "help" for help.

postgres=# \l
                                   List of databases
    Name     |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-------------+----------+----------+-------------+-------------+-----------------------
 pgupgradedb | postgres | UTF8     | zh_CN.UTF-8 | zh_CN.UTF-8 |
 postgres    | postgres | UTF8     | zh_CN.UTF-8 | zh_CN.UTF-8 |
 template0   | postgres | UTF8     | zh_CN.UTF-8 | zh_CN.UTF-8 | =c/postgres          +
             |          |          |             |             | postgres=CTc/postgres
 template1   | postgres | UTF8     | zh_CN.UTF-8 | zh_CN.UTF-8 | postgres=CTc/postgres+
             |          |          |             |             | =c/postgres
(4 rows)

postgres=# \c pgupgradedb
You are now connected to database "pgupgradedb" as user "postgres".
pgupgradedb=# \d+
                             List of relations
 Schema |       Name       |   Type   |  Owner   |    Size    | Description
--------+------------------+----------+----------+------------+-------------
 public | pgupgrade        | table    | postgres | 8192 bytes |
 public | pgupgrade_id_seq | sequence | postgres | 8192 bytes |
(2 rows)

pgupgradedb=# select * from pgupgrade;
 id | age |         inserttime
----+-----+----------------------------
  1 |  11 | 2018-06-07 14:34:27.237998
  2 |  22 | 2018-06-07 14:34:39.871966
(2 rows)

pgupgradedb=#

• 查询新数据库版本

pgupgradedb=# select version();
                                                       version
---------------------------------------------------------------------------------------------------------------------
 PostgreSQL 10.2 on x86_64-pc-linux-gnu, compiled by gcc (SUSE Linux) 4.3.4 [gcc-4_3-branch revision 152973], 64-bit
(1 row)
pgupgradedb=#

发现数据已经同步过来，现在可以停止原有数据库，修改10.2端口，重启服务，这样就完成了升级

小版本升级 10.2–> 10.3

• 下载需要升级的包 postgresql-10.3.tar.gz

• 上传到指定目录，解压

postgres@clw-db2:/home/postgres/pgsoft> tar -xf postgresql-10.3.tar.gz
postgres@clw-db2:/home/postgres/pgsoft>

• 编译检查,指定目录为本地 10.2的编译安装目录

postgres@clw-db2:/home/postgres/pgsoft/postgresql-10.3> ./configure --prefix=/opt/pgsql-10.2/

• 编译安装

make world 

• 安装

make install -world

• 查询此时数据库版本

postgres@clw-db2:~> /opt/pgsql-10.2/bin/psql -p 5439
psql (10.3, server 10.2)
Type "help" for help.

postgres=# select version();
                                                       version
---------------------------------------------------------------------------------------------------------------------
 PostgreSQL 10.2 on x86_64-pc-linux-gnu, compiled by gcc (SUSE Linux) 4.3.4 [gcc-4_3-branch revision 152973], 64-bit
(1 row)

• 重启服务

postgres@clw-db2:~> /opt/pgsql-10.2/bin/pg_ctl -D /pgdata/10.2/data/ restart
waiting for server to shut down.... done
server stopped
waiting for server to start....2018-06-07 15:33:13.158 CST [4765] LOG:  listening on IPv4 address "0.0.0.0", port 5439
2018-06-07 15:33:13.158 CST [4765] LOG:  listening on IPv6 address "::", port 5439
2018-06-07 15:33:13.179 CST [4765] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5439"
2018-06-07 15:33:13.223 CST [4765] LOG:  ending log output to stderr
2018-06-07 15:33:13.223 CST [4765] HINT:  Future log output will go to log destination "csvlog".
2018-06-07 15:33:13.232 CST [5092] LOG:  database system was shut down at 2018-06-07 15:33:12 CST
2018-06-07 15:33:13.294 CST [4765] LOG:  database system is ready to accept connections
 done
server started

• 查看此时系统版本

postgres@clw-db2:~> /opt/pgsql-10.2/bin/psql -p 5439
psql (10.3)
Type "help" for help.

postgres=# select version();
                                                       version
---------------------------------------------------------------------------------------------------------------------
 PostgreSQL 10.3 on x86_64-pc-linux-gnu, compiled by gcc (SUSE Linux) 4.3.4 [gcc-4_3-branch revision 152973], 64-bit
(1 row)

postgres=#

发现此时数据库版本升级为 10.3,则升级成功

pg_upgrade 升级9.6–》10.3

• 查看此时数据库版本

postgres=# select version();
                                                       version
----------------------------------------------------------------------------------------------------------------------
 PostgreSQL 9.6.8 on x86_64-pc-linux-gnu, compiled by gcc (SUSE Linux) 4.3.4 [gcc-4_3-branch revision 152973], 64-bit
(1 row)

postgres=#

• 下载需要升级的版本，此处为 postgresql-10.3.tar.gz

• 解压、安装与上述步骤相同，此处略

• 初始化数据库目录

postgres@clw-db2:/pgdata/10.2> mkdir /pgdata/10.3/data -p

postgres@clw-db2:/pgdata/10.2> /opt/pgsql-10/bin/initdb -D /pgdata/10.3/data/ -E UTF-8 --locale=zh_CN.UTF-8

• 停止原 9.6 数据库服务

• 备份旧数据的配置文件 pg_hba.conf 与 postgresql.conf

• 检查新旧版本j兼容性,先使用参数 --check

postgres@clw-db2:/pgdata/10.3/data> /opt/pgsql-10/bin/pg_upgrade -b /opt/pgsql-9.6/bin/ -B /opt/pgsql-10/bin/ -d /pgdata/9.6/poc/data/ -D /pgdata/10.3/data/ -k -c
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Checking for invalid "unknown" user columns                 ok
Checking for hash indexes                                   ok
lc_collate values for database "postgres" do not match:  old "en_US.UTF-8", new "zh_CN.UTF-8"
Failure, exiting
postgres@clw-db2:/pgdata/10.3/data>

• pg_upgrade 参数说明

-b 旧数据库的执行文件目录 -B 新数据库的执行文件目录
-d 旧数据库存放目录      -D 新数据库存放目录
-c 进行兼容性检查        -k 表示使用link的方式进行复制，而不是基于文件复制
-j cpu数量 ，根据自己机器指定数量 -P 指定新数据端口号
-U 指定新数据的管理员，默认为 `postgres` 

• 上述提示两次初始化数据库目录编码参数不一致，我们重新删除新数据库，重新初始化编码参数为 en_US.UTF-8 ,出现错误，则需要进行手动解决错误，直到没有错误,然后重新执行pg_upgrade命令

postgres@clw-db2:/pgdata/10.3> /opt/pgsql-10/bin/pg_upgrade -b /opt/pgsql-9.6/bin/  -B /opt/pgsql-10/bin/ -d /pgdata/9.6/poc/data/ -D /pgdata/10.3/data/ -k -j 8

• 看到下面表示升级成功

Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Checking for invalid "unknown" user columns                 ok
Creating dump of global objects                             ok
Creating dump of database schemas
                                                            ok
Checking for presence of required libraries                 ok
Checking database user is the install user                  ok
Checking for prepared transactions                          ok

If pg_upgrade fails after this point, you must re-initdb the
new cluster before continuing.

Performing Upgrade
------------------
Analyzing all rows in the new cluster                       ok
Freezing all rows in the new cluster                        ok
Deleting files from new pg_xact                             ok
Copying old pg_clog to new server                           ok
Setting next transaction ID and epoch for new cluster       ok
Deleting files from new pg_multixact/offsets                ok
Copying old pg_multixact/offsets to new server              ok
Deleting files from new pg_multixact/members                ok
Copying old pg_multixact/members to new server              ok
Setting next multixact ID and offset for new cluster        ok
Resetting WAL archives                                      ok
Setting frozenxid and minmxid counters in new cluster       ok
Restoring global objects in the new cluster                 ok
Restoring database schemas in the new cluster
                                                            ok
Adding ".old" suffix to old global/pg_control               ok

If you want to start the old cluster, you will need to remove
the ".old" suffix from /pgdata/9.6/poc/data/global/pg_control.old.
Because "link" mode was used, the old cluster cannot be safely
started once the new cluster has been started.

Linking user relation files
                                                            ok
Setting next OID for new cluster                            ok
Sync data directory to disk                                 ok
Creating script to analyze new cluster                      ok
Creating script to delete old cluster                       ok
Checking for hash indexes                                   ok

Upgrade Complete
----------------
Optimizer statistics are not transferred by pg_upgrade so,
once you start the new server, consider running:
    ./analyze_new_cluster.sh

Running this script will delete the old cluster's data files:
    ./delete_old_cluster.sh
postgres@clw-db2:/pgdata/10.3>

• 复制旧数据库的配置文件

postgres@clw-db2:/pgdata/10.3/data> cp /pgdata/9.6/poc/data/postgresql.conf /pgdata/9.6/poc/data/pg_hba.conf /pgdata/10.3/data/
postgres@clw-db2:/pgdata/10.3/data>

• 启动升级后的数据服务

postgres@clw-db2:/pgdata/10.3/data> /opt/pgsql-10/bin/pg_ctl -D /pgdata/10.3/data/ start
waiting for server to start....2018-06-07 16:44:38.921 CST [32465] LOG:  listening on IPv4 address "0.0.0.0", port 5438
2018-06-07 16:44:38.922 CST [32465] LOG:  listening on IPv6 address "::", port 5438
2018-06-07 16:44:38.937 CST [32465] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5438"
2018-06-07 16:44:38.971 CST [32465] LOG:  ending log output to stderr
2018-06-07 16:44:38.971 CST [32465] HINT:  Future log output will go to log destination "csvlog".
2018-06-07 16:44:38.984 CST [32501] LOG:  database system was shut down at 2018-06-07 16:34:00 CST
2018-06-07 16:44:39.029 CST [32465] LOG:  database system is ready to accept connections
 done
server started

• 验证版本和数据

postgres@clw-db2:/pgdata/10.3/data> psql -p 5438
psql (10.3)
Type "help" for help.

pgupgradedb=# \l
                                   List of databases
    Name     |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-------------+----------+----------+-------------+-------------+-----------------------
 pgupgradedb | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 postgres    | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
             |          |          |             |             | postgres=CTc/postgres
 template1   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | postgres=CTc/postgres+
             |          |          |             |             | =c/postgres
(4 rows)

pgupgradedb=# \d+
                             List of relations
 Schema |       Name       |   Type   |  Owner   |    Size    | Description
--------+------------------+----------+----------+------------+-------------
 public | pgupgrade        | table    | postgres | 8192 bytes |
 public | pgupgrade_id_seq | sequence | postgres | 8192 bytes |
(2 rows)

pgupgradedb=# select * from pgupgrade;
 id | age |         inserttime
----+-----+----------------------------
  1 |  11 | 2018-06-07 14:34:27.237998
  2 |  22 | 2018-06-07 14:34:39.871966
  3 |  33 | 2018-06-07 15:20:41.475839
(3 rows)

pgupgradedb=#

发现数据库和表、数据等都一样，至此升级成功。

注 意

以上升级仅为测试，实际在数据库大版本进行升级时，需要阅读官网关于新版班的相关注意事项，对比自己旧数据库，查看是否有需要变动或者修改的函数等。