Trafodion 离线备份与还原

众所周知，备份与还原是数据库维护中一项十分重要的操作，它可以保证数据库在异常失效时进行恢复操作，防止不必要的损失。
开源版本的Trafodion支持基于离线的备份与还原操作，而商业版的EsgynDB除了支持离线的备份与还原，更是支持在线的基于时间点的备份与还原。此文主要介绍离线备份与还原。
Trafodion离线备份与还原操作是通过HBase快照特征来实现的，HBase的快照是以一种轻量的方式而无须数据的复制来保证数据的当前状态。HBase快照的详细介绍请参考HBase官方文档http://blog.cloudera.com/blog/2013/03/introduction-to-apache-hbase-snapshots/，在此不作更多说明。快照可应用于Trafodion的所有表，包括元数据表，并通过HBase ExportSnapshot MapReduce Job导出到HDFS。当需要从HDFS还原备份文件时，快照同样通过ExportSnapshot MapReduce Job导入到目标系统进行还原。
下面详细介绍离线备份与还原相关事项，

• 离线备份

  • 脚本路径（同下，其中run_full_trafodion_backup.sh为备份脚本）

    [trafodion@pk-esg-2 ~]$ cdw
    [trafodion@pk-esg-2 esgynDB-2.2.0]$ cd hbase_utilities/backup_and_restore/
    [trafodion@pk-esg-2 backup_and_restore]$ ll
    total 44
    -rwxr-x--- 1 trafodion trafodion 13889 Nov 17 14:32 backup_restore_functions.sh
    -rwxr-x--- 1 trafodion trafodion 13154 Nov 17 14:32 run_full_trafodion_backup.sh
    -rwxr-x--- 1 trafodion trafodion  9730 Nov 17 14:32 run_full_trafodion_restore.sh

  • 示例

    ./run_full_trafodion_backup.sh -b backup_folder -u trafodion_user -h HBase_user -d hdfs_user -m mappers -n

  • Sudo权限
    运行脚本的用户需要sudo权限，以便能作为HBase用户，HDFS用户和Trafodion用户在免密码情况下运行脚本。

• 离线还原

  • 脚本路径（同上，其中run_full_trafodion_restore.sh为还原脚本）

    [trafodion@pk-esg-2 ~]$ cdw
    [trafodion@pk-esg-2 esgynDB-2.2.0]$ cd hbase_utilities/backup_and_restore/
    [trafodion@pk-esg-2 backup_and_restore]$ ll
    total 44
    -rwxr-x--- 1 trafodion trafodion 13889 Nov 17 14:32 backup_restore_functions.sh
    -rwxr-x--- 1 trafodion trafodion 13154 Nov 17 14:32 run_full_trafodion_backup.sh
    -rwxr-x--- 1 trafodion trafodion  9730 Nov 17 14:32 run_full_trafodion_restore.sh

  • 示例

    ./run_full_trafodion_restore.sh -b backup_folder -b backup_dir -u trafodion_user -h hbase_user -m mappers -n

  • Sudo权限
    同上

• 备份样例

//执行备份
[trafodion@n12 backup_and_restore]$ ./run_full_trafodion_backup.sh -n
logging output to: /home/trafodion/esgynDB-2.2.2/hbase_utilities/backup_and_restore/logs/run_traf_backup_20170414-1453.log
Checking if  /home/trafodion/esgynDB-2.2.2/hbase_utilities/backup_and_restore/logs exists...
environment: 2
WARNING: The user name running HBase service was not provided by user.
WARNING: The user name running HBase service could not be computed.
HBase user name: hbase
WARNING: The user name running HDFS service was not provided by user.
WARNING: The user name running HDFS service could not be computed.
HDFS user name: hdfs
Trafodion user name: Trafodion
OpenJDK 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
hbase.snapshot.enabled=true
Checking if Trafodion processes running.
***[INFO]: Trafodion is down.
OpenJDK 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
OpenJDK 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
hdfs_uri: hdfs://nameservice1
hdfs_backup_location:
hdfs_backup_location  hdfs://nameservice1/trafodion_backups/backup_20170414-1453
hadoop fs -test -e hdfs://nameservice1/trafodion_backups/backup_20170414-1453
hadoop fs -mkdir -p hdfs://nameservice1/trafodion_backups/backup_20170414-1453
hadoop fs -touchz hdfs://nameservice1/trafodion_backups/backup_20170414-1453/tmp
hadoop fs -rm -skipTrash hdfs://nameservice1/trafodion_backups/backup_20170414-1453/tmp
Deleted hdfs://nameservice1/trafodion_backups/backup_20170414-1453/tmp
Backup Folder: hdfs://nameservice1/trafodion_backups/backup_20170414-1453 .
hbase_trafodion_table_list: /home/trafodion/esgynDB-2.2.2/hbase_utilities/backup_and_restore/tmp/bckp_trafodion_table_list_20170414-1453.hbase
Getting list of Trafodion tables from HBase
OpenJDK 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
17/04/14 14:53:34 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
TABLE
TRAFODION.GZDB.AL_BATCH_INFO
TRAFODION.GZDB.AL_M_D_ADMINLEVEL
...
Generating hbase scripts for table: TRAFODION.GZDB.AL_BATCH_INFO
Generating hbase scripts for table: TRAFODION.GZDB.AL_M_D_ADMINLEVEL
...
Creating hbase snapshots ...
snapshot        'TRAFODION.GZDB.AL_BATCH_INFO'  , 'TRAFODION.GZDB.AL_BATCH_INFO_SNAP_20170414-1453'
snapshot        'TRAFODION.GZDB.AL_M_D_ADMINLEVEL'      , 'TRAFODION.GZDB.AL_M_D_ADMINLEVEL_SNAP_20170414-1453'
...
hbase shell /home/trafodion/esgynDB-2.2.2/hbase_utilities/backup_and_restore/tmp/bckp_create_snapshots_20170414-1453.hbase
OpenJDK 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
17/04/14 14:53:41 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
0 row(s) in 0.5350 seconds

0 row(s) in 0.3110 seconds
...
Starting Trafodion...
sudo: unknown user: Trafodion
sudo: unable to initialize policy plugin
Trafodion not started. Please start Trafodion at your convinience.
********************************************************************
Exporting TRAFODION.GZDB.AL_BATCH_INFO_SNAP_20170414-1453 ...
hbase org.trafodion.utility.backuprestore.TrafExportSnapshot -snapshot TRAFODION.GZDB.AL_BATCH_INFO_SNAP_20170414-1453 -copy-to hdfs://nameservice1/trafodion_backups/backup_20170414-1453/TRAFODION.GZDB.AL_BATCH_INFO_SNAP_20170414-1453 -mappers 0 -mr-lowlimit-mb 100
OpenJDK 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
17/04/14 14:54:50 INFO backuprestore.TrafExportSnapshot: Trafodion Export Snapshot Utility
17/04/14 14:54:51 INFO backuprestore.TrafExportSnapshot: Copy Snapshot Manifest
17/04/14 14:54:52 INFO backuprestore.TrafExportSnapshot: Loading Snapshot 'TRAFODION.GZDB.AL_BATCH_INFO_SNAP_20170414-1453' hfile list
17/04/14 14:54:52 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
17/04/14 14:54:52 INFO backuprestore.TrafExportSnapshot: Copy Snapshot Data Files
17/04/14 14:54:52 INFO backuprestore.TrafExportSnapshot: Finalize the Snapshot Export
17/04/14 14:54:52 INFO backuprestore.TrafExportSnapshot: Verify snapshot integrity
17/04/14 14:54:52 INFO Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS
17/04/14 14:54:52 INFO backuprestore.TrafExportSnapshot: Export Completed: TRAFODION.GZDB.AL_BATCH_INFO_SNAP_20170414-1453
...
traf_table_count: 284
var_backup_snapshot_count: 284
var_snapshot_file_count: 284
Backup complete.
//检查备份文件
[trafodion@n12 backup_and_restore]$ hadoop fs -ls /trafodion_backups
Found 1 items
drwxr-xr-x   - trafodion trafodion          0 2017-04-14 15:15 /trafodion_backups/backup_20170414-1453
//执行还原
[trafodion@n12 backup_and_restore]$ ./run_full_trafodion_restore.sh -b hdfs://nameservice1/user/trafodion/trafodion_backups/backup_20170414-1453