centos7 kafka_如何在CentOS 7上备份，导入和迁移Apache Kafka数据

centos7 kafka

The author selected the Tech Education Fund to receive a donation as part of the Write for DOnations program.

作者选择了Tech Education Fund作为“ Write for DOnations”计划的一部分来接受捐赠。

介绍 (Introduction)

Backing up your Apache Kafka data is an important practice that will help you recover from unintended data loss or bad data added to the cluster due to user error. Data dumps of cluster and topic data are an efficient way to perform backups and restorations.

备份Apache Kafka数据是一项重要的实践，它将帮助您从意外的数据丢失或由于用户错误而添加到群集的不良数据中恢复过来。 群集和主题数据的数据转储是执行备份和还原的有效方法。

Importing and migrating your backed up data to a separate server is helpful in situations where your Kafka instance becomes unusable due to server hardware or networking failures and you need to create a new Kafka instance with your old data. Importing and migrating backed up data is also useful when you are moving the Kafka instance to an upgraded or downgraded server due to a change in resource usage.

在由于服务器硬件或网络故障而使Kafka实例无法使用，并且您需要使用旧数据创建新的Kafka实例的情况下，将备份的数据导入并迁移到单独的服务器会很有帮助。 当由于资源使用情况的变化将Kafka实例移动到升级或降级的服务器时，导入和迁移备份数据也很有用。

In this tutorial, you will back up, import, and migrate your Kafka data on a single CentOS 7 installation as well as on multiple CentOS 7 installations on separate servers. ZooKeeper is a critical component of Kafka’s operation. It stores information about cluster state such as consumer data, partition data, and the state of other brokers in the cluster. As such, you will also back up ZooKeeper’s data in this tutorial.

在本教程中，您将在单个CentOS 7安装上以及在单独服务器上的多个CentOS 7安装上备份，导入和迁移Kafka数据。 ZooKeeper是Kafka操作的重要组成部分。 它存储有关群集状态的信息，例如使用者数据，分区数据以及群集中其他代理的状态。 这样，您还将在本教程中备份ZooKeeper的数据。

先决条件 (Prerequisites)

To follow along, you will need:

要继续进行，您将需要：

• A CentOS 7 server with at least 4GB of RAM and a non-root sudo user set up by following this tutorial.

  通过遵循本教程来设置具有至少4GB RAM的CentOS 7服务器和一个非root用户的sudo用户。

• A CentOS 7 server with Apache Kafka installed, to act as the source of the backup. Follow the How To Install Apache Kafka on CentOS 7 guide to set up your Kafka installation, if Kafka isn’t already installed on the source server.

  装有Apache Kafka的CentOS 7服务器，用作备份源。 如果源服务器上尚未安装Kafka，请按照“ 如何在CentOS 7上安装Apache Kafka”指南进行设置。

• OpenJDK 8 installed on the server. To install this version, follow these instructions on installing specific versions of OpenJDK.

  服务器上安装了OpenJDK 8。 要安装此版本，请按照以下说明安装特定版本的OpenJDK。

• Optional for Step 7 — Another CentOS 7 server with Apache Kafka installed, to act as the destination of the backup. Follow the article link in the previous prerequisite to install Kafka on the destination server. This prerequisite is required only if you are moving your Kafka data from one server to another. If you want to back up and import your Kafka data to a single server, you can skip this prerequisite.
  对于第7步是可选的-另一台安装了Apache Kafka的CentOS 7服务器用作备份的目标。 请按照之前的先决条件中的文章链接，在目标服务器上安装Kafka。 仅当您将Kafka数据从一台服务器移动到另一台服务器时，才需要此先决条件。 如果要备份并将Kafka数据导入到单个服务器，则可以跳过此先决条件。

第1步-创建测试主题并添加消息 (Step 1 — Creating a Test Topic and Adding Messages)

A Kafka message is the most basic unit of data storage in Kafka and is the entity that you will publish to and subscribe from Kafka. A Kafka topic is like a container for a group of related messages. When you subscribe to a particular topic, you will receive only messages that were published to that particular topic. In this section you will log in to the server that you would like to back up (the source server) and add a Kafka topic and a message so that you have some data populated for the backup.

Kafka 消息是Kafka中数据存储的最基本单位，并且是您要向Kafka发布和订阅的实体。 Kafka 主题就像一组相关消息的容器。 订阅特定主题时，您将仅收到发布到该特定主题的消息。 在本部分中，您将登录到要备份的服务器(源服务器)，并添加Kafka主题和消息，以便为备份填充一些数据。

This tutorial assumes you have installed Kafka in the home directory of the kafka user (/home/kafka/kafka). If your installation is in a different directory, modify the ~/kafka part in the following commands with your Kafka installation’s path, and for the commands throughout the rest of this tutorial.

本教程假定您已经在kafka用户的主目录( /home/kafka/kafka )中安装了Kafka。 如果您的安装位于其他目录中，请使用您的Kafka安装路径以及本教程其余部分中的命令修改以下命令中的~/ kafka部分。

SSH into the source server by executing:

通过执行以下命令，SSH进入源服务器：

• ssh sammy@source_server_ip

  ssh sammy @ source_server_ip

Run the following command to log in as the kafka user:

运行以下命令以kafka用户身份登录：

• sudo -iu kafka
  须藤-iu kafka

Create a topic named BackupTopic using the kafka-topics.sh shell utility file in your Kafka installation’s bin directory, by typing:

通过在Kafka安装目录的bin目录中使用kafka-topics.sh Shell实用程序文件创建一个名为BackupTopic的主题，方法是输入：

• ~/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic BackupTopic

  〜/ kafka /bin/kafka-topics.sh --create --zookeeper localhost：2181-复制因子1-分区1 --topic BackupTopic

Publish the string "Test Message 1" to the BackupTopic topic by using the ~/kafka/bin/kafka-console-producer.sh shell utility script.