aws rds 加密_如何以零停机时间和零数据丢失来加密AWS RDS MySQL副本集

aws rds 加密

Encryption of production databases can be a very challenging task. It is a very time-consuming operation and during that process, things may go wrong, which is the last thing that you want to do in a production environment. First of all, let’s look into the risks of doing such a procedure.

生产数据库的加密可能是一项非常具有挑战性的任务。 这是一个非常耗时的操作，在此过程中，可能会出错，这是您在生产环境中要做的最后一件事。 首先，让我们研究一下执行此过程的风险。

风险 (The Risks)

There are two major risks of doing database encryption. The first one is the potential downtime and the second is the data loss during the potential downtime.

进行数据库加密有两个主要风险。 第一个是潜在的停机时间，第二个是潜在停机期间的数据丢失。

To encrypt a database, one must take a snapshot of it, encrypt it, and then restore it from that snapshot, which can take several hours to complete depending on the size of the database. During that process, your production database may receive additional data that will not be present on the snapshot and one must also have a plan to migrate all new data after successful encryption.

要加密数据库，必须先对其进行快照，加密，然后从该快照中还原数据库，这可能需要几个小时才能完成，具体取决于数据库的大小。 在此过程中，您的生产数据库可能会收到快照中不会出现的其他数据，并且还必须制定计划，在成功加密后迁移所有新数据。

In AWS there isn’t a one-click way of doing RDS instance encryption and it always assumes the creation of a new RDS instance.

在AWS中，没有一键式进行RDS实例加密的方式，并且始终假定创建了一个新的RDS实例。

Let’s dig into our initial setup…

让我们深入了解我们的初始设置...

最初设定 (Initial Setup)

Initial setup

最初设定

In our initial setup, we will have one production database that will replicate all of its data into its replica which is running on read_only mode disabled.

在我们的初始设置中，我们将有一个生产数据库，它将所有数据复制到其副本中，该副本在禁用的read_only模式下运行。

The replica database is used as an analytics database. It has some analytics related tables where data is being inserted with AWS lambda and API gateway. The fact that production data is being replicated into that database will allow making complex analytics reports by joining analytics data with production data and create reports.

副本数据库用作分析数据库。 它具有一些与分析相关的表，通过AWS Lambda和API网关将数据插入其中。 将生产数据复制到该数据库中的事实将允许通过将分析数据与生产数据结合并创建报告来制作复杂的分析报告。

The main production database is an API database that serves all client applications.

主要生产数据库是为所有客户端应用程序提供服务的API数据库。

We will break off all encryption process into 3 steps.

我们将所有加密过程分为3个步骤。

步骤1：加密数据库 (Step 1: Encryption of the database)

The encryption of databases is pretty straight forward. There is an awesome tutorial here on how to do it, but before moving into that read this section till the end, here we have some IMPORTANT notes:

数据库的加密非常简单。 这里有一个很棒的教程，关于如何做到这一点，但是在继续之前，请阅读本节直到最后，这里有一些重要的注意事项：

• Complete all steps until step 13 included.
  完成所有步骤，直到包括步骤13。
• In step 12 the slave may have troubles connecting to the Master even if the all security group setup is done correctly. It turns out that if the RDS hostname is being used when adding an external master connection than the slave will not be able to connect to the master. To overcome this, you need to do any of the following actions:
  在步骤12中，即使所有安全组设置正确完成，从站也可能无法连接到主站。 事实证明，如果在添加外部主机连接时使用RDS主机名，则从服务器将无法连接到主机。 要解决此问题，您需要执行以下任一操作：

Action 1 (my favorite)

动作1(我的最爱)

Create an EC2 instance in the same VPC and then by using dig command find the private IP address of the RDS instance and use it instead. (Note that the IP address of the RDS instance may change suddenly, that’s why it is not shown in the RDS console). Although the private IP address is not very reliable, it is okay to use it, because we will get rid of this replication soon in the next steps.

在同一VPC中创建一个EC2实例，然后使用dig命令查找RDS实例的私有IP地址，然后改用它。 (请注意，RDS实例的IP地址可能会突然更改，因此RDS控制台中未显示它) 。 尽管专用IP地址不是很可靠，但是可以使用它，因为在接下来的步骤中我们将很快摆脱这种复制。

Action 2( the easy path )

动作2(简易路径)

Allow MySQL port from 0.0.0.0 and open your database to the internet. This approach is a lot easier and with this case, the RDS_HOSTNAME can be used without a problem. But keep in mind that opening database access to the internet is not the ideal solution from the security perspective, so use this method at your own risk.

允许从0.0.0.0版开始使用MySQL端口，并打开数据库到Internet。 这种方法要容易得多，在这种情况下，可以毫无问题地使用RDS_HOSTNAME。 但是请记住，从安全角度来看，开放数据库对Internet的访问不是理想的解决方案，因此使用此方法需要您自担风险。

After creating the encrypted replica, go to the AWS RDS console and create a read replica for that instance.

创建加密副本后，转到AWS RDS控制台并为该实例创建只读副本。

Note: If the create read replica setting is disabled, it means that you will need to enable automated backups on your instance.

注意：如果禁用了“创建只读副本”设置，则意味着您将需要在实例上启用自动备份。

Disable the read-onlyoption for the created replicas. (This can be done using RDS DB parameter groups)

对创建的副本禁用read-only选项。 (可以使用RDS DB参数组完成此操作)

The first replica is going to be the new production database and the second one analytics.

第一个副本将是新的生产数据库，第二个副本将是分析数据。

At the end of this step we will have the following setup:

在此步骤的最后，我们将进行以下设置：

Setup after completing Step 1

完成第1步后的设置

With this setup, you have real-time replication from your production database to encrypted replicas, and when switching from the production database to an encrypted production database you will not lose any data.

使用此设置，您可以从生产数据库实时复制到加密的副本，并且从生产数据库切换到加密的生产数据库时，您不会丢失任何数据。

The next step we will be the most challenging. We will replicate certain tables from the analytics database to the second encrypted replica.

下一步，我们将是最具挑战性的。 我们将某些表从分析数据库复制到第二个加密副本。

步骤2：处理分析 (Step 2: Dealing with analytics)

The only unique data that analytics database has is the data generated by AWS lambda which is stored in specific analytics tables, all other tables are production database tables that are being used just for joining inside SQL queries. Our main goal in this section is to establish a real-time replication connection between analytics database and encrypted analytics database (second encrypted replica) and replicate only analytics tables.

分析数据库具有的唯一唯一数据是由AWS lambda生成的数据，该数据存储在特定的分析表中，所有其他表都是生产数据库表，仅用于联接内部SQL查询。 本部分的主要目标是在分析数据库和加密的分析数据库(第二个加密副本)之间建立实时复制连接，并仅复制分析表。

AWS RDS does not support the replication of certain tables from one RDS instance to the other. To overcome this we will be using AWS DMS (database migration service).

AWS RDS不支持将某些表从一个RDS实例复制到另一个RDS实例。 为了解决这个问题，我们将使用AWS DMS(数据库迁移服务)。

AWS Database Migration Service helps you migrate databases to AWS quickly and securely. The source database remains fully operational during the migration, minimizing downtime to applications that rely on the database.

AWS Database Migration Service可帮助您将数据库快速安全地迁移到AWS。 源数据库在迁移期间保持完全可操作，从而最大程度地减少了依赖该数据库的应用程序的停机时间。

Navigate to DMS console then complete the following steps

导航到DMS控制台，然后完成以下步骤

• From the Replication, Instances menu create a replication instance
  从“复制，实例”菜单创建复制实例

• From the Endpoints, menu create the following endpoints: source endpoint using an analytics database as a source and target endpoint using encrypted analytics database as a target

  在“端点”菜单中，创建以下端点： 使用分析数据库作为源的源 端点和使用加密分析数据库作为目标的目标端点

• Complete the steps described in Using an Amazon-Managed MySQL-Compatible Database as a Source for AWS DMS

  完成使用与Amazon管理MySQL兼容数据库作为AWS DMS的源中所述的步骤

Note: Don’t forget to turn off the read-only option from the encrypted read-replicas

注意：不要忘记关闭加密的只读副本中的只读选项

Before we begin the actual replication process it is important to know that AWS DMS will transfer the data, but it will not transfer indexes on the tables, autoincrement settings, and so on. More detailed documentation on what will be transferred can be found here.

在开始实际的复制过程之前，重要的是要知道AWS DMS将传输数据，但不会传输表上的索引，自动增量设置等。 有关将要转移的内容的更多详细文档，请参见此处 。

To avoid such errors we will export table structure from the analytics table and create empty tables with all of the indexes being set up. This can be done using WorkBench or Navicat clients for MySQL.

为避免此类错误，我们将从分析表中导出表结构，并创建包含所有索引的空表。 可以使用MySQL的WorkBench或Navicat客户端来完成。

Hint: This is a unique opportunity to add extra indexes to your tables if they are very big. In general, it will take a lot of time to add an index into an existing big table, but now all data will be inserted from scratch. So there will not be a better time to do so.

提示：如果表很大，这是向表添加额外索引的独特机会。 通常，将索引添加到现有的大表中将花费大量时间，但是现在所有数据都将从头开始插入。 因此，没有比这更好的时间了。

Now that all empty tables are created on the target instance it is time to start the replication.

现在，所有空表都已在目标实例上创建，是时候开始复制了。

From the AWS DMS console, go to the dashboard then create a database migration task. All the fields are pretty straightforward to fill. The most important things to know is the following:

在AWS DMS控制台中，转到仪表板，然后创建数据库迁移任务。 所有字段都非常容易填写。 最重要的事情如下：

• Select Migrate existing data and replicate ongoing changes as Migration type. (This will ensure that your encrypted analytics database will stay in sync with the current analytics database)

  选择迁移现有数据并将正在进行的更改复制为迁移类型。 (这将确保您的加密分析数据库将与当前分析数据库保持同步)

• Select Do nothing as Target table preparation mode. (Will preserve the tables we have created on the target database and will insert data into them.)

  选择不执行任何操作作为目标表准备模式。 (将保留我们在目标数据库上创建的表，并将数据插入其中。)

• Enable CloudWatch logs (In case any error happens during the replication)

  启用CloudWatch日志 (如果复制期间发生任何错误)

• Enable validations (Will increase the overall migration time, but will ensure that data inserted to the target is identical to the data from source)

  启用验证 (将增加总体迁移时间，但将确保插入到目标的数据与来自源的数据相同)

Now click the create task and the replication process will start.

现在单击创建任务，复制过程将开始。

Hint: CloudWatch logs will provide useful information in case of errors during the replication process

提示：在复制过程中出现错误时，CloudWatch日志将提供有用的信息

Hint: If somehow CloudWatch logs are not present in the CloudWatch console then this article will help you a lot.

提示：如果CloudWatch控制台中不存在CloudWatch日志，那么 本文 将对您有很大帮助。

At this point, we will have the following setup:

此时，我们将进行以下设置：

Setup after completing Step 2

完成第2步后的设置

Now we have created the exact same initial setup with encrypted database instances, which are in sync with the original databases. It is time to switch databases.

现在，我们使用与原始数据库同步的加密数据库实例创建了完全相同的初始设置。 现在该切换数据库了。

步骤3：切换数据库 (Step 3: Switching the databases)

To complete the final setup we need to point our API to the encrypted production database. Continue this tutorial from step 13 where we stopped on the first step.

为了完成最终设置，我们需要将我们的API指向加密的生产数据库。 从第13步继续本教程，在第13步我们停止了。

Now the last thing we will do is to point analytics Lambda function to write to encrypted analytics database. Optionally after this step, you may remove the replication tasks and instances from AWS DMS.

现在，我们要做的最后一件事是指向分析Lambda函数以写入加密的分析数据库。 (可选)在此步骤之后，您可以从AWS DMS中删除复制任务和实例。

At the end of this final step we will have the following setup:

在最后一步的最后，我们将进行以下设置：

As you can see this setup will not cause any database downtime and during the migration process, no data will be lost.

如您所见，此设置不会导致任何数据库停机，并且在迁移过程中，不会丢失任何数据。

If you have read so far, congratulations, I hope it will help you in your projects.

到目前为止，如果您已阅读并恭喜，希望它对您的项目有所帮助。

翻译自: https://medium.com/swlh/how-to-encrypt-aws-rds-mysql-replica-set-with-zero-downtime-and-zero-data-loss-b145cbf1ecf0

aws rds 加密