Implement proactive data protection using Amazon EBS snapshots

All right, thank you everyone for coming and welcome to STG 226, Implementing Proactive Data Protection Using Amazon EBS Snapshots. I'm Sir, I'm a product manager with the Elastic Block Store team at AWS and along with me is Denton.

Hi, everyone. My name is Denton. I'm also a product manager with Elastic Block Store team. Um super excited today. I wonder w Subi and I are gonna share some of the features that we actually launched in the last month uh to help customers protect um their EBS volumes with EBS snapshots.

Um so firstly, we just want to give a very brief introduction of EBS snapshots and many of you may already be familiar with this. Um so EBS snapshots are used to protect EBS volumes. They're pointy time backups of EBS volumes and there are some properties about EB snapshot that you should be aware of.

Um firstly, they're incremental. So every time you're taking a snapshot of a volume that snapshot is only containing the change block data of the volumes. Uh when you create the snapshot, um the snapshots are also crash consistent, which means that any completed io that is on the disk at the time that you create the snapshot will be part of the snapshot. But any uh io that's in memory or that's in transit will not be part of the snapshot.

Uh but we actually have a very cool feature that I'll talk about later on to show you how to create uh application consistent snapshots that also contains the io of the um in memory. So snapshots can also be shared and copied across accounts and a bs regions. So a lot of our customers for disaster recovery are creating snapshots in one region and then copying to another region in case they need to load a, a volume up or load the workload up in the secondary uh disaster recovery region.

So there's four features that we're gonna try to cover in the next 20 minutes. Um the first feature um that I'm gonna talk about the first two. The first one is called DLM support for custom prescript and postscript automation.

Briefly. Why we created this feature is we have a lot of customers coming to us and they're telling us that they're running self managed databases on ec2 instances. So i mentioned earlier that um snapshots when they're created by default are crash consistent. So for self managed databases like mysql, post sql sap harder and windows applications, it's super important that you create application consistent snapshots so that when you create the volumes later on from those snapshots, they uh reflect the exact state of the volume when you create the snapshot.

So I'm just gonna go through a very brief demo um of how you create the application, consistent snapshots or how you automate it through data life cycle manager.

Um so this is the ec2 console. Uh you're probably already familiar with this. On the left hand side, we have life cycle manager. Um so I'm gonna go ahead and create a uh dlm policy and notice that I'm gonna target instances.

Um so in this case, we're actually working with the systems manager agent which resides on the instance which runs the commands to flash all the io and then pause the io in order to take the application consistent snapshot.

So the way dlm works is that it's got a single policy is gonna target instances based on the tag or multiple tags that you add. So in this case, I'm just gonna add a single tag. I'm gonna add a description, uh a test to the description. I'm just gonna skip over to the new section, which is you see here under advanced settings, the pre and post script section

um where customers can specify the different workloads that they're running on the instances that they're targeting with the policy. So if you're rounding sap hana or if you're running windows applications, you can just go ahead and click those tiles. And then what happens when the dlm policy runs is that for those applications, it's gonna automatically pause the io it's gonna flush the io to disk and then it's gonna initiate the snapshot. And then once the snapshot has been initiated, it's gonna start up the application again automatically.

So in, in that sense, once the snapshot has completed, you're getting application consistent snapshots. However, if you're running uh my sequel, post square sql, or if you have your own custom workload that might not fit into the these categories, what you can do is you can bring your own uh commands to data life cycle manager.

Uh earlier on, i created an ssm document for my sql. So what you do is you um just create an systems manager document and that document will contain all the commands required to freeze and uh flush io and you just submit it here into the policy. And so when the policy runs uh at the execution time, it's gonna look for the ssm document and it's gonna run those commands on the ac two instance by the systems manager agent.

Um so you can use it to create application consistent snapshots or if you have other use cases that you want to run on your ec2 instance, you can do that as well. Just before and after the snapshots are initiated. And also i should mention we have a template on our documentation page as well of what the ssm document should look like. So you can just copy and paste that across fill in the specific sections that you wanna run.

Um and then away you go. Um so that's the very first feature. The second feature i wanted to talk about um is called amazon data life cycle manager default policies. And the reason we created this policy type is because we heard from a lot of customers is that they just want a peace of mind, comprehensive data protection for all the resources in their account.

The way a lot of current backup mechanism works is that you create a policy, you create a backup plan is looking after a particular volume or an instance and it's creating a backup of that resource every hour, every day. The way that's different about default policies is that default policies is looking after a specific set of critical applications that you assign it and it's actually continuous in scanning, scanning those resources to see if there are recent backups.

So if there are recent backups already created by other backup mechanisms that you might already have running, then the default policy is not gonna take any action. So in this way, we're trying to save customers costs and those are the lower the number of resources that you have to manage. But for some reason if one of your other backup mechanisms fails, or if a tag gets missed from that resource, then a backup might not be created by those other mechanisms. And that's when the default policy will kick in. And then that'll, the default policy will then create the uh resources per retention.

So i'm gonna quickly then also show how to create a default policy. So the easiest way actually to create the default policy is you go to the ec2 dashboard um under data protection and security. You can see here, there's two types of uh default policies uh that you can create one for ebs snapshots and one for ebs back amazon machine images.

So go ahead and just click on the default policy creation and here you're setting the creation frequency and the retention period. And you're also specifying the resources, the noncritical applications that you wanna exclude.

So for example, if i only want to backup io two block express volume types, i can choose to exclude all the other volume types. Yeah. Yeah. And in that sense, the default policy will only be focusing and scanning on the io two block express volume types. You can also further add exclusion tags if you have temporary uh volumes that you do not wanna take backups of in your account.

So once you go ahead and add that you create the policy and it shows up in your ec2 dashboard as a policy that has to be created and is gonna be protecting all the critical applications. So really what we wanted to use is just to offer storage admins, peace of mind that you have some sort of backup protection mechanism that'll work no matter what

Hand over to Subi.

All right. So next feature that we're gonna talk about is block public access for amazon eb a snapshots. Now, one thing to know about snapshots is that they are private by default, but you can choose to share them with specific accounts or you can even choose to share them publicly. In that case, it becomes accessible to all aws users.

Now, most of our customers don't have any use cases for public sharing of snapshots. So for them, what they can do is they can block public access of amazon, ebay snapshots through a single account wide regional setting. It's very, very easy to enable and it's available in two modes. So let's get right to the demo and see how you can do that, right.

All right. So first, i'll go to the ec2 dashboard. That's where this setting resides. So under account attributes, you'll see data protection and security. And once you click on that, you'll see a bunch of different settings including default policies that we just talked about.

So once i scroll down, i can see block public access for ebay snapshots, we also recommend that you have block public access for amazon machine images enabled as well. So here i'll go in this case, uh i can see that my public access is not blocked. Uh and my users can share publicly snapshots in this region.

So i'll click on manage and then i'll select block public access now, i can enable it in two modes, block all public sharing or block new public sharing when i enable it and block new public sharing, all future public sharing is restricted. But if i have any existing public snapshots, they will continue to remain public in block all public sharing. It's a much tighter setting. And we recommend using block all public sharing if you don't have any use case for public sharing.

So in this case, if you have existing public snapshots, they won't be publicly accessible anymore. And any future public sharing will also be restricted. So in this case, i'll select block all public sharing and update the setting.

So as you can see, i've successfully enabled the setting. Uh it can usually take a few minutes. If you have existing public snapshots, you will receive an even bridge notification once the setting is enabled for you. And just like that, i can see that under public access, all sharing has been blocked. And now if i go to snapshots and if i select a snapshot and i try sharing it publicly, you can see that the public option is grayed out.

So no user in your account is allowed to share snapshots publicly for that account for that particular region.

So now we'll talk about the second feature uh that we released just about a few weeks ago, snapshot lock. So a lot of our customers have requirements to uh maintain their data in immutable format. Now, snapshots cannot be modified by default. But if you have delete permissions for a snapshot, you can delete them. So for it to be truly immutable, we've launched snapshot lock which allows you to lock a snapshot to prohibit the deletion of that snapshot.

There are certain use cases for which you might want to use this feature. So first of all, uh if you have regulatory requirements that require you to store your data in warm compliant format, which is right once read many where the data is truly mutable, then we recommend using this feature. In addition to that, if you just want to improve the security posture of your data, the security posture of your snapshots and prevent them from security security attacks such as ransomware, we recommend locking them because they cannot be deleted

um either accidentally or maliciously. Now let's get to the demo. Thank you.

So i'll go to the snapshots console, i'll select one of the snapshots and then i'll go to man at snapshot lock, i'll select lock snapshot. So this setting is very flexible in the sense that i have two modes, available. Governance mode is less restrictive. Whereas compliance mode is more restrictive.

When you lock a snapshot in governance mode, nobody will be allowed to delete it. But you can still give certain users permission to delete the lock or to change the lock configuration. So certain users can still go and reduce the log duration. Extend the log duration, delete the log or upgrade the log from governance to compliance in compliance mode. This is a much restrictive setting where no user will have access to modify the log settings or to delete the log. And in addition to that, the snapshot will also not be delete.

So i lock the snapshot in governance mode. Uh snapshot log is duration based. So i always have to select a log duration so i can lock it for a certain amount of days or i can select a, a log expiry time stamp at which the lock will expire. So in this case, i'll lock it for, let's say two days here, i can see the estimated expiry date. So once the lock expires, uh the snapshot can be related so you can lock it for the duration of the retention period of the snapshot. That's it.

Now, if i look a t the settings for this snapshot, i can see that it's logged in governance mode. And if i try to delete this snapshot, the deletion attempt will fail uh with the error message that the snapshot is now locked, so it cannot be deleted.

Now, let's select another snapshot and try locking it in governance mode. So when you select governance, uh when you select compliance mode, you have to select the log duration or the log expiry time stamp that you want. But in addition to that, you can also optionally select a cooling off period.

So a cooling off period gives you more flexibility in the sense that the log is still editable when the snapshot is in the cooling off period. So you can set a cooling off period between 1 to 72 hours during which the lock will still be editable. So in case you want more flexibility in case you change your mind or especially in cases where you're taking a multi volume snapshot and one of the snapshot fails and you need to delete the others as well. We recommend using a cooling off period.

So i'll lock it for two days again, i'll set a cooling off period of three hours. So in this case, i've successfully logged the snapshot. If i'll go back to the snapshots console, i can see that it's in the cooling off period. But then again, it's logged in compliance mode as well. Here is the remaining transcript formatted:

Now i'll select a third snapshot and in this case, i won't specify a cooling off period and lock it in compliance straight away. So i'm selecting a period of one day. I'm not specifying a cooling off period here.

So in this case, this snapshot has gone into compliance mode straight away because we haven't specified a cooling off period. And in this case, if i try to edit the lock, you can see that it's not editable. The only action that i'm allowed to take is extending the log duration.

I cannot delete the log, i cannot reduce the log duration, i cannot change the log mode from compliance back to governance. So it's a more stricter setting and if you have compliance reasons for which you might want to lock your data, we recommend using compliance mode.

All right. So that's it for our talk. Thank you so much for joining us. Uh these are some of the upcoming sessions uh for ebs as well as for ebs snapshots that we recommend that you attend. And as always, please take the sessions survey and let us know what you thought about the talk. Thank you.