Protect critical data with ease using Amazon EBS snapshots

All right, great. So before we dive into the agenda for today, uh by a show of hands, how many of you already use EBS snapshots? So it looks like almost all of you. So that's great.

So today, we are going to focus exclusively on EBS snapshots. We are going to talk about what they are, how you can use them and how they work. In addition to that, we are also going to talk about some of the security features that are available to you to improve the security and compliance posture of your snapshots.

Ben Denton is going to take over and he's going to talk about how you can take crash consistent snapshots and now how you can take application consistent snapshots using Data Life Cycle Manager, which is another capability that is available to you as part of EBS shotts.

We are also going to talk about some of the other features that are available to you to use with EB snapshots such as snapshots, archive and recycle bin.

So before we dive into a snapshots, let's spend a minute just talking about our overall storage portfolio. So at the heart of your data architecture is where you put your data. And AWS provides the most complete set of storage offerings to meet your requirements.

So as you can see in the middle, we have our core services. So we think of our storage portfolio in terms of three pillars, object block and file. So for object storage, you have Amazon S3, for block level storage, there's already own Amazon Elastic Block Store. And for file level storage, we have services like Amazon EFS and Amazon FSx.

On the right, you can see are hybrid and edge storage capabilities along with uh data movement services such as AWS, data sync and AWS storage getaway on the left, you can see our data protection capabilities uh such as AWS backup and Amazon Ebay snapshots. And together all these services come together to help you build the most secure compliant and auditable data estates. And all of these can be easily integrated with your hybrid on prem or edge based applications.

All right. So now let's double click into our Elastic Block Store offering. So let's first talk about our block storage portfolio. We think of our block storage portfolio in terms of easy to instance store and elastic block store.

So as an example of an ssd offering as part of instant to think of your i four instances as an example of an std offering, think of your d three instances on the elastic block store side as well. you have ssd offerings such as gp three and io two block express volumes. And you also have std offerings such as sc one and st one.

Now let's talk about EBS snapshots. So what are Amazon EB snapshots? So snapshots are nothing but point in time backups of ebs volumes. So what we mean by that is that each snapshot has all the information that is required to restore the volume in the state at the moment, the snapshot was taken.

So when you create a volume out of a snapshot, that volume begins as an exact replica of the original volume that was used to create the snapshot.

Secondly, snapshots are incremental. So only the change blocks from the last snapshot that you've taken are stored in the new snapshot that you're taking. So this helps you with two things. Firstly, um it minimizes the time that is required to create the snapshot. And secondly, uh it helps you save on costs because we are not duplicating data.

Thirdly, snapshots are crash consistent. So a lot of our customers have large data base and large file systems that span across multiple ebis volumes that might be attached to an ec2 instance. So with ebi snapshots, you can use a single api call to take crash consistent snapshots of all your ebs volumes or even a subset of your ebis volumes. So this saves you the work needed to let's say stop the instance, coordinate across all the volumes to make sure that the completed ios are persisted in the snapshot and lastly snapshots are very convenient and easy to share across accounts and across regions, especially for disaster recovery. use cases.

Now, let's talk about some of the use cases. Some of the most prominent use cases that we hear from customers for which they use ebs snapshots.

So the first use case and i've already spoken about this, it's backup and disaster recovery. So our customers will back up the data in the es volumes to a snapshot and then copy that data across accounts and across regions for disaster recovery workflows.

The second use case is what we call refresh, scale up and data handoff workflows. So a lot of times our customers are large organizations that have multiple business units and each of these business units are then associated with multiple accounts. So they want to maintain a centralized account where they develop amazon machine images or armies and then they want to distribute these army to other accounts so that everyone is consuming the latest army.

So this army might have the latest security patching, it might have approved agents for logging and monitoring purposes. So snapshots are very useful for that use case for distribution of these armies to the rest of the accounts within an organization.

The third use case is non ebs based backup and migration. So a lot of times our customers will back up the data in their corporate data centers to an ebs snapshot and then recover that in the cloud to an ebs volume. And it's extremely easy to back up your on prem data to ebs snapshots.

Lastly, there are a lot of third party solutions that have built automation on top of ebay snapshots, there are automation solutions, orchestration, security scanning and a lot of times our customers will create snapshots and then share it with vendor accounts for security scanning purposes or other workflows.

All right. So now let's talk about some of the basics related to snapshots. So we're going to start by talking about how incremental works. uh A lot of you might already be familiar with this, but uh it never hurts to just go back to the basics.

So uh in this example, as you can see, we have an ebs volume, the provision storage for the abs volume is one terabyte, but i haven't yet written any data to it. So if i snapshot this volume, the snapshot is also going to be empty because there is no data to snapshot and you won't be built for it.

But let's take another example. So in this example, i've now written three blocks of data to this volume. So if i take a snapshot of this volume, i will be billed for those three blocks and that snapshot one will contain these three blocks of data.

Now let's say another day passes, i write three more blocks of data to this volume blocks, 456 and take another snapshot. So the blocks in the snapshot will be blocks 45 and six as that is the delta between snapshot one and the current state of the volume. So you will only be billed for blocks 456 in this case and blocks 123 will be referenced from the previous snapshot snapshot one.

Uh just taking this example further, let's say i write another three blocks 789 and take a snapshot on the third day. Uh that third snapshot will also have three billable blocks, 789 blocks, 123 will be referenced from snapshot one and blocks 456 will be referenced from snapshot two.

So just as a refresher, some key things to remember uh related to snapshots. So firstly, they are full point in time backups of based volumes. Um they are incremental in nature and you are only built for the changed blocks between snapshots.

So now we come to some of the security features that are available that have been available for a long time and some that we've launched very recently to help you improve the security and compliance posture of your eb snapshots.

So a lot of times our customers tell us that backups are very critical for them. They serve as a critical line of defense against security threats and they are also very important for their compliance posture overall.

So we've launched some features and we have some features that will help you build more security and comply with either internal policies as well as external regulations.

So the first feature that we're going to talk about is encryption. So ebs provides encryption by integrating with aws key management system and the encryption that we offer is a 256 which is the industry standard.

Now, what do i mean by an encrypted ebs volume? So when we say that a volume is encrypted, what we mean is that the data at rest inside the volume is encrypted and the data in transit between the volume and the two instance is also encrypted.

In addition to that, if you create a volume out of that snapshot, that snapshot will also be encrypted by default. And let's say you then use that snapshot to create another volume. That volume is also going to be encrypted.

You can also encrypt a previously unencrypted snapshot by copying it and specifying the key with which you want to encrypt that snapshot, you can also re encrypt an already encrypted snapshot with a different key by copying it as well.

So uh an important concept to understand when it comes to encryption is the default key for your account. So every account uh has a regional key, which is the default k ms key for that account. Usually it's an aws managed key, but you are free to change that key to a customer managed key or any other key that you might wish.

Um so what this means is that whenever you encrypt a resource without specifying a key, the default k ms key is going to be used for encryption.

We have a very powerful feature available for enforcing encryption in ebs. We call it ebd or encryption by default. This is an account level setting and it applies to a singular region.

So when you enable this feature called encryption by default, what happens is that all the newly launched volumes will be encrypted by default. And since the newly launched volumes are encrypted, their snapshots will also be encrypted.

So this is a very easy way to ensure that all the new resources which are created in your account will be encrypted once you enable this feature.

Now let's talk about how you can achieve encryption of all your resources. So central security teams, they tell us that they have a mix of accounts where ebd might be enabled. ebd might not be enabled and then they have some existing resources, some of which might be encrypted and some of which might not be encrypted.

So these are the steps that you should go through if you want to achieve 100% encryption or do you want to encrypt all your resources?

So the first step that you want to start with is just identification and discovery. So you want to identify which resources are encrypted versus which are not. And then you want to identify which accounts have encryption by default enabled and which do not.

So for identifying resources that are encrypted, you can use um the describe api so you can use, describe volumes, describe snapshots and images and look at the encrypted parameter. And that will tell you whether a certain resource is encrypted or not.

In addition to that, you can also use aws conflict. So for those who don't know, aws don't know about aws conflict, it's a service that is separate from ebs, but it's, it's essentially a compliance management service.

So what it helps you do is it, it helps you track your resource configuration across time and you can use rules, aws config rules to flag resources which might be out of compliance.

So in this case, we already have a managed rule called encrypted volumes that you can use to flag volumes which might not be encrypted.

Now, to identify accounts which have ebd enabled or disabled, uh you can use get ebs encryption by default. You can also see it in the console

We also have an AWS Config managed rule for it called E2EBSEncryptionByDefault. You can also set up auto remediation for it. So what that means is that if EBD is disabled for your account, it will be auto remediated, meaning it will be turned on automatically.

The second step is identifying which key you want to use for encryption. So EBS offers two types of keys, customer managed key and AWS managed key. The key difference between the two types of keys is that it's much more convenient to share snapshots when you are using customer managed key versus AWS managed key. So there are some use cases in which you might want to use customer managed keys over AWS managed key. And we'll talk about those use cases in the subsequent slides.

So the first use case is backup and disaster recovery. So if this is your use case, you are likely copying or sharing your snapshots across regions and across accounts. So we recommend using a customer managed key.

The second use case as well if you have a centralized account that you are using to distribute AMIs, again, that involves sharing and copying. So we recommend using a customer managed key and for this third use case as well where you might be working with a third party solution and you might be sharing your snapshots with them. We recommend using customer managed keys.

So once you've identified what key type you want to use, then it's time to change the default KMS key that you want to use for EBS encryption. So that's very easy. And again, just as a reminder, it's applied on a per account per region basis. And you can easily modify your default KMS key and use the one that you match your use case.

Once you've identified the key that you want to use, you can then enable the EBD feature for the accounts that don't have it enabled. So it's very easy to enable, you can use, enable EBS encryption by default or you can use the console. It's an account level per region setting. You can also use AWS Config like we had discussed earlier.

Now, once you've enabled EBD, you are still left with existing unencrypted resources. So by enabling EBD for all your accounts, you have ensured that all the newly launched resources will be encrypted, but you are still left with unencrypted snapshots and unencrypted volumes. So to encrypt unencrypted volumes, what you can do is you can create a snapshot of that volume, you can encrypt it and then create a volume out of it, which will be a clone of the volume that you wanted to encrypt. Then for unencrypted snapshots, like we discussed earlier, you can easily copy them and encrypt it with the key that you desire.

So after following all of those steps, you will be able to make sure that all your resources are encrypted and you could repeat that for every account where EBD is not enabled or where resources are not encrypted.

Now, we're going to talk about sharing and permissions as they pertain to EBS snapshots. So one thing to know about EBS snapshots is that when you create an EBS snapshot, they are private by default. And what we mean by that is that they are visible or they are usable within the account, right? And the users of that account. So if you want another account to have access to it, you are going to have to modify permissions and share them with specific accounts. And this is private sharing. So you are privately sharing these snapshots with specific accounts, but you can also publicly share snapshots. Again, this is not something that we recommend doing unless you have legitimate use cases that require you to do that. But a lot of our customers, they have use cases for sharing test data with their customers or sharing trial software with their customers for which they might need to share snapshots publicly.

So when you share a snapshot publicly, it becomes visible to all AWS users. And any AWS user can use it. So by using it, what i mean is that they can copy it and then they can also create volumes out of it.

So here's one security feature that we launched recently called Block Public Access for EBS Snapshots. So you likely don't have any use cases for public sharing of snapshots. So in that case, we highly recommend that you enable this. This is an account level region level setting. And it really helps you lock down access to your data, right?

So it's available in two modes, Block all sharing mode and Block new sharing mode. So the difference between the two is with block new sharing mode. Let's say you have existing public snapshots, only future public sharing will be restricted. You'll still be able to share your existing snapshots publicly. So they will still remain publicly accessible. But if any user tries to share a new snapshot publicly, they won't be able to do that.

So if you have a use case where you need to share snapshots publicly, but you want to prevent any snapshots in the future from being shared publicly, then you can use this setting and then we have the more tighter setting called Block all sharing mode. So in this case, if you do have existing public snapshots, they will not be publicly accessible, so they will be blocked from being accessed publicly. And you will also be blocked from sharing any snapshots in the future publicly as well.

So out of the two, if you don't have any use cases for public sharing, we recommend enabling BPA or Block Public Access in Block all sharing mode.

So this is just a screenshot of where the setting resides. So most of you might be familiar with this, this is the AWS console. So under AWS console under Account Attributes, you'll have Data Protection and Security Settings. And once you go to that, you can see the Block Public Access setting. We also launched recently Block Public Access for AMIs. So we also recommend enabling that for your account. But for Block Public Access for EBS Snapshots. If you go to Manage, you will have the option of enabling Block all or Block new public sharing based on your requirements. You can choose the option that you want and then enable it.

Once you enable it, it just gets enabled within minutes. If you have public snapshots, it might take a few minutes to enable it. But you will receive an EventBridge notification once it's enabled for your account.

So as you can see in the screenshot, I've blocked public sharing and it says that all sharing has been blocked for this account for this, for a particular region. And then if you go to Snapshots and you try to modify permissions, you will see that the public sharing option has been grayed out. So no user in your account is allowed to publicly share snapshots.

The second feature that I'm going to talk about is EBS Snapshot Lock, which again we launched, I think about two weeks ago. So this is a very powerful feature if you want to prevent accidental or malicious deletions of your data. So when you lock a snapshot, what happens is that no one can delete it. So no user in your account or anybody who has access to it can delete it. It's very useful in certain use cases.

So let's go through the use cases in which you might want to lock your snapshots. So the first use case is when you would want to store your data in WORM compliant format, WORM means Write Once Read Many. So what that means is that the data cannot be modified and it cannot be deleted. So snapshots cannot be modified by default. But when you lock a snapshot, it also cannot be deleted. So using snapshot lock is a good way to enforce WORM compliance.

In addition to that, a lot of our financial and healthcare customers have requirements for storing data in this format to meet certain regulatory requirements such as SEC, CFTC, FINRA and others. So the Snapshot Lock feature is assessed by Cohasset Associates for using these, these environments. So it can be used for compliance with these regulations.

Lastly, from a security perspective, since backups are very important to a lot of our customers, it's a good practice to lock them because if you lock them, if there's a ransomware situation or if there is any security event it cannot be deleted, so your backups will stay intact.

So now getting into how it can be configured. One thing to know about Snapshot Lock is that it is always duration based. So you can lock a snapshot for a certain fixed duration and it can be anything from one day to up to 100 years based on your requirements.

The other thing to know about it is that it's available in two modes, depending on the kind of flexibility that you want. So the two modes that are available today are Compliance Mode and Governance Mode. So Compliance Mode is the restrictive mode and Governance Mode is the more flexible or the less restrictive mode.

So when you lock a snapshot in Compliance Mode, no user can delete the lock, no user can modify the lock configuration. But when you lock a snapshot in Governance Mode, you can grant certain users access to delete the lock or to change the lock configuration.

So we'll get into details on what these actions will be. But we recommend using Governance Mode if you want more flexibility. If you have use cases where you want to prevent deletions from most of your users, you want to have access to make sure that certain users still have access to delete the lock in case you need to delete the snapshot Compliance Mode is mostly for strict compliance use cases. WORM compliance, for example, where you want a more stricter setting so that no one is able to delete that snapshot before the lock expires.

So this is where the setting resides within the Snapshots console. Once you go to Actions, you have Manage Snapshot Lock option and once you go there, you can lock this snapshot. There are two modes available. As you can see Governance Mode and Compliance Mode.

When you choose Governance Mode, you have the option of specifying the lock duration, the lock duration can be specified in two ways. So first of all, you can lock a snapshot for a certain duration. So you can lock it for seven days. You can also specify a timestamp at which the lock should expire. So you can say that the lock should expire on January 2nd 2024.

So you have flexibility in terms of how you want to specify the duration.

So you can use either but not both. And once it's locked in governance mode, no one is allowed, no one will be able to delete it if the snapshot is locked. But then you can grant certain users access to delete this lock or to change the log configuration. So they can be allowed to reduce the duration to extend the duration, to upgrade the log from governance to compliance and so on.

This is for compliance mode. So when you choose compliance mode, uh similar to governance mode, you have to specify a long duration. You can also specify an expiry time stamp. Uh in addition to these parameters, you can also specify a cooling off period. So this field is entirely optional, but we recommend using this. So what what it helps you do is is that it gives you additional flexibility. So the lock will still remain editable for the cooling off period. So it can be specified from anything between 1 to 72 hours and in that period, the lock will remain editable. So you can go and make changes or you can delete the lock if you want. So in case you change your mind or uh especially in situations where you are taking multi volume snapshots and uh one of your snapshots fails and you might want to dilute the other snapshots as well. So we recommend using a cooling off period that might be suitable to you once the snapshot is locked, as you can see, uh if anyone tries to delete that snapshot, that call will fail, you will get an error saying that the snapshot is locked and it cannot be deleted.

Um and this is a screenshot of a snapshot which is logged in compliance mode. So as you can see, you are not allowed to make any changes except for extending the log duration. So the next feature that we're going to talk about is recycle bin. Recycle bin is a soft delete feature that we have available for snapshots as well as amazon machine images. So enabling this is very easy.

So um by a show of hands, how many of you have encountered situations where you accidentally deleted snapshots of amazon machine images and then you wanted to recover them later. Ok? Not a lot. Ok. There's one. All right. So yeah, a lot of customers have run into these issues where they end up accidentally deleting resources and they wish they had a soft delete option. So, recycle bin is that for you, uh recycle bin is again, very, very easy to set up. You can set up a retention rule. Um a retention rule can be set up at the account level or the tag level. So when you set that up, what that means is that i'm i'm setting a retention rule that all my snapshots and all my amazon machine images will be retained in the recycle bin for seven days after they are initially deleted. So during that seven day period, I can recover them if required after which they will be ejected out of the recycle bin and permanently deleted. So this gives you a soft delete option in case you deal with accidental deletions of resources.

In addition to that, another security control that we added to recycle bin last year at re invent was rule lock. So you can lock your retention rules as well. So especially in cases of account compromise where there might be malicious deletions of resources. You can rest assured that your rule cannot be altered, your rule cannot be modified or deleted and all your resources if deleted will go into the recycle bin and you can recover them from the recycle bin. So this is another good security control to use.

So that brings us to the end of our security features. Um now I'll invite denton on stage to talk about consistency of snapshots and application, consistent and crash consistent snapshots. Thank you.

Thanks, subby. Um so earlier on, in one of the slides subi mentioned that by default, when you're creating ebs snapshots, they're crash consistent. So i'm gonna take a couple of minutes to talk about exactly what crash consistent snapshots are and what's the difference between crash consistent snapshots and application consistent snapshots and why you might want to choose one over the other?

So crash consistent snapshots will contain the blocks of all completed io operations uh or the io that's on disk at the time that you initiate the snapshot. So any io that's in memory or in transit, that's not going to be on the snapshot when you create that snap a crash consistent snapshot. Whereas application consistent in snapshot, there's a little bit more co-ordination involved between the snapshot process and the application itself so that any in transit io or io in memory is going to be flushed first onto disk to make sure that they're on the disk. The uh any new io is gonna be temporarily frozen, the snapshots gonna be initiated. And that's how you guarantee sort of application application consistent snapshots.

So why would you wanna choose one over the other? So what we, when we speak to our customers, a lot of our customers tell us that when they're running self managed databases on ec2, if they're running sap hana, if they're running windows applications, it's super critical for those applications to retain all the data at the time. The snapshot is created. So for those customers, they wanna create application consistent snapshots.

Um and so why are we talking about application consistent snapshots today? We are actually going to introduce a feature a little bit later on that helps customers do exactly that to help customers automate the creation of application consistent snapshots, which actually builds on crash consistent snapshots offering that we had previously.

Um before I talk about the application consistent snapshot feature, I just want to talk a little bit more about the multi volume crash consistent snapshots because if you're not running those applications, maybe crash consistent snapshots is perfect for your use case.

So sylvia mentioned earlier on that, you know, there's a few ways to create snapshots of volumes. One is for example, if you use to create snapshot api you're creating a snapshot of a single volume, right? Um but there are also use cases where customers want to create a set of snapshots of all volumes that are attached to an ec2 instance. So they have a group uh of snapshots of that application at that exact state, right?

So when you're using the api, you're calling the great snapshots api to do that. Um and then once you do that, what you get is a collection of crash consistent multi volume snapshots of all the volumes that are attached to the instance.

Now you're probably looking at that diagram, you're thinking, you know, we have a similar set up um with my ac two instance, but in some situations, you don't really need to take the snapshot of the ebs of a boot volume, right? Or if you don't, if you have um, application temporary data on some of the ebs volumes or log data on those ebs volumes, you don't actually need to take snapshots of those volumes either.

So, what we uh have previously and what we launched at storage day last year was the ability to do exactly that is to define the subset of volumes that are attached to a single instance that you choose to create the set of crash consistent snapshots. And this is really for customers who want to reduce cost um of snapshots that they create then don't need or some of our customers actually previously were creating those snapshots and are manually deleting them. So we wanna help customers save from that process.

Ah so you can do that today using the crate snapshots api when you call that uh on a particular ec2 instance, just tell it you want to exclude the boot volume and any data volumes on that instance. Uh or if you're using amazon data life cycle manager, you can set policies to automatically exclude those volumes or set of volumes each time you create the crash consistent snapshots of those ec2 instances.

So what is amazon data life cycle manager? So, amazon data life cycle manager or dlm is a policy and tag based backup solution. Customers use it to define policies um that tell us how often they want to create snapshots or volumes. Uh how often they wanna create amazon machine images of ec2 instances and then how long they want to keep them for. So that's the basic of amazon dlm.

On top of that, they can also in the policy, define additional actions that they want to perform on the snapshots or the ami s once they create it. So basically any action that you want to take on the snapshot or ami you can define that in your dlm policy and the policies themselves. Uh there's no additional cost to creating policies. You can create as many as you like, you can uh define them, you can modify them. That's no additional cost. You're only paying for any ebs snapshots uh that are created through the policy.

So some of the actions that you can automate using the single dlm uh policy, a lot of our customers enable cross-region copy. So sue earlier mentioned that use case where you're creating a snapshot in your main region, but then you have a disaster recovery region somewhere else, then you can automatically create the snapshot in the main region and then copy it as soon as the first one completes in the main region,

um you can automatically share that with a different account. Um you can a automatically move it to a storage tier, ebs snapshots archive tier, which I'll talk about a little bit later on. Um and of course, you can also exclude the boot or the or the boot or data volumes um for the multie crash consistent snapshots.

So there are three main policy types and this is similar to the way customers create snapshots, right. So on the left hand side, there, you have the ability to automate the single snapshot of a single ebs volume. Um in the middle there, we spoke about the multi volume crash, consistent snapshots of all volumes that are attached to an ec2 instance. And on the right hand side, the ability to automatically create eb ds back amazon machine images as well as the underlying ebs snapshots.

So those are the three main policy types when using dlm.

So the feature that we announced earlier this month to help customers automate application consistent snapshots is actually building on the the middle policy type, the multi volume crash consistent snapshots. Um we're using dlm together with a db systems manager to coordinate those actions i mentioned earlier on the ec2 instance and i'll talk a little bit later on about how that coordinations uh is happening.

So the main use case for this feature, what we've called custom prescript and postscript automation is for customers who are running self managed databases. If you're running mysql, post square sql, sap hana, if you're running windows applications, you can use our inbuilt in-house built uh systems manager documents which has the commands to do the freeze and the flushing and the unfreeze or you can bring your own commands depending on whatever workload you're running on ec2 instances and you can set dlm to execute those commands on your behalf on the instance before the creation of the snapshots. And i'll talk a little bit about how that's happening.

What do you need to get started with the application consistent snapshots. So firstly for each ec2 instance that you're running, you'll need a systems manager agent installed and running. And what that is is it's sort of the interface if you like between the ec2 instance and the outside world. So we're using the ssm agent as a secure mechanism to pass instructions to the ec2 instance.

You need inm permissions for data life cycle manager to perform those actions. Uh you need the data life cycle manager policy which does the orchestration. And lastly, you need the ad bs systems manager run command document which contains all the instructions that you want to pass the ec2 instance.

And I'm just gonna briefly go through the main steps of the co-ordination of what's exactly happening sort of under the hood through this sort of architectural diagram. We have the main components of the feature here. We have the dlm policy, we have the snapshot process on the top left, we have the ec2 instance and then we have the systems manager command document.

So the first thing that happens is that when the policy is scheduled to create the snapshot at a specific time, it's gonna initiate the prescript portion of the systems manager document. And what we mean by the prescript portion is that it's the list of commands that will be run on the ec2 instance before the snapshot is initiated. So if you're running a self managed database, that's sort of like um flushing io from memory to disc and then it's temporarily pausing any new i os from happening.

We're going to uh the systems manager is going to pass the set of commands to the ec2 instance and run it v systems manager. Agent dlm is gonna get a signal back from the ssm document saying it succeeded or failed.

Um assuming it succeeded, we'll go on to call the um create snapshots api to create the set of multi volume, crash consistent snapshots and step five here, we're not actually waiting for the snapshot to complete, right? Because that can take a amount of time determining on the change rate of the particular volume.

So as soon as we initiate the volume as the snapshot on the volume, and we know that it initiated successfully, we're gonna initiate the post the post script portion of the ssm document and for self managed databases, that's telling the database you can start accepting io again.

So again, that gets passed through to the two instance by the systems manager agent, we get the signal back from their systems manager to the dlm. And then at some point in time, later on, once all this completes is when the snapshot completes

So once that snapshot completes, what we call that snapshot is an application consistent snapshot. And keep in mind steps 1 to 7 here is happening within seconds, right? Because most of the customers applications cannot withstand uh io pause for a significant amount of time. And you can see later on in the dlm policy, you can specify exactly that duration of time um that your application can withstand.

So I think I've spoken about all the main components. The last one I wanna mention is the systems manager document. So if you recall, this is the uh document that contains the instructions that you wanna pass the dlm will pass to the ec2 incident and be run by the ssm agent.

Um so please don't be intimidated by the block of code here. What I wanted to emphasize was just the two highlighted sections in yellow. Everything else basically is sort of dlm error handling and recognizing what's happening and communicating with the ec2 incident via the ssm agent. What we're really asking customers to provide us through the systems manager document is the prescript set of actions and the postscript set of actions.

And if you're ru running sap hano or windows applications, you don't even need this document. We have the inbuilt documents already. So you just click a button on the console and if you're running my sql and postscript sequel, we actually have sort of a template um for most standard versions of those databases available in our documentation. So you can just copy and paste that straight away.

So only if you have a slightly different uh database uh variation or a different application, you can just copy and paste the set of commands into this yellow section. Um and we have this template available in our documentation as well and then feed it into dlm and then dlm will do all the orchestration on behalf uh of your application.

And here are what looks like in the console. This is in the section where you're defining the data life cycle manager policy. You have the various tiles here, you have s a pv ss backup for windows applications and you sort of click them in and away you go um for the custom ssm document for your my sql and progress. There's some more things you can define including passing the ssm document.

So that in a nutshell is how the application consistent snapshot creation process works. And you're probably thinking great. I've set up my policy is starting to turn out snapshots. How do I know it's application consistent? So there's a few ways you can tell.

Um firstly is through tags. So through this process of creating application consis consistent snapshots through the policy, we are going to add uh two tags to the snapshot every snapshot that's created to if you see the success value next to the prescript and postscript keys um, of those tags.

Um if you see both of them, you, you can rest assured, you know, if your ssm document contained the right commands, then it's gonna be an application consistent snapshot. You can also monitor dlm policies through cloudwatch metrics in the console. And that'll tell you if there's any failures or s uh of the prescript or the postscript and you can set up sn ss notification if you like.

And we're also fully integrated with eventbridge. So every time a prescript takes an action, if it succeeds, if it fails, it's event, uh emitting an eventbridge event as well. So you can build dashboards and we're working on building some template dashboards to help customers with that as well.

So that was uh application consistent snapshot through data life cycle manager. Um i also wanted to introduce another feature that we launched a couple of weeks ago, uh couple of weeks ago called data life cycle manager, default policies. And what's super cool about default policies is that unlike other sort of backup mechanisms where you have a backup process and you've identified a volume and it's creating a snapshot of that volume every time.

What's different about default policies is that it is actually actively scanning the volume that you assign to the policy. And if it determines that there's already a backup that's been created by another mechanism, it's not gonna create another snapshot, right? So we're here really with default policies to try to save customers cost as well as a number of snapshots that you have to manage.

So we speak to a lot of storage, admins and account owners and we wanted them to use default policies as sort of like a catch all backup protection. You might have a bunch of different backup mechanisms already acting in your account. So if you can just enable default policies, if anything goes wrong with any of the other mechanisms, the default policies will kick in and i'll talk a little bit more about how that's happening.

So the two main use cases um for default policy, one is just a really simple way to enable automated backup protection for all ec2 instances and ebs volumes. And the second one is sort of like the comprehensive backup protection if you have multiple teams and users using the same account.

So going to the first use case again, going back to the ec2 console subi mentioned earlier, how you can go to the data protection and security section of the ec2 dashboard and there you'll see data life cycle manager, default policies. You can create one policy for ebs snapshots and one for ebs back amazon machine images. And with literally two clicks from the ec2 dashboard, you can create a default policy that is protecting all ebs volumes in your account.

So in this case, it's creating a snapshot every day and retaining it for seven days, right? So it's a super simple way, just a couple of clicks and all your volumes or all your instances in your account are protected. So that's the first use case.

And then you can go back to the dashboard and you can see the default policies is enabled or not as well as the configuration. Now, the second use case here is, is targeted for storage admins. We hear a lot of storage, a admins telling us that they um responsible for a number of accounts which are used by multiple teams.

Now, each team has its own backup mechanism, backup process and different workloads and different volumes that they need or need not backup. And so this is a very common sort of situation that our storage admins uh run into. And you imagine this but multiply by 10 teams and thousands of volumes.

So what d four policies allow the storage admins to do is craft a, a specific policy that only targets those critical applications. So anything that customers or the end users don't need to create snapshots of you can exclude that from your default policy. So your default policy is only focusing on these critical applications.

And again, it's scanning these volumes. If the other backup mechanisms are working correctly, they're taking the hourly daily backups. The default policy is not gonna create any new resources. So how do you do that sort of configuration and confining into critical workloads.

Again. Going back to that screen in the section here for exclusion parameters is where you can s specify. For example, if you don't want to backup any boot volumes. Um in the example above, i only wanted to backup io two block express volumes so i can exclude all the other volume types from my default policy.

And i can also add tags of temporary da uh volumes that i don't want to take snapshots of what's also really cool about default policies is that we're now fully integrated with the volume creation and the launch instance wizard uh resource creation processes by ec through console.

What we heard a lot from customers previously was that when they're creating these resources through console, they often don't know if the resources are going to be backed up. It's only later on. They see snapshots start being created from these volumes or ami started creating for these instances that they know they've config configured it successfully.

So what we're trying to do with this um integration with the console is to bring that closer to the creation of the volume. So you'll notice that this is before the customer clicks on the crate volume button. They can see that a the default policy is enabled the creation and retention of the default policy and whether this particular volume that they're about to create is going to be included or excluded in the default policy.

In this case, it's excluded because you notice that i've added a tag here which corresponded to a tag from the previous page of the default policy that i wanted to exclude. And similarly for the launch instance, visit as well, we have the same um for ebs backed army policies.

Another cool nifty feature that i wanted to share with customers. An improvement in the ec2 console is on the ebs volumes page. What we heard from customers was that a lot of customers have hundreds of thousands of volumes in their account, but they don't know which volumes have recent backups.

So they don't know how closely the backups are being uh adhered to in terms of their organization's requirements. So what you noticed today if you go to the console, if you click on ebs volumes on the left hand side, you'll see a new section on the right hand side here, which is a summary of all the volumes in your account.

And how many of those have ebs snapshots created within the last 24 hours. So you can use that as a quick view to see, hey, if there's any volumes that don't have snapshot, that might be alarming and might need to do something about that, for example, create default policies.

And what we can do here with the console to rate is you can down select using the search button up the top uh s search field up the top here. And in this case, i'm only focusing on io two block express and io one volumes because those are the most important volumes for me.

And here it's showing me uh those four volumes specifically and three of those have recent backup. So you can use that search field at the top to look at, you know, attached volumes, different volume types and uh volumes in different states as well.

So just to summarize, i think the default policy, the key call out here is that um it works in conjunction with other backup mechanisms. Um so please consider using it just for peace of mind to know that there's something there that will be creating daily backups in case any of your other backup mechanisms um are not working.

Lastly, i do want to just also mention ebs snapshots archive. Um so we actually launched ebs snapshots archived here a couple of years ago um at re invent, but cost and ebs snapshots continued to be on top of our customers minds, right?

So i just wanted to remind customers that we do have eb snapshots archived here, which is a lower cost storage tier for ebs snapshots aimed at customers who have long-term retention requirements of snapshots greater than 90 days.

Um we get asked a lot from customers like what are the key appropriate candidates for snapshots that are suitable for archive? And i think there's really main two main types.

One is if you have uh end of project snapshot that you wanna keep around and it's the only snapshot for the volume and you wanna keep it for at least 90 days. Then that's a great candidate for moving to archive here because you're gonna get the storage savings.

The other uh set of snapshots is uh those snapshots with a change rate greater than 25%. Um so you can use ebs direct api and we have some instructions on our documentation on how you can determine the difference in change rate between two snapshots. And if it's greater than 25% then you'll get cost savings as well.

Um when you move it to the eps snapshots archive tier, um and as of this week, you can now automate that process with ad bs backup. If you're already using ad bs backup, you can create the snapshot, uh the monthly snapshot and then move it to archive tier and also you can do it with data life cycle manager just to recap.

Um what we've spoken about today. Um number one, please, please please do consider using ebs snapshots to backup your mission critical workloads on ebs volumes. Um sui spoke about the security features using block public access to control, who has access to your snapshots and snapshot lock to make sure they can't be deleted.

We spoke about data livecycle manager to automate crash, consistent and application consistent snapshots and why you wanna use one over the other. And of course, we have eb snapshots archived here to meet compliance requirements and lower cost.

Just want to give a quick plug for some of the upcoming ebs snapshot sessions. Um if you're around the venetian tomorrow at 11 subi and i will be there giving a quick talk, uh some of these uh features again today, but in more of a relaxed uh hands on uh mode um talking about the uh features that we mentioned and thank you for your time this evening and hope you enjoy the rest of the evening.