Network-attached storage in the cloud with Amazon FSx

All right, everybody. Uh thanks for joining today's session. Uh we're gonna go ahead and get started. Uh my name is David Stein. I work uh with AWS doing, go to market for our Amazon uh FSX and file services and I'm joined today by I'm Prashant Bli. I'm the general manager for Amazon FSX for Windows and FSX for Open ZFS and great.

Well, we'll go over a handful of slides here and talk about network attached storage in the cloud with Amazon FSX and then we'll be joined by Taylor with Phillip 66 to talk about some customer use cases about how they migrated to the cloud with some of our file storage services at the end.

All right. So, um to get started, I just want to spend a few minutes talking about um how we think about solving data challenges and as it relates to your file storage and, and migrating that to AWS.

Um at AWS, we know um more than ever that innovation is uh relies on data and the ability to leverage that data effectively. Um that's why we say that data is the basis of modern innovation and because of data data's role. Organizations are adding data at an unprecedented rate driven by a wide variety of use cases. We estimate or IDC estimates that over 100 zettabytes of data has been added to customers environments in 2002, I'm sorry, 2022. And they expect that number to double by 2026. And a lot of it's being driven by file storage. As you can see by that last bar, it takes up a large portion of the amount of data that customers are growing in their environments.

And what's driving that acceleration is a lot of use cases around media, files and images and videos. We see a lot of AI and ML adoption and all the data that's associated with it. We've got the the use of simulated and virtual environments for simulation and the proliferation of IoT and other data generating devices that's adding a lot of data to, to the cloud and on premises environments.

But if you look at what's driving the file storage specifically, there's a few key buckets to think about as to why, why a lot of this data storage is file. Um and file storage is very ubiquitous in a lot of environments that because it has such a rich history and development of over 50 years, it's a very mature technology. It provides a default interface that applications and developers can use when they're making their applications that again has also been around for a long period of time and it has the right levels of permissions and abilities to share data between multiple users and applications.

So file data provides, provides a simple shared platform that dozens or hundreds or sometimes thousands of users and applications are able to leverage and it also provides high performance to provide near local latencies. So that when dozens or hundreds or thousands of applications or users are accessing these files, it works just as if it's like a local local file on their on their environment.

And if we think about the different types of file use cases that we're seeing and customers are coming to us and asking how can you help move these, these workloads to AWS. It ranges across a broad spectrum of use cases. We see enterprise IT use cases. These are things like corporate applications, database servers where you need shared file storage for like clustered databases. We see user shares and team shares where you're dealing with office documents, media, files, engineering files, line of business applications for things like financial industry that might be doing like risk modeling, energy companies that are doing seismic and reservoir simulations, media and entertainment companies that are doing video editing, video rendering and special effects, machine learning and AI and simulations um SAS applications which can be born in the cloud and need simple, easy to use shared file storage software, build environments, um backup and disaster recovery use cases um where you need to point a backup, maybe from like a database server or maybe you've got a file share that you need to create secondary or tertiary copies to meet your, your data's your business' recovery point and recovery time objectives and meet your compliance standards, data science and analytics where you're sharing files across multiple data science workstations and applications that need to share multiple protocols. Dev ops platforms where you need to share file storage between different dev ops platforms. As you're developing and testing code before you deploy and you need that simple shared storage. So you don't have to keep copying and duplicating data. All of this is driving file storage.

Um we see it born in the cloud, we see it on prem and customers are liking looking to migrate to the cloud to take advantage of all the the elasticity and agility that the cloud has to offer. But as this data grows at this unprecedented scale, it becomes increasingly challenges or customers tell us it becomes increasingly challenged to predict future growth. When you're purchasing on premises storage, you have to carefully size it and predict growth for the next 3 to 5 years and make sure that you have the right amount of capacity and throughput provision for your users.

It's also challenging to maintain resiliency and security and we talk about resiliency. It includes keeping your application up, making sure it's secure and putting all the infrastructure in place to do that. And resiliency also can mean secondary infrastructure for backups and disaster recovery. Um and oftentimes secondary data centers as well as additional hardware and software to set up and manage, um it becomes hard to optimize storage cost. You have to choose between different tiers of storage. You know, what's the right amount of SSD or HDD? What's the right amount of throughput and performance on my filers or, or file servers that we need to provision for? Um it becomes challenging to gain insights from your data as well. Um so if your data lives in a singular platform and you don't have um the extra things in your environment to be able to gain insights from your data. Um it, it becomes difficult for your company to extract extra value out of it as well.

And so with fully-managed NAS in the cloud and what customers look at when they move file storage to the cloud is something that's fully managed that they're able to take advantage of to get all those benefits of the cloud so that they could go from worrying about that time to set up the time to set up and, and provision all of that infrastructure on premises, the cost associated with not just setting it up and provisioning, but also keeping it running the complexity of keeping it running. And that means keeping your infrastructure going, it means doing patches and updates. It means migrating to a new storage platform and you do a refresh every 3-4 or five years.

So with fully managed NAS in the cloud, you get to go from worrying about that time cost and complexity to focusing on innovation for your business. And what this means is that you're able to extract better value out of your data. And so we have storage administrators, we have application administrators that are able to easily attach file storage in the cloud without just having and focus on more important tasks of their business. And not the menial task of having to do things like replace failed drives on a weekend or dealing with emergency patches and updates.

The benefits of a fully managed file service in the cloud is we take care of a lot of that overhead for you as well as optimizing for that time and cost. However, switching NAS solutions can take weeks, months years, each NAS solution or each file storage solution on prem offers unique data access features. It offers unique performance profiles that you have to fine tune for your company. And performance needs. Oftentimes with the NAS rates, multiple types of departments divisions that have individual performance profiles and moving that you know, requires careful sizing and also unique data management APIs to do things like handle the management of your data that's specific to the software or appliance that you're using and dealing with the data access features and and the uh dealing with snapshots and, and how to keep your data uh protected and all those kinds of things that are unique to the vendor and it requires rearchitecting and retraining. And it's not just NAS solutions, it's are you thinking about even moving to other storage platforms? You know, we'll have customers that look to take advantage of, of different types of storage platforms in the cloud. But it does involve this rearchitecture and customers when they're migrating to the cloud, typically tell us that they want the easiest path to get it into the cloud using the same foundation that they already have been built on over over the years in their on premises environment.

So to help with this, we launched a service or a platform, I should say in 2018 called FSX, FSX stands for File System X and the X stands for your choice of workload. And so what do we mean by your choice? It means the ability to launch run and scale these feature-rich high performance cost effective um and fully managed storage using common third-party file systems that you're already using in your environment.

We have four FSX services available today. FSX for NetApp ONTAP, which is a, a very popular uh uh NAS appliance platform that customers have been using for decades. Um it's a very mature technology FSX for Open CFS, which is an open source NFS. Linux based file service platform that's also been in development for 20 or so years based off the common NFS file protocol, FSX for Windows File Server, built off a Windows, OS and FSX for Luster, which is an open source Linux based platform that offers high performance and and parallel performance for a variety of analytics applications and machine learning and anything where you need really high paralyzed throughput and performance.

So the idea with your choice is you get to choose a file system that most closely resembles either exactly what you're using today or very similar in a set of features and functionality. And you don't have to worry about, you don't have to worry about a different set of features or retraining or rearchitecting. You get to take advantage of existing software that you know, and you're familiar with.

So we're going to focus on three file systems for today's presentation. When we think about on premises, file storage, customers are telling us are typically coming from a NetApp ONTAP environment, maybe some Windows file servers with various types of storage platforms behind it or maybe a self managed Linux or NFS file server with also various types of storage platforms for it. And the idea with these three platforms is you're able to, to leverage the same technologies that you've been using on prem without having to rearchitect, recertify, retrain or re invent the wheel.

We can easily provision a new file system in the cloud that's fully managed without having to worry about managing the underlying infrastructure, the compute the storage, the networking, the backup, the encryption, the security. Um you could just tell us how much capacity you need. Um how much performance you need. We'll take care of the provisioning and we'll keep it up and running for you. Um and we'll upgrade the hardware and the software behind the scenes, it's subtracted from you and you get various types of options about different performance profiles and different types of um capacities. And you're able to fine tune that as you go.

Um the idea is we want to make it as easy as possible to migrate these workloads to AWS with like for like functionality. Um so what does like for like mean there's a lot of words on the slide, I'm not going to read through all of them, but a couple of key themes is that you're able to get uh when we say like for like, and, and what are the same features that you get with your current storage? Um you're able to get a lot of the same storage efficiencies that you're using today. Um things like data duplication and compression. Um being able to take advantage of tier. Uh for example, with our FSX for NetApp product, you're able to take care of all the same data management uh features as well. So with Windows File Server, you get all the PowerShell commands with a NetApp ONTAP, you get all of the same um API to manage ONTAP that you're familiar with um for managing your, your access patterns and um all the administrative features.

Um so data access, you get Active Directory integration, uh multiple protocols with FSX for NetApp ONTAP, including SMB NFS and iSCSI. Um you get things like uh SMB multichannel for data protection. You get to take advantage of snapshots local. So users can do restorage, you get backups that you could keep on S3 for long term backup and recovery. You could do things like antivirus with FSX for NetApp ONS app, you could do file access, auditing with several of our file systems and you could also extend from on premises as well. So it's not always just a migration. We have customers that will point backups and DR to us for secondary copies and shut down that infrastructure. We also have customers taking advantage of bursting into the cloud. So they could take advantage of additional compute to run some sort of jobs faster. And a common example of that is a NetApp ONTAP infrastructure deployment that can leverage a FlexCache feature with NetApp to burst into AWS, migrate the data easily spin up additional compute to be able to run a job faster and various other types of scenarios you might want to burst to the cloud.

Um so here's a couple of customer examples. We'll have a few of these throughout the presentation. I'll cover two right now. Um both of these are actually Australian customers. So if there's any Australian customers in the audience, um we've got John Holland Group, um who's a large um um um manufacturing uh customer uh based out of Australia. And they had uh migrated to AWS, but they were doing self managed file storage um just on EC2 and EBS with, with some Windows file servers. Um this was, I believe before FSX had come out when they set this up. Um and they had a plan once FSX came out, they saw the benefit of being able to use fully managed. Um so they were uh had a plan to migrate to FSX over the course of a year. Unfortunately, there was an outage that caused their application servers to go down and they had estimated that it was going to take 3 to 4 weeks to get this back up and running. So our AWS account teams and ProServe teams were able to work with them to take the backups and restore into FSX for Windows File Server and do this in just a matter of 48 hours. So they're able to get 90 business critical applications up and running very quickly. And so it shows that even though they were planning to do this migration over a year, it's unfortunate that an incident kind of pushes further, but it does show that it was able to because it's like for like, because they didn't have to do any of this rearchitecting. They were able to get this up and running way quicker and get their business critical applications back online. Um very quickly.

Another example is New South Wales Government has eHealth and they had a 1.3 petabytes of medical imaging files stored on an on premises. NetApp, they wanted to migrate to AWS. Um this was shortly when after FSX for NetApp ONTAP was announced about two years ago and they were able to migrate this over petabyte of data and see a 10x improvement and save 16 million in benefits due to avoid a cost of managing their own hardware and infrastructure

So they were able to retire their on premises infrastructure and migrate to AWS, reduce the operational overhead and management of having to manage their on presence infrastructure and still get better performance because they were getting all the benefits of an FSX or NetApp file system running on AWS architect AWS infrastructure.

Um so this is another great example. They're able to just use native NetApp um software to snap mirror to be able to replicate. Um which makes it very easy to migrate from on prem to AWS. Uh a benefit of something like NetApp ONTAP is it appears in your console just like another uh FSX for NetApp ONTAP is that it appears in your NetApp um uh blue XP or console. It's just another NetApp array.

Alright. Um oops, sorry, these are for pros, I'm going to hand it over to Prosch. He's going to talk to you a little bit more about a deep dive in our services and some recently announced features. Thanks so much.

Let's see uh about uh the where we purpose built Amazon FSX to serve as the easiest migration path for your existing network attached storage uh workloads. So we already talked about the like for like storage that we provide with Amazon FSX. But let's have a closer look at FSX in terms of uh the benefits and value it provides beyond serving as the easiest path for migration.

So beyond just that, we provide leading edge performance and scale, which allows you to run highly demanding high performance applications, we provide a broad set of cost optimization options. Um and this allows you to optimize and fine tune your TCO for your application or workload needs because there's a variety, we recognize that there's a variety of workload needs out there um in terms of performance, in terms of features and capabilities and in terms of access patterns and uh usage patterns. And so we really want to enable you to fine tune the total cost of ownership according to your workload needs. And then there's also a whole bunch of infrastructure as code automation support we provide with FSX which allows you to really simplify your deployment architectures and it allows you to improve agility by allowing your teams to self serve their storage needs in a lot of cases as opposed to having a centralized storage administration team being the bottleneck in terms of enabling your teams to have their storage for their workloads. And then we provide a, a variety of different native service integrations within AWS. Um this is really allowing you to leverage the broadest and deepest set of capabilities that we have on AWS as a whole. Um and so this allows you to do more with your data and to innovate on behalf of your business.

So let's dive into um each of these in turn. So what do we mean by leading edge performance and scale? This is talking about beyond just taking the existing performance profiles of these um mature well developed file systems that we talked about earlier and providing those performance profiles in terms of, you know, the near local latency of file operations, the high throughput and high IOPS capabilities, the paralyzed cluster capabilities in the case of FSX for Luster and in the case of the recently launched FSX for ONTAP scale out file systems, which we launched just two days back. In addition to taking advantage of the existing file systems, performance profiles and delivering them to you as is just as if you were using that file system on premises or elsewhere, we actually make the performance and scale even better by building on the latest AWS technologies across compute storage, you know, dis technologies as well as network technologies.

So examples of this are how we build on the Graviton, the AWS Graviton processor for our file server compute instances. Uh how we build on the Nitro technology and the SRD technology to provide really high speed networking with uh consistent low latencies and things like that. And these are just examples of how we are able to deliver continual improvement in price performance and scale by building on the latest AWS technologies, not just leveraging these existing file system technologies.

So here are some examples of what I mean by continual improvement in the last re:Invent and re:Invent 2022 we launched two X higher SSD throughput and IOPS on both FSX for ONTAP and on FSX for Open CFS. And we launched support for NVMe caching in August of this year. During the AWS Storage Day event, we actually launched six X higher through five X higher IOPS and the ability to provision IOPS separately from storage in order to fine tune your TCO on FSX for Windows. And then most recently, of course, just two days back, we launched scale out file systems on FSX for ONTAP which allows you to drive 8 to 9 X higher SSD through and IOPS and five X higher SSD storage scalability on a single file system.

So with this, you can now have up to a petabyte of SSD storage per file system and you can drive up to 36 gigabytes per second of through and this was just launched two days back. So this is what I mean by a continual improvement over time by building on the latest AWS technologies.

So here's a glimpse of how performance and scale look across our service portfolio. Um I won't go through all of these in detail, but suffice it to say that we deliver sub millisecond latencies. We deliver multiple gigabytes per second to a few tens of gigabytes per second of throughput profile system. We deliver anywhere between hundreds of thousands and millions of IOPS per file system. And we deliver storage scalability at the hundreds of terabytes profile system in the scale up mode and in the petabyte scale in the scale out mode.

Um in addition to all of these things on FSX for NetApp ONTAP, we also deliver capacity pool storage, that's virtually unlimited and that's fully elastic.

Here's an example of a customer that was able to leverage the performance and scale that we deliver on our service change. Healthcare deployed hundreds of their SQL Server databases uh by leveraging FSX for Windows file server. And by doing that, they were actually able to get the sustained IOPS performance they needed. But in addition to that, they were able to save $3 million in sequel licensing costs. And the way they did this was by essentially taking an architecture that they used to use premises and then fundamentally changing it because of what we enable on AWS with the unique multi AZ deployment capability and with the ability of using FSX for Windows shared storage in lieu of SAN storage or whatever other solution they were using earlier.

And by doing that, they were actually able to deliver $3 million in sequel licensing costs by being able to leverage SQL Server Standard instead of SQL Server Enterprise where they didn't need Enterprise and so on. They were able to get highly available and highly performance storage while also meeting their RTO RPO requirements.

So next, let's dive into what we mean by cost optimization capabilities. So there's a whole bunch of cost optimization capabilities that the file systems themselves deliver things like data compression, data deduplication, thin provisioned volumes, storage quotas. There's also data compaction, there's a whole bunch of capabilities that these existing file systems already have built and matured and innovated on over the last few decades.

So we continue to deliver those things because of our like for like storage model. But beyond just the file system native capabilities, we actually add a whole bunch of other cost optimization capabilities, what I call as FSX capabilities in this slide.

So for example, we allow you to choose storage type between SSD and HtD so that you can fine tune your TCO for your application needs. We allow you to choose deployment type between Single AZ and Multi AZ across our portfolio. Um you can, you can choose Single AZ for test and development workloads and Multi AZ for production workloads. As an example, we deliver fully elastic backup storage across our portfolio and we deliver automatic data tiering to fully elastic capacity pool storage in the case of a six for NetApp ONTAP.

And we also deliver dynamic scaling of storage and performance. So what I mean by this is first of all, you can configure performance like throughput and IOPS and so on separately from storage. Meaning if you have a workload where your storage is only, let's say a couple of terabytes, like let's take SQL Server as an example, let's say you just have a three terabyte database use case. But for those three terabytes, you actually need hundreds of thousands of IOPS and you need multiple gigabytes per second of throughput. You're actually now free to completely separate these two concerns and have storage be the exact size you want it to be and the performance be the exact levels you want it to be.

So first of all, we allow you to configure them separately so that you can fine tune the costs. And then secondly, we allow you to dynamically scale it and scale them separately as well. You can scale storage up, you can scale throughput up and so on. So that as your application needs your, your user needs, keep changing and growing, you can grow the storage along with it.

Here's an example of what we mean by combining the file system native capabilities with the FSX capabilities of cost optimization.

So on FSX for NetApp, ONTAP, we provide two storage tiers. We provide the SSD storage tier and the capacity pool storage tier with SSD storage, you get sub millisecond latencies and it's optimized for the most frequently accessed data. Your most active data think of it as your active working set on capacity pool storage, we provide fully elastic storage that automatically grows and shrinks as your overall file system grows and shrinks. And it's cost optimized for infrequently access data.

So think of this as the 80% of your file system that's infrequently accessed and think of the SSD as serving the 20% that's frequently accessed. So using this, you're already able to save pretty significantly in terms of costs. If you have 20% of data in SSD and 80% in capacity pool, what you're getting here is the best of both worlds, meaning you get the SSD performance for your active active data or frequently access data and you get the fraction of the cost of SSD for your infrequently access data.

And now if you go ahead and combine that FSX capability with some of the file system, native capabilities like data compression and deduplication and so on, now you're talking about two thirds savings over two thirds savings. So that in a sense adds up to something like 88% cost savings, which basically brings your storage cost all in your TCO to something like three cents per gigabyte month.

Here's an example of a customer that was able to leverage these cost optimization capabilities. Our CM moved their storage over to FSX for NetApp ONTAP and they were able to leverage built in compression compaction deduplication all of the storage efficiency features of NetApp ONTAP and combined with the FSX cost optimization capabilities, they delivered 46% reduction in storage costs. They also had five X improvement in their database speed because of the performance we deliver and overall they had 80% reduction in refresh costs.

So next, let's dive into infrastructure as code automation support that we deliver. So we actually allow you to deploy infrastructure as code with CloudFormation and with a whole bunch of other automation uh support. What this allows you to do is it allows you to standardize and automate your storage deployments. It allows you to automate test and deploy infrastructure with your continuous deployment uh architectures.

We also enable end users to create their own file systems, your teams, your application teams, your user, end user teams can go and now provision file systems on their own and they can use pre-approved templates to do so. And this frees up the teams to actually be self driving their own workload needs rather than coming to a centralized point of friction.

So here's an example of what I mean by that, we see typically two types of five system deployment patterns in AWS. We see centrally owned storage where there's a shared services, VPC and there's a bunch of application VPCs and the storage owners can spend less time managing infrastructure and more time on boarding and supporting their application teams. That's one kind, the other kind we see which is actually very unique to what you can do within AWS as opposed to what you can do on premises is end user owned storage where each of the application VPCs have their own storage that's stood up by the teams themselves.

And here storage owners create repeatable patterns and they create pre-approved templates. And the application teams can leverage those things. They can now scale and test even faster. And this helps improve agility within your organization. It really allows your teams to innovate at a fast pace without being bogged down by centralized IT support.

Here's an example of a company that was actually able to leverage this automation support and deliver tremendous value for their customers. Technology One is the largest enterprise size provider in Australia. Um they delivered something like thousands of single tenant customer database environments, uh SQL Server, database environments um using this kind of uh an automated deployment architecture and they ended up with uh uh uh an architecture that really allowed them to have a globally consistent storage solution.

And what this allows allowed them to do is um basically achieve comprehensive compliance controls and scale in a customer specific way across tens of thousands of database environments uh across their customers.

So um let's let's look at native service integrations. So you can use Amazon FSX file systems from the entire range of compute instances, VMs containers HPC clusters, those sorts of things across AWS. So it's including EC2 VMware Cloud on AWS, Amazon AppStream 2.0 Amazon WorkSpaces AWS SageMaker, EKS Batch, the whole deal.

There's a whole range of databases, media production and intelligent search services that we integrate with natively as well. Most importantly, there's a variety of security log analytics and fraud detection services that we natively integrate with.

For example, you can actually use file access auditing on FSX for Windows file server. The audit logs can be sent over into Amazon Kinesis or into CloudWatch Logs. And I'll go into a little more detail on that. In the next slide, there's a range of data movement and protection services that we integrate with as well including AWS Backup and then last but not the least, there's a range of monitoring visualization and automation services that we integrate with.

So together, what this allows you to do is really do more with your data and innovate on behalf of your customers and dream up new customer experiences and really gain more insights from your data in ways that you couldn't have before.

So let's dive into one example of a native service integration. So on FSX for Windows file server, we support file access, audit logging. And the way you see the audit logs is you choose either Amazon CloudWatch Logs or Amazon Kinesis Data Firehose as the destination for your file access, audit logs.

So whenever an end user or an application is accessing a file or a folder, the fact that that user attempted to read or write or create or delete or whatever else, a file or a folder gets logged as an access audit log. Now, even if that attempt resulted in a failure because they lacked the permissions or whatever else, even that gets logged, even failed attempts get logged.

All of those log entries go into either one of these two. And then from CloudWatch Logs, for example, you can use CloudWatch Logs, Insights to view and query logs. You can trigger CloudWatch alarms or Lambda functions to do interesting things. You can export the logs to S3 for archival and processing. And then from Kinesis Data Firehose, you can actually integrate the logs with AWS partner solutions like Splunk and Datadog.

You can stream the logs to ElasticSearch or to Redshift for further processing of the logs. You can also stream the logs to S3 to trigger real time processing and reporting of these audit logs. So there are lots of new interesting things you can do with your data and with your access logs that you couldn't have done before through these native service integrations.

So having looked at um a little bit of a dive deep of FSx, let's look into um, what's new with our service. Here's a glimpse of all of the launches that we have done. At least the major launches that we have done since Reinvent last year. I won't go through each of these in detail. It's just meant to show you that the pace of innovation is strong within our service and within AWS in general.

And if you just look at Amazon FSx launches alone, you basically are looking at something like one launch every two weeks on average over the year. So that gives you a sense for how we are continually innovating on behalf of our customers.

So on FSx for NetApp ONTAP, we, we launched the scale out capabilities just two days back. And with that, you can now um drive 36 gigabytes per second in place of what you used to be able to drive with four gigabytes per second on a single scale up file system.

You can drive six x higher right throughput, uh where you used to drive one, you can now drive up to six gigabytes per second of right? And then in terms of SSD IOPS, you can drive seven x higher SSD IOPS, you used to be able to drive up to 160 k per file system. You can now drive 1.2 million per file system.

How do we do this? So on today's file systems, meaning the scale up file systems. Um this is how a file system looks. You have a pair of file servers uh that are serving as an HA pair, a high availability pair. Um and you can access it via NFS SMB or iSCSI. In the event that a file server goes down, the secondary file server takes over as the active and the active passive HA pair together provide high availability with the new file system clusters.

The scale out file system clusters that we've launched, we now take that HA pair and we scale it out to up to six HA pairs per file system. And with that, you are able to drive in aggregate anywhere between 2 to 6 HA pairs in aggregate, you're able to drive 36 gigabytes per second, read six gigabytes per second, right throughput.

And then uh the other launch that I'd like to talk about in a little more detail is uh on FSx for Open ZFS, we launched on demand data replication across file systems. So we actually launched uh this capability just two days back as well.

With this, what you can do is on a single file system, you can take uh you can have a volume that you can take multiple snapshots of. And then with the new feature you can now have on demand data replication of that volume into a replica file system.

So with this, uh the service will take care of calculating the incremental data that needs to be replicated. All of the data is replicated at the block level. And so it takes a point in time, consistent snapshot and it takes incremental data at the block level since the last application and it transfers over just those bits of data to give you the most efficient replication in terms of both time and space.

And so with this feature, we are talking about um the capability being powered by ZFS and receive capability which is a native file system capability of Open ZFS. It provides an easy to use, fully managed and resilient replication mechanism.

So the service takes care of things like actually establishing the network connectivity, maintaining the network connectivity, detecting intermittent failures and recovering from them, resuming transfer where needed. It also takes care of things like data encryption address in transit and it takes care of things like access control of who can do what in terms of replication of data.

You can use this for a variety of use cases like synchronization of your development environments with your production environments, you can use it to set up read replicas to provide scale out read performance. You can use it to have a whole bunch of data experimentation workflows where you want to have a control experiment where you want to spawn a bunch of experimental data sets and and do the experiment and pick the winner and replicate back the winner into the production environment.

So next, let's dive into how a customer has actually been benefiting from our service for that. I'd like to invite back David Stein and also our guest for today, Taylor Savage.

All right, great. Uh thanks. So, hopefully that was helpful. Um I'm gonna go through uh interview here with Taylor and then we'll be around for questions afterwards. Uh we'll wrap up after this.

Um so Taylor, uh if you could please introduce yourself, my name is Taylor Savage. Uh I work for Phillips 66 downstream energy company in energy space. And uh I, I work on the team that manages our global storage and backup infrastructure. So specifically, my role is primarily in our NetApp space. I manage our both on prem NetApp environment as well as our, our cloud environment as well. So been doing that for about 11 years.

Great. Awesome, a lot of NetApp file storage. So I understand Phillips 66 has been going through kind of a cloud migration journey over the years. Can you tell us a little bit about that journey and how file storage fits in with it?

That's right. Yeah, I think it was probably six or seven years ago. We started our cloud migration in earnest and at the time, there were just some various properties of the energy market that caused it, but essentially we were going through like a lot of other companies were at the time a digital transformation. And so kind of the push for leader from leadership was to be a cloud first company, a cloud native company. And to move all of our workloads that we were running on prem to AWS. And the bigger ask was to at least at, at the start rather than refactoring our entire environment. They wanted us to figure out a way to take the workloads we had on prem and provide the same services in the cloud that we were providing on prem and to be able to make that migration as non disruptive as possible.

So basically trying to figure out what services existed in the cloud. That could be a one for one map for what we had on prem. So in the storage space, particularly, that was difficult for us because at the time, this was like 2016, 2017, we did not have a solution in AWS that was providing NetApp services. The exact same way we were doing on prem.

Um so we're a leverage NetApp customer. Basically all of our file storage runs on NetApp storage as well as our DR environment, all the provisioning. We do all the management, we do runs in the NetApp ecosystem. So we have all sorts of workloads in NetApp. We've anything a normal corporation, we have anything from regular team file shares or PowerPoint drives all the way to our research center has instrumentation applications our refinery has drone images and various turnaround documents that they store on NetApp.

So just like I said, a general leverage NetApp infrastructure. And so as we were looking to go to the cloud again, we were struggling because we didn't necessarily want to move our NFS workloads to EFS and our, our SMB workloads to uh Windows FSx, we would have had to again change the way we do provisioning, change the way we do permission and we just didn't have that time and uh support to be able to do that.

And so because FSx for ONTAP didn't exist at the time, the solution we went with was some folks may know, but NetApp has a Cloud Volumes ONTAP solution, which is great. It provided exactly what we needed because it's basically ONTAP OS running on top of two hardware uh virtualized hardware. And so it provided some of the things we needed at the time to enable that migration without as much as, as little disruption as possible. But there were some challenges with CVO that we've, we've experienced. And once FSx for ONTAP was announced, we were all really excited because it provided those exact same requirements that we had. It was literally a one for one mapping for what we were doing on prem.

We didn't have to retrain our guys. We didn't have to buy new hardware or refactor the way we were doing storage. So we were excited when ONTAP was announced have been working with it for a few years.

OK. Nice. Yeah. So a good range of use cases. It sounds like, you know, general purpose. You've had probably some, some mission critical files in there that were really important to the business.

S a our entire probably should have mentioned that our entire ERP environment runs on accounting and, and all that sort of stuff runs on NetApp as well. So it's pretty critical for us to enable that migration.

Yeah, great, great. Yeah, NetApp has been great because they've had their software in the marketplace for a long time. They've been very cloud forward thinking and FSx is a great natural extension where it's fully managed license is included, so less overhead to set up and manage. So it's great. You're able to make that easy, easy migration over what?

And I also understand you've got some other on premises infrastructure since then. You've had some other on premises infrastructure, primary and DR that you migrated over. Can you talk to us a little bit about that?

Right. So when FX ONTAP came out, it started getting certified by all the various vendors. We started to look at what's the easiest way to begin utilizing this service. And primarily we were looking at converting everything we were running in CVO over to FSx ONTAP as well as everything we still had on prem that had yet to be migrated. We want everything to be in FSx ONTAP where possible for those workloads that are running in AWS.

And so as we were kind of discussing, what are the easiest things to do? The, the lowest hanging fruit for us was our DR environment. So again, I mentioned we're, we're entirely NetApp ecosystem in terms of our file server DR environment. So today, we do all of our DR from our global refineries, we snap mirror back to uh to our BART's, Oklahoma and Houston corporate locations.

So we have refineries, you know, on the gulf coast, they run on VMware, NFS provided data stores from NetApp on VMware. We take those data stores replicate them back via snap mirror to our DR location. If a hurricane is coming for the coast, we stand up those data stores in our DR location and run on kind of warm virtual infrastructure there.

So as we're looking at converting that environment to ONTAP, we thought that would be the easiest environment because of the way that FSx ONTAP works, we can enable snap mirror from our on prem devices up to FSx ONTAP kind of the same way

Person 1: And basically, instead of pointing our data stores to our DR location on-prem, we literally just repointing them to an FSx ONTAP instance. We let that data migrate, it was several 100 terabytes. And once that was done, we just essentially turned off the old SnapMirror replication. And now we were fully cloud native you know, in the cloud for our DR environment.

And because we don't have, we're not provisioning our DR storage as production storage, there was no impact to, to any user. So that was kind of the first easy use case for us was to get our entire DR environment going to FSx ONTAP. Nice, easy way to get rid of like secondary infrastructure. So all those offices go straight to FSx, NetApp where they go to a central location that's then replicated.

No, they go straight to FSx ONTAP. So we've got, you know, Montana California, Louisiana. It all go straight to a single FSx ONTAP cluster running in our production environment.

And then the other big piece we, so we've continued to work on moving, obviously, not only our DR but the rest of our on-prem and our cloud CVO environment onto FSx ONTAP. So we've been working that, but in the meantime, this summer, we had another incident where we acquired a midstream energy company out of Denver and they luckily were a NetApp customer as well.

So they ran on a single physical NetApp instance in their Denver data center. And when we learned, we were requiring that company, we were told we need to get this data migrated. We need to shut down their data center basically is what we were asked to do and that was going to be done by the end of the summer.

So we had a couple of months to figure out what are we going to do with this data. And because we already have this effort to move our on-tap data, our, our, our CVO data and our ONT data at FSx ONTAP. And we're a cloud first company. And we're like, well, let's just move it to ONTAP primarily.

So even though we only had a few production workloads running in FSx ONTAP at the time, we just decided, let's just go all in. And so it was about 132 terabytes of of NetApp data. And again, just kind of general critical workloads team shares, it was ran the whole gamut of data types. And so it was about 100 and 32 terabytes of data.

We did kind of a lot of the same way we did it in DR we just snap mirror all those volumes up to an FSx ONTAP production instance. And it took about a week I think, to move the data, it probably could have gone quicker because I kind of slow rolled it. But it took about a week for me to move the data.

Once the data was moved, we took an outage weekend. It was a Saturday night and we just literally cut all the snap mirrors off, you know, ran a final update, cut all the snap mirrors off and then changed over the DNS names the file pads and that was it we were done.

So we were able to get that entire 132 terabytes of, of our storage environment for that company acquisition moved in a couple of weeks and we supported that migration. Now, of course, that that effort is still going on. But at least from the storage perspective, we're, we're able to support the migration of that hardware and decommissioning of that data center.

You make it sound pretty quick and easy. Hopefully, it was, it was for that workload. When you're, when you're just snap mirroring, that's what's great about ONTAP is it can just receive those snap mirrors. And so when you have workloads where you're relying on snap, it's easy to just cut that over. So that was what was very useful for us.

Yeah. And even when we met yesterday, I think you're saying that you even had another thing this past weekend where you had to migrate some things over pretty quickly as well. We were just talking about that right before we walked up. But there was an incident on Monday night where I was getting ready to come here and we had an incident at a research center, our scientists, their instrument monitoring application went down and there was a Windows patch, I think that was pushed out and it disabled some NTLM authentication.

And we had some issues with this particular workload. They couldn't access their NetApp storage. And so I was talking to my supervisor because I was kind of worried about it because I was trying to come here and, and he said, yeah, let's just move it up to FSx just like the rest of this other production workload we were working on.

And so we did, we just pointed, I think it was like 330,000 files. It took me about an hour to move and uh I was like trying to get my kids dinner ready and migrate this data at the same time. But yeah, it was great. We, we moved it in about an hour and I called the guy and said, hey, I've switched over your DFS links and, and that was done.

So it was it enabled that, you know, that particular migration fairly, fairly easily. So that's great. Hopefully the kids got a good dinner. Cool. And so I, I like the story, right? It's easy and it's kind of some of the echoes, some of the things that me and Prashanth were talking about around, you know, like for like migration and migration.

What now that you've been running for a little while, like, have you seen any other benefits? Like what's the experience been like? And what benefits have you seen?

Person 2: Yeah. Yeah, that's a good question. I probably should have mentioned most of that earlier. So I mentioned that there are some challenges with CVO and CVO is a great solution really happy with what NetApp has done there and, and because they provided something that we really needed at the time, but the problem with whether it's CVO or on-prem is of course you're managing licensing, you're managing expansion, especially with CVO storage expansion is pretty, it can be difficult if you have really large workloads.

Again, I mentioned your managed licensing. There's just some, there's just some challenges that make those environments a little bit harder to, to manage. And so with FSx ONTAP, I mentioned earlier, we don't have to retrain our guys to whether it's provisioning or DR with SnapMirror. It's just, it's the exact same OS that we log into.

From our perspective, it's no different. We just open a Putty session and log into ONTAP and it looks no different than an on-prem or C system. So another big thing for us is the capacity tier talking about it earlier. That's been really awesome. We hadn't really utilized that feature with our on-prem environment. It's a little bit more difficult to set up.

But with ONTAP, it's just in the console, you enable it at, at the time you provision the instance. And so for our DR environment, like I said, it was a couple two or 300 terabytes on one of them and it had moved 90% to capacity tier. So it was quite a cost reduction in terms of our DR storage.

And then with our production workloads, I can't remember the last time I looked, you know, it was still moving data but it had gotten to 50% of data into capacity tier before it had finished. So uh that that's another big benefit for us.

Um so those are some of the things that cost optimization less kind of. Yeah, we, we did the math because we were curious about what the cost differences would be between CVO with even with the licensing and with ONTAP. And I think the the rough kind of napkin math we did was it would be at least 80% of the cost assuming some of the workloads migrate it to capacity tier the way we wanted to. So it's definitely it's definitely cheaper and easier to manage.

Um it's again, it's a native workload which is a requirement for us in terms of our cloud first approach upgrades. Patching is huge for us. Patching is a whole process as I'm sure most people are aware, especially on the NetApps with the HA. It takes a lot of uh change control and communicating to other teams when we do these upgrades.

And so to be able to have essentially a managed upgrade process for us has been, has been huge, that's been a huge benefit. So those are just some of the kind of major level things that have been good for us. Maybe do some of the more menial things that you're having to work on in the past. So you can kind of focus on assuming some more important things for the business capacity expansion is also a literally a drop down and you just change the number and you're done, you've got more capacity. So that's a big one as well.

Cool. Awesome. So what's next? Anything else coming down the pipe?

Person 1: Yeah, I think our biggest effort right now is probably in our VMware cloud infrastructure. We started moving, I say we are, our VMware team started moving workloads in the VMware cloud this past year. And we're looking at we started to test it. We have it in some of our European environments, but we're starting to look at utilizing FSx ONTAP for VMC storage to kind of replicate the way the data stores or for for everything really for the data STS FSx ONTAP is the data storage for the VMware cloud workload servers.

Uh just to kind of replicate the way that we do DR for our refineries today where we have, you know, a NetApp provided data store running on VMware. We can kind of replicate that and be able to DR entire, you know, refinery sites into the cloud. So we're excited about that.

Um we've also been having a lot of discussions especially in the refining space about security, some of the various security issues that have been happening. So again, there's a lot of discussions around SnapLock and some of the other features that increase security that, that a lot of vendors but especially NetApp is starting to support.

That's great. Yeah, VM data storage is something we're starting to hear a lot. You know, it was certified VMware last year and it helps isolate the storage independent of the compute. So you could size your VMware instances more appropriately and then scale your storage as you grow. So that's great to hear.

And SnapLock was something that we supported mid sometime this summer, which allows a lot of companies to meet some compliance needs with creating immutable objects too. So that's great to hear. Awesome.

Well, it's super exciting to hear about the cloud journey. We um love hearing, you know, the success stories and, and it's been great to see Phillip 66 on this journey over the last, you know, year or so. Um thanks for taking the time and coming out. We really appreciate it.