What’s new with Amazon EC2

Thank you everyone for joining us for uh what's new with amazon ec2. It's always great being back in Las Vegas for re:Invent. I just love this week as we have a ton of innovative announcements. It's great to talk to customers. My name is Martin Beeby. I lead Amazon EC2 and networking services. I'm joined by:

Hi, I'm Mark B. I'm a principal product marketing manager. I'm excited to be here today to talk to you about some of the latest innovations that we have in Amazon EC2.

Martin: Yes. So this is my sixth re:Invent and I'm always, it amazes me how much innovation comes out this week. Um and just the, the amount of, you know, the growth in this conference and the, the growth in our, our products over the years. Um again, this is Amazon EC2. And if you watch the keynote this morning, you probably heard Adam in the keynote talk a lot about reinvention, reinventing storage, reinventing uh chips, reinventing cloud. If you think about cloud computing itself, it really was a reinvention of compute and the way that customers um utilized compute uh before uh maybe a lot of you had data centers in your buildings where you had to procure and configure and set up uh compute yourself. Um cloud computing really is a reimagining of how compute can be used. Um now customers can just access compute via web services and uh get on demand access to it at a low cost.

So before I go into what's new with Amazon EC2, I always like to take a step back and think about where we came from. This is a snapshot of the blog when we launched our very first EC2 instance back in August of 2006. This is the uh bar the blog from our chief evangelist Jeff Barr from his uh pool side in Cabo San Lucas, where we launched what's now called the m1.small. Um back then, this was a really innovative instance. It was a 1.7 GHz Xeon processor with 1.75 gigabytes of RAM. It doesn't sound like much now, but really, it was really amazing back then and back then, customers paid 10 cents per clock hour to, to use this thing.

Now, you know, when we built Amazon EC2, we thought customers want three things, right? They would want to um access, compute on demand so they could spin up and spin down um compute resources as they needed. They wanted to pay for only what they used and they would only need one instance. Well, we were certainly wrong about that last point. From the very first instance that we launched. Uh we have certainly grown. Uh today, we have launched over 30 billion instances that equates to over 100 million instances launched every single day that speaks wonders about not just the innovation in our uh EC2 platform and technology, but also our operation excellence in the amount of work that it requires to be able to scale. Um this seamlessly, it took us about 2.5 years to reached the one millionth instance being launched. And now we launch over 100 million instances every day, just tremendous growth over time.

So this is one thing. But really, you know, when you think about Amazon EC2, people think about the reliability security and availability of it. And when I talk about availability, it's really about our global availability, right? 32 regions with more coming soon. Um and when we think about our regions, it's not just one data center in a single country. It's really um every region has at least three availability zones and each availability zone has multiple data centers um associated with it. These data centers of these availability zones are miles apart. They have redundancy in power and water and network and connectivity so that if anything goes wrong, your applications and your business doesn't go down, right?

So if you had a uh a region or availability zone, all in the same building and there was an earthquake for some companies that would mean that that region would go down for us. It doesn't because we have um our data centers planned out and mapped out miles apart with this redundancy built in. And then it's not just about regions, but we also through talking to customers, we we understood the need of customers. We understand we don't have regions everywhere. So and in certain cases where workloads need to be closer to the actual data center for latency reasons or customers just want a faster access to the actual infrastructure. We also have something called avail uh local zones. And we, since our launch, we have offered 35 local zones and we continue to add more and more local zones uh with each chasing year.

So from the very beginning, we had two big principles, right? The first was to provide customers with the tools and services to securely and reliably run virtually any workload. And the second was to innovate continuously to increase performance while lowering overall cost. You know, we've kept that those two principles over the years. And today from that very first m1.small instance i talked about, we have over 750 instance types for virtue le workload. And we're gonna go into some of these instance types um throughout the presentation, but 750 just think about that. That's, that's quite a bit.

Now, you might think 750. That's a lot. How do i choose or how do i know which instance is? Right for me. Well, there, there's a couple of ways, let me first uh talk about the, the nomenclature of how to understand the instances, right? This each instance has varying ratios of compute and memory and storage and network. Um the first two letters or, you know, for in this example, m5 stands for the instance family uh five is just the fifth generation of the general purpose instance family. The small letter after the uh first two letters is the processor. So in this case, this is an intel processor, but we also have processors from AMD and processors, our own Graviton processor as well. And then the last letter is the option, whether it's something like local attached storage or enhanced networking or faster EBS. And then finally, last is the what we call the t-shirt size.

So as i said before, our instances are offered in different ratios. So if, if it's an m5.small, that's a 4 to 1 memory to vCPU ratio. So in xlarge, it's just a scaling of that 4 to 1 ratio. Now art later will talk about a new service that we have, we, we had offered um part of our, our two family that would um help you pick the instance based on questions that you asked the, the, the the system like perhaps what is the best instance for HPC workloads or something like that? We also have another service called Compute Optimizer that uses your historical usage to help identify and help you optimize what instances are best for, for you. But we'll save that for art.

So part of our success over the years is not just our innovation but our innovation with our partners. I want to invite Art on stage to talk a little bit about that.

Art: All right, thanks Martin. And once again, thanks everybody for joining us here today. So, on this slide, I just want to cover that. Amazon is the only company, the only cloud provider that actually offers all of these operated, all of these, excuse me, chip partners here. So we have x86 and that comprises our uh instances from Intel and AMD. We have ARM based instances that are designed here by Amazon and I'll talk a little bit about those. Those are our Graviton family of compute instances. And finally, we have Mac instances which are also ARM based instances actually as well. And I'll talk a little bit about what we've offered in the Mac space as well.

And then thanks Martin for the teaser. We'll get to a little bit, I'll try to do a little demo of some of the features that were launched this morning, uh the later on in this uh presentation here. But let's start with our Intel based instances and what we've been doing in that space. I think one of the things that we talked about, you know, a lot of us have heard, I'll say Gen AI first here, uh today we talked a lot about Gen AI stuff, but I want to talk, we're gonna focus here initially on compute and what we have offered and what we can offer.

So, Intel has been a partner since we launched Amazon EC2 back in 2006. So 17 years of partnership and as a matter of fact of those 750 instances that Martin talked about a full 400 of those instances are Intel based. And the image that everyone is taking a picture of right now is the latest set of processors that we have on the Intel side here. I've particularly focused. If you look at the right hand side of the screen here on the seventh generation processors, our seventh generation Intel processors are based upon Intel Sapphire Rapids or fourth generation Xeon based processors. These are actually custom to AWS and the performance of these processors is roughly 15% better than you can get on any other cloud process provider running Sapphire Rapids at this time.

So this is the beginning of my discussion about that seventh generation here. And let me give you a quick customer example. This came into me just a couple of weeks here prior to re:Invent. So this is from Tufts Medicine. They started migrating over from the sixth generation instances here and excuse me, from the fifth generation instances. So m5n actually over to the seventh generation here, they've been able to, because of the, the performance here available on the n the seventh generation instances, they've been able to increase the amount of density that they used, for instance, by over 68% that additional 68% has resulted in a savings and annual projected savings of about $1.2 million simply by migrating from that fifth generation over to the current seventh generation instances.

So I think it's a great example of how migrating from one family, an earlier generation to the current generation can result in additional cost savings for you. Here is within the x86 family. I'm gonna talk, explore a couple of the unique instances that we launched within the ex within the um in within the seventh generation family here first.

So we all launched a unique seventh generation instance called our Flex instance. Excuse me, the Flex instances have been enabled by our AWS Nitro system, something I'm going to define and talk to you all a little bit about as we get down uh to later in the discussion here. But the Flex generations, the Flex instances is probably one of the easiest ways to get started using the seventh generation instances without trading off on performance. And these instances offer 19% more price per performance than our m6i generation. So moving from m5 to upgrading to the Flex instances will give you 19% better performance. Excuse me and some of our top customers have already migrated to the Flex instances.

Now, excuse me, also within the seventh generation family and announced actually during our on Sunday this week is our one of our most recent previews here. This is also off of Sapphire Rapids. This is our u7i instance family, the u7i instances use the fourth generation Intel Xeon processor using Sapphire Rapids. And they're here, they're based for scaling up in databases. So large databases in the high memory instance size.

Thank you. And if we look at that these instances offer up to 32 terabytes. So 30% more memory than our existing u1"

So u one is uh the current family and u seven i is what we're going to just to um we're changing, we changed the naming um between the original u launch here we but what this instance has in addition to 32 terabytes of memory, we also have increase the amount of ebs bandwidth by 2.5 times to 100 gigabits per second. And this instance supports io two block express and it is going to receive over 125% better performance than the prior u one generation. So i encourage anybody who has large memory uh workloads to take a look at the u seven i preview that's now launched and is out there.

So this isn't the only part of our portfolio. So i've talked here a little bit about our first x 86 provider. Let me talk a little bit more about our a md instances. So am d we started our work with ad in 2018 and aws was the first major cloud provider to offer amd based instances. Within the last two years, we've offered two updated generations of instances in 2022. When martin and i were here, i was talking about the milan based instances in the sixth generation family. And this year, i'm proud to announce that we have launched 1/7 generation in genoa. If you actually take a look and compare the m six a instances over to the seventh generation family, there is an incredible performance improvement that's out there. But i'll just call a mind here to the current generations that are out here. We have a complete portfolio on the a md side, which includes a um gp u based instance, which is the g four ad instance. We have a high performance compute instance and we'll talk about that high performance compute a little bit later, martin will give us some information about that and the full mc and r family on the seventh generation side.

So a little bit more about that seventh generation family, specifically the m instances here, these are genoa based for those who have followed some of the md work and going from that m six a to an m seven a delivers a full 50% more performance. So within going from 2022 to 2023 we offered 50% more performance here. For these instances. We are cm in our instances offer twice as much memory bandwidth in the seventh generation as we did on the sixth generation as well. And they offer better performance than comparable amd instances in the cloud.

So i talked here about our x 86 portfolio first. I'd like to talk a little bit about our arm based portfolio starting with our graviton based instances we've now launched. So in 2018, we launched our first uh graviton based instance. And since then, we've actually launched four generations. Well, getting ahead of myself here because the fourth generation was announced this morning by adam and I'll talk a little bit about that fourth generation as we talk here. These are pictures of that first through fourth generation here only because i'm a chip guy and i couldn't resist having pictures of chips on a slide somewhere. Um these are the, those are the real chips first. We launched graviton two that, that's out there. This was in 2020 our 2020 graviton two instances delivered 40% better price per performance versus the prior instances. Today, there are over 12 different graviton instances based off of that graviton two family and there's today, more than 50,000 aws customers that are using graviton.

So what represents our current graviton family? We launched our graviton three processors, which is also a full complemented graviton family. Here, we have you see on the right side, the cm and r instances comparable to the cm and r instances i talked about on the intel side. This is the seventh generation. So graviton three corresponds to that seventh generation of instances as well. And of our customer base, 100 out of our top 100 customers leverage graviton and over 50,000 customers are using graviton on a daily basis.

A lot of people though ask martin and i, ok. So graviton is a different architecture than the x 86. Maybe i have x 86 in my data center. Maybe i'm interested in saving something and migrating over. I'd like some assistance. Maybe exactly what do you support. So on the left hand side of the screen, we have a full complement of the linux operating systems that are supported here. Everything from ubuntu to amazon linux. Well, all of the latest generations in the center here, we make it easy. We support most of uh containers based applications including for those of you who are interested in using a serverless solution. You can use one of our more managed solutions here. We have the a ks or cs and then finally integrated this with all of our, many of our services here at amazon.

And the other thing is, you know, four or five years ago, if i got on this stage and talked about efficiency, i talked to you about the efficiency and how good i am at designing architectures on cpus and stuff like that. But since then, everyone is also concerned about power, sustainability. A lot of us and companies have sustainability goals. One amazing thing about the architecture of graviton is is we are it is able to use 60% less energy than that prior generation. And that is derived from efficient, efficient architecture of the system. Now, customers such as zendesk have actually used aws graviton and they've seen a 50% reduction in their monthly carbon emissions.

Now, one other thing about efficiency of architecture design, i'm gonna talk a little bit about efficiency. Efficiency also derives performance as well. And my efficiently using the architecture that i have allows me to deliver better performance for customers. And that's one of the things that we deliver here with our graviton based architecture.

The other thing is, is is efficiency also shows up in the design on the data center side, you know, dave brown, who is the vice president of compute and networking and amazon, all he says to me is this, you know, the best type of energy is energy that you didn't actually use. And that means, you know, energy and power to power, the data center as well uh from a sustainability perspective. So that's one of the things that we've been able to do with the architecting here on graviton.

So one other important thing, i don't know if anybody out there has used or leverages sap or sap hana database. But earlier this month, sap announced that they are ho the data running on aws graviton three asap excuse me has seen a 30% better compute performance and 50% higher memory using this. And by choosing to run sap hana in the cloud, they are seeing a 45% reduction to their carbon footprint. So really impressive here with the reduction in that but also the performance. And just on a personal note, um a lot of people who use sap hanna had asked me questions about it. How can i use it on graviton? So i think you know that now that sap is directly running it here, i think it's a great positioning for anybody who is interested in running sap hana here.

So just announced this morning was graviton four. Graviton four is our most powerful and energy efficient chip that we have built thus far. And when it comes to silicon design, we have found ourselves taking a unique approach that i have characterized earlier. One of the things. So what does graviton four deliver here? So we offer up to 50% more cores than the prior generation. Graviton four is also the first processor that we have built that handles multi socket coherency and it can scale our instances up to 33 times more, excuse me, and three times more memory than our prior generation. So that can deliver for high performance databases, memory cache and big analytic databases that can be used here.

And that's not all with graviton four. We've also heard everyone has questions about security. So to validate real world workloads, we're also trying to raise the bar by unsecured by validating monitoring and protecting every stage of our host boot process. I'll talk more about how the nitro system helps with security and delivers a secure message to everyone. But this is validating what feedback that we have heard from customers. We have fully encrypted the high speed hardware of graviton instances here with graviton four also announced this morning was the preview for our eighth generation. This is our 1st 8th generation instance and i encourage everyone to take a look at ragrag is a memory optimized instance.

So rm is general purpose r is memory optimized and c is compute optimized. So we're starting off the graviton family here. This time with the rag, we get over 30% better compute performance compared to the r seven g family. And i mentioned some of the other statistics that are here. So sign up for the preview, uh you can actually just google graviton four or rag and the preview link will pop up i did it before the meeting before here. So i know it works. Otherwise i was gonna put a qr code in here.

But finally, our last arm based instances are mac instances with over 34 million developers that are out there. Mac is a large community of people that have u are using it. We have been one of the first, we are the only cloud provider actually to offer mac instances. And this year we expanded that to offer the mac two mini and the mac two pro mini. These have provided a more than 35% faster performance compared with our m one mac instances that were launched. prior customers are now able to achieve faster build times without having to install or manage that infrastructure.

Here, here are a sampling of some of the customers that are out there. So customers such as pinterest, goldman, sachs, ry gaines, all probably familiar names to those of you in this room here. They have achieved up to a four x improvement in performance by migrating their instances to the cloud and an over three x increase in parallel workloads. And it has resulted in up to 80% reduction in build failures simply by using the m instances here in the cloud. So it saves a lot of resources and time.

These aren't the only instances that we're talking about here. I'm going to turn it back to martin to talk about some of these higher performance workloads that are out there, martin.

Thanks art. Whereas a lot of the uh computing instances that are mentioned are more for generalized computing instances. There's another class of instances called high scale instances. And some of these high scale instances are some of the ones that were the most difficult to bring the cloud workloads. Like hbc. Some customers thought would never come to the cloud because it's so it requires such high performing infrastructure.

If you've heard of hpc, um it's really all around us. It's used every day from things like computational fluid dynamics to drug discovery, to the genomics, to um electronic design automation. It really is part of all of our lives even if you haven't heard of it. And we have a entire portfolio of ec2 instances um again from our partners a md and intel um as well as our own graviton based instances

Art had mentioned the seventh generation instances earlier and this is really just a continuation of that seventh generation instances.

Uh we have um uh worked on these instances over the years and really we've them for the different types of hpc workloads out there, whether it's compute intensive workloads or memory intensive workloads or data intensive hpc workloads. And our work has been truly recognized.

We've over the years, won several awards and been recognized by hpc wire this year. Again for the sixth year running, we recognize it as the best hpc platform. Um in addition, to that. We've won several other awards for specific use cases and customer choice awards as well. So really proud of our h bc portfolio.

Another high scale workload you might have heard of this year is generative. Now, you've heard a lot about generative a i this morning for sure. But more and more companies are using um generative a i and machine learning to really change the way the business operates to be, make their employees more a more productive, to provide better customer experiences and to really optimize what they're doing.

Now, when we think about um uh generative a i, there's a machine learning, there's really two types of workloads that, that we focus on here. The first is training, which is the type of workload where you're building and refining your, your model. And then the second type of workload is inference which is actually deploying those models and putting it in production. I'll talk a little bit more about this.

So we have uh portfolio uh for training as well as inference. We partner with um many partners from nvidia uh for our gp u based instances, intel haron for um for training as well. Um we, we, we recently announced uh partnership with qualcomm as well to bring their a i accelerators to, to the cloud. But we also build our own uh chips trainum and inferential that i'll talk about in a bit.

But first, let me talk about uh intel, i'm sorry, uh talk about nvidia, you know, machine learning and generative. i has seen a tremendous growth over the years. If you look back to uh about a decade ago, uh models were relatively small. Uh for example, in 2013, the models towards 2012, the model size was about 60 million parameters that might seem big. But today's to today's standards, compared to today's standards, that's relatively small and you didn't actually need that much compute to handle those um uh those model sizes. In fact, you could get by with only two gp us today.

However, model sizes have grown orders of magnitude. And today we're seeing models in the hundreds of billions if not trillions of parameters in size. And customers are wanting clusters of up to tens of thousands of gp us. Um so we've worked closely with nvidia over the years to develop a whole portfolio of gp u based instances. The most recent one that we delivered earlier this year was amazon ec2 p five instances.

Now, these instances are really powerful. They're based off of their uh h 100 gp us, which at the time was the most powerful gp us out there. Um they, they are powered by eight nvidia h 100 sensor core processors with up to four x higher performance and 40% lower cost over previous generation. They're highly scalable, right? You can um scale them up to 20,000 gp us with our uh ultra cluster technology which results to 20 exa flops of aggregate compute performance. That is just amazing. That's really like a supercomputer in the cloud.

In addition to that, we link everything together with our e fa network to give it really fast, 3200 gigabit per second, uh high speed networking. So really amazing there. But this is just one example you might have heard this morning that we are deepening our partnership with nvidia and really delivering um making aws the best place to run gp us.

So part of the announcement, the deepened partnership this morning when jensen came on stage and jensen wong ceo and founder of nvidia um came on stage. We are announcing um more gp us and new newer nvidia technologies coming to the aws cloud. This includes um the l four, the l 40 s and the h 200 gp us that will really help with um advanced graphics video uh and uh a iml and generative a i workloads.

But we're also working on building the first um uh cluster with the gh 200 which is the grace hopper architecture. No cloud provider has done this yet. Uh if you're not familiar with grace hopper is a combination, uh it's really a super chip, it combines grace wishes. nvidia's arm based cpu architecture with their hopper um gp u architecture. So it's really a super chip and we're working to bring that to aws to really give our customers a super powerful cluster.

But beyond that, we are also bringing dg x cloud to us and we're working with nvidia to build what we call project sa which is going to be the world's largest and fastest a i gp u based supercomputer that will really help customers including nvidia themselves, build and train their models and run experiments in the cloud. So really exciting announcement this morning and you know, those are all coming in um 2024. Uh so we look forward to um de delivering those and seeing what our customers as well as invita can do with that.

So, really exciting there. Now gp us are super powerful and they're super popular, but they're really hard to get um in the sense that uh because everyone uh wants to do a iml generative a i, they're um grabbing as many, they're, they're trying to acquire as many gp us as possible. And this is true even when they don't need all the time, right?

So, uh for example, if you were going to uh need gp us for a month, but then you, you might be worried that you don't, you're not going to get access to it. You, you might just grab it for all the times even when you're not using it. Um earlier this year. Actually, just a couple of weeks ago, we announced a new service called amazon e ec2 capacity blocks for me. And this is basically a reservation systems that help cause customers get access to gp u instances.

Um the usage is very straightforward. It's very much like when you book a hotel or a rental car, uh you just have to specify the start date that you need those gp us, the duration of how long you need those gp us and the amount of capacity that you need. And it really uh provides customers with reliable and predictable access to the gp u so that they can run the short duration gp u um uh training and experiments when they need it.

Um and, and that predictability really helps our customers and really helps us democratize access to these gp us again here, just an example, if you needed uh gp us starting on um december 11th and you needed it for a five day duration, you would specify start on december 11th for five days and then the amount of gp us that you need so really simple and straightforward. And um so far, we've heard great feedback from our customers and you know, the ability to get gp us really is gonna change how, how customers uh approach a iml.

But given the growth in machine learning that i talked about, you know, gp us are, are powerful. Um but customers always want uh want more and they're, they're also looking at cost with the uh sizing and complexities of a iml models and general a i models growing orders of magnitude, the cost of compute is actually increasing as well.

So earlier this year, we announced our training platform, uh aws trainum and uh trn one and trn one n. These are uh you know, these provide customers with lower training costs up to 50% cost to train savings. Um they are deployed in ultra clusters so you can deploy them in up to 64,000 clusters of 64,000 um training and chips. And they offer up to 1600 gigabit per second of e fa version to networking, bandwidth for really fast um communication together.

And they're supported by the most popular machine learning train uh frameworks out there, including things like pytorch and tensorflow and coming soon uh jacks. So um those support of those um popular uh frameworks really help customers get started quickly and easily.

But of course, we didn't stop there this morning. We also announced train m two coming soon. These are our second generation purpose built uh chips for generative a i and uh ml training. And these are even more performant than our first generation 14 x faster than training one. Um they are gonna be deployed in even larger size ultra clusters, ultra clusters of up to 100,000 trainum two chips.

Uh and they're really ideal for really large scale generator i models with hundreds of billions of parameters uh in, in size. So again, really for the those large larger models and when you need that super performance and of course, just like our trainum chips, they are supported by the most popular uh a iml frameworks out there.

Now, a lot of the attention today is a focus around training because we're really at the start of um machine learning and general a i where customers are, are trying to build the best models out there. But once the the models are built and they're, they're at scale. Um the most of the cost is really going to be around inference.

Now, if you think about um an application like alexa, for example, right, we train the model for alexa uh periodically throughout the year. But think about how many inferences, how many times are you asking alexa a question like how's the weather? What time is it play my favorite song. We're literally getting millions and hundreds of millions of inferences to alexa every single day.

So the cost of inference in the long term is gonna outweigh the cost of training. So that's why our first chip we built was inferential one. And uh earlier this year, we announced the second generation of inferential, which is inferential two.

Now, these are uh super powerful, high performance energy efficient and low cost inference uh chips with up to 384 gigabytes of um of memory of a high bandwidth, uh high speed accelerated memory four x higher throughput and 10 x lower uh latency than our previous generation, if one instances and really, again, optimized for large uh large models, 100 billion uh plus parameter models at scale.

Now, chips and processors are one thing. But then it's really when we, when we think about ec2 and we think about what's made uh uh cloud computing popular, uh it's really about the whole system in addition to the processors and the instances, i'm gonna bring art back on stage and he's gonna talk about uh nitro and the underlying infrastructure.

All right. All right. All right, thanks presentation. We talked about the 750 instances that exist that being a lot of instances. But one of the core capabilities that have enabled us to do this is the development of our nitro system.

Back in 2012, 2013, we started realizing that we were how a lot of other companies in the cloud space work here is, is, you know, we have to do some management in order to parcel out and sell cp out to everyone. We started realizing that the management function was becoming more increasingly more consuming and that management function was resulting in that fully 25% of our capacity was being used by us to manage the software.

So we needed to figure out a way to rearchitect our entire system here in order to give better performance in the cloud. So we introduced the aws nitro system back in 2017, 2018. And it is the foundation of our modern instances, our modern ec2 instances. And i actually, i used to come here with a little bit of chart. And you could see once we started launching in 2017, you can see this massive increase in the total number of instances that we're able to launch.

The reason we're able to do that is we've now officially separated our management function away from the cp, away from the hardware function. And this is across the board. This has enabled us to launch more instances faster and it also has enabled us to actually develop a system that provides even better security throughout our entire architecture.

So the system is comprised of a nitro card here which has a series of chips on it, a nitro chip and a thin hypervisor that's on top and then inside that a security chip that makes sure that when we boot up and anything is happening that we verify the hardware and the software, no aws employee has access to data that's out there, that's part of this architecture.

But what this also does, it allows us to do a lot of behind the scenes work to do upgrades to the system without impacting customers and reduces the management function. So fully today, we actually, if you take a look at our instances, we sell 100% of that cpu to the customer and you get near bare metal performance and it is the nitrous system that is en enable that as well.

So what have we done since we be, since we started here? It's, it's hard to believe. But within our networking bandwidth here, back in 2006, we offered a stunning one gigabit per second in network bandwidth

Now, I assure you in 2006, if we were all sitting in this room, although I could have fit all of our customers in just this one room, we can't do that any longer today. That one gigabit per second was literally state of the art today. We look back and kind of chuckle about it, but since then, we have now offered up to 3200 gigabits per second, where needed here.

That's the type of performance though that's absolutely required. And Martin talked about the training in two instances here. We are now offering up to 6400 gigabits per second in that networking bandwidth here. So these numbers are, we're not just adding more networking because we have nothing else to do on a Thursday afternoon. But we're adding this because more. This is the demand that customers are supplying.

And this is you can see the arc of what's happening and I had to do this kind of build here. Otherwise the one gigabit on the far left looked even obnoxiously smaller as well. So just to make sure that we can see kind of what has happened here and what's happening with compute as well.

So not only this, we're talking about network first, but also the other thing, many of you customers are likely really interested in is this access to storage as well. So we've also changed the amount of memory, the amount of bandwidth that we have here to EBS for our EBS. And we've seen a great deal of interest in more performance here in our Elastic Block Store.

So in order to meet that demand, we now offer up to 100 gigabits per second with 400,000 IOPS here and that's across the board. So I mentioned memory here and I want to kind of close a little bit some of the Nitro System here by talking about the final piece of silicon if you will, I don't know that memory is necessarily everybody thinks about it as silicon, but we develop our own SSDs here.

Solid state drives here at Amazon. Why? Because the performance of these knowing our own cloud based system, you design, we're designing something here specific to the cloud. And one of the things we decided when we were doing this is that customers really concerned about latency and latency in accessing memory.

And there's two types of latency, there's just general latency and tail latency, which is the variability within that latency that happens there. So, one of our goals here was to introduce these SSDs here to provide the best possible experience and you know, latency and access to the memory translate into performance issues for customers because the longer it takes to get something from memory to be able to act on it in the CPU, the longer it takes you to do and accomplish a task.

So these SSDs that we introduced offer 60% better latency and we've also reduced that tail latency or the variability within the latency to create more predictable outcomes by 75%. So those are all part of that Nitro system, but that's not all the Nitro system affords.

I alluded to some of the security elements that are here and I want to just briefly talk about that security element is here that we finished. It's enhanced security that we've delivered. So when we launched in 2017, as I said, we restricted all operator access to data. So that means that no employee, not Martin, not myself, not anyone else you'll meet here can actually take a look at any data that's ever been exposed within the system. No AWS employee has that access.

This also offers us the ability to securely patch and modify the system without having any impact to customers by doing this offline. And the other thing is this, we are so committed to this, we have no SSH access. So there's no purpose access of any kind to, to the system at all here and we have no APIs that are provided to access this data.

And it's something, by the way, I might mention that we also have included in our terms and conditions, you know, the small font, tiny font that you probably read and click, accept or hopefully you accept when you start to use it. We have actually now included this information as part of those terms in service that this is something that we sign up to model uh uh when you work with AWS.

So the other thing is a lot of times customers have a need for additional security, develop something, excuse me, called Nitro Enclaves. So imagine a situation where you're at work and there's employees who need to access for perhaps personally identifiable or PII type information. And but this type of information is on a need to know basis pretty much and you know, certain employees, let's say I go to my local bank and I go up to the teller that teller may need to have access to my bank account information and to the the balance that's in that bank.

But another employee of that bank may have an interest in knowing the total aggregate amount of money that is being stored at the bank, but they shouldn't have access to my bank account and my bank information. So they have an interest in the total data, but they don't have the interest, they don't have the rights to the specific data about Art Bao.

And in that type of instance, you can protect that personally identifiable information so that somebody can get access to that result and product and only the right people can see or have access to the private information that's there. That's one of the reasons that we have developed and we had launched Nitro Enclaves here as well.

So back to my uh security element as I close out this section here is, is that uh this QR code here includes a link to the full report. We were, we were so convinced of this. We actually went with the an a third party cybersecurity company called the NCC group uh that did an evaluation of our, our architecture to certify that it is indeed no operator has access to any of this information. And so you can take a look at that.

And then the other thing that I want to mention, we are trying to be even more transparent about our architecture and about the security elements. So you can also look up our white paper on the Nitro systems security if you're so inclined and it's available publicly available as well for our site.

So I want to conclude today though, by talking about something that I think is really fun is some of our AI generated features in the natural language processing space. Oh, we talked about those 750 instances earlier and how can I make it even easier for you to select those instances? So I'm going, we're going to use, we are using generative AI here using Amazon Q, which can help you select additional in your instances and instance type.

I'm including an example here that I ran actually last week when I was setting this up here. So I typed in this query using natural language processing. And the reason I typed something this specific is because I want to get a specific answer out, but I want to show you how you can make you make use of this system to help select or define instances.

So here's my question, I want to deploy, deploy a large database on a web app getting high performance. Ok? So it's gonna think here for a second about it and give me some result. I don't plan to read this entire thing to all of you. But what it has done here is it provided me a series of answers. It's actually even ordered them based upon the criteria that I've typed in.

So the first year is what it thinks is the most recommended i put in high performance as one of the key things i put in database. So the first thing it did was recommend the latest AMD seventh generation R7A instances. But maybe there were some other qualifiers that I didn't include in my question. I mean, all of us have done that in a when communicating with a chat bot here.

So I typed in an additional question here to say what if I wanted them to be Intel based instead? What is the response then? So here it provides a more detailed response. Thanks for a second and provided a response that looks like I went forward here to my next question. Um I also offered a question here on video encoding as well to just take a look at a completely different workload. Also including a video encoding with high performance workload that I wanted to include.

Same thing here. They'll provide a response back and it provides three responses. And also the blurb at the bottom is a description about why it chose that response. And it shows that based upon that information that you provided, I like to use that blurb myself to say, ok, do I need more information? Do I need to ask it another question? In this case, I also asked a secondary question here, what instant choices provide better availability? Martin mentioned, you know, some availability things with GPUs particularly in this video space.

So I thought that this might be kind of a uh a great response to or a great question to ask it. So it now provides some information in a textual format here on that. And the final AI feature that I want to talk about today is called Console to Code. So a lot of people use the Amazon console to pro for prototyping purposes and think about a situation where you come to this meeting and you listen to this discussion here and you probably think in the back of your mind, I will remember this in about two hours from now.

There's certain aspects of it. Like, what did Art say? Like I have to revisit that presentation or something like that? Well, a lot of customers go into the console, do an act, do perform a task and later on realize, ok, this was successful. I want to duplicate this next time I come in to the console, I want to get a start.

So one of the things we've done using our Well Architected framework is is we have a way to actually go into the console to generate code so that you can take the work that you did prototyping and actually have a snippet of code to be able to replicate that later on and use that as a start as you go to do production based models.

So these are some screenshots that I took earlier today of the console itself. If you look on the left hand side there, you can see Console to Code, you can just simply click on that here. And you can see though in the center for those of you familiar with the console, you can see recorded actions here or in the center. So these are the actions that it's been recording that you've been taking.

In this case, I chose the first two actions you can see them lower highlighted here and you'll be able to select on them. And then on the upper right hand side, you can choose to select the type of code that you might want to generate. In this case, I have TypeScript code, but you can also choose Python or Java. Uh Dave Brown did an example earlier today with Python and so Adam also did the example with Python, but I chose TypeScript code because it actually is the most popular uh that our users or developers using the console tend to use.

So after you select that it will actually generate the TypeScript code for you and show it here in the screen and you can download that as a developer for usage later. The goal here is to take work that you did in the console and exploratory work that you did and be able to use that later on so that you don't forget it. This isn't right. This isn't geared as production based code, but as a starting point for you.

So I realize that today, Martin and I have covered a lot of ground here with the Amazon C2 instances. We started off talking about the x86 the Graviton instances, the Mac instances, we talked a little bit about HPC instances. And here we've ended with some of these examples that we had some of the newest features using generative AI.

In addition, this is we are not finished innovating here at Amazon. And you know, next year, if you show up here again, we're gonna have even more innovations as well to take a look at. So I want to thank you all for coming here.

And before I leave you, I have one final thing. If you enjoy taking a look and you enjoy learning about AWS. I encourage you to take a look at our new badge. Compute. This allows you the opportunity to learn a little bit more about compute while earning a free badge. And this is our new badge system that is free to take a look at.

So this is part of uh can be part of your learning journey and you can go to AWS Skill Builder and get a free badge. You can place the badge on, you know, your resume or on LinkedIn or whatnot uh that gets handed to you after you have taken uh some of this uh it's taken, excuse me, this uh this badge, excuse me. Um and there's some course work there.

So if you're not tired of hearing me speak uh as well, you can watch some more videos of me as well and answer questions about it. I want to thank all of you for coming out to Vegas and joining us here today. Please don't forget to take the session survey. We really do review all the feedback that has been provided here and I want to thank Martin for joining me and thank all.